Heroku makes it easy to use DJ in your application. You can read more about how background jobs work. There are two steps to using DJ: the first is to configure your application to use it, and the second is to run one or more worker processes in your app.

Setting Up Delayed Job

Delayed Job, also known as DJ, is a queueing system for Rails written by Tobias Lütke of Shopify. It stores jobs in a table named delayed_jobs in your app’s database.

To use, install the DJ plugin in your Rails app:

$ script/plugin install git://github.com/collectiveidea/delayed_job.git

Create a database migration for the delayed_jobs table:

$ script/generate migration create_delayed_jobs

Edit the created migration to contain:

class CreateDelayedJobs < ActiveRecord::Migration
  def self.up
    create_table :delayed_jobs, :force => true do |table|
      table.integer  :priority, :default => 0      # jobs can jump to the front of
      table.integer  :attempts, :default => 0      # retries, but still fail eventually
      table.text     :handler                      # YAML object dump
      table.text     :last_error                   # last failure
      table.datetime :run_at                       # schedule for later
      table.datetime :locked_at                    # set when client working this job
      table.datetime :failed_at                    # set when all retries have failed
      table.text     :locked_by                    # who is working on this object
      table.timestamps
    end
  end
  def self.down
    drop_table :delayed_jobs
  end
end

Run your migrations, and now you’re ready to put something in your queue.

Continuing the feed example from the previous section, you might have a class like this:

class Feed < ActiveRecord::Base
  def perform
    xml = RestClient.get url
    # ... do work to read and parse the feed and place it in the database
  end
end

DJ can queue any object that responds to a perform method. In this case, calling perform will cause the feed to go out and fetch the latest info from the URL, and store the results in the database. Now we can queue it with this bit of code:

Delayed::Job.enqueue Feed.create!(:url => 'http://adam.blog.heroku.com/feed')

This returns immediately, as no work has been done yet. The job is now on the queue, which you can see in your Rails console:

>> Delayed::Job.find(:first)
=> #<Delayed::Job id: 8, priority: 0, attempts: 0, handler: "--- !ruby/obj ...

The job won’t execute until a worker process exists to consume it. Start the worker process like this:

$ rake jobs:work
(in /home/adam/feedreader)
*** Starting job worker host:silver pid:4227
1 jobs processed at 7.0823 j/s, 0 failed ...

The worker process finds the job in queue, and performs it. If you leave this running and add more jobs to the queue, you’ll see those jobs being processed by the worker. Press Ctrl-C to exit when you no longer wish to process jobs.

Running DJ Workers

Delayed Job workers can be declared in your Procfile and run just like any other type of dyno on Cedar:

Procfile

worker: bundle exec rake jobs:work

Then, scale up:

$ heroku scale worker=1

If you’re on the older Aspen or Bamboo stacks, see Running Delayed Job Workers on Aspen/Bamboo.

Further Reading

• Tobias Lütke’s introductory blog post on DJ
• The delayed_job github project
• John Nunemaker’s DJ tutorial
• Building a Queue-Backed Feed Reader
• The Rails Way on Delayed Job