Delayed Job (DJ)

Last Updated: 07 December 2013

background delayed job dj queueing ruby

Table of Contents

You can use Delayed Job 2.1 or later on Heroku.

Delayed Job, also known as DJ, makes it easy to add background tasks to your Rails applications on Heroku. You can also use Resque and many other popular background queueing libraries. Delayed Job uses your database as a queue to process background jobs. If your app has a high database load using DelayedJob may not be a good background queueing library for your app. To get started using Delayed Job you need to configure your application and then run a worker process in your app.

If you have questions about Ruby on Heroku, consider discussing it in the Ruby on Heroku forums.

Setting up Delayed Job

To use Delayed Job with PostgreSQL we’ll need to first add the gem to your Gemfile

gem 'delayed_job_active_record'

Run bundle install and then you need to create the table for queuing jobs.

$ rails generate delayed_job:active_record

Then migrate your database

$ rake db:migrate

You then need to tell your application to process jobs put into your job queue, you can do that by adding this to your Procfile:

worker:  bundle exec rake jobs:work

Now when you start your application using Foreman it will start processing your job queue.

$ foreman start

Queuing jobs

Now that you have Delayed Job set up on your system you’ll want to put jobs in the queue. There are a few different ways to do this, for all of them refer to delayed_job documentation

Delayed Job adds a delay method to ActiveRecord objects causing that method to be executed in the background. So if you have a blog post model

class Post < ActiveRecord::Base
  def send_to_twitter!
    Twitter.update("#{self.title} #{self.url}")
  end
end

Now you can call the method send_to_twitter! directly

Post.find(9).send_to_twitter!

Or you can delay that method by using the delay method which will add the method to the back of you queue.

Post.find(9).delay.send_to_twitter!

Queuing emails

Since sending email is not instantaneous it rarely makes sense to require a user to wait on it being sent. You can use the delay method on your mailers directly. So if we have a mailer called UserMailer and we are sending out a registration email it might look like this:

UserMailer.send_registration_mail(email, name).deliver

we can use the delay method on the mailer to send the email in a background task instead

UserMailer.delay.send_registration_mail(email, name)

It is considered a good practice to send all emails in the background.

Deploy

Now that our code is working locally, you’ll want to deploy your code to Heroku.

$ git push heroku master
Counting objects: 4, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 315 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)

-----> Heroku receiving push
-----> Ruby/Rails app detected
-----> Installing dependencies using Bundler version 1.2.0.pre
       Running: bundle install --without development:test --path vendor/bundle --binstubs bin/ --deployment
       Using rake (0.9.2.2)
       ...
       Using sass (3.1.19)
       Using sass-rails (3.2.5)
       Using twitter-bootstrap-rails (2.0.1)
       Using uglifier (1.2.5)
       Using wicked (0.1.6)
       Your bundle is complete! It was installed into ./vendor/bundle
       Cleaning up the bundler cache.
-----> Writing config/database.yml to read from DATABASE_URL
-----> Preparing app for Rails asset pipeline
       Please see this article for troubleshooting help:
       http://devcenter.heroku.com/articles/rails31_heroku_cedar#troubleshooting
-----> Rails plugin injection
       Injecting rails_log_stdout
       Injecting rails3_serve_static_assets
-----> Discovering process types
       Procfile declares types      -> web
       Default types for Ruby/Rails -> console, rake, worker
-----> Compiled slug size is 15.1MB
-----> Launching... done, v9
       http://furious-robot-218.herokuapp.com deployed to Heroku

To git@heroku.com:furious-robot-218
   47ae11e..0d13ad1  master -> master

Don’t forget to migrate your database

$ heroku run db:migrate

Heroku should be running your worker process. You should be able to see it when running:

$ heroku ps

If you don’t see your worker you may need to scale it up with the heroku command:

$ heroku ps:scale worker=1

Debugging

To see the last element in your queue you can run the following from the console:

$ heroku run console
>> Delayed::Job.last
=> #<Delayed::Job id: 8, priority: 0, attempts: 0, handler: "--- !ruby/obj ...

The job won’t execute until a worker process exists to consume it. Be sure a worker process is started and running:

$ heroku ps
Process   State      Command

web.1     up for 1h  bundle exec thin start -p $PORT
worker.1  up for 1h  bundle exec rake jobs:work

View worker process output by filtering the logs with the -p flag.

$ heroku logs -p worker -t
2012-04-26T20:25:37+00:00 app[worker.1]: ...

The worker process can be manually invoked for further isolation.

$ heroku run bundle exec rake jobs:work
*** Starting job worker host:silver pid:4227
1 jobs processed at 7.0823 j/s, 0 failed ...

If a job fails it will have a last_error field, you can use this to get information on why it failed.

job = Delayed::Job.where("last_error is not null").last
puts job.last_error
"{OAuthException: Error validating access token: ...

Further reading