This add-on is operated by Crazy Ant Labs

Run scheduled tasks on your Heroku Apps.

Cron To Go

Last updated 25 June 2020

Cron To Go is an add-on for running scheduled jobs on one-off dynos using cron expressions. Cron To Go combines the simplicity of using cron to schedule jobs with the scale and elasticity of the cloud. Instead of running always-on processes, use Cron To Go to start one-off dynos only when you need them. Another benefit of using Cron To Go over code based schedulers is that you don’t need to deploy new code when you have to make changes or additions to your scheduled jobs - simply add, edit or delete a job from the Cron To Go dashboard.

Provisioning the add-on

Cron To Go can be attached to a Heroku application via the CLI:

$ heroku addons:create crontogo
-----> Adding crontogo to sharp-mountain-4005... done, v18 (free)

After you provision Cron To Go, you can use its dashboard to define and schedule jobs.

Free Trial

The free trial allows you to experiment with all features of the silver plan for 7 days. After 7 days, the jobs are disabled and you will only be able to run ad-hoc jobs or enable the jobs after you upgrade to a paid plan.

Popular use cases

• Running periodic SQL statements on Heroku Postgres. You can refresh materialized views, run cleanup statements to get rid of stale or old data or perform BI processes such as in-database transformations. The following example makes use of psql command line with the DATABASE_URL environment variable to refresh a materialized view: psql $DATABASE_URL -c "REFRESH MATERIALIZED VIEW vw_account_counters"
• Hitting HTTP endpoints. You can make HTTP requests to your web API or 3rd party API endpoints easily using Curl to start application processes. The following example uses curl to hit a 3rd party endpoint: curl http://itsthisforthat.com/api.php?json
• Scheduling periodical batch jobs such as grabbing orders from Shopify, generating weekly reports or sending email notifications to application users by running your own code. The examples below show how to run Rails and Node apps using Cron To Go.

Dashboard

The Cron To Go dashboard allows you to:

• Create jobs
• Import jobs from other scheduling systems
• Edit job definitions
• Delete jobs
• Duplicate a job to create a new job based on an existing job
• Pause an active job
• Resume an inactive job
• View job logs
• View job history - see previous executions status and duration
• Change notification settings

You can access the dashboard via the CLI:

$ heroku addons:open crontogo
Opening crontogo for sharp-mountain-4005

or by visiting the Heroku Dashboard and selecting the application in question. Select Cron To Go from the Add-ons menu.

Importing jobs

Click the import button to smoothly migrate from other scheduling systems. With a single click you can import all jobs defined in your app’s Heroku Scheduler. This allows you to define your job schedule using cron expressions and manage all jobs in a single place.

Defining Jobs

A job in Cron To Go consists of the following:

• Optional nickname (for example, Daily ETL).
• Job schedule, in cron expression format. The smallest interval you can specify is every minute.
• Timezone. UTC is the default timezone.

• Command to execute.
• Size of one-off dyno to execute the command in. Read more about available one-off dyno types.
• One-off dyno timeout in seconds. Sets the time for the one-off dyno to expire, after which it will be killed. You can read more about it here.

When triggered, these jobs run in one-off dynos and show up in your logs as a dyno named crontogo_[job_id].

Cron expression examples

You can use crontab.guru to generate other cron expressions.

Monitoring and logging

One-off dynos that execute as part of Cron To Go output their logs with process crontogo_[job_id].

To retrieve the logs for the dynos started by Cron To Go run the following command:

$ heroku logs -t -a <app-name> -d crontogo

Use the dashboard to view the streamed dyno log from Heroku. For more production-ready persistence of logs, you can add one of the Heroku platform’s available logging add-ons to your app.

Notifications

Cron To Go monitors your job executions and determines execution status by Heroku’s API response status code and the process exit code. When a process exits with code 0 (zero), execution is successful. Otherwise (API response status code is not 2xx or process exit code greater than 0), the job is marked as failed.

By default, with plans that support notifications, app members will be notified by email every time a job starts to fail and then when it finished successfully after one or more consecutive failures.

You can change these settings by clicking Notifications in dashboard:

• Change the notification trigger, to only get emails when jobs start to fail.
• Set recipient emails manually (the preferred approach is to use your email platform to create an alias and manage group membership instead of managing email lists in Cron To Go).
• Silence notifications by deselecting both app members and email recipients.

Webhooks

Webhooks enable you to receive notifications whenever particular events occur with your jobs. You can subscribe to notifications for the following events:

• Job Execution Failed - occurs whenever a job execution fails to complete.
• Job Execution Started - occurs whenever a job execution is started.
• Job Execution Succeeded - occurs whenever a job execution completes successfully.

Webhook notifications are sent as HTTPS POST requests to a URL of your choosing. To integrate with webhooks, you need to implement a server endpoint that receives and handles these requests.

To subscribe to webhooks, go to the webhooks interface in the dashboard by clicking Webhooks. Then click the Add webhook button.

In the dialog that opens, fill out the following:

• Nickname (optional) - a descriptive name for your webhook.
• Endpoint URL - HTTPS URL of your server endpoint that will receive all webhook notifications.
• Authorization Header (optional) - a custom Authorization header that will be included with all webhook notifications.
• Topics - select the types of notifications you want to be informed about. You must include at least one topic.

And click Add webhook.

Securing Webhooks

When a webhook is created, a signing secret is generated and displayed once.

Each request is signed by Cron To Go in the X-Hub-Signature header. If you’d like to verify the authenticity of our request, copy the and use it to verify the webhook signature. You can rotate the signing secret at any time and get a new one.

You may also use an authorization header to verify that the request came from Cron To Go. If set, the authorization header is passed through in the Authorization header in the request. It should be validated using the authorization mechanism of your choice on your server.

Managing Webhooks

After creating a webhook, you may do the following:

• Pause/Resume - temporarily pause webhook notifications or resume them.
• Rotate secret - see Securing Webhooks
• Ping webhook - manually send a test event to your endpoint
• View deliveries - View a log of the notifications that Cron To Go has enqueued to send. Each notification has a status (one of Succeeded, Failed, ‘Pending’), Created timestamp, ID, Topic (one of job.execution.failed, job.execution.started, job.execution.succeeded, webhook.ping) and Duration. You may also view the Request payload, Response code and Response body.

Receiving Webhooks

When a webhook event that you’ve subscribed to occurs, Cron To Go sends a POST request to your server endpoint with the details of the event.

You can verify the authenticity of these requests in the following ways:

• The request’s Authorization header matches the value you provided when subscribing to notifications.
• The request’s X-Hub-Signature header contains the HMAC SHA256 signature of the request body (signed with the secret value provided when subscribing).

A resulting webhook notification request resembles the following:

POST https://webhook.site/394f2074-e56f-4110-7bf7-ca14a1f48b7c
Authorization: Bearer 01234567-89ab-cdef-0123-456789abcdef
X-Hub-Signature: cLcN5U5x+jHEkANnVaaRwBw7yE4uv4pXdjcY9Cajc7M=

{
  "Metadata": {
    "Webhook": {
      "Id": "6b1c6f0c-52c1-47a6-9344-57a4579ced68"
    },
    "Delivery": {
      "Id": "c7ce90e6-5708-43b1-a052-13991f45c771"
    },
    "Attempt": {
      "Id": "2e241efe-d22f-4fe7-aab9-adcde63fca6d"
    },
    "Event": {
      "Id": "2e579289-4c4a-4085-9b43-74020865cdda",
      "Topic": "webhook.ping"
    }
  },
  "CreatedAt": 1590911947688,
  "Resource": "webhook",
  "PreviousData": null,
  "Data": {
    "Topics": [
      "job.execution.succeeded",
      "job.execution.failed",
      "job.execution.started"
    ],
    "State": "paused",
    "Alias": "Test Webhook",
    "CreatedAt": 1590501031122,
    "Id": "6b1c6f0c-52c1-47a6-9344-57a4579ced68",
    "OrganizationId": "d57060b1-23fe-2d59-afd0-7f56d9e1fc55",
    "UpdatedAt": 1590911941179,
    "Url": "https://webhook.site/394f2074-e56f-4110-7bf7-ca14a1f48b7c"
  },
  "Id": "2e579289-4c4a-4085-9b43-74020865cdda",
  "Topic": "webhook.ping",
  "UpdatedAt": 1590911947688
}

You should always respond with a 200-level status code to indicate that you received the notification. Cron To Go ignores the body of your response, so a 204 status with an empty body is ideal:

204 No Content

job.execution.started Event Format

{
  "Id": "22e45a18-0a91-470d-b0d4-846b2fa832cb",
  "Topic": "job.execution.started",
  "CreatedAt": 1590591047428,
  "UpdatedAt": 1590591047428,
  "Resource": "JobExecution",
  "PreviousData": null,
  "Data": {
    "AttemptsCount": 1,
    "TriggerType": "manual",
    "Target": {
      "Type": "dyno",
      "TimeToLive": 1800,
      "AppId": "22e1e196-fbc6-4091-af57-8d7ad63aa7d3",
      "Command": "python run_etl.py",
      "Size": "Hobby"
    },
  }
  "State": "running",
  "CreatedAt": 1590591046983,
  "Id": "43668791-ce11-4994-9e91-fc8fda310614",
  "OrganizationId": "d57060b1-23fe-2d59-afd0-7f56d9e1fc55",
  "NextAttemptAt": null,
  "UpdatedAt": 1590591046983,
  "JobId": "6069e1c0-280d-22a3-93ce-159d9a60924d",
  "LastAttempt": {
    "State": "running",
    "CreatedAt": 1590591046512,
    "Id": "a28c2010-549c-4253-a869-42b748998477",
    "StatusCode": 201
  },
  "Metadata": {
    "Webhook": {
      "Id": "6b1c6f0c-52c1-47a6-9344-57a4579ced68"
    },
    "Delivery": {
      "Id": "e69d0b8a-447d-4ddc-a21a-704630272353"
    },
    "Attempt": {
      "Id": "7ecb9953-dd36-4309-8db6-0043f989b7aa"
    },
    "Event": {
      "Id": "22e45a18-0a91-470d-b0d4-846b2fa832cb",
      "Topic": "job.execution.started"
    }
  },
}

job.execution.succeeded Event Format

{
  "Id": "5b8f2e1c-4a53-4db6-b6ae-a88786f77459",
  "Topic": "job.execution.succeeded",
  "CreatedAt": 1590591016589,
  "UpdatedAt": 1590591016589,
  "Resource": "JobExecution",
  "PreviousData": null,
  "Data": {
    "AttemptsCount": 1,
    "TriggerType": "schedule",
    "Target": {
      "Type": "dyno",
      "TimeToLive": 1800,
      "AppId": "22e1e196-fbc6-4091-af57-8d7ad63aa7d3",
      "Command": "python run_etl.py",
      "Size": "Hobby"
    },
    "State": "succeeded",
    "CreatedAt": 1590591011407,
    "Id": "acc83806-505c-46a1-b9a9-04025f7d66fe",
    "OrganizationId": "d57060b1-23fe-2d59-afd0-7f56d9e1fc55",
    "NextAttemptAt": null,
    "UpdatedAt": 1590591016222,
    "JobId": "6069e1c0-280d-22a3-93ce-159d9a60924d",
    "LastAttempt": {
      "ExitStatus": 0,
      "State": "succeeded",
      "CreatedAt": 1590591011017,
      "Id": "2f858f6e-1adb-46ac-945f-b86230ec2076",
      "StatusCode": 201
    }
  },
  "Metadata": {
    "Webhook": {
      "Id": "6b1c6f0c-52c1-47a6-9344-57a4579ced68"
    },
    "Delivery": {
      "Id": "277d0c61-1eff-46f2-8e8a-22d9c144d11c"
    },
    "Attempt": {
      "Id": "6c6128fd-ca7f-4828-997f-0084cc15d4b1"
    },
    "Event": {
      "Id": "5b8f2e1c-4a53-4db6-b6ae-a88786f77459",
      "Topic": "job.execution.succeeded"
    }
  }
}

job.execution.failed Event Format

{
  "Id": "265a4af1-7db2-41ad-a3d1-79f7ef2b20ac",
  "Topic": "job.execution.failed",
  "CreatedAt": 1590591051225,
  "UpdatedAt": 1590591051225,
  "Resource": "JobExecution",
  "PreviousData": null,
  "Data": {
    "AttemptsCount": 1,
    "TriggerType": "manual",
    "Target": {
      "Type": "dyno",
      "TimeToLive": 1800,
      "AppId": "22e1e196-fbc6-4091-af57-8d7ad63aa7d3",
      "Command": "python run_etl.py",
      "Size": "Hobby"
    },
    "State": "failed",
    "CreatedAt": 1590591046983,
    "Id": "43668791-ce11-4994-9e91-fc8fda310614",
    "OrganizationId": "d57060b1-23fe-2d59-afd0-7f56d9e1fc55",
    "NextAttemptAt": null,
    "UpdatedAt": 1590591051024,
    "JobId": "6069e1c0-280d-22a3-93ce-159d9a60924d",
    "LastAttempt": {
      "ExitStatus": 7,
      "Message": "Process exited with status 7",
      "State": "failed",
      "CreatedAt": 1590591046512,
      "Id": "a28c2010-549c-4253-a869-42b748998477",
      "StatusCode": 201
    }
  },
  "Metadata": {
    "Webhook": {
      "Id": "6b1c6f0c-52c1-47a6-9344-57a4579ced68"
    },
    "Delivery": {
      "Id": "a03167ee-e26c-47dc-a798-f3fa4eb2a41d"
    },
    "Attempt": {
      "Id": "c56f1bbf-b154-4f0b-96dd-d500946cbd55"
    },
    "Event": {
      "Id": "265a4af1-7db2-41ad-a3d1-79f7ef2b20ac",
      "Topic": "job.execution.failed"
    }
  }
}

webhook.ping Event Format

{
  "Id": "2e579289-4c4a-4085-9b43-74020865cdda",
  "Topic": "webhook.ping",
  "CreatedAt": 1590911947688,
  "UpdatedAt": 1590911947688,
  "Resource": "webhook",
  "PreviousData": null,
  "Data": {
    "Topics": [
      "job.execution.succeeded",
      "job.execution.failed",
      "job.execution.started"
    ],
    "State": "paused",
    "Alias": "Test Webhook",
    "CreatedAt": 1590501031122,
    "Id": "6b1c6f0c-52c1-47a6-9344-57a4579ced68",
    "OrganizationId": "d57060b1-23fe-2d59-afd0-7f56d9e1fc55",
    "UpdatedAt": 1590911941179,
    "Url": "https://webhook.site/394f2074-e56f-4110-7bf7-ca14a1f48b7c"
  },
  "Metadata": {
    "Webhook": {
      "Id": "6b1c6f0c-52c1-47a6-9344-57a4579ced68"
    },
    "Delivery": {
      "Id": "c7ce90e6-5708-43b1-a052-13991f45c771"
    },
    "Attempt": {
      "Id": "2e241efe-d22f-4fe7-aab9-adcde63fca6d"
    },
    "Event": {
      "Id": "2e579289-4c4a-4085-9b43-74020865cdda",
      "Topic": "webhook.ping"
    }
  }
}

Long running jobs

You can control the maximum execution time for a job using the timeout setting for the job. You can make sure only one instance of a job executes by setting the timeout to a value lower than the frequency in which the jobs executes as set by the cron expression.

API Access

The environment variables CRONTOGO_API_KEY and CRONTOGO_ORGANIZATION_ID are added to the app when Cron To Go add-on is provisioned and are used for API access. API access is currently in beta. Reach out to us at support@crazyantlabs.com if you’d like to join the beta.

Using with Rails

Cron To Go can run any command that can be run in your application. In Rails, the convention is to set up rake tasks. To create tasks in Rails, copy the code below to lib/tasks/scheduler.rake and customize it to fit your needs:

desc "This task is called by the Cron To Go"
task :eat_breakfast => :environment do
  puts "Eating breakfast..."
  Kitchen.eat
  puts "done."
end

task :exercise => :environment do
  puts "Starting workout..."
  Gym.workout
  puts "done."
end

After writing a task and deploying it to Heroku, you can test it using heroku run:

$ heroku run rake eat_breakfast

After testing on Heroku, you can add a job to Cron To Go that specifies rake eat_breakfast as the command to run.

Using with Node.js

Cron To Go can run any command that can be run in your application. For Node.js, you can simply add a file to your app called schedules.js and execute it:

module.exports.eat_breakfast = function()
{
        console.log('Eating breakfast...');
        Kitchen.eat();
        console.log('done.');
}

require('make-runnable');

After writing your task and deploying it to Heroku, test it using heroku run:

$ heroku run node schedules.js eat_breakfast

After testing on Heroku, you can add a job to Cron To Go that specifies node schedules.js eat_breakfast as the command to run.

Debugging failed jobs

Jobs may fail because of Heroku errors or application errors.

Heroku errors may have indicate a service-wide error (e.g. "Service temporarily unavailable. Please try again later") or errors that have to do with the dyno type you chose ("Requested type Standard-1X is not available. Please use Hobby" or "Cannot run more than 1 Free size dynos."). In such cases, make sure to select a dyno type that conforms with your app and that you adhere to the limit on concurrent dynos of the selected type: either change your job schedules do make sure there are less concurrent jobs using this dyno type, or ask Heroku to increase your limit. See here for more information.

When your job fails due to application errors, you can check your application logs in Cron To Go by clicking Actions -> View Job Logs or by searching for the job id in your logging add-on.

You may also want to reproduce the error by running the job command using heroku run - simply copy the command and run it in your terminal to find the culprit. For example:

$ heroku run python db-cleanup.py --size Hobby --timeout 1800 --app cyberdyne-backoffice
Running python db-cleanup.py --timeout 1800 on ⬢ cyberdyne-backoffice... up, run.9986 (Hobby)
python: can't open file 'db-cleanup.py': [Errno 2] No such file or directory

Frequently Asked Questions

Q: In Cron To Go, I can only select Hobby/Free Dynos, but when trying to run the job I get the following error: Requested type Hobby is not available. Please use Standard-1X, etc.

A: Cron To Go shows the dyno types available for your app. If you’re getting this message, you probably have to add a procfile to your app. After adding it, you will be able to select the appropriate dyno type.

Q: I get the error Cannot run more than X <type> size dynos. What can I do?

A: There are limits imposed by Heroku on the concurrent number of one-off dynos. To remedy these issues, either change the dyno type, the frequency of your jobs or ask Heroku to increase your limits. In addition, if you use private space dyno types with low timeout values, this can also be the cause (See Long Running Jobs).

Q: I’m using Cron To Go with PHP and I don’t see anything in my job logs. What’s wrong?

A: Artisan seems to append > '/dev/null' 2>&1 to the command you run. Possible solutions are outlined here.

Migrating between plans

Use the heroku addons:upgrade command to migrate to a new plan.

$ heroku addons:upgrade crontogo:starter
-----> Upgrading crontogo:newplan to sharp-mountain-4005... done, v18 ($49/mo)
       Your plan has been updated to: crontogo:starter

Removing the add-on

You can remove Cron To Go via the CLI:

$ heroku addons:destroy crontogo
-----> Removing crontogo from sharp-mountain-4005... done, v20 (free)

Known issues

Cron To Go and Container Registry

If you are using Cron To Go and Container Registry as your deployment method, your process must be accessible from the web image. If your process requires a custom process type, please reach out to us at support@crazyantlabs.com to join the private beta.

Support

All Cron To Go support and runtime issues should be submitted via one of the Heroku Support channels. Any non-support related issues or product feedback is welcome at support@crazyantlabs.com.