Declaring and Scaling Process Types with Procfile
Table of Contents
Procfile is a mechanism for declaring what commands are run when your web or worker dynos are run on the Heroku platform. It follows the process model on the large scale of the Heroku dyno manifold You can use Procfile to declare any other process types, such as a multiple types of workers, or a singleton process like a clock or a consumer of the Twitter streaming API.
Procfile and the process management commands (heroku run and heroku scale) are only available on the Cedar stack.
Process Types as Templates
A Procfile is a list of process types in an application. Each process type is a declaration of a command that is executed when a process of that process type is executed.
All the language and frameworks on the Cedar stack declare a web process type, which starts the application server. Rails 3 has the following process type:
web: bundle exec rails server -p $PORT
Clojure’s web process type looks something like this:
web: lein run -m demo.web $PORT
You can reference other environment variables populated by Heroku, most usefully the $PORT variable, in the command.
A Maven-generated batch file executing the Tomcat Java application server:
web: sh target/bin/webapp
For many apps, these defaults will be sufficient. For more sophisticated apps, and to adhere to the recommended approach of more explicitly declaring of your application’s required runtime processes, you may wish to define your process types. For example, Rails applications are supplied with an additional process type of this sort:
worker: bundle exec rake jobs:work
Declaring Process Types
Process types are declared via a file named Procfile placed in the root of your app. Its format is one process type per line, with each line containing:
<process type>: <command>
<process type> is a name for your process, such as web, worker, urgentworker, clock, etc. <command> is a command line to launch the process, such as rake jobs:work.
Developing Locally with Foreman
It’s important when developing and debugging an application that the local development environment is executed in the same manner as the remote environments. This ensures that incompatibilities and hard to find bugs are caught before deploying to production and treats the application as a holistic unit instead of a series of individual commands working independently.
Foreman is a command-line tool for running Procfile-backed apps. It’s installed automatically by the Heroku Toolbelt, and also available as a Ruby gem.
If you had a Procfile with both a web and worker processes, Foreman will start one of each process type, with the output interleaved on your terminal:
Run your app with Foreman:
$ foreman start
18:06:23 web.1 | started with pid 47219
18:06:23 worker.1 | started with pid 47220
18:06:25 worker.1 | (in /Users/adam/myapp)
18:06:27 web.1 | => Awesome web application server output
Your web process loads on port 5000 because this is what Foreman provides as a default in the $PORT env var. It’s important that your web process respect this value, since it’s used by the Heroku platform when you deploy.
You can now test the app locally. Press Ctrl-C when done to shut it down.
Deploying to Heroku
A Procfile is not necessary to deploy apps written in most languages supported by Heroku. The platform automatically detects the language, and creates a default web process type to boot the application server.
Creating an explicit Procfile is recommended for greater control and flexibility over your app.
For Heroku to use your Procfile, add the Procfile to the root of your application push to Heroku:
$ git add .
$ git commit -m "Procfile"
$ git push heroku
...
-----> Procfile declares process types: web, worker
Compiled slug size is 10.4MB
-----> Launching... done
http://strong-stone-297.herokuapp.com deployed to Heroku
To git@heroku.com:strong-stone-297.git
* [new branch] master -> master
Use heroku ps to determine the number of processes that are executing. The list indicates the process type in the left column, and the command corresponding to that process type in the right column:
$ heroku ps
Process State Command
------------ ------------------ ------------------------------
web.1 up for 2m bundle exec rails server mongrel -..
Use heroku logs to view an aggregated list of log messages from all process types.
$ heroku logs
2011-04-26T01:24:20-07:00 heroku[slugc]: Slug compilation finished
2011-04-26T01:24:22+00:00 heroku[web.1]: Running process with command: `bundle exec rails server mongrel -p 46999`
2011-04-25T18:24:22-07:00 heroku[web.1]: State changed from starting to bouncing
2011-04-25T18:24:22-07:00 heroku[web.1]: State changed from created to starting
2011-04-25T18:24:29-07:00 heroku[web.1]: State changed from starting to up
2011-04-26T01:24:29+00:00 app[web.1]: => Booting Mongrel
2011-04-26T01:24:29+00:00 app[web.1]: => Rails 3.0.5 application starting in production on http://0.0.0.0:46999
2011-04-26T01:24:29+00:00 app[web.1]: => Call with -d to detach
2011-04-26T01:24:29+00:00 app[web.1]: => Ctrl-C to shutdown server
Scaling a Process Type
Heroku runs one web process for you automatically, but other process types don’t start by default. To launch a worker, you need to scale it up to one dyno:
$ heroku scale worker=1
Scaling worker processes... done, now running 1
Check ps to see the new process type running, for example:
$ heroku ps
Process State Command
------------ ------------------ ------------------------------
web.1 up for 5m bundle exec rails server mongrel -..
worker.1 up for 5s env QUEUE=* bundle exec rake resque:work
Use heroku logs -ps worker to view just the messages from the worker process type:
$ heroku logs -ps worker
2011-04-25T18:33:25-07:00 heroku[worker.1]: State changed from created to starting
2011-04-26T01:33:26+00:00 heroku[worker.1]: Running process with command: `env QUEUE=* bundle exec rake resque:work`
2011-04-25T18:33:29-07:00 heroku[worker.1]: State changed from starting to up
2011-04-26T01:33:29+00:00 app[worker.1]: (in /app)
The output we see here matches our local output from Foreman, interleaved with system messages from the Heroku process manager about the state of the process.
You can scale up higher with the same command. For example, two web processes and four worker processes:
$ heroku scale web=2 worker=4
Scaling web processes... done, now running 2
Scaling worker processes... done, now running 4
$ heroku ps
Process State Command
------------ ------------------ ------------------------------
web.1 up for 7m bundle exec rails server mongrel -..
web.2 up for 2s bundle exec rails server mongrel -..
worker.1 up for 7m env QUEUE=* bundle exec rake resque:work
worker.2 up for 3s env QUEUE=* bundle exec rake resque:work
worker.3 up for 2s env QUEUE=* bundle exec rake resque:work
worker.4 up for 3s env QUEUE=* bundle exec rake resque:work
More Process Type Examples
The Procfile model of running processes is extremely flexible. You can run any number of processes with whatever arbitrary commands you want, and scale each independently.
For example, using Ruby you could run two types of queue workers, each consuming different queues:
worker: env QUEUE=* bundle exec rake resque:work
urgentworker: env QUEUE=urgent bundle exec rake resque:work
These can then be scaled independently.
$ heroku scale worker=1 urgentworker=5