Heroku

How It Works

Getting Started with Django on Heroku/Cedar

Last Updated: 20 January 2012

django python

Table of Contents

This guide will show you how to deploy a Python/Django application to Heroku using a Postgres database. For other Python apps, see Getting Started with Python on Heroku/Cedar.

Prerequisites

Start a Django App Inside a Virtualenv

Start by making an empty top-level directory for the project:

$ mkdir hellodjango && cd hellodjango

Create a Virtualenv (v0.7):

$ virtualenv venv --distribute
New python executable in venv/bin/python
Installing distribute...............done.
Installing pip...............done.

To activate the new environment, you’ll need to source it:

Windows users can run vevn\Scripts\activate.bat for the same effect.

$ source venv/bin/activate

This will change your prompt to include the project name. (You must source the virtualenv environment for each terminal session where you wish to run your app.)

Install dependencies with pip:

$ pip install Django psycopg2
Downloading/unpacking Django
  Downloading Django-1.3.1.tar.gz (6.5Mb): 6.5Mb downloaded
  Running setup.py egg_info for package Django

Downloading/unpacking psycopg2
  Downloading psycopg2-2.4.4.tar.gz (648Kb): 648Kb downloaded
  Running setup.py egg_info for package psycopg2

Installing collected packages: Django, psycopg2
  Running setup.py install for Django
  Running setup.py install for psycopg2
    ...
Successfully installed Django psycopg2
Cleaning up...

Now we can create a Django app in a subdirectory:

$ django-admin.py startproject hellodjango

Note that we’ve packaged the app, placing the Django app in a subdirectory of our main project. This keeps our main Django app separate from Virtualenv artifacts and other Django apps we might want to add to the directory.

Test that the app runs locally by starting up the Django development webserver:

$ python hellodjango/manage.py runserver
Validating models...

0 errors found
Django version 1.3, using settings 'hellodjango.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

Now you have a working copy of our application with dependencies. Create a pip requirements file which declares our required Python modules:

$ pip freeze > requirements.txt

Make sure that there aren’t any “editable” modules (lines that start with a -e) in this list. They should never be used in production.

All packages required should be declared explicitly in requirements.txt:

$ cat requirements.txt
Django==1.3
psycopg2==2.4.2

Commit to Git

Exclude Virtualenv artifacts from source control tracking:

GitHub provides an excellent Python gitignore file that can be installed system-wide.

.gitignore

venv
*.pyc

You will use Git to deploy code to Heroku. You don’t have to use it as your main source control system; Git can coexist with Mercurial, Subversion, or other revision control systems.

Initialize a Git repo and commit the project (if you aren’t already tracking your project with Git):

$ git init
Initialized empty Git repository in /Users/adam/hellodjango/.git/
$ git add .
$ git commit -m "my django app"
[master (root-commit) 8c07531] my django app
 5 files changed, 184 insertions(+), 0 deletions(-)
 create mode 100644 .gitignore
 create mode 100644 hellodjango/__init__.py
 create mode 100644 hellodjango/manage.py
 create mode 100644 hellodjango/settings.py
 create mode 100644 hellodjango/urls.py
 create mode 100644 requirements.txt

Deploy to Heroku

Create an app on the Cedar stack:

$ heroku create --stack cedar
Creating afternoon-sword-29... done, stack is cedar
http://afternoon-sword-29.herokuapp.com/ | git@heroku.com:afternoon-sword-29.git
Git remote heroku added

Deploy your app:

$ git push heroku master
Counting objects: 9, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (9/9), 3.01 KiB, done.
Total 9 (delta 0), reused 0 (delta 0)

-----> Heroku receiving push
-----> Python/Django app detected
-----> Preparing virtualenv version 1.7
       New python executable in ./bin/python2.7
       Also creating executable in ./bin/python
       Installing distribute............done.
       Installing pip...............done.
-----> Byte-compiling code
-----> Django settings injection
       Injecting code into hellodjango/settings.py to read from DATABASE_URL
-----> Installing dependencies using pip version 1.2.0
       Downloading/unpacking Django==1.3 (from -r requirements.txt (line 1))
       ...
       Successfully installed Django psycopg2
       Cleaning up...
-----> Discovering process types
       Procfile declares types         -> (none)
       Default types for Python/Django ->web
-----> Compiled slug size is 8.0MB
-----> Launching... done, v3
       http://afternoon-sword-29.herokuapp.com deployed to Heroku

To git@heroku.com:afternoon-sword-29.git
* [new branch]      master -> master

Check to see that your web process is up:

$ heroku ps
Process       State               Command
------------  ------------------  ------------------------------
web.1         up for 4s           python hellodjango/manage.py r..

View your logs:

$ heroku logs
2011-08-27T07:58:00+00:00 heroku[web.1]: Starting process with command `python hellodjango/manage.py runserver 0.0.0.0:8642 --noreload`
2011-08-27T07:58:00+00:00 app[web.1]: Validating models...
2011-08-27T07:58:00+00:00 app[web.1]:
2011-08-27T07:58:00+00:00 app[web.1]: 0 errors found
2011-08-27T07:58:00+00:00 app[web.1]: Django version 1.3, using settings 'hellodjango.settings'
2011-08-27T07:58:00+00:00 app[web.1]: Development server is running at http://0.0.0.0:8642/
2011-08-27T07:58:00+00:00 app[web.1]: Quit the server with CONTROL-C.
2011-08-27T07:58:01+00:00 heroku[web.1]: State changed from starting to up

Finally, visit your app on the web:

$ heroku open

You should see the familiar “It worked!” Django welcome page.

Syncing the Database

The heroku run command lets you run one-off admin procesess. You can use this to sync the Django models with the database schema:

$ heroku run python hellodjango/manage.py syncdb
Running python hellodjango/manage.py syncdb attached to terminal... up, run.1
Creating tables ...
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_user_permissions
Creating table auth_user_groups
Creating table auth_user
Creating table auth_message
Creating table django_content_type
Creating table django_session
Creating table django_site

You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no): yes
Username (Leave blank to use 'u59044'): adam
E-mail address: adam@example.com
Password:
Password (again):
Superuser created successfully.
Installing custom SQL ...
Installing indexes ...
No fixtures found.

Using the Django Shell

Similarly, you can use heroku run to get a Django shell for executing arbitrary code against your deployed app:

$ heroku run python hellodjango/manage.py shell
Running python hellodjango/manage.py shell attached to terminal... up, run.3
Python 2.7.1 (r271:86832, Jun 26 2011, 01:08:11)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from django.contrib.auth.models import User
>>> User.objects.all()
[<User: adam>]
>>>

Postgres Database Config

When you deploy a Django application, the compile process appends the following code to your settings.py to use the DATABASE_URL environment variable:

import os
import sys
import urlparse

# Register database schemes in URLs.
urlparse.uses_netloc.append('postgres')
urlparse.uses_netloc.append('mysql')

try:

    # Check to make sure DATABASES is set in settings.py file.
    # If not default to {}

    if 'DATABASES' not in locals():
        DATABASES = {}

    if 'DATABASE_URL' in os.environ:
        url = urlparse.urlparse(os.environ['DATABASE_URL'])

        # Ensure default database exists.
        DATABASES['default'] = DATABASES.get('default', {})

        # Update with environment configuration.
        DATABASES['default'].update({
            'NAME': url.path[1:],
            'USER': url.username,
            'PASSWORD': url.password,
            'HOST': url.hostname,
            'PORT': url.port,
        })
        if url.scheme == 'postgres':
            DATABASES['default']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'

        if url.scheme == 'mysql':
            DATABASES['default']['ENGINE'] = 'django.db.backends.mysql'
except Exception:
    print 'Unexpected error:', sys.exc_info()

You don’t need to do anything special to configure this, the above information is purely for reference.

Using A Different WSGI Server

The examples above used the development HTTP server for Django. For production apps, you should use a production-ready embedded webserver Gunicorn and gevent.

Let’s try adding Gunicorn to our Django app by adding the gunicorn dependency to our pip file:

requirements.txt

Django==1.3
psycopg2==2.4.4
gunicorn==0.13.4

Update dependencies with pip install -r requirements.txt.

Add to your INSTALLED_APPS:

settings.py

'gunicorn',

Next, create a Procfile to declare the command to launch the web process:

Procfile

web: python hellodjango/manage.py run_gunicorn -b "0.0.0.0:$PORT" -w 3

Run locally with Foreman:

$ foreman start
01:14:46 web.1     | started with pid 89897
01:14:46 web.1     | 2011-08-27 01:14:46 [89899] [INFO] Starting gunicorn 0.13.4
01:14:46 web.1     | 2011-08-27 01:14:46 [89899] [INFO] Listening at: http://0.0.0.0:5000 (89899)
01:14:46 web.1     | 2011-08-27 01:14:46 [89899] [INFO] Using worker: sync
01:14:46 web.1     | 2011-08-27 01:14:46 [89900] [INFO] Booting worker with pid: 89900

Deploy:

$ git add .
$ git commit -m "use gunicorn"
$ git push heroku

We can see that Gunicorn is running from the logs:

$ heroku logs

Running a Worker

The Procfile format lets you run any number of different process types. Let’s add a background worker to our Django app, using the Celery queueing system and the Django-kombu package for using the database as a queueing backend.

Install django-celery and django-kombu into your Virtualenv, which will also pull all of the Celery and Kombu dependencies:

$ pip install django-celery django-kombu
Downloading/unpacking django-celery
...
Successfully installed amqplib anyjson celery django-celery django-picklefield importlib kombu ordereddict pyparsing python-dateutil
Cleaning up...

Freeze the new package list:

$ pip freeze > requirements.txt

Add Kombu and Celery to your INSTALLED_APPS:

settings.py

'djcelery',
'djkombu',

And append the following Celery configuration to the same file:

settings.py

import djcelery
djcelery.setup_loader()
BROKER_BACKEND = "djkombu.transport.DatabaseTransport"
CELERY_RESULT_DBURI = DATABASES['default']

Finally, create or extend your Procfile to include a process type for your worker:

Procfile

worker: python hellodjango/manage.py celeryd -E -B --loglevel=INFO

Sync your local database to create the Celery and Kombu tables:

$ python hellodjango/manage.py syncdb

Run locally with Foreman:

$ foreman start
17:08:13 web.1     | started with pid 15345
17:08:13 worker.1  | started with pid 15346
17:08:13 web.1     | Validating models...
17:08:13 worker.1  | /Users/adam/hellodjango/lib/python2.6/site-packages/djcelery/loaders.py:84: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
17:08:13 worker.1  |   warnings.warn("Using settings.DEBUG leads to a memory leak, never "
17:08:13 worker.1  | [2011-09-12 19:08:13,651: WARNING/MainProcess]
17:08:13 worker.1  |
17:08:13 worker.1  |  -------------- celery@valkyrie v2.3.2
17:08:13 worker.1  | ---- **** -----
17:08:13 worker.1  | --- * ***  * -- [Configuration]
17:08:13 worker.1  | -- * - **** ---   . broker:      djkombu.transport.DatabaseTransport://localhost//
17:08:13 worker.1  | - ** ----------   . loader:      djcelery.loaders.DjangoLoader
17:08:13 worker.1  | - ** ----------   . logfile:     [stderr]@INFO
17:08:13 worker.1  | - ** ----------   . concurrency: 4
17:08:13 worker.1  | - ** ----------   . events:      ON
17:08:13 worker.1  | - *** --- * ---   . beat:        ON
17:08:13 worker.1  | -- ******* ----
17:08:13 worker.1  | --- ***** ----- [Queues]
17:08:13 worker.1  |  --------------   . celery:      exchange:celery (direct) binding:celery
17:08:13 worker.1  |

The worker is running locally, so now you can commit:

$ echo celerybeat-schedule.db >> .gitignore
$ git add .
$ git commit -m "use a celery worker"
[master f82a1bd] use a celery worker
 4 files changed, 29 insertions(+), 7 deletions(-)

Deploy to Heroku, note that the compile process picks up our new worker process type:

$ git push heroku master
...
-----> Discovering process types
       Procfile declares types -> web, worker
...

Create the Celery tables in your Heroku app’s database:

$ heroku run python hellodjango/manage.py syncdb
Running python hellodjango/manage.py syncdb attached to terminal... up, run.5
Creating tables ...
Creating table celery_taskmeta
Creating table celery_tasksetmeta
Creating table djcelery_intervalschedule
...

Running more than one dyno for an extended period may incur charges to your account. Read more about dyno-hour costs.

Now we’re ready to scale up a worker:

$ heroku scale worker=1
Scaling worker processes... done, now running 1

$ heroku ps
Process       State               Command
------------  ------------------  ------------------------------
run.5         complete for 1m     python hellodjango/manage.py syncdb
web.1         up for 2m           python hellodjango/manage.py run_g..
worker.1      up for 42s          python hellodjango/manage.py celer..

Tail the logs to watch the worker do its work:

$ heroku logs --tail --ps worker
2011-09-13T00:18:36+00:00 heroku[worker.1]: State changed from created to starting
2011-09-13T00:18:44+00:00 heroku[worker.1]: Starting process with command `python hellodjango/manage.py celeryd -E -B --loglevel=INFO`
2011-09-13T00:18:45+00:00 heroku[worker.1]: State changed from starting to up
2011-09-13T00:18:45+00:00 app[worker.1]: /app/lib/python2.7/site-packages/djcelery/loaders.py:84: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
2011-09-13T00:18:45+00:00 app[worker.1]:   warnings.warn("Using settings.DEBUG leads to a memory leak, never "
2011-09-13T00:18:45+00:00 app[worker.1]: [2011-09-12 19:18:45,455: WARNING/MainProcess] -------------- celery@78a75dcb-aa69-4ebd-9cf0-e3f4161d4622 v2.3.2
2011-09-13T00:18:45+00:00 app[worker.1]: ---- **** -----
2011-09-13T00:18:45+00:00 app[worker.1]: --- * ***  * -- [Configuration]
2011-09-13T00:18:45+00:00 app[worker.1]: -- * - **** ---   . broker:      djkombu.transport.DatabaseTransport://localhost//
2011-09-13T00:18:45+00:00 app[worker.1]: - ** ----------   . loader:      djcelery.loaders.DjangoLoader
2011-09-13T00:18:45+00:00 app[worker.1]: - ** ----------   . logfile:     [stderr]@INFO
2011-09-13T00:18:45+00:00 app[worker.1]: - ** ----------   . concurrency: 4
2011-09-13T00:18:45+00:00 app[worker.1]: - ** ----------   . events:      ON
2011-09-13T00:18:45+00:00 app[worker.1]: - *** --- * ---   . beat:        ON
2011-09-13T00:18:45+00:00 app[worker.1]: -- ******* ----
2011-09-13T00:18:45+00:00 app[worker.1]: --- ***** ----- [Queues]
2011-09-13T00:18:45+00:00 app[worker.1]:  --------------   . celery:      exchange:celery (direct) binding:celery

You can now start adding jobs to your queue and the worker will work them, which will be visible in the log output.