Getting Started with Django on Heroku/Cedar
Table of Contents
This guide will show you how to deploy a Python/Django application to Heroku using a Postgres database. For other Python apps, see Getting Started with Python on Heroku/Cedar.
Prerequisites
- Python, virtualenv, pip, and the Heroku toolbelt as described in Getting Started with Python on Heroku/Cedar.
- An installed version of Postgres to test locally
Start a Django App Inside a Virtualenv
Start by making an empty top-level directory for the project:
$ mkdir hellodjango && cd hellodjango
Create a Virtualenv (v0.7):
$ virtualenv venv --distribute
New python executable in venv/bin/python
Installing distribute...............done.
Installing pip...............done.
To activate the new environment, you’ll need to source it:
Windows users can run vevn\Scripts\activate.bat for the same effect.
$ source venv/bin/activate
This will change your prompt to include the project name. (You must source the virtualenv environment for each terminal session where you wish to run your app.)
Install dependencies with pip:
$ pip install Django psycopg2
Downloading/unpacking Django
Downloading Django-1.3.1.tar.gz (6.5Mb): 6.5Mb downloaded
Running setup.py egg_info for package Django
Downloading/unpacking psycopg2
Downloading psycopg2-2.4.4.tar.gz (648Kb): 648Kb downloaded
Running setup.py egg_info for package psycopg2
Installing collected packages: Django, psycopg2
Running setup.py install for Django
Running setup.py install for psycopg2
...
Successfully installed Django psycopg2
Cleaning up...
Now we can create a Django app in a subdirectory:
$ django-admin.py startproject hellodjango
Note that we’ve packaged the app, placing the Django app in a subdirectory of our main project. This keeps our main Django app separate from Virtualenv artifacts and other Django apps we might want to add to the directory.
Test that the app runs locally by starting up the Django development webserver:
$ python hellodjango/manage.py runserver
Validating models...
0 errors found
Django version 1.3, using settings 'hellodjango.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
Now you have a working copy of our application with dependencies. Create a pip requirements file which declares our required Python modules:
$ pip freeze > requirements.txt
Make sure that there aren’t any “editable” modules (lines that start with a -e) in this list. They should never be used in production.
All packages required should be declared explicitly in requirements.txt:
$ cat requirements.txt
Django==1.3
psycopg2==2.4.2
Commit to Git
Exclude Virtualenv artifacts from source control tracking:
GitHub provides an excellent Python gitignore file that can be installed system-wide.
.gitignore
venv
*.pyc
You will use Git to deploy code to Heroku. You don’t have to use it as your main source control system; Git can coexist with Mercurial, Subversion, or other revision control systems.
Initialize a Git repo and commit the project (if you aren’t already tracking your project with Git):
$ git init
Initialized empty Git repository in /Users/adam/hellodjango/.git/
$ git add .
$ git commit -m "my django app"
[master (root-commit) 8c07531] my django app
5 files changed, 184 insertions(+), 0 deletions(-)
create mode 100644 .gitignore
create mode 100644 hellodjango/__init__.py
create mode 100644 hellodjango/manage.py
create mode 100644 hellodjango/settings.py
create mode 100644 hellodjango/urls.py
create mode 100644 requirements.txt
Deploy to Heroku
Create an app on the Cedar stack:
$ heroku create --stack cedar
Creating afternoon-sword-29... done, stack is cedar
http://afternoon-sword-29.herokuapp.com/ | git@heroku.com:afternoon-sword-29.git
Git remote heroku added
Deploy your app:
$ git push heroku master
Counting objects: 9, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (9/9), 3.01 KiB, done.
Total 9 (delta 0), reused 0 (delta 0)
-----> Heroku receiving push
-----> Python/Django app detected
-----> Preparing virtualenv version 1.7
New python executable in ./bin/python2.7
Also creating executable in ./bin/python
Installing distribute............done.
Installing pip...............done.
-----> Byte-compiling code
-----> Django settings injection
Injecting code into hellodjango/settings.py to read from DATABASE_URL
-----> Installing dependencies using pip version 1.2.0
Downloading/unpacking Django==1.3 (from -r requirements.txt (line 1))
...
Successfully installed Django psycopg2
Cleaning up...
-----> Discovering process types
Procfile declares types -> (none)
Default types for Python/Django ->web
-----> Compiled slug size is 8.0MB
-----> Launching... done, v3
http://afternoon-sword-29.herokuapp.com deployed to Heroku
To git@heroku.com:afternoon-sword-29.git
* [new branch] master -> master
Check to see that your web process is up:
$ heroku ps
Process State Command
------------ ------------------ ------------------------------
web.1 up for 4s python hellodjango/manage.py r..
View your logs:
$ heroku logs
2011-08-27T07:58:00+00:00 heroku[web.1]: Starting process with command `python hellodjango/manage.py runserver 0.0.0.0:8642 --noreload`
2011-08-27T07:58:00+00:00 app[web.1]: Validating models...
2011-08-27T07:58:00+00:00 app[web.1]:
2011-08-27T07:58:00+00:00 app[web.1]: 0 errors found
2011-08-27T07:58:00+00:00 app[web.1]: Django version 1.3, using settings 'hellodjango.settings'
2011-08-27T07:58:00+00:00 app[web.1]: Development server is running at http://0.0.0.0:8642/
2011-08-27T07:58:00+00:00 app[web.1]: Quit the server with CONTROL-C.
2011-08-27T07:58:01+00:00 heroku[web.1]: State changed from starting to up
Finally, visit your app on the web:
$ heroku open
You should see the familiar “It worked!” Django welcome page.
Syncing the Database
The heroku run command lets you run one-off admin procesess. You can use this to sync the Django models with the database schema:
$ heroku run python hellodjango/manage.py syncdb
Running python hellodjango/manage.py syncdb attached to terminal... up, run.1
Creating tables ...
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_user_permissions
Creating table auth_user_groups
Creating table auth_user
Creating table auth_message
Creating table django_content_type
Creating table django_session
Creating table django_site
You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no): yes
Username (Leave blank to use 'u59044'): adam
E-mail address: adam@example.com
Password:
Password (again):
Superuser created successfully.
Installing custom SQL ...
Installing indexes ...
No fixtures found.
Using the Django Shell
Similarly, you can use heroku run to get a Django shell for executing arbitrary code against your deployed app:
$ heroku run python hellodjango/manage.py shell
Running python hellodjango/manage.py shell attached to terminal... up, run.3
Python 2.7.1 (r271:86832, Jun 26 2011, 01:08:11)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from django.contrib.auth.models import User
>>> User.objects.all()
[<User: adam>]
>>>
Postgres Database Config
When you deploy a Django application, the compile process appends the following code to your settings.py to use the DATABASE_URL environment variable:
import os
import sys
import urlparse
# Register database schemes in URLs.
urlparse.uses_netloc.append('postgres')
urlparse.uses_netloc.append('mysql')
try:
# Check to make sure DATABASES is set in settings.py file.
# If not default to {}
if 'DATABASES' not in locals():
DATABASES = {}
if 'DATABASE_URL' in os.environ:
url = urlparse.urlparse(os.environ['DATABASE_URL'])
# Ensure default database exists.
DATABASES['default'] = DATABASES.get('default', {})
# Update with environment configuration.
DATABASES['default'].update({
'NAME': url.path[1:],
'USER': url.username,
'PASSWORD': url.password,
'HOST': url.hostname,
'PORT': url.port,
})
if url.scheme == 'postgres':
DATABASES['default']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'
if url.scheme == 'mysql':
DATABASES['default']['ENGINE'] = 'django.db.backends.mysql'
except Exception:
print 'Unexpected error:', sys.exc_info()
You don’t need to do anything special to configure this, the above information is purely for reference.
Using A Different WSGI Server
The examples above used the development HTTP server for Django. For production apps, you should use a production-ready embedded webserver Gunicorn and gevent.
Let’s try adding Gunicorn to our Django app by adding the gunicorn dependency to our pip file:
requirements.txt
Django==1.3
psycopg2==2.4.4
gunicorn==0.13.4
Update dependencies with pip install -r requirements.txt.
Add to your INSTALLED_APPS:
settings.py
'gunicorn',
Next, create a Procfile to declare the command to launch the web process:
Procfile
web: python hellodjango/manage.py run_gunicorn -b "0.0.0.0:$PORT" -w 3
Run locally with Foreman:
$ foreman start
01:14:46 web.1 | started with pid 89897
01:14:46 web.1 | 2011-08-27 01:14:46 [89899] [INFO] Starting gunicorn 0.13.4
01:14:46 web.1 | 2011-08-27 01:14:46 [89899] [INFO] Listening at: http://0.0.0.0:5000 (89899)
01:14:46 web.1 | 2011-08-27 01:14:46 [89899] [INFO] Using worker: sync
01:14:46 web.1 | 2011-08-27 01:14:46 [89900] [INFO] Booting worker with pid: 89900
Deploy:
$ git add .
$ git commit -m "use gunicorn"
$ git push heroku
We can see that Gunicorn is running from the logs:
$ heroku logs
Running a Worker
The Procfile format lets you run any number of different process types. Let’s add a background worker to our Django app, using the Celery queueing system and the Django-kombu package for using the database as a queueing backend.
Install django-celery and django-kombu into your Virtualenv, which will also pull all of the Celery and Kombu dependencies:
$ pip install django-celery django-kombu
Downloading/unpacking django-celery
...
Successfully installed amqplib anyjson celery django-celery django-picklefield importlib kombu ordereddict pyparsing python-dateutil
Cleaning up...
Freeze the new package list:
$ pip freeze > requirements.txt
Add Kombu and Celery to your INSTALLED_APPS:
settings.py
'djcelery',
'djkombu',
And append the following Celery configuration to the same file:
settings.py
import djcelery
djcelery.setup_loader()
BROKER_BACKEND = "djkombu.transport.DatabaseTransport"
CELERY_RESULT_DBURI = DATABASES['default']
Finally, create or extend your Procfile to include a process type for your worker:
Procfile
worker: python hellodjango/manage.py celeryd -E -B --loglevel=INFO
Sync your local database to create the Celery and Kombu tables:
$ python hellodjango/manage.py syncdb
Run locally with Foreman:
$ foreman start
17:08:13 web.1 | started with pid 15345
17:08:13 worker.1 | started with pid 15346
17:08:13 web.1 | Validating models...
17:08:13 worker.1 | /Users/adam/hellodjango/lib/python2.6/site-packages/djcelery/loaders.py:84: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
17:08:13 worker.1 | warnings.warn("Using settings.DEBUG leads to a memory leak, never "
17:08:13 worker.1 | [2011-09-12 19:08:13,651: WARNING/MainProcess]
17:08:13 worker.1 |
17:08:13 worker.1 | -------------- celery@valkyrie v2.3.2
17:08:13 worker.1 | ---- **** -----
17:08:13 worker.1 | --- * *** * -- [Configuration]
17:08:13 worker.1 | -- * - **** --- . broker: djkombu.transport.DatabaseTransport://localhost//
17:08:13 worker.1 | - ** ---------- . loader: djcelery.loaders.DjangoLoader
17:08:13 worker.1 | - ** ---------- . logfile: [stderr]@INFO
17:08:13 worker.1 | - ** ---------- . concurrency: 4
17:08:13 worker.1 | - ** ---------- . events: ON
17:08:13 worker.1 | - *** --- * --- . beat: ON
17:08:13 worker.1 | -- ******* ----
17:08:13 worker.1 | --- ***** ----- [Queues]
17:08:13 worker.1 | -------------- . celery: exchange:celery (direct) binding:celery
17:08:13 worker.1 |
The worker is running locally, so now you can commit:
$ echo celerybeat-schedule.db >> .gitignore
$ git add .
$ git commit -m "use a celery worker"
[master f82a1bd] use a celery worker
4 files changed, 29 insertions(+), 7 deletions(-)
Deploy to Heroku, note that the compile process picks up our new worker process type:
$ git push heroku master
...
-----> Discovering process types
Procfile declares types -> web, worker
...
Create the Celery tables in your Heroku app’s database:
$ heroku run python hellodjango/manage.py syncdb
Running python hellodjango/manage.py syncdb attached to terminal... up, run.5
Creating tables ...
Creating table celery_taskmeta
Creating table celery_tasksetmeta
Creating table djcelery_intervalschedule
...
Running more than one dyno for an extended period may incur charges to your account. Read more about dyno-hour costs.
Now we’re ready to scale up a worker:
$ heroku scale worker=1
Scaling worker processes... done, now running 1
$ heroku ps
Process State Command
------------ ------------------ ------------------------------
run.5 complete for 1m python hellodjango/manage.py syncdb
web.1 up for 2m python hellodjango/manage.py run_g..
worker.1 up for 42s python hellodjango/manage.py celer..
Tail the logs to watch the worker do its work:
$ heroku logs --tail --ps worker
2011-09-13T00:18:36+00:00 heroku[worker.1]: State changed from created to starting
2011-09-13T00:18:44+00:00 heroku[worker.1]: Starting process with command `python hellodjango/manage.py celeryd -E -B --loglevel=INFO`
2011-09-13T00:18:45+00:00 heroku[worker.1]: State changed from starting to up
2011-09-13T00:18:45+00:00 app[worker.1]: /app/lib/python2.7/site-packages/djcelery/loaders.py:84: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
2011-09-13T00:18:45+00:00 app[worker.1]: warnings.warn("Using settings.DEBUG leads to a memory leak, never "
2011-09-13T00:18:45+00:00 app[worker.1]: [2011-09-12 19:18:45,455: WARNING/MainProcess] -------------- celery@78a75dcb-aa69-4ebd-9cf0-e3f4161d4622 v2.3.2
2011-09-13T00:18:45+00:00 app[worker.1]: ---- **** -----
2011-09-13T00:18:45+00:00 app[worker.1]: --- * *** * -- [Configuration]
2011-09-13T00:18:45+00:00 app[worker.1]: -- * - **** --- . broker: djkombu.transport.DatabaseTransport://localhost//
2011-09-13T00:18:45+00:00 app[worker.1]: - ** ---------- . loader: djcelery.loaders.DjangoLoader
2011-09-13T00:18:45+00:00 app[worker.1]: - ** ---------- . logfile: [stderr]@INFO
2011-09-13T00:18:45+00:00 app[worker.1]: - ** ---------- . concurrency: 4
2011-09-13T00:18:45+00:00 app[worker.1]: - ** ---------- . events: ON
2011-09-13T00:18:45+00:00 app[worker.1]: - *** --- * --- . beat: ON
2011-09-13T00:18:45+00:00 app[worker.1]: -- ******* ----
2011-09-13T00:18:45+00:00 app[worker.1]: --- ***** ----- [Queues]
2011-09-13T00:18:45+00:00 app[worker.1]: -------------- . celery: exchange:celery (direct) binding:celery
You can now start adding jobs to your queue and the worker will work them, which will be visible in the log output.