10/4/13

Deploying Django on Heroku - Minuti by Mike Tigas

Minuti
LoudthinkingfromthemindofMikeTigas.

DeployingDjangoonHeroku
Alert: Extremely long tutorial post ahead. If you want to follow along, Ive made an example project available here. Said example site is running here. An extremely abridged tldr version of this (containing just the shell and code snippets, sans explanation) is located here: tldr.markdown. I've been toying around with Heroku in my spare time over the past couple weeks because "NoOps" is the new hotness and the promise is cool enough: wouldnt it be great if you could write and deploy a high-performance website without having to micromanage the infrastructure? (See also: erosion and whatnot.) In any case, the pricing structure of Heroku (750 free hours) is such that you can run a low-end, low-traffic website in Heroku for free, which is useful for trying it out. The Heroku Django starter doc isn't bad, but leaves out some bits that I think are important in any production environment: Using the correct
g u n i c o r nworker

class: gunicorn recommends you either run

behind a buffering proxy server (i.e. nginx) or run one of the "async" worker classes. The Cedar 'herokuapp.com' HTTP stack directly connects requests to the backend for flexibility (think WebSockets and the like), but doesnt provide gzip or buffering of requests. (In fact, the Heroku Django docs mention using gunicorn+gevent, but dont actually configure gevent in the relevant

https://mike.tig.as/blog/2012/02/13/deploying-django-on-heroku/

1/12

10/4/13

Deploying Django on Heroku - Minuti by Mike Tigas

examples.) Handling of static assets and uploaded files. I have seen this question asked in a few places and the combination of Django 1.3+, django-storages, and boto make this extremely painless to set up. So Ive decided to tinker with Heroku and write a step-by-step tutorial as I go. The following assumes youre fairly proficient with Django and these steps are only useful for getting a barebones proof of concept site up and running. I do provide the database and caching bits for you, so you can use this as a stepping stone for trying out more full-featured projects on Heroku. (Note: The free Heroku database is a shared server with only 5MB of raw data storage. The free memcached instance is likewise a tiny 5MB instance. These are toy websites were deploying here for free. The 20GB shared database is $15/month; you can also host your own dedicated EC2 postgres instance if youd rather not go all-out with a Heroku dedicated DB.)

PRELIMINARIES
System dependencies Im going to assume you have a working copy of git, Python 2.7.X, pip, and virtualenv on your local system. If you don't, you should install them via homebrew. If you don't have homebrew, visit the documentation and run the one-line install. (If you do have brew installed, now would be a great time to update it. Run Install git.
b r e w i n s t a l l g i t b r e wu p d a t e .)

Now install Python, add it to your PATH, and add that new PATH to
. z s h r cor

. b a s h _ p r o f i l e

so that this works in the future. (If you use ZSH or another shell, do this to your similar file.)

b r e w i n s t a l l p y t h o n

https://mike.tig.as/blog/2012/02/13/deploying-django-on-heroku/

2/12

10/4/13

Deploying Django on Heroku - Minuti by Mike Tigas

e x p o r t P A T H = / u s r / l o c a l / s h a r e / p y t h o n : $ P A T H e c h o " e x p o r t P A T H = / u s r / l o c a l / s h a r e / p y t h o n : \ $ P A T H " > > ~ / . b a s h _ p r o f i l e

Then install pip and virtualenv.

e a s y _ i n s t a l l p i p p i p i n s t a l l v i r t u a l e n v

Setting up Heroku Register a Heroku account first. Now install the h e r o k uand f o r e m a ncommands:
s u d o g e m i n s t a l l f o r e m a n h e r o k u s u d o u p d a t e _ r u b y g e m s

(Note: The Heroku docs tell you to use their toolbelt package to install these packages, but Ive encountered errors with
f o r e m a nunless

Ive s u d og e mi n s t a l l ' dit.

The gems get you the same thing, anyway.) Once installed, run the h e r o k ul o g i ncommand, which allows you to run commands against your Heroku account. (The Heroku Toolbelt page has an example of the login bit under "Getting Started".) If youre keeping score at home, here are the things you need to move on: git python 2.7.x pip virtualenv heroku foreman

BOOTSTRAPPING A HEROKU PYTHON SITE

https://mike.tig.as/blog/2012/02/13/deploying-django-on-heroku/

3/12

10/4/13

Deploying Django on Heroku - Minuti by Mike Tigas

Well start by choosing an app name. (Change these values, please.)

# T h e " a p p n a m e " t h a t t h i s w i l l g e t i n t h e H e r o k u c o n t r o l p a n e l . A l s o # d e t e r m i n e s d i r e c t o r y n a m e s a n d y o u r " P R O J E C T _ N A M E . h e r o k u a p p . c o m " # d e f a u l t d o m a i n . e x p o r t P R O J E C T _ N A M E = " m y t e s t a p p " # T h e p y t h o n m o d u l e n a m e f o r y o u r D j a n g o s i t e . S e p a r a t e f r o m a b o v e s i n c e # p y t h o n a p p n a m e s s h o u l d u s e u n d e r s c o r e s r a t h e r t h a n d a s h e s . e x p o r t P Y T H O N _ A P P _ N A M E = " m y _ t e s t _ a p p "

I like to put my projects in a ~ / C o d edirectory, but you can change this to place your projects whever you normally would:
# S e t u p a h e r o k u $ P R O J E C T _ N A M E v i r t u a l e n v i n t h e ~ / C o d e d i r e c t o r y . c d ~ / C o d e v i r t u a l e n v n o s i t e p a c k a g e s h e r o k u $ P R O J E C T _ N A M E

Im going to gloss over the fine details on how to use virtualenv, but you should be able to follow along here if youve ever done customization to your . b a s h _ p r o f i l e ,
. z s h r c ,

or similar shell init file.

# M o d i f y t h e ` a c t i v a t e ` f i l e w i t h s o m e s a n i t y e n s u r i n g d e f a u l t s , l i k e # i g n o r i n g a n y s y s t e m l e v e l P Y T H O N P A T H a n d D J A N G O _ S E T T I N G S _ M O D U L E . c d h e r o k u $ P R O J E C T _ N A M E e c h o " e x p o r t P R O J E C T _ N A M E = \ " $ P R O J E C T _ N A M E \ " " > > b i n / a c t i v a t e e c h o " e x p o r t P Y T H O N _ A P P _ N A M E = \ " $ P Y T H O N _ A P P _ N A M E \ " " > > b i n / a c t i v a t e e c h o " e x p o r t P I P _ R E S P E C T _ V I R T U A L E N V = t r u e " > > b i n / a c t i v a t e e c h o " e x p o r t P Y T H O N P A T H = \ " \ $ V I R T U A L _ E N V / r e p o / s r c \ " " > > b i n / a c t i v a t e e c h o " u n s e t D J A N G O _ S E T T I N G S _ M O D U L E " > > b i n / a c t i v a t e # A c t i v a t e t h e e n v i r o n m e n t . s o u r c e b i n / a c t i v a t e

Now were in an isolated virtualenv environment (since we started it with

p a c k a g e s ),

n o s i t e -

and we can

p i pi n s t a l lto

our heart's content and those packages will be

P I P _ R E S P E C T _ V I R T U A L E N V ).

installed within this isolated sandbox (since we set

(If youve never used virtualenv before: If you want to open this virtualenv later, just run
c d~ / C o d e / ( P R O J E C TN A M E ) /and

then

s o u r c eb i n / a c t i v a t e .)

Now well start up a repository to store our project and work our way through

https://mike.tig.as/blog/2012/02/13/deploying-django-on-heroku/

4/12

10/4/13

Deploying Django on Heroku - Minuti by Mike Tigas

installing Django and our gunicorn server bits:

# I n i t i a l i z e a g i t r e p o s i t o r y i n t h e ` r e p o ` s u b d i r e c t o r y o f t h i s v i r t u a l e n v g i t i n i t r e p o c d r e p o # S t a r t t h i s g i t r e p o w i t h m y P y t h o n . g i t i g n o r e o f c h o i c e . # S e e i t a t h t t p s : / / g i s t . g i t h u b . c o m / 1 8 0 6 6 4 3 / f o r n o t e s . c u r l s L O h t t p s : / / r a w . g i t h u b . c o m / g i s t / 1 8 0 6 6 4 3 / . g i t i g n o r e g i t a d d . g i t i g n o r e g i t c o m m i t m " i n i t i a l c o m m i t , . g i t i g n o r e " # C r e a t e a ` s r c ` d i r e c t o r y w i t h i n o u r r e p o . m k d i r s r c # I n s t a l l D j a n g o ( 1 . 3 . X ) , g u n i c o r n ( 0 . 1 3 . X ) , g e v e n t ( 0 . 1 3 . X ) , a n d t h e g r e e n l e t # d e p e n d e n c y . e c h o " d j a n g o = = 1 . 3 . 1 " > r e q u i r e m e n t s . t x t e c h o " g u n i c o r n = = 0 . 1 3 . 4 " > > r e q u i r e m e n t s . t x t e c h o " g e v e n t = = 0 . 1 3 . 4 " > > r e q u i r e m e n t s . t x t e c h o " g r e e n l e t = = 0 . 3 . 4 " > > r e q u i r e m e n t s . t x t p i p i n s t a l l r r e q u i r e m e n t s . t x t

The s r cdirectory will be where our Python sources live. Itll be a place on
P Y T H O N P A T H ,

so root-level modules (and things that arent pip-installable) can be

placed there. (I prefer this to putting everything on the root level of the repository - as is done in the Heroku docs -- for matters of keeping a well-organized source tree.) Well set up a plain Django project inside:
# E n t e r t h e ` s r c ` d i r a n d c r e a t e a d j a n g o p r o j e c t c d $ V I R T U A L _ E N V / r e p o / s r c $ V I R T U A L _ E N V / b i n / d j a n g o a d m i n . p y s t a r t p r o j e c t $ P Y T H O N _ A P P _ N A M E c d $ V I R T U A L _ E N V / r e p o

Now, well configure a procfile, which describes the processes that will power our services. (Well, just one now for our web service.)

# U n l i k e t h e g u n i c o r n d e f i n e d i n H e r o k u ' s D j a n g o e x a m p l e , w e ' r e g o i n g # t o u s e o n e o f t h e a s y n c w o r k e r c l a s s e s , " g e v e n t " . U s i n g a n a s y n c w o r k e r c l a s s # i s r e c o m m e n d e d w h e n s e r v i n g t r a f f i c d i r e c t l y t o g u n i c o r n ( w h i c h i s w h a t # h a p p e n s u n d e r t h e H e r o k u C e d a r s t a c k ) . e c h o " w e b : g u n i c o r n _ d j a n g o b 0 . 0 . 0 . 0 : \ $ P O R T w 9 k g e v e n t m a x r e q u e s t s 2 5 0 p r e l o a d s r c / $

https://mike.tig.as/blog/2012/02/13/deploying-django-on-heroku/

5/12

10/4/13

Deploying Django on Heroku - Minuti by Mike Tigas

The w e bservice is special-cased to provide a $ P O R Tenvironment variable, which is where Heroku will send your web traffic. Ive set up some sane defaults (9 workers, 250 requests per worker before restarting them) for Gunicorn that you can configure for yourself later. Now well commit this bare Django project and test it locally.
# C o m m i t e v e r y t h i n g w e h a v e i n h e r e . g i t a d d . g i t c o m m i t m " b a s e d j a n g o s i t e " # T e s t o u t o u r s e t u p . f o r e m a n s t a r t

Were using

f o r e m a n(man

page), which reads the Procfile and simulates running the

service on Heroku. Opening http://127.0.0.1:5000/ in the browser should display the standard "It Worked!" page. Now, lets try to get this running in the cloud:
# C r e a t e a H e r o k u i n s t a n c e f o r t h i s s i t e h e r o k u c r e a t e s c e d a r $ P R O J E C T _ N A M E # M a k e s u r e t o a d d ` s r c ` t o t h e P Y T H O N P A T H o n o u r s e r v e r . ( W e a d d e d t h i s t o o u r # l o c a l a c t i v a t e f i l e , b u t i t n e e d s t o b e a p p l i e d t o H e r o k u , t o o . ) h e r o k u c o n f i g : a d d P Y T H O N P A T H = / s r c # D e p l o y t h i s p r o j e c t t o H e r o k u g i t p u s h h e r o k u m a s t e r

You should now be able to hit

h t t p : / / P R O J E C T _ N A M E . h e r o k u a p p . c o m /and

see that the

Django instance is running. Some things to try: Check

h e r o k up sto

see the status of the processes you have running.

See h e r o k ul o g sto see access or error logs. ( h e r o k ul o g stacts like the t a i l command and sends you a constant stream of log lines.)

CONFIGURING A DATABASE AND SERVING STATIC FILES

Now we've got a website running at
h t t p : / / P R O J E C T _ N A M E . h e r o k u a p p . c o m /that

has

https://mike.tig.as/blog/2012/02/13/deploying-django-on-heroku/

6/12

10/4/13

Deploying Django on Heroku - Minuti by Mike Tigas

no database and cannot serve static assets. Well work on both, by enabling the admin (since the Django 1.3+ admin site requires staticfiles and Users set up in the database). You can add a free, shared database account to your Heroku app by running this command:
h e r o k u a d d o n s : a d d s h a r e d d a t a b a s e : 5 m b

If you run

h e r o k uc o n f i gyoull

see the D A T A B A S E _ U R L , which contains your databases

username, password, hostname, and database name. (Well be using this environment var to configure our database in Django shortly.) For static storage, Im going to use b o t oand d j a n g o s t o r a g e sto store files in Amazon S3. You should check out the Amazon AWS site and register an account if you dont already have one. Then, go to the Security Credentials page to grab yourself an "Access Key ID" and a "Secret Access Key". (Keep these values: well add this to our settings soon.) At this point, you should also log into the AWS S3 Console and create a bucket to store your static files. (Also keep this around for settings.) Heroku uses the US Standard (US East) region, so place your bucket there for performance and lowest cost bandwidth within an AWS region is free of charge. Install psycopg2, boto, and django-storages:
c d $ V I R T U A L _ E N V / r e p o e c h o " p s y c o p g 2 " > > r e q u i r e m e n t s . t x t e c h o " b o t o = = 2 . 2 . 1 " > > r e q u i r e m e n t s . t x t e c h o " d j a n g o s t o r a g e s = = 1 . 1 . 4 " > > r e q u i r e m e n t s . t x t p i p i n s t a l l r r e q u i r e m e n t s . t x t

Open up s r c / $ P Y T H O N _ A P P _ N A M E / s e t t i n g s . p yand add ' s t o r a g e s 'to your

I N S T A L L E D _ A P P S .

Uncomment

d j a n g o . c o n t r i b . a d m i n ,

too.

Then, add the following lines to the bottom of your settings file, filling in your own
A W S _ A C C E S S _ K E Y _ I D ,A W S _ S E C R E T _ A C C E S S _ K E Y ,

and A W S _ S T O R A G E _ B U C K E T _ N A M E .

https://mike.tig.as/blog/2012/02/13/deploying-django-on-heroku/

7/12

10/4/13

Deploying Django on Heroku - Minuti by Mike Tigas

D E F A U L T _ F I L E _ S T O R A G E = ' s t o r a g e s . b a c k e n d s . s 3 b o t o . S 3 B o t o S t o r a g e ' S T A T I C F I L E S _ S T O R A G E = D E F A U L T _ F I L E _ S T O R A G E A W S _ A C C E S S _ K E Y _ I D = ' ' A W S _ S E C R E T _ A C C E S S _ K E Y = ' ' A W S _ S T O R A G E _ B U C K E T _ N A M E = ' ' S T A T I C _ U R L = ' / / s 3 . a m a z o n a w s . c o m / % s / ' % A W S _ S T O R A G E _ B U C K E T _ N A M E A D M I N _ M E D I A _ P R E F I X = S T A T I C _ U R L + ' a d m i n / '

Copy and paste the following lines into the end of your settings file to enable database configuration by reading the D A T A B A S E _ U R Lenvironment var.
i m p o r t o s i m p o r t s y s i m p o r t u r l p a r s e # R e g i s t e r d a t a b a s e s c h e m e s i n U R L s . u r l p a r s e . u s e s _ n e t l o c . a p p e n d ( ' p o s t g r e s ' ) u r l p a r s e . u s e s _ n e t l o c . a p p e n d ( ' m y s q l ' ) t r y : i f ' D A T A B A S E S ' n o t i n l o c a l s ( ) : D A T A B A S E S = { } i f ' D A T A B A S E _ U R L ' i n o s . e n v i r o n : u r l = u r l p a r s e . u r l p a r s e ( o s . e n v i r o n [ ' D A T A B A S E _ U R L ' ] ) # E n s u r e d e f a u l t d a t a b a s e e x i s t s . D A T A B A S E S [ ' d e f a u l t ' ] = D A T A B A S E S . g e t ( ' d e f a u l t ' , { } ) # U p d a t e w i t h e n v i r o n m e n t c o n f i g u r a t i o n . D A T A B A S E S [ ' d e f a u l t ' ] . u p d a t e ( { ' N A M E ' : u r l . p a t h [ 1 : ] , ' U S E R ' : u r l . u s e r n a m e , ' P A S S W O R D ' : u r l . p a s s w o r d , ' H O S T ' : u r l . h o s t n a m e , ' P O R T ' : u r l . p o r t , } ) i f u r l . s c h e m e = = ' p o s t g r e s ' : D A T A B A S E S [ ' d e f a u l t ' ] [ ' E N G I N E ' ] = ' d j a n g o . d b . b a c k e n d s . p o s t g r e s q l _ p s y c o p g 2 ' i f u r l . s c h e m e = = ' m y s q l ' : D A T A B A S E S [ ' d e f a u l t ' ] [ ' E N G I N E ' ] = ' d j a n g o . d b . b a c k e n d s . m y s q l ' e x c e p t E x c e p t i o n : p r i n t ' U n e x p e c t e d e r r o r : ' , s y s . e x c _ i n f o ( )

(These have been copied from the Heroku Django starter doc. In cases where your Django app is on the root level of the repo, this code would automatically be appended to your settings file, but hey, were going for explicit instructions here to

https://mike.tig.as/blog/2012/02/13/deploying-django-on-heroku/

8/12

10/4/13

Deploying Django on Heroku - Minuti by Mike Tigas

try to cut through the magic.) Open up s r c / $ P Y T H O N _ A P P _ N A M E / u r l s . p yand uncomment the lines for the admin. Now commit and push.
g i t a d d . g i t c o m m i t m " e n a b l e a d m i n a n d b o t o b a c k e d s t o r a g e "

At this point you'll probably want to deploy your static files

h e r o k u r u n " P Y T H O N P A T H = / s r c p y t h o n s r c / $ P Y T H O N _ A P P _ N A M E / m a n a g e . p y c o l l e c t s t a t i c n o i n p u t "

(Note: the P Y T H O N P A T H = / s r cenv var needs to be set manually since manage.py doesnt seem to get it when using would live directly in PYTHONPATH.) And then syncdb to initialze your database and create a user account for yourself.
h e r o k u r u n " P Y T H O N P A T H = / s r c p y t h o n s r c / $ P Y T H O N _ A P P _ N A M E / m a n a g e . p y s y n c d b n o i n p u t " h e r o k u r u n " P Y T H O N P A T H = / s r c p y t h o n s r c / $ P Y T H O N _ A P P _ N A M E / m a n a g e . p y c r e a t e s u p e r u s e r " s r cwhich h e r o k ur u n .

The new default project layout in

Django 1.4 would make this step obsolete; in the Django 1.4 case, our m a n a g e . p y would cleanly put that directory on the implied

Now try to open up h t t p : / / P R O J E C T _ N A M E . h e r o k u a p p . c o m / a d m i n / . The page should load, complete with the normal styling (served from your S3 bucket). You should also be able to log in with the username and password you just created.

OTHER HELPFUL BITS

Ive found that its easiest to put templates in-app when using this workflow. If you need to use the old-fashioned workflow of putting all of your templates under one directory, you can move them to
' y o u r _ p y t h o n _ a p p _ n a m e 'to s r c / $ P Y T H O N _ A P P _ N A M E / t e m p l a t e s /and

then add

your I N S T A L L E D _ A P P S .

https://mike.tig.as/blog/2012/02/13/deploying-django-on-heroku/

9/12

10/4/13

Deploying Django on Heroku - Minuti by Mike Tigas

You can set up memcached similar to how you hooked up PostgreSQL. First, add it to your account:
h e r o k u a d d o n s : a d d m e m c a c h e : 5 m b

Then add p y l i b m cand d j a n g o p y l i b m c s a s lto your requirements.

c d $ V I R T U A L _ E N V / r e p o e c h o " p y l i b m c = = 1 . 2 . 2 " > > r e q u i r e m e n t s . t x t e c h o " d j a n g o p y l i b m c s a s l = = 0 . 2 . 4 " > > r e q u i r e m e n t s . t x t p i p i n s t a l l r r e q u i r e m e n t s . t x t

The d j a n g o p y l i b m c s a s lpackage is required to automatically configure memcached on Heroku (including the server, username, and password). All you have to do is point your settings file to its cache class:
C A C H E S = { ' d e f a u l t ' : { ' B A C K E N D ' : ' d j a n g o _ p y l i b m c . m e m c a c h e d . P y L i b M C C a c h e ' } }

Uploaded media (i.e. things in a F i l e F i e l dor I m a g e F i e l d ) will get thrown into your S3 bucket automatically. As per the Django file docs, using
o b j e c t . s o m e _ f i l e _ f i e l d . u r lwill

return the URL of the file, as stored in S3, so you

M E D I A _ U R L .

can use that property in templates without having to worry about

< i m gs r c = " { {o b j . i m a g e _ f i e l d . u r l} } " / >

(i.e.

see this demo page and the source of

that view for a full example.) You can get basic "piggyback" SSL support (where your app runs at
h t t p s : / / * . h e r o k u a p p . c o m / )

by adding that addon:

h e r o k u a d d o n s : a d d s s l : p i g g y b a c k

https://mike.tig.as/blog/2012/02/13/deploying-django-on-heroku/

10/12

10/4/13

Deploying Django on Heroku - Minuti by Mike Tigas

This simply sets up the HTTPS path but doesnt enforce it: to require SSL youll need to use some sort of Django middleware to redirect non-SSL requests. (Ive whipped up this one that can be set as the first middleware in access the domain via SSL.)
M I D D L E W A R E _ C L A S S E S .

It also sets

the S t r i c t T r a n s p o r t S e c u r i t yheader which tells complaint browsers to ONLY

POSTSCRIPT
Ive only been toying with the Heroku (Cedar) stack for about two weeks now, and its been pretty interesting so far. Its very cool to be able to provision, deploy, and scale a cloud-based website within a shell, without dealing with the underlying Linux systems much (if at all). (With built-in robustness, too: Heroku attempts auto restarts of crashed processes once every ten minutes.) Im not aware of any major Python/Django-running sites that deploy to Heroku in production, but the Cedar stack and the Python support along with it is fairly new. (Heroku does seem to be pretty popular for a fair bit of mostly Ruby-based sites.) While the costs seem high at first glance compared to a purely shared host or AWS by itself (about $36/mo per dyno after the first one, databases and addons on their own steep scale), the cost is theoretically balanced out by lessening the need of a "true" sysadmin staff since the infrastructure from hardware to OS, all the way up to the Python application is entirely outsourced. This does have its own ups and downs (that I wont get into since Im still relatively new to the platform), but in terms of raw cost, a fully decked 48-dyno (24 web, 24 worker) operation with a Ronin-class dedicated database would run you about $22,692 a year (which probably compares pretty favorably to a combination of hosting and IT staff costs in a more standard environment). On the other hand, I havent yet load-tested Django/Heroku with a more legit, fullfeatured website, so the performance factor in the cost analysis is still a big question mark. February 13, 2012 permalink share

https://mike.tig.as/blog/2012/02/13/deploying-django-on-heroku/

11/12

10/4/13

Deploying Django on Heroku - Minuti by Mike Tigas

PyPyonHeroku

Lookingforsomething?

PreviousPost

Archive
MikeTigasisaKnightMozillaOpenNewsFellowandaWeb/mobile applicationsdeveloperwithexperienceinthemediaindustry(portfolio).
PGP:0x6E0E9923&0x3082B5A3,OTR:0xB0846D0B&0x17F5E551

20012013MikeTigas.CCAttribution3.0.Seelicense. PoweredbyDjangoanddjangomedusa.Seecolophon.
AlsoavailableviaTor:tigas3l7uusztiqu.onion

https://mike.tig.as/blog/2012/02/13/deploying-django-on-heroku/

12/12