Sexy: NGiNX + uWSGI deployment for Django with virtualenv
Deploying NGiNX together with uWSGI and virtualenv for use in Your Django driven apps could be a bit confusing if You have no previous (or little) experience in using mentioned software.
This tutorial will show You how to install and configure neat stack for Django apps utilising NGiNX, uWSGI and virtualenv.
I will be using Ubuntu Server 10.10 (32bit in a virtualbox container) for this purpose.
0. Prerequisites
We’ll need root-level access. If we are in the sudoers file (or group we are belonging to is) following command will do the trick:
sudo -sI’m assuming You already have installed build-essential packages, if not you ought to type following command:
aptitude install build-essential1. uWSGI
Note:To compile uWSG under ubuntu You’ll need libxml2-dev and python-dev packages so install them beforehand
mkdir /usr/src/uwsgi
cd /usr/src/uwsgi
wget http://projects.unbit.it/downloads/uwsgi-0.9.6.8.tar.gz
tar xfz uwsgi-0.9.6.8.tar.gz
cd uwsgi-0.9.6.8
make
cp ./uwsgi /usr/local/sbin/Firstly we create a directory for our uUWSGI source (for keeping our stuff in a proper place). After changing to it as working dir we download (wget) the uWSGI tarball, extract, compile and copy the resulting binary file ‘uwsgi’ into /usr/local/sbin dir.
Since Ubuntu uses Upstart for managing services we won’t use old-styled /etc/init.d/ script. Instead we will have something like this:
# file: /etc/init/uwsgi.conf
description "uWSGI starter"
start on (local-filesystems
and runlevel [2345])
stop on runlevel [016]
respawn
exec /usr/local/sbin/uwsgi \
--uid www-data \
--socket 127.0.0.1:5050 \
--master \
--logto /var/log/uwsgi_main.log \
--logdate \
--optimize 2 \
--processes 2 \
--harakiri 120 \
--vhost \
--no-siteYou’ll need to copy it and paste it into /etc/init/uwsgi.conf file, for which purpose You could use any text editor of choice, like nano:
nano /etc/init/uwsgi.confNow I’ll the meaning of each option we are using to start uWSGI.
- –uid www-data sets the user that will be used to execute the uWSGI binary, ‘www-data’ is generally the common name for the user to whom are assigned the processes of server software providing services associated with HTTP protocol (eg Apache, Nginx, Lighttpd, etc.)
- –socket 127.0.0.1:5050 this option tells the uWSGI binary to bind to our servers’ loopback interface, and listen on port 5050. uWSGI can also use UNIX sockets but in this example we won’t be using it.
- –master enables master uWSGI managing process.
- –logto /var/log/uwsgi_main.log tells uWSGi where to store messages regarding its use
- –logdate adds timestamps to loglines
- –optimize 2 sets python optimization level to 2 (if You need debbuging information or Your app is using docstrings for any execution-related purpose omit this line)
- –processes 2 tells uWSGI to start two workers, You could set it to a higher number but generally it’s good to set it to exact number of cores of a CPU installed on Your server.
- –harakiri 120
- –vhost enables “virtual hosting” or rather dynamic python interpreter starting capabilities (we’ll be using it to run our django apps later)
- –no-site tells uWSGI to omit system wide python site-packages directory (it generally is crucial if we want to use virtualenv)
2. NGiNX
To build NGiNX we’ll need following packages installed:
aptitude install libpcre3-dev libpcrecpp0 libssl-dev zlib1g-dev libgeoip-devThen, type (or paste) following commands in order:
mkdir /usr/src/nginx
cd /usr/src/nginx
wget http://nginx.org/download/nginx-0.8.54.tar.gz
tar xfz nginx-0.8.54.tar.gz
cd nginx-0.8.54/Next, we’ll configure NGiNX:
./configure --pid-path=/var/run/nginx.pid \
--conf-path=/etc/nginx/nginx.conf \
--sbin-path=/usr/local/sbin \
--user=www-data \
--group=www-data \
--http-log-path=/var/log/nginx/access.log \
--error-log-path=/var/log/nginx/error.log \
--with-http_stub_status_module \
--with-ipv6 \
--with-http_ssl_module \
--with-http_realip_module \
--with-sha1-asm \
--with-sha1=/usr/lib \
--http-fastcgi-temp-path=/var/tmp/nginx/fcgi/ \
--http-proxy-temp-path=/var/tmp/nginx/proxy/ \
--http-client-body-temp-path=/var/tmp/nginx/client/ \
--with-http_geoip_module \
--with-http_gzip_static_module \
--with-http_sub_module \
--with-http_addition_module \
--with-file-aio \
--without-mail_smtp_module
and then install it, together with configuring tmp dir for nginx temporary data:
make && make install
mkdir /var/tmp/nginx
chown www-data /var/tmp/nginxAgain we’ll use upstart for managing our nginx process, so paste following code into /etc/init/nginx.conf file:
# file: /etc/init/nginx.conf
description "NGiNX starter"
start on (local-filesystems
and runlevel [2345])
stop on runlevel [016]
respawn
exec /usr/local/sbin/nginxNext we’ll need to create three dirs for our sites configuration
cd /etc/nginx
mkdir sites-available sites-disabled sites-enabledNext paste following code into backends.conf file (we’ll use it to store our uwsgi configuration):
upstream uwsgi_main {
server 127.0.0.1:5050;
}After that paste following two lines right before the “}” mark located at the end of /etc/nginx/nginx.conf file:
include /etc/nginx/backends.conf;
include /etc/nginx/sites-enabled/*;The next thing is creating configuration file holding information about our django website, it could be something simple like this:
server {
listen 80;
server_name .mywebsite.com;
set $home /home/manynights/mywebsite.com/pydocs;
access_log /home/manynights/mywebsite.com/logs/nginx/access.log;
error_log /home/manynights/mywebsite.com/logs/nginx/error.log;
client_max_body_size 10m;
keepalive_timeout 120;
location / {
uwsgi_pass uwsgi_main;
include uwsgi_params;
uwsgi_param UWSGI_CHDIR $home/deploy;
uwsgi_param UWSGI_SCRIPT deploy;
uwsgi_param UWSGI_PYHOME $home/env;
root $home;
}
location /media/ {
root $home;
autoindex on;
error_page 404 = "404";
}
}You should paste above code to /etc/nginx/sites-available/mywebsite.com and after that symlink it into sites-enabled directory:
ln -s /etc/nginx/sites-available/mywebsite.com /etc/nginx/sites-enabled/mywebsite.comNow, we’ll create mywebsite.com directory to hold it’s files (note that for base dir for my apps I’ll be using my users home dir: /home/manynights, thus I will be using another terminal so there won’t be any weird situation like having problems with access because root would be the holder of owner permissions for my apps’ dir):
cd ~/
mkdir mywebsite.com
cd mywebsite.com
mkdir pydocs logs logs/nginxWe should set proper permissions, I’m using setfacl, but chmod is enough too:
setfacl -Rm u:www-data:rwx logs
setfacl -Rm u:www-data:rx pydocs3. virtualenv
On Ubuntu you can install virtualenv by typing following command in your prompt:
sudo aptitude install python-virtualenvIt will install the virtualenv package itself and packages that are required.
cd ~/mywebsite.com/pydocs
virtualenv --no-site-packages env
virtualenv --relocatable env
Important thing: “virtualenv –relocatable env” must be issued after creating the environment itself, in my case the virtualenv environments name is env.
Note: You could also create a shell script to do it all for you, from the website dir’s creation right to the virtualenv preparation.
4. Django
To install latest stable Django version (1.3 at the moment) in our virtualenv we ought to issue following commands:
cd ~/mywebsite.com/pydocs
. env/bin/activate
mkdir env/src
wget -O env/src/django.tgz http://www.djangoproject.com/download/1.3/tarball/
cd env/src
tar xfz django.tgz
cd Django-1.3
python setup.py install
5. App configuration
The remaining step is starting our django project and configure our app properly.
cd ~/mywebsite.com/pydocs
. env/bin/activate
django-admin.py startproject pro
mv pro/* .
rm -rf pro
sed -i "s/pro\.urls/urls/g" settings.py
We are using django-admin.py from our virtualenv environment to create (startproject) our project in pro directory and then move its contents to our pydocs dir.
Note:django-admin.py script doesn’t allow starting our project in current dir.
While still working on our pydocs dir we’ll also copy django admin media to media dir:
mkdir media
cp -R env/lib/python2.6/site-packages/django/contrib/admin/media/* media/As You probably already noticed we’ll be needing deploy script to make our django project smoothly work with NGiNX and uWSGI:
mkdir deploy
nano deploy/deploy.pyPaste following code into it:
import sys
import site
import os
envpath = '/home/manynights/mywebsite.com/pydocs/env/lib/python2.6/site-packages'
prev_sys_path = list(sys.path)
site.addsitedir(envpath)
sys.path.append(os.path.dirname(os.path.abspath(__file__)) + '/..')
new_sys_path = [p for p in sys.path if p not in prev_sys_path]
for item in new_sys_path:
sys.path.remove(item)
sys.path[:0] = new_sys_path
from django.core.handlers.wsgi import WSGIHandler
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
application = WSGIHandler()
That’s all. If you have any questions feel free to ask.
No related posts.
12 Comments
Thanks for the info,
It will be more beneficial if there are also informations for nginx.conf script and virtualenv along with.
I’ve already did but the cache refresh didn’t make in in time ;P
Hi and thanks for the tutorial. It is a long time that I am looking for a through, easy to follow, and flawless tutorial to make this ‘sexy’ combination working. So I carefully followed your instructions, but ended up with this error:
uWSGI Error
wsgi application not found
Any ideas?
Did You named the deploy script properly? are the ACL properties set for www-data user? and could You post the logs from uWSGI and your nginx config files? (pastebin would be appreciated).
I went through everything you mentioned and am not sure where to go from here. I ran the deploy.py script but nothing output. my site shows a 404 so I’m just wondering if there’s a place to store the site files to test to see if nginx is working? does deploy.py start nginx or do I need to run another script to get it going?
i didn’t mention anything about running deploy.py script – it’s only a wrapper for connecting uwsgi with your django project (using wsgi). you must start both uwsgi and nginx by issuing following commands:
start nginx
start uwsgi
also, i’ve followed steps in my own tutorial on a fresh virtual machine and have had no problems whatsoever p_p
Hi manynights,
Followed the guide and very happy by the end of it. Running into some issues now though. Running multiple sites has this strange issue where only 2 sites run at a time??? Any suggestions or ideas as to why? The 2 sites will work and the others just give a 502 error. I even tried to up the Processors in the uwsgi.conf but still no success. I’ve checked the uwsgi log file and some of the sites are giving me errors about pyscopg for the postgre database but what makes this strange is that the site will work on and off.
The tutorial is great and works fine so far. I did have to add “expect fork” before “respawn” in order to keep nginx from going crazy on startup, though.
Without it, nginx kept being started, finding out that port 80 was already in use (by itself) and dying, ad nauseam. A quick look at /var/log/nginx/error.log revealed the issue and the example upstart conf at http://wiki.nginx.org/Upstart pointed me in the right direction.
I suspect this tutorial also works with multiple sites? I didn’t know uWSGI instances could be shared like that.
yes, it does work for multiple sites. I use it in production where there are many small sites (mainly static content + some forms). thanks for your appretiation ;p
your deploy script has an indention error.
sys.path.remove(item) needs to be indented.