HowToPythonWSGI¶

Background: What is WSGI?¶

The official WSGI documentation (http://wsgi.readthedocs.io/en/latest/) has the following definition of WSGI at
http://wsgi.readthedocs.io/en/latest/what.html

WSGI is the Web Server Gateway Interface. It is a specification that describes how a web server communicates with web applications, and how web applications can be chained together to process one request.

The above would be much clearer if it stated that WSGI is "a specification that describes how a Python-capable web server communicates with Python web applications running within the Python server. WSGI describes an API in Python for the Python interpreter to run a callable Python function (which implements a web app) to handle a request, and for how that Python function responds to the request.

If a web server is a Python-based web server or has an embedded Python interpreter built into the web server, then the web server can implement WSGI API within the web server process. If not, then the web server must use a different gateway protocol (HTTP, CGI, FastCGI, SCGI, uwsgi, ...) to communicate to a Python process (which supports that same gateway protocol), and then the Python process can translate the request from the gateway protocol to the WSGI API.

lighttpd and Python WSGI¶

lighttpd is single-threaded and does not have any plans to build an embedded Python interpreter into the lighttpd server since arbitrary Python application code can cause the entire lighttpd server to block. Therefore, lighttpd must use another gateway protocol to run Python applications which implement the WSGI API.

• HTTP: lighttpd can use mod_proxy to connect to a Python server via HTTP
• FastCGI: lighttpd can use mod_fastcgi to connect to a Python server via FastCGI
• SCGI: lighttpd can use mod_scgi to connect to a Python server via SCGI (scgi.protocol = "scgi" (default))
• uwsgi: lighttpd can use mod_scgi to connect to a Python server via uwsgi (scgi.protocol = "uwsgi" (since lighttpd 1.4.42))
• CGI: lighttpd can use mod_cgi to start a Python program via CGI

A Python server must be run as an independent process to serve a Python WSGI application (unless lighttpd is configured to run the Python process as a CGI). There are many Python servers which support one or more of the above protocols and also support Python WSGI applications. The WSGI website lists some Python servers at http://wsgi.readthedocs.io/en/latest/servers.html

There are such a voluminous number of options that a few simple, easily accessible options are described below to help a new Python WSGI app developer get started.

Before proceeding, please see
https://docs.python.org/3/howto/webservers.html "HOWTO Use Python in the web" if not familiar with writing web applications.

Python WSGI apps via CGI¶

CGI is often the slowest way to run a Python WSGI application, but is also often the among the easiest to set up. The Python 2.5 or later standard library ships with wsgiref (https://docs.python.org/3/library/wsgiref.html). Here is a basic "Hello World!" Python WSGI application run by the wsgiref CGIHandler. This can be put into a file, marked executable, and run by a web server as a CGI program, e.g. using mod_cgi.

#!/usr/bin/env python
import wsgiref.handlers

def application(environ, start_response):
    start_response('200 OK', [('Content-Type', 'text/plain;charset=utf-8')])
    return ['Hello World!\n']

if __name__ == '__main__':
    wsgiref.handlers.CGIHandler().run(application)

A slightly more detailed explanation of the above can be found at http://henry.precheur.org/python/how_to_serve_cgi.html

Python WSGI apps via HTTP¶

Another way to run Python WSGI applications is from within a Python server which accepts HTTP requests. Many, many Python-based web servers are available with differing features and performance characteristics, but for simplicity, Python 2.5 or later standard library provides wsgiref.simple_server (https://docs.python.org/3/library/wsgiref.html#module-wsgiref.simple_server). Even though a Python-based web server may be able to serve all client HTTP requests to the web site, it is common to have a featureful web server handling static content, authentication, etc. in front of the Python-based web server which focuses on handling the Python WSGI application. lighttpd can serve a variety of client requests, and can be configured to use mod_proxy to send specific client requests to the backend Python-based web server when those client requests should be handled by the Python WSGI application.

The example given in https://docs.python.org/3/library/wsgiref.html#examples

#!/usr/bin/env python
from wsgiref.simple_server import make_server

# Every WSGI application must have an application object - a callable
# object that accepts two arguments. For that purpose, we're going to
# use a function (note that you're not limited to a function, you can
# use a class for example). The first argument passed to the function
# is a dictionary containing CGI-style environment variables and the
# second variable is the callable object (see PEP 333).
def hello_world_app(environ, start_response):
    status = '200 OK'  # HTTP Status
    headers = [('Content-Type', 'text/plain;charset=utf-8')]  # HTTP Headers
    start_response(status, headers)

    # The returned object is going to be printed
    return [b"Hello World"]

httpd = make_server('', 8000, hello_world_app)
print("Serving on port 8000...")

# Serve until process is killed
httpd.serve_forever()

To serve client requests, a startup (and shutdown) script must start up the Python-based web server as a daemon, just as something must start up the lighttpd web server as a daemon.

Python WSGI apps via uwsgi, SCGI, FastCGI, or HTTP using the uWSGI server¶

There are a plethora of Python-based servers supporting one or more of the gateway protocols (e.g. uwsgi, SCGI, FastCGI, HTTP, ...). The WSGI website lists some Python servers at http://wsgi.readthedocs.io/en/latest/servers.html

As with serving Python WSGI apps via HTTP (see section above), a Python server must be started as a daemon, separate from the web server daemon in order to run Python WSGI apps via other gateway protocols, including uwsgi, SCGI, and FastCGI.

For this example, let's pick the uWSGI server to run a simple Python WSGI hello_world_app. First install the uWSGI server:

• Fedora: $ dnf install uwsgi uwsgi-plugin-python
• Debian: $ apt-get install uwsgi uwsgi-plugin-python

Create a simple "Hello World!" Python WSGI application in a file /path/to/htdocs/hello_world_app.py (and replace /path/to/htdocs/ with your own path).

def application(environ, start_response):
    start_response('200 OK', [('Content-Type', 'text/plain;charset=utf-8')])
    return ['Hello World!\n']

Here is a basic command for a startup script to run uWSGI server on localhost on port 8080 to handle the uwsgi protocol (lowercase "uwsgi"):

uwsgi -L --uwsgi-socket 127.0.0.1:8080 --plugin python --chdir /path/to/htdocs -w hello_world_app

With the above, lighttpd can be configured to use mod_scgi to send requests for the hello_world_app to 127.0.0.1:8080. lighttpd 1.4.42 or later needs lighttpd.conf scgi.protocol = "uwsgi" to tell lighttpd to use the slightly more efficient uwsgi protocol instead of the default scgi protocol.

uWSGI server can use the SCGI protocol with lighttpd mod_scgi, though the uwsgi protocol is slightly more efficient.

uwsgi -L --scgi-socket 127.0.0.1:8080 --plugin python --chdir /path/to/htdocs -w hello_world_app

uWSGI server can use the FastCGI protocol with lighttpd mod_fastcgi.

uwsgi -L --fastcgi-socket 127.0.0.1:8080 --plugin python --chdir /path/to/htdocs -w hello_world_app

uWSGI server can use the HTTP protocol with lighttpd Docs_ModProxy.

uwsgi -L --http-socket 127.0.0.1:8080 --plugin python --chdir /path/to/htdocs -w hello_world_app

Of course, "/path/to/htdocs" should be replaced in the above examples with the actual path where hello_world_app.py is located.

The uWSGI server has many feature to control memory usage, parallelism, and more. Lots of uWSGI doc is available at http://uwsgi-docs.readthedocs.io/en/latest/

Aside: for slightly faster performace, both lighttpd and uWSGI can be configured to use a unix domain socket instead of a TCP socket, e.g. uwsgi --<xxx>-socket /path/to/yourapplication.sock ... (where <xxx> is the protocol, as in examples above).

Python WSGI apps via FastCGI using the flipflop server¶

The flipflop server can be used to run a Python WSGI app to which requests are sent from lighttpd using mod_fastcgi. Download the flipflop server (https://github.com/Kozea/flipflop) (https://pypi.python.org/pypi/flipflop) and create the following wrapper script hello_world_app.fcgi next to hello_world_app.py (from example above).

#!/usr/bin/env python
from hello_world_app import application
from flipflop import WSGIServer
WSGIServer(application).run()

Alternatively,

#!/usr/bin/env python
#: optional path to your local python site-packages folder
import sys
sys.path.insert(0, '<your_local_path>/lib/python2.6/site-packages')

def application(environ, start_response):
    start_response('200 OK', [('Content-Type', 'text/plain;charset=utf-8')])
    return ['Hello World!\n']

if __name__ == '__main__':
    from flipflop import WSGIServer
    WSGIServer(application).run()

Configure lighttpd mod_fastcgi to start the script when lighttpd starts up, or add the script to your own startup scripts so that it is available to lighttpd.

Python WSGI apps via FastCGI or SCGI using the flup server¶

flipflop is a stripped down version of flup (https://www.saddi.com/software/flup/) (https://pypi.python.org/pypi/flup/1.0). flup provides a FastCGI to WSGI server when using flup.server.fcgi.

from hello_world_app import application
from flup.server.fcgi import WSGIServer
WSGIServer(application).run()

flup can also run as an SCGI to WSGI server whe using flup.server.scgi.

from hello_world_app import application
from flup.server.scgi import WSGIServer
WSGIServer(application).run()

To run flup with Python 3, the flup-py3 package is needed. Alternatively, pip install flup6

Other references¶

Nicholas Piël published Benchmark of Python WSGI Servers on March 15, 2010. http://nichol.as/benchmark-of-python-web-servers and the post has a very nice set of examples configuring different Python WSGI servers, though please note that the benchmarks are over 6 years old at this time.