Project

General

Profile

Feature #1523

Add WSGI support to mod_proxy backends

Added by Anonymous almost 9 years ago. Updated 2 months ago.

Status:
Wontfix
Priority:
High
Assignee:
-
Category:
mod_proxy
Target version:
Start date:
Due date:
% Done:

0%

Missing in 1.5.x:

Description

WSGI is the way of deploying python applications. There are of course several ways of deploying, such as FastCGI, SCGI and HTTP - but all these are middleware to WSGI and therefore "sub-optimal". Both Apache (mod_wsgi) and Nginx (mod_wsgi) currently supports this.

Consider this ticket a humble request (or perhaps a placeholder for my future patch :) to incorporate WSGI as a backend to mod_proxy for the upcoming lighttpd 1.5.

-- Lfe

History

#1 Updated by darix almost 9 years ago

  • Status changed from New to Fixed
  • Resolution set to invalid

we got mod_proxy_backend_scgi.c already. ever thought on using that?

#2 Updated by Anonymous almost 9 years ago

  • Status changed from Fixed to Need Feedback
  • Resolution deleted (invalid)

As the OP mentioned, using SCGI requires a middleware library like FLUP to connect SCGI->FLUP->WSGI. Given WSGI is the way forward for "at least" Python deployments, and the current availability of mod_wsgi for apache and nginx, I think this should be looked at, or at least considered carefully before closing INVALID so abruptly.

The incentive and desire to remove Apache out of our web application achitectures is quite high on my TODO list, as I'm sure is the same for other system admins / developers out there.

I would also be looking at a mod_wsgi rather than integrating into mod_proxy.

#3 Updated by Anonymous over 8 years ago

+ from me..

I use Lighttpd as webserver on all our servers.

We have many web applications written in python - but the tinkering with stuff like "flup" etc. sometimes let me think of switching back to Apache, because it has a mod_wsgi which works great in my tests..

Or I use Nginx, who I didn't used till now, but seems to be a new concurrent to Lighttpd.

But as I really like Lighttpd, especially the configuration, me and very much other web programmers who doesnt do their web applications in PHP, or lame CGI scripts, I want to stay with your webserver - would really appreciate it, if there will be a mod_wsgi soon, like on the other state-of-the-art web servers today.

Also a +1 that it should be a mod_wsgi instead of a mod_proxy wsgi thing..

-- mm

#4 Updated by Anonymous over 8 years ago

+1 for mod_wsgi

-- matin

#5 Updated by Anonymous over 8 years ago

wrong mail address :(

-- matin

#6 Updated by Anonymous over 8 years ago

-- martin

#7 Updated by Anonymous over 8 years ago

wsgi requires us to link python into lighttpd and execute the scripts inside lighttpd. that means lighttpd blocks when ever your script executes and cant do anything else. nginx implements it that way and require the users to use multiple workers to handle multiple requests. given that lighttpd and nginx share some fundamental designs i dont see it happen anytime soon.

what is wrong with scgi/fcgi and python?

#8 Updated by hoffie over 8 years ago

We recently discussed that on IRC again, and the result was mostly the same what the anonymous person in comment 8 said.
mod_wsgi is just a special mod_python, and mod_{anythingthatblocks} is impossible to implement in lighty. nginx/mod_wsgi might be fun for testing, but nothing more and nothing less.
If you need a quick way to deploy WSGI-compliant applications, simply use python's built-in web server, which is especially targetted at this task.
For production deployments, use SCGI or FastCGI (or even proxy to HTTP).
After all, WSGI is just a API specification, not a protocol like FastCGI/SCGI.

I tend to close this as WONTFIX...

#9 Updated by Anonymous over 8 years ago

I'll try to work this out and hopefully can then provide somewhat generic instructions. After all I'm mostly stressed by all the different options to deploy stuff and in most cases you can't find a good example.

Personally I'm fine with either decision - just decide so that I know what direction to expect (can't speak for others) :)

even wiki:Docs:ModFastCGI doesn't tell about python (see end of page) :(

-- martin

#10 Updated by Anonymous over 8 years ago

you see mod_fastcgi and mod_scgi (or there respective counterparts in 1.5) have some kind of generic scheme how they work. now all you need is a small python script which handles the fcgi/scgi loop and calls your logic code. a bit googling brings up http://cleverdevil.org/computing/24/python-fastcgi-wsgi-and-lighttpd for example. which has a nice generic example for you.

flup is another one. but in the end i would expect your framework like django/turbogears/nevow will ship with a scgi/wscgi script or some standalone webserver, which you could access with mod_proxy.

#11 Updated by stbuehler over 8 years ago

  • Status changed from Need Feedback to Fixed
  • Resolution set to wontfix

We will never include blocking modules for generating content. And no, we will not start threading or forking for this. Use apache if you think that would be the right way.

#12 Updated by stbuehler about 8 years ago

  • Status changed from Fixed to Wontfix

#13 Updated by gstrauss 2 months ago

  • Description updated (diff)

Anonymous wrote:

I'll try to work this out and hopefully can then provide somewhat generic instructions. After all I'm mostly stressed by all the different options to deploy stuff and in most cases you can't find a good example.

Unfortunately, this is still true today, more than 8 years later. Any experienced Python programmer reading this should try really hard to understand that the flexibility of the Python frameworks are great, but the dearth of accessible documentation for newcomers is very frustrating to many people. The WSGI documentation is most useful for those who already know how to do things. Most Python documentation surrounding WSGI is very hard to approach for those new to WSGI.

From the perspective of lighttpd, I'll summarize and reiterate some of the answers already provided.

The "GI" in many of these protocols is an acronym for "gateway interface" protocol. A gateway interface is a description how to communicate across a gateway (e.g. between different processes). For example, CGI ("common gateway interface") is a description of how a web server (which speaks an HTTP protocol to the client) can communicate the client request (HTTP protocol) to a separate process which understands the CGI protocol. A web server sets up the environment, starts the backend CGI process to handle the request, and reads the response from the backend process.

WSGI is even more constrained, and in my opinion, is a Python API and not a gateway interface. WSGI describes an API in Python for Python to run a callable Python application routine to handle a request, and for how that application routine responds. If a web server is a Python-based web server or has the Python interpreter built into the web server, then the web server can implement WSGI API within the web server process. If not, then the web server must use a different gateway interface to communicate to a Python process (which supports that same gateway interface) which can then translate the request to the WSGI API.

lighttpd is single-threaded and does not have any plans to build Python into the lighttpd server since arbitrary Python application code can cause the entire lighttpd server to block.

If you can find the right documentation, it is quite easy to have lighttpd (or other webserver) use any of a number of gateway interfaces to speak to a Python process, which then calls the WSGI API, e.g.

  process     protocol    process
  --------    --------    --------------
  lighttpd -> HTTP     -> Python process (-> WSGI app)
  lighttpd -> FastCGI  -> Python process (-> WSGI app)
  lighttpd -> SCGI     -> Python process (-> WSGI app)
  lighttpd -> uwsgi    -> Python process (-> WSGI app)
  ...

As an aside, uwsgi (lowercase) is a protocol. uWSGI is a featureful application server framework which implements the uwsgi protocol as well as HTTP, FastCGI, SCGI, and other protocols.

#14 Updated by gstrauss 2 months ago

https://redmine.lighttpd.net/projects/lighttpd/wiki/HowToPythonWSGI

lighttpd is single-threaded and does not have any plans to build an embedded Python interpreter into the lighttpd server since arbitrary Python application code can cause the entire lighttpd server to block. Therefore, lighttpd must use another gateway protocol (HTTP, FastCGI, SCGI, uwsgi, CGI, ...) to run Python applications which implement the WSGI API.

A Python server must be run as an independent process to serve a Python WSGI application (unless lighttpd is configured to run the Python process as a CGI). There are many Python servers which support one or more of the above protocols and also support Python WSGI applications. The WSGI website lists some Python servers at http://wsgi.readthedocs.io/en/latest/servers.html

Also available in: Atom