Lighttpd

"bin-path" in lighttpd starts up a backend server with the listening socket on stdin. The backend server is expected to accept() and serve connections.

Your script fails to understand this.

Any ideas for what we might be doing wrong

You made a wild and uneducated guess how to write the provided (Perl) examples into Python.

You should have taken a few moments of your own time to ask a search engine how to write a python websocket server. After all, you're not the first person to try to write a websocket server in python. There are a huge number of examples and even simple python libraries that have already been written.

Hi gstrauss,

I was afraid this was the case, but python has no construct for being able to bind or accept on sys.stdin. I tried many variations on this before posting here. In python, sys.stdin is not treated as equivalent to a socket, and unlike C and perl (apparently), you cannot construct a socket of any type from stdin. I had tried several approaches, even resorting to explicitly creating the socket from /dev/stdin, but this attempt throws a python error. Additionally I tried to explicitly use the socket file that lighttpd creates, but you can't bind to a socket file that already exists so that is clearly not a workable solution.

Given that, I was hoping that there was some esoteric workaround that allows a socket to be created and accept connections. Google has failed to provide any answers for anything remotely related to this problem. I figured since lighttpd is the first place I have seen stdin hacked in this fashion that someone here had also figured out how to make this work with python as it is only the single most popular programming language. Apparently I was mistaken.

As for your snarky and rude follow up, I should point out that I did quite extensive searching for python websockets solutions and they are almost all built on other webserver libraries and provide standalone webservers to implement the websockets interface (tornado, wstunnel, simply_websockets_server, etc...). In point of fact, I have an existing prototype implementation that uses the tornado websockets implementation. I wanted to switch to using lighttpd's implementation so that I would not have to dual configure mtls with a second webserver and could leverage the existing lighttpd instance.

Given the above, I can only conclude your advice is that I should simply stop trying to use lighttpd websockets implementation with python as it doesn't work and go use someone else's webserver. The decision to use stdin in this fashion is counter to 25 years of CGI implementation, and appears to be incompatible with the single most common language in use today. It may have seemed to make sense at the time, but time is showing it to be a lousy solution. A better solution would be for two options to be made available: a CGI compatible mode that establishes a new instance of the user provided handler for every connection without the need for the user to implement any kind of socket handling. Stdin and stdout could be used in this mode exactly as it is in other cgi applications. In fact this could be thought of, and probably implemented, as CGI where the client closes the connection instead of the server. For more complex problems a second option is one that connects to a named socket that is created by a single global instance of the users application (with the name of the socket provided to lighttpd in the configs). This approach allows any language, even python, to create a socket on the named file and follow industry best practices for socket handling.

If you are going to preach from on-high, make sure you really are on the high ground.

Passing the bound socket as fd 0 comes from FastCGI, but can be used in other environments as well. systemd socket activation is a very close idea (but supports multiple sockets).

https://git.lighttpd.net/lighttpd/lighttpd2/src/branch/master/tests/run-scgi-envcheck.py#L60 has an example of how to use a socket on fd 0 in python, which seems to work fine in the tests.

Imho having the webserver spawn backends is never a "production-grade" setup; you really should spawn services as separate units, and also run them with a different user (which lighttpd can't do for you). This especially works fine with systemd socket activation, where a unix socket can already be restricted to the webserver, and the service itself run as a different user. (You still can (and imho should) start the service at system boot, not "on-demand socket activated".)

Thank you Stbuehler,

the python you linked appears to be exactly the workaround I was looking for.

I was afraid this was the case, but python has no construct for being able to bind or accept on sys.stdin.

False. Stop acting like an ignorant fool and learn how to use a search engine.
You probably spent more time on your wall of text post than you did in a search engine.
Python has a websockets library. Amazing (to you, at least), the name of the library is "websockets".
https://pypi.org/project/websockets/
https://websockets.readthedocs.io/en/stable/reference/server.html#websockets.server.WebSocketServer

... Any other keyword arguments are passed the event loop’s create_server() method.
... You can set sock to a socket that you created outside of websockets.

https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.create_server

This site is for lighttpd, not for python and not a kindergarten site for "how to use a search engine".

I misinterpreted. I'll leave my post for my embarrasement.
With mod_wstunnel, you're not writing a websocket server.

However, a mod_wstunnel backend is a server, as I wrote in my first post, and your script looked nothing like a server that performed accept() on sockets. That remains true. What also remains true is that python can use sockets on a file descriptor, including sys.stdin.

Gstrauss,

I agree with you on both counts, and as Stbuehler demonstrated with the link above, getting a socket from stdin is possible, but requires undocumented knowledge of python and Unix in general. First, that stdin is always filehandle 0 (is it really?), second that you can provide the fileno to the python socket constructor, but only with the explicit fileno= prefix.

I'm sorry my initial question seemed overly simplistic, and I'm only posting this followup such that the explicit knowledge in the previous paragraph will be publicly available for the next person whom google brings here, and maybe we can save that person and yourself a little trouble down the road.

Otherwise, I do thank you for your help, and appreciate anyone who undertakes to help those that end up in these kinds of forums.