Project

General

Profile

[Solved] Is there a simple way of implementing Server-Side Events using lighttpd?

Added by Deweloper over 7 years ago

Hello,

I'd like to implement a service which would send a text/event-stream content as described at https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#Examples .
This could be done inside a FastCGI responder backend, but each client connection (long-lasting) would need it's own instance of the backend process.
Does the FastCGI protocol allow duplicating a fd and continuing sending response in parallel with processing next calls to FCGI_Accept()?
Are there any other options than porting mod_fd_transfer to current version of lighttpd?


Replies (11)

RE: Is there a simple way of implementing Server-Side Events using lighttpd? - Added by gstrauss over 7 years ago

This could be done inside a FastCGI responder backend, but each client connection (long-lasting) would need it's own instance of the backend process.

Not true. You can write a FastCGI backend which accepts multiple connections and multiplexes across them.

Does the FastCGI protocol allow duplicating a fd and continuing sending response in parallel with processing next calls to FCGI_Accept()?

Duplicating the fd? No. Multiplexing responses? Yes. However, lighttpd creates a new socket connection for each request instead of sending multiple requests on the same connection. One reason for this is simplicity. Another is that FastCGI protocol does not have flow control. (HTTP/2 has flow control). It is possible to have one lighttpd server process and one FastCGI backend process, using 2 fds per client (one from client to lighttpd and one from lighttpd to FastCGI backend).

Are there any other options than porting mod_fd_transfer to current version of lighttpd?

With recent changes in lighttpd 1.4.40, it should be easier to port mod_fd_transfer to lighttpd 1.4.40, perhaps without even needing extra hooks in lighttpd. Another option is HTTP/2, but sadly you'll have to use a different server since lighttpd does not currently support HTTP/2.

RE: Is there a simple way of implementing Server-Side Events using lighttpd? - Added by Deweloper over 7 years ago

So it seems that just the API of libfcgi is limited to a single request at a time, but not the FastCGI protocol itself.
Thanks.

RE: [Solved] Is there a simple way of implementing Server-Side Events using lighttpd? - Added by christophe.lohr@cegetel.net almost 7 years ago

Hi,
I also have a question about creating "event-stream".
Well, my question is not about managing multiple requests in the same time, but about real-time effects.

Attached is a piece of code to test "event-stream" (based on https://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_ModFastCGI#Troubleshooting)
I wished keep open the connection and to push updates of a variable to feed an ajax client.

Unfortunately the client does not receive updates in real-time (i.e, on the FCGX_FFlush(request.out)).
The client receive all updates at once, at the end (on the FCGX_Finish_r(&request)).

So, what did I miss?

Many thanks for your advices.
Best regards

RE: [Solved] Is there a simple way of implementing Server-Side Events using lighttpd? - Added by christophe.lohr@cegetel.net almost 7 years ago

Hi,
May I ask another question?
I can't see how to know when the web client leaves and closes the connexion to lighttpd.
The call to FCGX_FPrintF does not return any error. The fastcgi socket is still alive.
The SIGPIPE signal is already handled by the fcgi library (with an empty handler ;-) ) (nevertheless, the sigpipe signal is not useful in case of multiple requests in parallel)

So, what is the best way to do?
No more magic configuration option? ;-)

Best regards

RE: [Solved] Is there a simple way of implementing Server-Side Events using lighttpd? - Added by gstrauss almost 7 years ago

Just as the lighttpd configuration option is documented, so is the fcgi library that you are using. As this is not a forum for the fcgi library, I suggest looking in the documented header at FCGX_FPrintF() and FCGX_FFlush().

RE: [Solved] Is there a simple way of implementing Server-Side Events using lighttpd? - Added by christophe.lohr@cegetel.net almost 7 years ago

Hi,
I'm looking a the fastcgi protocol specification: https://fastcgi-archives.github.io/FastCGI_Specification.html#S5.4
If I understand well, when the web client leaves, the server:
1) sends an FCGI_ABORT_REQUEST to the fast-cgi application
2) sends a SIGPIPE to the application
3) closes the (unix) sockets to the application in case of not-multiplexed requests (but after receiving back a FCGI_REQUEST_COMPLETE?)

Isn't it ?

Well, I use the plain old fcgi library. I look at the code. It seems that the above-mentionned points are not implemented. (Or not on the responsibility of the library ;-) )
Nevertheless, I can code something to handle termination of the client, in the context of my app.
The simplest way to do that is to check if the socket is still alive or not. But I can also check incoming FCGI_ABORT_REQUESTs, if needed.
What does the lighttpd actually do when the client leaves?

Thank you for your explanations.
Best regards.

RE: [Solved] Is there a simple way of implementing Server-Side Events using lighttpd? - Added by gstrauss almost 7 years ago

If I understand well, when the web client leaves, the server:

It is apparent to me that you do not understand this very well. It is also apparent to me that you have barely tested your code. Finally, it is apparent to me that you have not looked in the documented header fcgiapp.h (e.g. /usr/include/fcgiapp.h) at FCGX_FFlush() to which I pointed you in my previous post.

1) sends an FCGI_ABORT_REQUEST to the fast-cgi application

Maybe; not required.

2) sends a SIGPIPE to the application

No. Your app receives a SIGPIPE if your app writes to a broken pipe or socket, and has not been configured to handle SIGPIPE.

3) closes the (unix) sockets to the application in case of not-multiplexed requests (but after receiving back a FCGI_REQUEST_COMPLETE?)

Maybe; FCGI_REQUEST_COMPLETE is not required

What does the lighttpd actually do when the client leaves?

lighttpd closes the socket. That's it.

As I mentioned above, this is a forum for lighttpd. There are numerous fastcgi library implementation, though you seem to think there is only one. My final response in this thread is this: write some code and test it. You have many incorrect assumptions above, and many of your questions could be answered by writing a small amount of code and testing it.

RE: [Solved] Is there a simple way of implementing Server-Side Events using lighttpd? - Added by christophe.lohr@cegetel.net almost 7 years ago

Dear,
even if you have posted your "final response", do you allow me to continue a little bit? ;-)

The attached code accepts several clients, and sends a data to each of them, time to time.

According to my tests, when a client leaves, lighttpd does not send a SIGPIPE immediately, but rather when another client arrives.
It doesn't send a FCGI_ABORT_REQUEST, and doesn't close the socket.

Note that the FCGX_FFlush() doesn't return any error. Yes, I started with libfcgi, I continue with it... for now.

Nevertheless, thank you for your feedback and your patience.

Best regards.

    (1-11/11)