utilities » weighttp

(this bug was restored from backup)

There are shortcomings in how weighttp handles chunked responses, specifically when the chunked parts are received across multiple reads.

Patch submitted in https://github.com/lighttpd/weighttp/pull/9

A snipped of syscalls from Happstack (running the pong program referenced above) receiving request and sending chunked response:

recvfrom(11, "GET / HTTP/1.1\r\nHost: any\r\nConne"..., 65536, 0, NULL, NULL) = 48
sendto(11, "HTTP/1.1 200 OK\r\nTransfer-Encodi"..., 169, 0, NULL, 0) = 169
sendto(11, "4\r\n", 3, 0, NULL, 0) = 3
sendto(11, "PONG", 4, 0, NULL, 0) = 4
sendto(11, "\r\n", 2, 0, NULL, 0) = 2
sendto(11, "0\r\n\r\n", 5, 0, NULL, 0) = 5
close(11)

Regarding Happstack:
Now then, Happstack could also use some improvement for better performance, using writev() instead of multiple syscalls to write the chunked parts. Additionally, before calling close(fd) on a client connection, Happstack ought to shutdown(fd, SHUT_WR) and then read() from the connection until EOF (or until a configured timeout), so that the HTTP client has a better chance of receiving the final packets of the response, instead of a TCP RST should the client attempt to write more to the socket (e.g. for HTTP pipelining or should the server send the HTTP response while the client is still sending request body). Another thing: Happstack running on Linux calls epoll_ctl with EPOLL_CTL_MOD (which fails on a new connection) before falling back to epoll_ctl with EPOLL_CTL_ADD. Happstack should know the connection is new because it just called accept(). One more thing, directly related to weighttp, is that the performance of Happstack serving weighttp requests is terrible when weighttp keep-alive is enabled -- Happstack is 100x slower than weighttp without keep-alive! Repro with weighttp -n 1000 with and without -k. This is likely a problem with Happstack since weighttp -k against lighttpd performs much better -- the expected behavior -- than weighttp without -k.

Happstack is also very busy with syscalls for thread IPC and short timers. Happstack should wait for OS events to wake it when there is nothing to do, instead of waking itself so frequently with timers.

In any case, use weighttp, once weighttp more robustly handles chunked encoding, makes it easier to spot these Happstack performance deficiencies.

The issue in this ticket has been fixed.

stbuehler: please review additional patches pending in https://github.com/lighttpd/weighttp/pull/9