Lighty not closing idle connections correctly?
I use Lighttpd for serving static files on an extremely busy website. When looking into some issues with downloads from the server that it runs on, I noticed the following:
First, a screenshot of server-status at 19 m 52 s: http://www.modyourpanties.com/hosting/6109_090309122705skuld19m52.jpg
Next, a screenshot of server-status at 27 m 50 s: http://www.modyourpanties.com/hosting/6110_090309123008skuld27m50.jpg
The thing that strikes me is that the "Written" column hasn't changed - on the first file it's stuck on 2070055/2207997 - yet these 2 snapshots are taken 8 mins apart!
I my config I have the following:
server.max-keep-alive-requests = 4
server.max-keep-alive-idle = 4
server.max-read-idle = 60
server.max-write-idle = 120
So I'm expecting the idle writes to close after 120 seconds... but they don't. This behaviour, btw, happens all the way down the list with very many connections getting "stuck" in the same write amount. Only a restart clears them, but not all connections are like this and some do indeed get removed correctly. I'm concerned that the "stale connections" is basically going up till I hit my server limit of 500 connections and not getting cleared down correctly.
I'm running Lighttpd 1.4.20 on CentOS 5 (64bit), on a server with 2gb ram. I can supply a full config if needed.
Does anybody have any ideas on this one?
I currently have a server with nearly 2.5k connections, of which about 2000 of them have times way over the timeout. Some of them are at 20,000 seconds.
So I guess there is no fix for this... :)
please upgrade to current (1.4.22) version, we won´t discuss on older one´s as they might already be fixed.
on closer look at the server-status this simply might be a bandwidth problem.
with ~5gig in 20 mins this might overload your network and connections simply sum up to > 500. i´d disable keep-alive completely and it should be quite safe to allow more than 1k or 2k connections.
beside this forum there´s also #lighttpd on irc.freenode.net and make sure to report back to this thread if your problem was solved.
Thanks for the reply - I'd already upgraded to 1.4.22 and the same issue happened. I also temporarily removed the connection cap, just to see what happened and as you can see from my previous post, it just kept adding connections and kept adding...
I'll try disabling keep alive and see if that helps - it's already on a very low value though.
Thanks for the info about the IRC channel. :)
PS I have 2 high load high traffic lighttpd servers running on seperate boxes. The other one only serves images and thumbnails and doesn't seem to have the same issue even though they are identical boxes with identical lighty installs (only the config file differs slightly). It's only on my main fileserver that I have this issue.
Okay after doing some investigate work it appears this issue is caused by the server.kbytes-per-second value being set. Turning this off (even in conjunction with connection.kbytes-per-second, and evasive) solved the above issue. Turning it back on saw the issue come back.
This is a key thing for me since I dont use traffic shaping at the kernel level, but instead use Lighttpd's method.