Project

General

Profile

Connection limit issues, lighty not honoring timeouts

Added by Morgon almost 14 years ago

Hi there,

I'm having a little trouble understanding some of the 'idle' configuration options, as I thought this would limit how long a 'handle-req' status could take. I'm getting loads of 'connection limit reached' errors on a server with a ton of idle CPU/RAM available.

Right now the server's only been up 6 minutes and already the max of 800 connections has been reached, the vast majority of them in the 'handle-req' state. I lowered my max-write-idle and max-read-idle to 20 and 5, respectively, but these handle-req's are already up to 250 seconds at time of writing (from a server that's been up 9 minutes).

I can up the max connections, but those will just get full, as well. What can I do to make lighty play nice?

Relevant config options:

server.max-keep-alive-requests = 4 # Added to try to mitigate the issue. Helped, but only slightly.
server.max-keep-alive-idle = 4 # Added with above
server.max-write-idle = 20
server.max-read-idle = 5
server.max-fds = 64000
server.max-connections=800

....
    fastcgi.server = (
        "local" => (
          "socket" => "/tmp/php.socket" + var.PID,
          "bin-path" => "/usr/bin/php-cgi",
          "broken-scriptfilename" => "enable",
          "docroot" => "/path/to/docroot" 
          "check-local" => "disable",
          "allow-x-send-file" => "disable",
          "allow-x-sendfile" => "disable",
          "min-procs" => 3,
          "max-procs" => 6,
          "bin-environment" => ( "PHP_FCGI_CHILDREN" => "50", "PHP_FCGI_MAX_REQUESTS" => "1024" ),
          "bin-copy-environment" => ( "PATH", "SHELL", "USER" )
     )

I can somewhat delay it by putting PHP_FCGI_MAX_REQUESTS to something stupidly low, like 2, but the issue eventually materializes.

PHP 5.3.2, lighttpd 1.4.26, CentOS 5.4

Edit: I even tried 1.2.27 RC1 only because there was a supposed bugfix for 'connections piling up', but it seems it's specific to FreeBSD; as such, it did not correct my issue.

Thanks for your time!


Replies (3)

RE: Connection limit issues, lighty not honoring timeouts - Added by nitrox almost 14 years ago

Idle state is basically our keep-alive. The handle-req state indicates a problem with your php tho and thats sth. where it seems to be heavily broken. 6x50 php procs -> 300 procs in total? Who are you? Facebook?

Review your php stuff, there´s nothing lighty or any other httpd could do about, you either have way too much requests (and many people think so) or your code is way too slow or is doing strange stuff, like grabbing data from remote sources or crawling remote pages or fetch some sort of data via curl and alike (and tho many people think they have too many requests) its basically a crappy setup or crappy way of doing things.

Setting PHP_FCGI_MAX_REQUESTS to 2 is indeed stupid, you could use cgi instead and there would be no need for fastcgi.

RE: Connection limit issues, lighty not honoring timeouts - Added by Morgon almost 14 years ago

Thanks for the.. response.

The thing that annoys me the most is that this software configuration (minus some minor version differences) and code is the same across all 5 servers. None of the others have a problem. While I'm not Facebook, I do a ton of traffic on these machines (~13 million daily requests of a non-HTML application), and the procs are needed, especially during peak.

I guess I'll try backing my PHP 5.3.2 to 5.3.1 (which is the only version difference between my machines) and see if that helps.

Just so we're clear, you're saying that handle-req is the state that listens for PHP only? There are no lighty considerations for this particular value? I'd agree with you that it was something terribly wrong with my PHP code if it was eating CPU; but the machine is relatively idle.

Thanks again for your time.

RE: Connection limit issues, lighty not honoring timeouts - Added by nitrox almost 14 years ago

If by idle you mean they just wait for sth. without doing much work - there´s still sth. wrong. 5 Servers for ~30 r/s is alot too. Well, i don´t know your app nor the problems php might cause you, but where in this chain can lighty help to solve it? It just hands it over to php and waits if it gets sth. back to serve your clients.

Btw. we are at #lighttpd on irc.freenode.net, tho this sounds like a problem with php we might be able to exclude lighty from this equation and give some hints. If you found some help there, please report back to this thread.

    (1-3/3)