mod_proxy_core + mod_proxy_backend_http may leak connections
I tried to use mod_proxy_core + mod_proxy_backend_http link lighttpd with apache.
I killed apache and restart it (both lighttpd and apache were serving at the monent, 500 req/s), and I found lighttpd was not stable. Its cpu usage suddenly raised to a high level for a few seconds or even minutes, then falled to normal level, but after some minutes it raised again, and so on.
I shutdown lighttpd, set proxy-core.debug = 1, and did it angain. This time I found lighttpd kept printing "proxy_state_engine: state=6" in its errorlog and never stopped.
At last, I found an interesting thing. When the httpd backend returns an incomplete response, something like
"HTTP/1.1 200 OK\r\nContent-Length: 100\r\n"(obviously it's not finished yet), without any other headers or content-body, lighttpd would keep printing "proxy_state_engine: state=6". The reason is that the leak connections, caused by irregular response, occupy positions in srv->connections and never relase them. As the number of leak connections increases, lighttpd process's cpu usage can be up to 100%.
I suggest treat the incomplete response, which returned by backend httpd server, as a blank page.
Updated by liming almost 10 years ago
- File mod_proxy_core.diff mod_proxy_core.diff added
- Status changed from Fixed to Patch Pending
- Assignee set to stbuehler
- % Done changed from 100 to 0
Hi, I find this patch will cause a new bug, because sess->recv->is_closed is not reset to 0 when return HANDLER_COMEBACK
If you set
proxy-core.backends = ("18.104.22.168:1111", "22.214.171.124:1112")
proxy-core.protocol = "http"
proxy-core.balancer = "static"
126.96.36.199:1111 is not a valid backend, but 188.8.131.52:1112 is a working apache.
First, try 184.108.40.206:1111 and it fails (connection is refused). Second, try 220.127.116.11:1112 and succeeds connecting. However, lighttpd cannot receive any data from apache because sess->recv->is_closed = 1 after 18.104.22.168:1111 fails.
The new patch can fix this bug.
Also available in: Atom