Bug #784
closedmod_fastcgi crash ?
Description
lightty unexpectedly crashed - nothing in system log, load was about 1.5-1.6 whcih is normal for that time, ~1.5 out of 2G RAM was used for buffers and cache.
fast-cgi threads are spawned externally, there are 8 masters wach running 5 childs:
mci6 ~ # ps faux | grep cgi lighttpd 25904 0.0 0.0 42116 704 ? Ss Jul21 0:01 /usr/bin/php-cgi lighttpd 1280 2.4 0.2 42640 4480 ? S 16:53 0:31 \_ /usr/bin/php-cgi lighttpd 1281 2.4 0.2 42640 4476 ? S 16:54 0:29 \_ /usr/bin/php-cgi lighttpd 1285 2.3 0.2 42652 4524 ? S 16:54 0:27 \_ /usr/bin/php-cgi lighttpd 1289 2.3 0.2 42652 4456 ? S 16:55 0:26 \_ /usr/bin/php-cgi lighttpd 1292 2.4 0.2 42640 4456 ? S 16:55 0:26 \_ /usr/bin/php-cgi lighttpd 25963 0.0 0.0 42116 704 ? Ss Jul21 0:00 /usr/bin/php-cgi lighttpd 1351 2.4 0.2 42640 4448 ? S 17:02 0:17 \_ /usr/bin/php-cgi lighttpd 1352 2.4 0.2 42640 4464 ? S 17:02 0:17 \_ /usr/bin/php-cgi lighttpd 1361 2.3 0.2 42644 4428 ? S 17:03 0:14 \_ /usr/bin/php-cgi lighttpd 1362 2.5 0.2 42644 4456 ? S 17:03 0:15 \_ /usr/bin/php-cgi lighttpd 1363 2.3 0.2 42640 4424 ? S 17:05 0:12 \_ /usr/bin/php-cgi lighttpd 26022 0.0 0.0 42112 692 ? Ss Jul21 0:00 /usr/bin/php-cgi lighttpd 1274 2.3 0.2 42640 4440 ? S 16:52 0:31 \_ /usr/bin/php-cgi lighttpd 1279 2.3 0.2 42640 4424 ? S 16:53 0:29 \_ /usr/bin/php-cgi lighttpd 1283 2.4 0.2 42640 4456 ? S 16:54 0:28 \_ /usr/bin/php-cgi lighttpd 1284 2.3 0.2 42648 4468 ? S 16:54 0:27 \_ /usr/bin/php-cgi lighttpd 1296 2.2 0.2 42644 4452 ? S 16:56 0:23 \_ /usr/bin/php-cgi lighttpd 26081 0.0 0.0 42116 696 ? Ss Jul21 0:00 /usr/bin/php-cgi lighttpd 1290 2.2 0.2 42640 4472 ? S 16:55 0:25 \_ /usr/bin/php-cgi lighttpd 1299 2.2 0.2 42648 4444 ? S 16:57 0:22 \_ /usr/bin/php-cgi lighttpd 1314 2.3 0.2 42652 4492 ? S 16:59 0:21 \_ /usr/bin/php-cgi lighttpd 1354 2.2 0.2 42652 4432 ? S 17:03 0:15 \_ /usr/bin/php-cgi lighttpd 1395 2.3 0.2 42648 4376 ? S 17:07 0:09 \_ /usr/bin/php-cgi lighttpd 26140 0.0 0.0 42116 696 ? Ss Jul21 0:00 /usr/bin/php-cgi lighttpd 1264 2.1 0.2 42656 4456 ? S 16:50 0:31 \_ /usr/bin/php-cgi lighttpd 1265 2.2 0.2 42648 4504 ? S 16:50 0:32 \_ /usr/bin/php-cgi lighttpd 1398 2.2 0.2 42640 4424 ? S 17:08 0:07 \_ /usr/bin/php-cgi lighttpd 1416 2.1 0.2 42640 4344 ? S 17:11 0:04 \_ /usr/bin/php-cgi lighttpd 1417 2.2 0.2 42648 4360 ? S 17:11 0:03 \_ /usr/bin/php-cgi lighttpd 26199 0.0 0.0 42116 696 ? Ss Jul21 0:00 /usr/bin/php-cgi lighttpd 1278 2.2 0.2 42640 4392 ? S 16:53 0:28 \_ /usr/bin/php-cgi lighttpd 1282 2.1 0.2 42648 4432 ? S 16:54 0:25 \_ /usr/bin/php-cgi lighttpd 1287 2.2 0.2 42660 4468 ? S 16:54 0:26 \_ /usr/bin/php-cgi lighttpd 1295 2.2 0.2 42660 4480 ? S 16:56 0:23 \_ /usr/bin/php-cgi lighttpd 1360 2.3 0.2 42668 4480 ? S 17:03 0:14 \_ /usr/bin/php-cgi lighttpd 26258 0.0 0.0 42112 692 ? Ss Jul21 0:00 /usr/bin/php-cgi lighttpd 1251 2.1 0.2 42636 4492 ? S 16:49 0:32 \_ /usr/bin/php-cgi lighttpd 1301 2.1 0.2 42636 4468 ? S 16:57 0:21 \_ /usr/bin/php-cgi lighttpd 1312 2.1 0.2 42636 4428 ? S 16:58 0:20 \_ /usr/bin/php-cgi lighttpd 1316 2.1 0.2 42644 4432 ? S 16:59 0:19 \_ /usr/bin/php-cgi lighttpd 1342 2.1 0.2 42636 4444 ? S 17:00 0:17 \_ /usr/bin/php-cgi lighttpd 26317 0.0 0.0 42116 620 ? Ss Jul21 0:00 /usr/bin/php-cgi lighttpd 1209 2.1 0.2 42644 4452 ? S 16:42 0:40 \_ /usr/bin/php-cgi lighttpd 1223 2.1 0.2 42648 4708 ? S 16:45 0:37 \_ /usr/bin/php-cgi lighttpd 1235 2.0 0.2 42644 4596 ? S 16:46 0:34 \_ /usr/bin/php-cgi lighttpd 1267 2.1 0.2 42640 4480 ? S 16:51 0:29 \_ /usr/bin/php-cgi lighttpd 1275 2.2 0.2 42648 4480 ? S 16:52 0:29 \_ /usr/bin/php-cgi
after restart - lighty started using them just fine like nothing happened
the only thing out of ordinary - DB was overused at that point of time and some queries were timing out/taking too long (same as previous two days)... however there are 3 boxes with identical hardware/software/configuration sharing the load, only this one had the issue
# lighttpd -v lighttpd-1.4.10 (ssl) - a light and fast webserver Build-Date: Apr 6 2006 16:15:33
# tail /var/log/lighttpd/access.log l66.32.72.109 wapi3.webkinz.com - [31/Jul/2006:11:36:27 -0400] "POST /API/sindex.php HTTP/1.1" 200 109 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)" 84.172.206.75 wapi3.webkinz.com - [31/Jul/2006:11:36:27 -0400] "POST /API/sindex.php HTTP/1.1" 200 334 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FunWebProducts; (R1 1.3); .NET CLR 1.1.4322)" 142.161.231.167 wapi3.webkinz.com - [31/Jul/2006:11:36:28 -0400] "POST /API/sindex.php HTTP/1.1" 500 369 "http://www.webkinz.com/SWF/main2.v30_5.swf" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" 209.115.215.127 wapi3.webkinz.com - [31/Jul/2006:11:36:28 -0400] "POST /API/sindex.php HTTP/1.1" 500 369 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)" 24.47.250.38 wapi3.webkinz.com - [31/Jul/2006:11:36:28 -0400] "POST /API/sindex.php HTTP/1.1" 500 369 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)" 24.66.94.140 wapi3.webkinz.com - [31/Jul/2006:11:36:28 -0400] "POST /API/sindex.php HTTP/1.1" 500 369 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; DigExt)" 68.93.109.125 wapi3.webkinz.com - [31/Jul/2006:11:36:28 -0400] "POST /API/sindex.php HTTP/1.1" 500 369 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; YPC 3.2.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727)" 72.74.232.42 wapi3.webkinz.com - [31/Jul/2006:11:36:28 -0400] "POST /API/sindex.php HTTP/1.1" 500 369 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)" 68.79.254.45 wapi3.webkinz.com - [31/Jul/2006:11:36:28 -0400] "POST /API/sindex.php HTTP/1.1" 500 369 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)" 69.175.94.108 wapi3.webkinz.com - [31/Jul/2006:11:36:28 -0400] "POST /API/sindex.php HTTP/1.1" 500 - "http://www.webkinz.com/SWF/main2.v33_0.swf" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
# cat /var/log/lighttpd/error.log 2006-07-31 00:01:53: (log.c.129) logfiles cycled 2006-07-31 11:36:28: (mod_fastcgi.c.2878) backend is overloaded, we disable it for a 2 seconds and send the request to another backend instead: reconnects: 0 load: 135 /tmp/php.sock.1 2006-07-31 11:36:28: (mod_fastcgi.c.2878) backend is overloaded, we disable it for a 2 seconds and send the request to another backend instead: reconnects: 1 load: 135 /tmp/php.sock.2 2006-07-31 11:36:28: (mod_fastcgi.c.2878) backend is overloaded, we disable it for a 2 seconds and send the request to another backend instead: reconnects: 2 load: 135 /tmp/php.sock.3 2006-07-31 11:36:28: (mod_fastcgi.c.2878) backend is overloaded, we disable it for a 2 seconds and send the request to another backend instead: reconnects: 3 load: 135 /tmp/php.sock.4 2006-07-31 11:36:28: (mod_fastcgi.c.2878) backend is overloaded, we disable it for a 2 seconds and send the request to another backend instead: reconnects: 4 load: 135 /tmp/php.sock.5 2006-07-31 11:36:28: (mod_fastcgi.c.2878) backend is overloaded, we disable it for a 2 seconds and send the request to another backend instead: reconnects: 5 load: 135 /tmp/php.sock.6 2006-07-31 11:36:28: (mod_fastcgi.c.2878) backend is overloaded, we disable it for a 2 seconds and send the request to another backend instead: reconnects: 0 load: 135 /tmp/php.sock.7 2006-07-31 11:36:28: (mod_fastcgi.c.2878) backend is overloaded, we disable it for a 2 seconds and send the request to another backend instead: reconnects: 0 load: 135 /tmp/php.sock.8 2006-07-31 11:36:28: (mod_fastcgi.c.3540) no fcgi-handler found for: /API/sindex.php 2006-07-31 11:36:28: (mod_fastcgi.c.3540) no fcgi-handler found for: /API/sindex.php 2006-07-31 11:36:28: (mod_fastcgi.c.3540) no fcgi-handler found for: /API/sindex.php 2006-07-31 11:36:28: (mod_fastcgi.c.3540) no fcgi-handler found for: /API/sindex.php 2006-07-31 11:36:28: (mod_fastcgi.c.3540) no fcgi-handler found for: /API/sindex.php 2006-07-31 11:36:28: (network_linux_sendfile.c.93) writev failed: Bad file descriptor 2442 2006-07-31 11:36:28: (connections.c.536) connection closed: write failed on fd 2442 2006-07-31 11:36:28: (connections.c.132) (warning) close: 2442 Bad file descriptor
-- webmaster
Updated by Anonymous about 17 years ago
2006-07-31 00:01:53: (log.c.129) logfiles cycled
2006-07-31 11:36:28: (mod_fastcgi.c.2878) backend is overloaded, we disable it for a 2 seconds and send the request to another backend instead: reconnects: 0 load: 135 /tmp/php.sock.1
2006-07-31 11:36:28: (mod_fastcgi.c.2878) backend is overloaded, we disable it for a 2 seconds and send the request to another backend instead: reconnects: 1 load: 135 /tmp/php.sock.2
2006-07-31 11:36:28: (mod_fastcgi.c.2878) backend is overloaded, we disable it for a 2 seconds and send the request to another backend instead: reconnects: 2 load: 135 /tmp/php.sock.3
2006-07-31 11:36:28: (mod_fastcgi.c.2878) backend is overloaded, we disable it for a 2 seconds and send the request to another backend instead: reconnects: 3 load: 135 /tmp/php.sock.4
2006-07-31 11:36:28: (mod_fastcgi.c.2878) backend is overloaded, we disable it for a 2 seconds and send the request to another backend instead: reconnects: 4 load: 135 /tmp/php.sock.5
2006-07-31 11:36:28: (mod_fastcgi.c.2878) backend is overloaded, we disable it for a 2 seconds and send the request to another backend instead: reconnects: 5 load: 135 /tmp/php.sock.6
2006-07-31 11:36:28: (mod_fastcgi.c.2878) backend is overloaded, we disable it for a 2 seconds and send the request to another backend instead: reconnects: 0 load: 135 /tmp/php.sock.7
2006-07-31 11:36:28: (mod_fastcgi.c.2878) backend is overloaded, we disable it for a 2 seconds and send the request to another backend instead: reconnects: 0 load: 135 /tmp/php.sock.8
2006-07-31 11:36:28: (mod_fastcgi.c.3540) no fcgi-handler found for: /API/sindex.php
2006-07-31 11:36:28: (mod_fastcgi.c.3540) no fcgi-handler found for: /API/sindex.php
2006-07-31 11:36:28: (mod_fastcgi.c.3540) no fcgi-handler found for: /API/sindex.php
2006-07-31 11:36:28: (mod_fastcgi.c.3540) no fcgi-handler found for: /API/sindex.php
2006-07-31 11:36:28: (mod_fastcgi.c.3540) no fcgi-handler found for: /API/sindex.php
2006-07-31 11:36:28: (network_linux_sendfile.c.93) writev failed: Bad file descriptor 2442
2006-07-31 11:36:28: (connections.c.536) connection closed: write failed on fd 2442
2006-07-31 11:36:28: (connections.c.132) (warning) close: 2442 Bad file descriptor
Updated by Anonymous over 16 years ago
Possibly related to bug : http://trac.lighttpd.net/trac/ticket/897
And:
http://bugs.php.net/bug.php?id=41229
http://bugs.php.net/bug.php?id=43610
-- evert
Updated by stbuehler over 16 years ago
- Status changed from New to Fixed
- Resolution set to duplicate
Also available in: Atom