Project

General

Profile

[Solved] FreeBSD, php-fpm sbwait

Added by 2ge over 3 years ago

Hi all,

I got similar problem as described here:
https://redmine.lighttpd.net/boards/2/topics/141?r=639

some debug info:

# uname -a
FreeBSD xxx.xxx 10.1-STABLE FreeBSD 10.1-STABLE #0 r274924M: Wed Oct 26 13:48:56 CEST 2016     xxx@xxx:/usr/obj/usr/src/sys/SUPERMICRO  amd64
(we are running 11-STABLE, same behavior)

# ps alx | grep php-fpm | grep -v grep | grep sbwait
   80 19050 85706   0  28  0  374684   72936 sbwait IJ    -    0:00.56 php-fpm: pool www (php-fpm)
   80 44186 85706   0  46  0  370588   73252 sbwait IJ    -    0:05.48 php-fpm: pool www (php-fpm)
   80 44223 85706   0  52  0  374684   77228 sbwait IJ    -    0:02.21 php-fpm: pool www (php-fpm)
   80 44281 85706   0  52  0  362396   64776 sbwait IJ    -    0:00.74 php-fpm: pool www (php-fpm)
   80 45183 85706   0  20  0  354204   55572 sbwait IJ    -    0:04.23 php-fpm: pool www (php-fpm)
   80 48745 85706   0  45  0  370588   72488 sbwait IJ    -    0:06.81 php-fpm: pool www (php-fpm)
   80 52109 85706   0  32  0  374684   73924 sbwait IJ    -    0:00.88 php-fpm: pool www (php-fpm)

# php-fpm -v
PHP 5.6.27 (fpm-fcgi) (built: Oct 26 2016 18:52:30)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies
# lighttpd -v
lighttpd/1.4.42 (ssl) - a light and fast webserver
Build-Date: Oct 21 2016 14:22:41

in lighttpd.conf:

server.max-fds = 32768
server.max-keep-alive-requests = 4
server.max-keep-alive-idle = 2
server.max-read-idle = 60
server.max-write-idle = 360
server.max-connections = 10240
server.event-handler = "freebsd-kqueue" 
server.network-backend = "writev" 

/server-status:
https://www.dropbox.com/s/sp6qy5e75h0fbch/Screenshot%202016-10-31%2002.47.26.png?dl=0

# gdb /usr/local/sbin/php-fpm 19050
GNU gdb 6.1.1 [FreeBSD]
...
0x0000000801df43f8 in _write () from /lib/libc.so.7
(gdb) bt
#0  0x0000000801df43f8 in _write () from /lib/libc.so.7
#1  0x00000008012bf136 in pthread_suspend_all_np () from /lib/libthr.so.3
#2  0x00000000006152ff in fcgi_flush ()
#3  0x0000000000615636 in fcgi_write ()
#4  0x000000000061e4a0 in sapi_cgibin_ub_write ()
#5  0x0000000000547b2c in php_output_op ()
#6  0x0000000000548422 in php_output_stack_pop ()
#7  0x000000000054852a in php_output_end_all ()
#8  0x0000000000534e37 in php_request_shutdown ()
#9  0x000000000061dcbc in main ()
(gdb) f 0
#0  0x0000000801df43f8 in _write () from /lib/libc.so.7
(gdb) info frame
Stack level 0, frame at 0x7fffffffc5b0:
 rip = 0x801df43f8 in _write; saved rip 0x8012bf136
 called by frame at 0x7fffffffc5f0
 Arglist at 0x7fffffffc5a0, args: 
 Locals at 0x7fffffffc5a0, Previous frame's sp is 0x7fffffffc5b0
 Saved registers:
  rip at 0x7fffffffc5a8

Basically I have to restart php-fpm every couple of hours. I set time out in php.ini, it doesn't help. Those processes just staying there and taking resources.

Thanks for any help


Replies (3)

RE: FreeBSD, php-fpm sbwait - Added by gstrauss over 3 years ago

Please try adding this to lighttpd.conf global scope: server.stream-response-body = 1

You might also compile and try lighttpd git master, which is scheduled to be released as 1.4.43 tomorrow (Monday)
See DevelGit doc and git clone https://git.lighttpd.net/lighttpd/lighttpd1.4.git

RE: FreeBSD, php-fpm sbwait - Added by gstrauss over 3 years ago

It appears that others have had similar problems with FreeBSD 10.3-STABLE https://forums.freebsd.org/threads/58241/

You might try FreeBSD 11.0, which was recently released.

The issue here is unlikely to be related to https://redmine.lighttpd.net/boards/2/topics/141?r=639 since lighttpd 1.4.40 and later detect sendfile() failure on unix domain sockets on FreeBSD, and will fall back to writev().

RE: [Solved] FreeBSD, php-fpm sbwait - Added by gstrauss almost 3 years ago

server.event-handler = "freebsd-kqueue"

The problem should go away if you use server.event-handler = "poll"

The issue is fixed in 12440e89 which will be part of lighttpd 1.4.46. Details in #2788, but basically, lighttpd had been using kqueue in edge-triggered mode, but expecting level-triggered behavior. The patch changes lighttpd usage of kqueue to be level-triggered.

    (1-3/3)