Project

General

Profile

[UE] lighttpd.service: Killing process 29502 (php-cgi) with signal SIGKILL. | lighttpd.service: Failed with result 'timeout'.

Added by Anonymous about 2 years ago

Hi all, I hope you are all well and can help me. I have been using Lighttpd on my pi for a while now. I'm on to a pi 4. Here is my problem, in last few days I've noticed my service drops. Cloudflare reports errors and uptime robots reports my site is down. It would go down for 1-2 minutes maybe every 3-4 hours. When I look at the log for when the site is down I see this:

Apr 11 14:37:11 raspberrypi systemd[1]: lighttpd.service: State 'stop-final-sigterm' timed out. Killing.
Apr 11 14:37:11 raspberrypi systemd[1]: lighttpd.service: Killing process 29499 (php-cgi) with signal SIGKILL.
Apr 11 14:37:11 raspberrypi systemd[1]: lighttpd.service: Killing process 29500 (php-cgi) with signal SIGKILL.
Apr 11 14:37:11 raspberrypi systemd[1]: lighttpd.service: Killing process 29501 (php-cgi) with signal SIGKILL.
Apr 11 14:37:11 raspberrypi systemd[1]: lighttpd.service: Killing process 29502 (php-cgi) with signal SIGKILL.
Apr 11 14:37:11 raspberrypi systemd[1]: lighttpd.service: Failed with result 'timeout'.
Apr 11 14:37:11 raspberrypi systemd[1]: Stopped Lighttpd Daemon.
Apr 11 14:37:11 raspberrypi systemd[1]: Starting Lighttpd Daemon...
Apr 11 14:37:11 raspberrypi lighttpd[5379]: WARNING: include-conf-enabled.pl is deprecated and slated for removal.
Apr 11 14:37:11 raspberrypi lighttpd[5379]:          Replace in lighttpd.conf with:
Apr 11 14:37:11 raspberrypi lighttpd[5379]:            include "/etc/lighttpd/conf-enabled/*.conf" 
Apr 11 14:37:11 raspberrypi systemd[1]: Started Lighttpd Daemon.
Apr 11 14:37:12 raspberrypi lighttpd[5386]: WARNING: include-conf-enabled.pl is deprecated and slated for removal.
Apr 11 14:37:12 raspberrypi lighttpd[5386]:          Replace in lighttpd.conf with:
Apr 11 14:37:12 raspberrypi lighttpd[5386]:            include "/etc/lighttpd/conf-enabled/*.conf" 

How can I try to figure out what is causing the problem? Many thanks.


Replies (8)

RE: lighttpd.service: Killing process 29502 (php-cgi) with signal SIGKILL. | lighttpd.service: Failed with result 'timeout'. - Added by gstrauss about 2 years ago

journalctl --unit=lighttpd

Something on your pi is running a systemctl command, likely every 3-4 hours, potentially to restart lighttpd, and that is timing out for some reason.

Apr 11 14:37:11 raspberrypi systemd[1]: lighttpd.service: State 'stop-final-sigterm' timed out. Killing.
Apr 11 14:37:11 raspberrypi systemd[1]: lighttpd.service: Killing process 29499 (php-cgi) with signal SIGKILL.
Apr 11 14:37:11 raspberrypi systemd[1]: lighttpd.service: Killing process 29500 (php-cgi) with signal SIGKILL.
Apr 11 14:37:11 raspberrypi systemd[1]: lighttpd.service: Killing process 29501 (php-cgi) with signal SIGKILL.
Apr 11 14:37:11 raspberrypi systemd[1]: lighttpd.service: Killing process 29502 (php-cgi) with signal SIGKILL.
Apr 11 14:37:11 raspberrypi systemd[1]: lighttpd.service: Failed with result 'timeout'.

If lighttpd is being sent a graceful shutdown signal, then lighttpd 1.4.56 and later support server.feature-flags += ("server.graceful-shutdown-timeout" => 4) after which lighttpd will kill existing connections that lighttpd was waiting on to gracefully complete.

However, you're probably running an very old version of lighttpd -- latest lighttpd is lighttpd 1.4.64. Unfortunately, too many distros, including pi's, Ubuntu, and Debian, think that "maintenance" is never upgrading unless you upgrade your entire system to the latest pi/Ubuntu/Debian release.

RE: lighttpd.service: Killing process 29502 (php-cgi) with signal SIGKILL. | lighttpd.service: Failed with result 'timeout'. - Added by Anonymous about 2 years ago

Thank you for your reply. I notice I have a cloudflare script that updates my DNS records that has a restart lighttpd command in it if there is no 200 response. I am going to remove that to see if that is the cause. I am not sure why it would suddenly be a cause of the problem but I'm going to see what happens. Many thanks.

May I ask, what is the best way to update it?

RE: lighttpd.service: Killing process 29502 (php-cgi) with signal SIGKILL. | lighttpd.service: Failed with result 'timeout'. - Added by Anonymous about 2 years ago

PS: I am not sure if you needed this but I ran journalctl --unit=lighttpd

pi@raspberrypi:~ $ journalctl --unit=lighttpd
-- Logs begin at Mon 2022-04-11 15:38:52 IST, end at Mon 2022-04-11 17:10:36 IST. --
Apr 11 15:39:02 raspberrypi systemd1: Starting Lighttpd Daemon...
Apr 11 15:39:02 raspberrypi systemd1: Started Lighttpd Daemon.
Apr 11 16:17:41 raspberrypi systemd1: Stopping Lighttpd Daemon...
Apr 11 16:19:12 raspberrypi systemd1: lighttpd.service: State 'stop-final-sigterm' timed out. Killing.
Apr 11 16:19:12 raspberrypi systemd1: lighttpd.service: Killing process 689 (php-cgi) with signal SIGKILL.
Apr 11 16:19:12 raspberrypi systemd1: lighttpd.service: Killing process 690 (php-cgi) with signal SIGKILL.
Apr 11 16:19:12 raspberrypi systemd1: lighttpd.service: Killing process 691 (php-cgi) with signal SIGKILL.
Apr 11 16:19:12 raspberrypi systemd1: lighttpd.service: Killing process 692 (php-cgi) with signal SIGKILL.
Apr 11 16:19:12 raspberrypi systemd1: lighttpd.service: Failed with result 'timeout'.
Apr 11 16:19:12 raspberrypi systemd1: Stopped Lighttpd Daemon.
Apr 11 16:19:12 raspberrypi systemd1: Starting Lighttpd Daemon...
Apr 11 16:19:12 raspberrypi systemd1: Started Lighttpd Daemon.
Apr 11 16:57:41 raspberrypi systemd1: Stopping Lighttpd Daemon...
Apr 11 16:59:11 raspberrypi systemd1: lighttpd.service: State 'stop-final-sigterm' timed out. Killing.
Apr 11 16:59:11 raspberrypi systemd1: lighttpd.service: Killing process 11642 (php-cgi) with signal SIGKILL.
Apr 11 16:59:11 raspberrypi systemd1: lighttpd.service: Killing process 11643 (php-cgi) with signal SIGKILL.
Apr 11 16:59:11 raspberrypi systemd1: lighttpd.service: Killing process 11644 (php-cgi) with signal SIGKILL.
Apr 11 16:59:11 raspberrypi systemd1: lighttpd.service: Killing process 11645 (php-cgi) with signal SIGKILL.
Apr 11 16:59:11 raspberrypi systemd1: lighttpd.service: Failed with result 'timeout'.
Apr 11 16:59:11 raspberrypi systemd1: Stopped Lighttpd Daemon.
Apr 11 16:59:11 raspberrypi systemd1: Starting Lighttpd Daemon...
Apr 11 16:59:11 raspberrypi systemd1: Started Lighttpd Daemon.

RE: lighttpd.service: Killing process 29502 (php-cgi) with signal SIGKILL. | lighttpd.service: Failed with result 'timeout'. - Added by gstrauss about 2 years ago

Thank you for your reply. I notice I have a cloudflare script that updates my DNS records that has a restart lighttpd command in it if there is no 200 response. I am going to remove that to see if that is the cause. I am not sure why it would suddenly be a cause of the problem but I'm going to see what happens. Many thanks.

Please look in your lighttpd error log to see if there are any hints why your lighttpd instance is not restarting. See if lighttpd reports that it received a signal (from systemctl) and if lighttpd is shutting down. If there is nothing there, then you might look into why systemctl is not properly signalling the lighttpd process. Is systemctl service misdirected to the wrong lihttpd process pid? I do not know, but am making suggestions for you to look into.

May I ask, what is the best way to update it?

You can politely request that the packager of the lighttpd package on Raspberry Pi update the package. That is unlikely to bear fruit, but is the proper channel.

If you take a more technical route, you can build and (optionally) package lighttpd yourself. See InstallFromSource For your current issue, this does not sound like an issue in lighttpd, but rather a configuration issue somewhere, so upgrading lighttpd is unlikely to make a difference.

RE: lighttpd.service: Killing process 29502 (php-cgi) with signal SIGKILL. | lighttpd.service: Failed with result 'timeout'. - Added by Anonymous about 2 years ago

Thank you so much :), I'm not great with reading logs, what do you think this means?

2022-04-11 14:33:59: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: Undefined array key "policy-page" in /var/www1/wordpress/wp-content/plugins/gdpr/templates/priva$
2022-04-11 14:33:59: (mod_fastcgi.c.421) FastCGI-stderr: PHP Warning: Undefined array key "policy-page" in /var/www1/wordpress/wp-content/plugins/gdpr/templates/priva$
2022-04-11 14:35:40: (server.c.2059) server stopped by UID = 0 PID = 1
2022-04-11 14:37:11: (server.c.1464) server started (lighttpd/1.4.53)

2022-04-11 14:40:00: (mod_fastcgi.c.421) FastCGI-stderr: PHP Deprecated: Optional parameter $thumbnail declared before required parameter $excerpt is implicitly treat$
2022-04-11 14:40:00: (mod_fastcgi.c.421) FastCGI-stderr: PHP Deprecated: Optional parameter $column declared before required parameter $excerpt is implicitly treated $
2

RE: lighttpd.service: Killing process 29502 (php-cgi) with signal SIGKILL. | lighttpd.service: Failed with result 'timeout'. - Added by gstrauss about 2 years ago

I'm not great with reading logs, what do you think this means?

2022-04-11 14:35:40: (server.c.2059) server stopped by UID = 0 PID = 1
2022-04-11 14:37:11: (server.c.1464) server started (lighttpd/1.4.53)

It means that lighttpd received a signal, likely SIGTERM, from pid 1 (systemd on your system) and stopped. Almost two minutes later, lighttpd was started.

FYI: I don't have much patience for people that give up without demonstrating any effort. Good luck!

RE: lighttpd.service: Killing process 29502 (php-cgi) with signal SIGKILL. | lighttpd.service: Failed with result 'timeout'. - Added by Anonymous about 2 years ago

Thank you very much for your help, I figured it means something stopped it and started it but I'm not sure what. Also, I'm afraid my ignorance has caused offense.

RE: lighttpd.service: Killing process 29502 (php-cgi) with signal SIGKILL. | lighttpd.service: Failed with result 'timeout'. - Added by gstrauss about 2 years ago

I figured it means something stopped it and started it but I'm not sure what.

"server stopped" in the lighttpd error log literally means "server stopped"
"server started" in the lighttpd error log literally means "server started"

Also, I'm afraid my ignorance has caused offense.

No offense taken, though no further responses are likely to be forthcoming. Good luck!

    (1-8/8)