Lighttpd

If you run lighttpd in a chroot, then SIGUSR1 may cause lighttpd to exit.

If you run lighttpd daemonized but with command line arguments which are not full paths, then lighttpd daemonizes and changes directory to /, and then lighttpd may exit after SIGUSR1 if those relative paths are invalid (after chdir /).

Check the way that you configure and daemonize lighttpd. lighttpd -f /etc/lighttpd/lighttpd.conf -p

Regarding SIGHUP, the original authors of lighttpd used SIGHUP as a signal to rotate logs and that remains an historical inconsistency with common conventions.

Thanks a lot gstrauss! I found the problem with your help.

As it turns out, there were two issues with my configuration:
1. I was using relative path in some places; and
2. the the private key for the TLS certificate was not readable by user `www`.

So it appears that lighttpd reads the TLS certificate with root privilege at start, drops the privilege to `www` afterwards (if configured to do so I guess), and by the time it encounters a SIGUSR1 signal, it can only read the private key as user `www`, and exits when that fails.

As a follow up question, is there a way to make lighttpd log such failures in the error log?

As a follow up question, is there a way to make lighttpd log such failures in the error log?

lighttpd does issue trace. Check your error log. If lighttpd running as user 'www' and/or in a chroot does not have permission to reopen and write to the error log, you might have configured lighttpd in such a way that lighttpd fails when trying to write the error trace. As an alternative, you can configure lighttpd to send error trace to syslog and can manage the error log that way.

In short, if you configure lighttpd to chroot and/or to drop privileges, then you need to keep in mind that lighttpd can no longer gracefully restart outside the chroot, or with privileges that have already been dropped! In those cases, you should gracefully stop and restart lighttpd.

I did a little bit of testing by deliberately removing the read permission for the TLS certificates, but I can't seem to get lighttpd to log the error during signal handling.

Outside of signal handling, errors seem to get logged to the error log just fine. When sent a SIGUSR1 signal, I was expecting something along the line of:

...fdevent.c) fdevent_load_file() path/to/privkey.pem: Permission denied

but the only lines that seem to appear in the error log are:

...server.c)[note] graceful shutdown started
...server.c)Server stopped by UID = XXXXX PID = YYYYY

Here, lighttpd is invoked as a regular user without chroot, and the error log is owned by the user with write permission.

Could you provide any insights into what the cause may be that lighttpd is not logging the error?

Thanks again!

Could you provide any insights into what the cause may be that lighttpd is not logging the error?

I do not like repeating myself. You ignored what I previously said about the error log.

Running lighttpd -D ... with no error log (so trace goes to console stderr), starting lighttpd, removing permission from key.pm, and then sending SIGUSR1 to lighttpd:
2025-02-14 18:27:05: (fdevent.c.946) fdevent_load_file() /path/to/key.pem: Permission denied

I read your reply carefully, so I really didn't ignore what you said. In fact, specifically to rule out the problems that you pointed out for the test, lighttpd is invoked by a regular user, without chroot or privilege dropping, etc.

Running lighttpd -D ... with no error log (so trace goes to console stderr), starting lighttpd, removing permission from key.pm, and then sending SIGUSR1 to lighttpd:

2025-02-14 18:27:05: (fdevent.c.946) fdevent_load_file() /path/to/key.pem: Permission denied

The problem is that this line seems to ignore server.errorlog configuration and always go to stderr, where as the server.c.XXX lines go to server.errorlog? I'm confused why it doesn't go to server.errorlog...

I only have the following lines in lighttpd.conf:

server.document-root = "/home/tim/www/" 
server.errorlog = "/home/tim/error.log" 
server.port = 8080
server.modules = ("mod_openssl")
ssl.engine = "enable" 
ssl.pemfile = "/home/tim/cert.pem" 
ssl.privkey = "/home/tim/privkey.pem" 

Look at the code. The code performs steps in a deterministic order. That order is in the code. Your assumptions are incorrect.

lighttpd TLS modules are initialized before privileges are dropped, in order to be able to read privileged files, if so configured.

The error log has not yet been opened at the time that TLS modules are initialized.

I can see your post history on this site. It is not worth wasting my breath continuing this "conversation".

I can see your post history on this site. It is not worth wasting my breath continuing this "conversation".

Putting aside your demeaning attitude here, I'm not trying to strike up a "conversation". I have a specific question regarding lighttpd usage, which is how to get fdevent_load_file() /path/to/key.pem: Permission denied and alike that occurs upon a SIGUSR1 signal written to the error log. To which you answered:

Check your error log. If lighttpd running as user 'www' and/or in a chroot does not have permission to reopen and write to the error log...

which I interpret as --- if lighttpd is not running in a chroot and does have permission to reopen and write to the error log, then the above mentioned error message should appear in the error log. Based on this interpretation, I set up a test to see if I can make such error message appear in the error log, but I wasn't able to, therefore I asked for your further input.

Then you said in your own test, your were able to get the error message written to stderr, to which I basically responded that I too was able to get that into stderr, but have not been able to get it into the error log.

So the question remains how to get fdevent_load_file() /path/to/key.pem: Permission denied into the error log.

After that, I'm losing you completely.

That order is in the code. Your assumptions are incorrect.

Which order and which assumption I made are you referring to?

lighttpd TLS modules are initialized before privileges are dropped, in order to be able to read privileged files, if so configured.

That's exactly one of the assumptions I made, isn't it it? I hypothesized:

So it appears that lighttpd reads the TLS certificate with root privilege at start, drops the privilege to `www` afterwards...

So which assumption I made is incorrect? I'm not at all trying to be rhetorical, I'm genuinely confused.

The error log has not yet been opened at the time that TLS modules are initialized.

Does this mean that fdevent_load_file() /path/to/key.pem: Permission denied can not be written to the error log, since it "has not yet been opened"?

For those who may be reading in the future, I figured it out.

Any error (such as TLS certificate loading errors) that happens between startup (could be a fresh start or a SIGUSR1 triggered graceful restart) and the time when error log is (re)opened (the call to config_log_error_open()) is written to file descriptor STDERR_FILENO. STDERR_FILENO is hooked to server.breakagelog (instead of server.errorlog) after the first call to config_log_error_open() during a fresh start, therefore these type of errors are written to stderr during a fresh start, and to the breakage log (as opposed to the error log) during SIGUSR1 triggered graceful restarts.

OP did not read and follow How to get support

OP's lighttpd.conf was not shared and OP never mentioned (in post above) using server.breakagelog, which causes lighttpd to redirect stderr in the lighttpd process, and which is inherited by child processes such as CGI.

OP never mentioned (in post above) using server.breakagelog

I did not mention using server.breakagelog because I was not using it until right before my very last post, that's why I couldn't find traces of the above mentioned error anywhere in the log.

As far as I understand, the current behavior is that if server.breakagelog is not configured, any error message that happens between a SIGUSR1 triggered graceful restart and the call to config_log_error_open() in server.c is silently dropped without being logged anywhere (even if server.errlog is also configured).

Considering that when SIGUSR1 triggered graceful restart occurs, there is no fundamental difference between errors that happen before and after config_log_error_open(), my personal view is that, for whatever it is worth, they should both go to server.errorlog, which is different from the current behavior.

@gstrauss
If what I said or did not say during the course of this thread caused offense to you by any chance, I would like you to know that it was unintentional, and I apologize for that. Thanks for maintaining the software, and for helping me with my questions.

As I stated above, lighttpd sends trace to stderr before the error log is opened.
The trace is not silently dropped. YOU and YOUR SYSTEM might silently drop the error trace if you are not otherwise capturing that trace.

I hope that you are not so naive that you are unable to understand that a professional configuration collects error trace across multiple machines and might not use a local on-disk error log at all. It's clear that you haven't considered that since you have a poorly informed personal view that you think others want to hear. We do not.

... a professional configuration collects error trace across multiple machines and might not use a local on-disk error log at all ...

This is completely orthogonal to what I was trying to convey. What I was saying is that, excluding the error messages from the child processes such as CGI*, there are three factors ( independent from lighttpd.conf) that influence how error messages are currently handled by lighttpd:
1. whether lighttpd is invoked to daemonize (0) or not daemonize (1); and
2. whether the error occurs before (0) or after (1) the call to config_log_error_open() in the server main loop; and
3. whether its a fresh start (0) or a graceful restart (1).

Representing each of the 2^3=8 scenarios as a triple of the three factors, how lighttpd currently handles the error messages falls into the following five cases:
A. (*, 0, 0): goes to stderr;
B. (1, 1, *): goes to server.errorlog if configured, otherwise go to stderr;
C. (0, 1, *): goes to server.errorlog if configured, othersise dropped;
D. (0, 0, 1): goes to server.breakagelog if configured, otherwise dropped;
E. (1, 0, 1): goes to server.breakagelog if configured, otherwise goes to stderr;

What I attempted to convey is that, in my humble but nevertheless unwelcome opinion, a more desirable behavior is that:
- case D collapses into case C; and
- case E modified to: goes to server.breakagelog server.errorlog if configured, otherwise goes to stderr;
This way, the purpose of breakage log becomes cleaner, as it's now exclusively for error messages from the child processes.

As you see, this does not in any way affect a professional deployment's ability to send error trace over a network, so I do not see the relevance of that.

I understand that my opinions are unwelcome, but since it has already been expressed this time, I do want to try my best to make sure that it is not misunderstood.

It appears you may have taken a junior programming class, but it does not seem to me that you have completed any project which involved bootstrapping.

I will code review and consider any patches submitted, but note that my code reviews are usually thorough and involve whole-system design, something your opinions have yet to cover, as you continue to be focused on your own narrow use case. Your attempt at a matrix is misleading and incomplete, and your major character flaw is posting in such as way as to suggest that it is complete. That is haughty, not humble. You have been told numerous times that lighttpd does not discard error trace -- YOUR system might -- so your matrix including the phrase "otherwise dropped" is false. Your continued posting have not helped you credibility, and I detest people posting crap and suggesting "prove me wrong". I have thus far continued to respond to your "prove me wrong" disinformation to highlight your incompetence. Please stop posting here. You are not being "humble" by continuing to post, and you look like an idiot to most people reading that delusional self-projection about your "humble opinion", which is anything but that. This topic started with some novice mistakes like using relative paths along with daemonizing, and not realizing the implications of dropping privileges and file access to privileged files. When that was pointed out, you fixed it. Good. However, please note that set the bar for your level of expertise and the subsequent value of your technical opinions on the specific subject at hand.

I understand that you, in your own esteemed opinion, would like error trace to be handled differently. Stop suggesting that your opinion is the right way. Prove it with suggested patches, if you can. You might start by reading the documentation for server.breakagelog, which is not enabled by default. Your post suggests that you want to change the documented behavior of server.breakagelog. I suggest instead that you learn how to capture stderr from your daemon processes.

In absence of server.errorlog, lighttpd error trace goes to lighttpd stderr. If you want full control over the error stream, stop here and handle it.

If you configure an error log, then lighttpd handles that during lighttpd configuration processing, server restarts, and signalling for log rotation.
There is complexity and order of operations. Error trace goes to process stderr before the error log is opened. Deal with it.

If server.breakagelog is configured, lighttpd redirects lighttpd stderr to that log, and that remains so for restarted processes; the original stderr has been replaced. This is documented very directly and succinctly in server.breakagelog.

lighttpd provides a variety of options for the error stream, including error log, breakage log, syslog, and piped loggers. Not all features of lighttpd error log processing and graceful restart are available for every single usage mode of lighttpd, e.g. chroot and dropping privileges complicate matters.