Lighttpd

Please read https://redmine.lighttpd.net/boards/2/topics/5

Your request for assistance is severely lacking in necessary details.

Do I miss any configuration?

Ubuntu and Debian are negligent in keeping stable releases patched to current lighttpd releases. lighttpd 1.4.45 was released over 3 years ago. The latest release is lighttpd 1.4.55 and there have been important bug fixes and security patches made in the past 3+ years

Can you reproduce the behavior you are seeing when using lighttpd 1.4.55?

Hi,
I am using ROCK Pi S Ubuntu

# uname -a
uname -a
Linux rockpis 4.4.143-34-rockchip-g3c9d2019dba7 #1 SMP PREEMPT Tue Nov 5 01:39:21 UTC 2019 aarch64 aarch64 aarch64 GNU/Linux

# /etc/init.d/lighttpd -v
/etc/init.d/lighttpd -v
lighttpd/1.4.54 - a light and fast webserver

The following screen shows the cgi program (/var/www/cgi-bin/execgi-bin.cgi) is not terminated.

root       364  0.0  0.2   1912  1184 pts/0    Ss   16:12   0:00 /bin/sh -
root       411  0.0  0.0      0     0 ?        S    16:26   0:00 [kworker/3:2]
root       428  0.0  0.0      0     0 ?        S    16:26   0:00 [kworker/1:2]
root       473  0.0  0.0      0     0 ?        S    16:33   0:00 [kworker/1:1]
root       474  0.0  0.0      0     0 ?        S    16:33   0:00 [kworker/u8:1]
root       477  0.0  0.0      0     0 ?        S    16:33   0:00 [kworker/2:0]
www-data   479  0.0  0.6   4192  2900 pts/0    S    16:33   0:00 /etc/init.d/lighttpd -D -f /etc/lighttpd/lighttpd.conf -m /usr/
root       485 82.0  0.8 391500  3584 pts/0    Sl+  16:33   5:47 ./exellrp-new
www-data   498  0.0  0.2   4368  1116 pts/0    S    16:35   0:00 /var/www/cgi-bin/execgi-bin.cgi
root       501  0.2  1.3  11860  6044 ?        Ss   16:40   0:00 sshd: joohong [priv]
joohong    503  0.2  1.4  12232  6428 ?        Ss   16:40   0:00 /lib/systemd/systemd --user
joohong    504  0.0  0.7 162212  3352 ?        S    16:40   0:00 (sd-pam)
joohong    526  0.1  0.8  11860  3848 ?        R    16:40   0:00 sshd: joohong@pts/1
joohong    527  0.0  0.6   3776  3000 pts/1    Ss   16:40   0:00 -bash
joohong    533  0.0  0.5   5288  2388 pts/1    R+   16:41   0:00 ps -aux
joohong@rockpis:~$

I also attached the lighttpd.conf

Regards,

Joo

Thank you for the additional info.

lighttpd closes the pipe from the CGI program output and sends SIGTERM to the CGI program when lighttpd detects that the client has disconnected. Please strace the lighttpd daemon and/or your CGI program to see if the signal is being sent to the CGI. Check if your CGI is blocking or discarding the signal.

If your CGI program is writing response data, the writes will fail with EPIPE after lighttpd closes the pipe. If your CGI program is not in the process of writing response data, then lighttpd read/write timeouts apply, e.g. server.max-write-idle Please see the documentation

(BTW, my mention of old Ubuntu systems stemmed from your January post of running lighttpd 1.4.45 https://redmine.lighttpd.net/boards/2/topics/8900)

It seems like I do not have "strace" command in my embedded linux environment.

I put some debug code (log) in my CGI program to check if it does receive the SIGTERM from the server:

volatile sig_atomic_t done = 0;
static FILE *fp;

void term(int signum)
{
        fp = fopen("/home/rock/upload/llrp.log", "a");
    if (fp == NULL)
    {
        exit(-1);
    }
    fprintf(fp, "receive signal SIG NUM = %d\n", signum);
    fclose(fp);
    //fprintf(fp, "receive signal SIGTERM\n");
    IReaderApiClose(CCLI::handle);
        done = 1;
}

int main(void) 
{
    IReader *handle;
    int ret;
    int region;
    const char *cgi_env = getenv("QUERY_STRING");

    char cgi_buffer[128];

    sprintf(cgi_buffer, "%s", cgi_env);

    for (unsigned int i = 0; i < strlen(cgi_buffer); i++)
    {
        if (cgi_buffer[i] == '&')
        {
            cgi_buffer[i] = ' ';
        }
    }
    std::string cgi_string(cgi_buffer);

    fp = fopen("/home/rock/upload/llrp.log", "a");
    if (fp == NULL)
    {
        exit(-1);
    }
    fprintf(fp, "Open log...\n");
    fclose(fp);
        struct sigaction action;
        memset(&action, 0, sizeof(struct sigaction));
        action.sa_handler = term;
        sigaction(SIGTERM, &action, NULL);
        sigaction(SIGHUP, &action, NULL);
        sigaction(SIGUSR1, &action, NULL);
        sigaction(SIGKILL, &action, NULL);
        sigaction(SIGINT, &action, NULL);

    CCLI::handle = IReaderApiInit();

    if (NULL == CCLI::handle)
    {
        exit(-1);
    }
    //printf("connecting...\n");
    ret = IReaderApiConnect(CCLI::handle, (char *)"127.0.0.1");
    if (IREADER_SUCCESS != ret)
    {
        // printf("Connect IReader Fails");
        IReaderApiClose(CCLI::handle);
        exit(-1);
    }

    CCLI::process_cli_command(cgi_string);

    if (done == 0)
    {
        IReaderApiClose(CCLI::handle);
    }

    return 0;
}

I open the log file (llrp.log) and see no signal is caught. Then from the shell I issue a kill command to kill the CGI PID and I check the llrp.log file again and it shows a signal 15 is caught. This indicates my CGI program does not block or discard the signal...

Currently, my CGI program does not write any data. When It does, as you have pointed out, the CGI program will be closed because of the broken PIPE.

Thank for your help.

Regards,

Joo

    const char *cgi_env = getenv("QUERY_STRING");

    char cgi_buffer[128];

    sprintf(cgi_buffer, "%s", cgi_env);

The above code is vulnerable to buffer overflow if the query string is longer than 127 characters. NEVER use sprintf(). ALWAYS distrust data from external sources.

You should copy into std::string and modify '&' to ' ' in the std::string
std::string cgi_string(cgi_env);

It appears that lighttpd is receiving POLLRDHUP event on client fd. When lighttpd goes to check if the client socket is half-closed or fully-closed, lighttpd finds SOL_TCP TCP_INFO tcpi.tcpi_state == TCP_CLOSE_WAIT, which lighttpd interprets as half-closed. I need to look into this further.

For all socket-based lighttpd backends, lighttpd propagates the TCP_FIN. All dynamic lighttpd backends besides mod_cgi use sockets. (mod_fastcgi, mod_proxy, mod_scgi, mod_sockproxy, mod_wstunnel)

However, for mod_cgi, lighttpd uses pipes, and the stdin pipe of the CGI script might be connected to a temporary file or /dev/null if there was no client request body. In both cases, the stdin pipe of the CGI is no longer connected to lighttpd. Even if it were, we would want to send any pending input to the pipe before signalling the CGI of the receipt of POLLRDHUP.

How might lighttpd safely signal the equivalent of a socket TCP_FIN to a process (executable) running the CGI? There is no one obviously right answer which is why lighttpd does not implement one. POLLRDHUP from client may or may not have any meaning to the specific CGI, and some people might want the CGI killed upon receipt of POLLRDHUP, and others might not.

I am not a huge fan of infinite configuration knobs, but am willing to be convinced that I should add a config option to lighttpd -- cgi.kill-on-rdhup or something like that -- to send the CGI a TERM signal when a POLLRDHUP is received from the client. (Hint: asking nicely will not be sufficient to convince me)

Another alternative is to write your C/C++ backend as an SCGI script and use mod_scgi instead of mod_cgi. The code to write an SCGI backend in C is fairly straightforward and I wrote a simple SCGI backend scgi_responder.c that is used for testing lighttpd. lighttpd will propagate the TCP_FIN to the SCGI backend.

[Edit] clarification: lighttpd will propagate TCP_FIN to socket-based backends only if enabled "tcp-fin-propagate" => "enable" in the backend host definition. Please refer to the generic options in the mod_fastcgi options fastcgi.server. These options apply to all lighttpd socket-based dynamic backends.

I am trying if I can do a quick patch with bo luck.

When the browser closes the session (TCP_FIN is sent), I do not see the function cgi_handle_fdevent is called (I added some debug log).
I am wondering where is the right place to look at?

Thanks

Joo

In gw_backend.c:gw_handle_subrequest() is the shared code for lighttpd dynamic socket backends. It contains

    if (r->conf.stream_request_body & FDEVENT_STREAM_REQUEST_TCP_FIN)
        gw_conditional_tcp_fin(hctx, r);

Near the end of mod_cgi.c:mod_cgi_handle_subrequest(), you would do something like the following (untested)

    if (r->conf.stream_request_body & FDEVENT_STREAM_REQUEST_TCP_FIN) {
        /* XXX: TODO: check if any cached request body has been sent to CGI; see gw_conditional_tcp_fin() (for sockets) */
        if (hctx->pid > 0) {
                cgi_pid_kill(p, hctx->pid);
        }
        /* above will send kill but otherwise continue.  To send kill and stop reading from CGI
         *   cgi_connection_close(hctx);
         * which includes the cgi_pid_kill() */
    }

The behavior in lighttpd dynamic socket backends sends any data received from client before propagating TCP_FIN. The sample (untested) code above, tries to do similar for CGI. If you want to give up immediately, use cgi_connection_close(hctx);

Again, if you're going to spend time on this, I would recommend using mod_scgi and settings "tcp-fin-propagate" => "enable" for the scgi.server host, as the functionality you seek is already available there.

Thank you so much for the help. I will use mod_scgi at later time.
I ran some test with the patch listed above, the flow does not seem to enter mod_cgi_handle_subrequest when TCP_FIN is received. Are there any server settings that need to be configured?

Thanks again.

Joo

the flow does not seem to enter mod_cgi_handle_subrequest when TCP_FIN is received

You are incorrect.

I have pointed you at the correct place in the code.

The (untested) code snippet I posted is triggered on TCP_FIN, but what it does when it receives the TCP_FIN needs additional work. As you haven't made any attempt to convince me why mod_cgi should support this feature in a flexible way, I am done here.

I will use mod_scgi at later time.

That time has arrived. Good luck.

"As you haven't made any attempt to convince me why mod_cgi should support this feature in a flexible way, I am done here."

Our code is currently based on the CGI model. It works stable and is tested except this little issue. Intuitively, this feature should be there by default or in a flexible way. I believe other designs that use the CGI model will also love to have this feature supported. (it's kind of weird to have the backend session still open while the front end session is gone).

Anyway, I ran more testing again using the patch. It seems like it works the first time. After the first time, I create the session again and then kill, that function is not called.

Thanks,

Joo