Project

General

Profile

[UE] lighttpd v1.4.67 - HTTP2 response not going out properly.

Added by gautham_t 11 months ago

The OS is RedHat 7.2.

We currently are trying to bring up lighttpd v1.4.67 for testing HTTP2 functionality.

We have been using lighttpd v1.4.55 for an embedded system. Until we get the HTTP2 functionality right,
we have added a suffix of 2 to lighttpd v1.4.67 (i.e lighttpd2, mod_openssl_2, mod_auth_2, etc) as both
versions will be copied to the system but one version will be started based on a config.

Tried two tests. first test over HTTP/2 and second test over HTTP/1.1
The request of HTTP/1.1 is received, processed and response sent back properly.

Over HTTP2, the request is received, processed but the response is not sent back properly.
The HTTP2 frame like HEADERS1: 200 OK, DATA1 (text/plain) is not present.
Rather we see the TCP PSH frame (frame 10 in httpv2-get.pcap)
Getting a GOAWAY from the client with frame_size_error.

In the plugin we write the payload to the write queue as follows:

buffer* b = buffer_init();
buffer_copy_string_len(b, (const char*)resp.getPayLoad(), resp.getPayLoadLen());
chunkqueue_append_buffer(con->write_queue, b);
buffer_free(b);

and also call the following:
joblist_append(con);

It is working for plain HTTP. Looks like writing to write_queue for HTTP2 is not going out properly.

Have attached the lighttpd.conf, pcaps, snippets of error.log and command, output of the curl requests.

Please can you suggest. What additional step do we need to do for HTTP2 response?

Regards,
Gautham

lighttpd.conf (1.23 KB) lighttpd.conf
httpv1-get.pcap (1.24 KB) httpv1-get.pcap pcap of HTTP v1.1 test
httpv2-get.pcap (2.11 KB) httpv2-get.pcap pcap of HTTP v2 test
errorLog-Httpv1-get.txt (16.5 KB) errorLog-Httpv1-get.txt snippet of error.log for HTTP v1.1 test
errorLog-Httpv2-get.txt (7.98 KB) errorLog-Httpv2-get.txt snippet of error.log for HTTP v2 test
curl-httpv1-get-cmd-output.txt (539 Bytes) curl-httpv1-get-cmd-output.txt command and output of HTTP v1.1 test
curl-httpv2-get-cmd-output.txt (706 Bytes) curl-httpv2-get-cmd-output.txt command and output of HTTP v2 test

Replies (7)

RE: lighttpd v1.4.67 - HTTP2 response not going out properly. - Added by avij 11 months ago

You may want to retest with lighttpd 1.4.69, which has these two lines in the changelog, among others:

[core] fix HTTP/2 HEADERS frame parsing bug
[core] remove extra HTTP/2 HEADERS frame len check

RE: lighttpd v1.4.67 - HTTP2 response not going out properly. - Added by gstrauss 11 months ago

In the plugin we write the payload to the write queue as follows:

You are using a custom plugin (mod_mxa?). Why did your post not start with that information?

Why are you posting in the Support forum instead of in the Development forum?

lighttpd 1.4.55 was released Jan 2020, over three years ago.
Since then, HTTP/2 support was added, and a developer might reasonably hypothesize that maybe some changes were made to lighttpd internals to support HTTP/2.

You probably have some incorrect assumption, incompatibility, or bug in your code which happens not to work with lighttpd 1.4.56 or later with HTTP/2. That bug is probably easy to identify if you are competent at troubleshooting, and probably has something to do with using con->write_queue instead of r->write_queue.

What additional step do we need to do for HTTP2 response?

Have you looked at any other lighttpd modules to see what they do? And then looked to see what your code is or isn't doing?

Based on the small snippet of code you have shared, whoever wrote that code was not a very good programmer. There was never a good reason to allocate a buffer, copy the payload, and then copy the buffer into the chunkqueue. chunkqueue_append_mem() has been in lighttpd since the first release of lighttpd 1.4.0 back in 2005.

In any case, you should be using http_chunk_append_mem() instead of chunkqueue_append_buffer() or chunkqueue_append_mem(). http_chunk_append_mem() has also been in lighttpd since at least the first release of lighttpd 1.4.0.

RE: [UE] lighttpd v1.4.67 - HTTP2 response not going out properly. - Added by gstrauss 11 months ago

For future maintenance, your company might consider migrating mod_mxa -- whatever it is -- to a FastCGI or SCGI script written in a language that your company can support. lighttpd is very fast even with the extra step of communicating over a unix domain socket to a FastCGI or SCGI or other backend.

RE: [UE] lighttpd v1.4.67 - HTTP2 response not going out properly. - Added by gautham_t 11 months ago

Appreciate your help. After changing to r->write_queue, the HTTP2 response was sent out properly.

I am facing another issue. I will open a new thread in Development forum.

RE: [UE] lighttpd v1.4.67 - HTTP2 response not going out properly. - Added by gstrauss 11 months ago

Appreciate your help. After changing to r->write_queue, the HTTP2 response was sent out properly.

You failed to read and understand my response that the person doing the development then (and now) did it wrong.
You would not have had a problem if you used http_chunk_append_mem().

I am facing another issue. I will open a new thread in Development forum.

Given your inability to take the time to understand my post above, I might choose not to respond further in the forums.

Your company can send an RFP if you'd like to pay for professional support for your project needs on modifying your custom plugin.

It is possible that your custom plugin provides no advantages to your company other than maintenance headaches, and maybe could have been written as a script.

RE: [UE] lighttpd v1.4.67 - HTTP2 response not going out properly. - Added by gautham_t 11 months ago

Please can you share how to reach out to you for professional support.
Can we reach you over email? The company representative can check with you.

Regards,
Gautham

RE: [UE] lighttpd v1.4.67 - HTTP2 response not going out properly. - Added by gstrauss 11 months ago

If you clone the git repository at https://git.lighttpd.net/lighttpd/lighttpd1.4, my email address is in the commit messages: (gstrauss) gluelogic.com

BTW, the error log shared in https://redmine.lighttpd.net/boards/3/topics/10998 suggests to me that there are further errors in your company's custom module.

    (1-7/7)