Bug #3166: GET requests hangs on big files after 10% using HTTP/2 - Lighttpd - lighty labs

Actions

Copy link

Bug #3166

closed

GET requests hangs on big files after 10% using HTTP/2

Added by flynn over 1 year ago. Updated over 1 year ago.

Status:

Fixed

Priority:

Normal

Category:

core

Target version:

1.4.66

ASK QUESTIONS IN Forums:

Description

I found this issue first using php with fastcgi over https, but it is reproducible without both:

minimal lighttpd configuration using version 1.4.65
files significant larger than 4GB (2GB is too small)
request the big files with HTTP/2. e.g. with curl curl --http2 http://1.2.3.4/bla --output bla
the download stalls at about 10% of the filesize totally, the connection remains open, curl hangs, nothing else happens
(I tried one file with 5,5GB which stalls at 564MB, another file with 23GB stalls at 2,2GB)
the same behaviour using firefox
request with HTTP/1.1 terminate without error and correct size (which is not an usable option using a browser)

Actions

Copy link

Updated by gstrauss over 1 year ago

This optimization will be part of the next lighttpd release (lighttpd 1.4.66). Does it help?
https://git.lighttpd.net/lighttpd/lighttpd1.4/commit/f86b448799fc7f55274a1d61583ff3d390f58e0b

Also, please check that you are running curl 7.84 or later. (See #3089)

Actions

Copy link

Updated by flynn over 1 year ago

The used curl version is 7.84.0.
The problem is also reproducable with firefox browser on Linux and Windows.
Both clients hang after 10% downloaded filesize.

Chromium has the same problem, so it is not an issue of a single client.

I applied the patch above to version 1.4.65, the problem remains: curl hangs at the exact same position.

Actions

Copy link

Updated by flynn over 1 year ago

I enabled debug.log-state-handling = "enable", it just stops writing log messages:

2022-08-04 18:08:12: (connections.c.958) fd:7 id:1 state:write
2022-08-04 18:08:12: (connections.c.958) fd:7 id:1 state:write
2022-08-04 18:08:12: (connections.c.958) fd:7 id:1 state:write
2022-08-04 18:08:12: (connections.c.958) fd:7 id:1 state:write
2022-08-04 18:08:12: (connections.c.958) fd:7 id:1 state:write
2022-08-04 18:08:12: (connections.c.958) fd:7 id:1 state:write
2022-08-04 18:08:12: (connections.c.958) fd:7 id:1 state:write

Actions

Copy link

Updated by gstrauss over 1 year ago

I see similar behavior, but have not tracked down the issue yet.

Actions

Copy link

Updated by gstrauss over 1 year ago

I have not had much time to dig into this, but will do so later today.

The issue appears at exactly 4 GiB in size. Things work with one byte fewer.
This suggests to me that there is a uint32_t calculation somewhere which should be off_t.

Actions

Copy link

Updated by gstrauss over 1 year ago

Status changed from New to Patch Pending

Thanks for the bug report. This looks to be a regression since lighttpd 1.4.60. Here is a patch.

--- a/src/h2.c
+++ b/src/h2.c
@@ -2636,8 +2636,8 @@ h2_send_cqdata (request_st * const r, connection * const con, chunkqueue * const
     if (h2r->h2_swin < 0) return 0;
     if ((int32_t)dlen > r->h2_swin)   dlen = (uint32_t)r->h2_swin;
     if ((int32_t)dlen > h2r->h2_swin) dlen = (uint32_t)h2r->h2_swin;
-    const uint32_t cqlen = (uint32_t)chunkqueue_length(cq);
-    if (dlen > cqlen) dlen = cqlen;
+    const off_t cqlen = chunkqueue_length(cq);
+    if ((int32_t)dlen > cqlen) dlen = (uint32_t)cqlen;
     /*(note: must temporarily disable next line when running h2spec since
      * some h2spec tests expect 1-byte DATA frame, not a deferred response)*/
     else if (dlen < 2048 && cqlen >= 2048) return 0;

Actions

Copy link