Project

General

Profile

Actions

Bug #3166

closed

GET requests hangs on big files after 10% using HTTP/2

Added by flynn 4 months ago. Updated 4 months ago.

Status:
Fixed
Priority:
Normal
Category:
core
Target version:
ASK QUESTIONS IN Forums:
No

Description

I found this issue first using php with fastcgi over https, but it is reproducible without both:
  • minimal lighttpd configuration using version 1.4.65
  • files significant larger than 4GB (2GB is too small)
  • request the big files with HTTP/2. e.g. with curl curl --http2 http://1.2.3.4/bla --output bla
  • the download stalls at about 10% of the filesize totally, the connection remains open, curl hangs, nothing else happens
    (I tried one file with 5,5GB which stalls at 564MB, another file with 23GB stalls at 2,2GB)
  • the same behaviour using firefox
  • request with HTTP/1.1 terminate without error and correct size (which is not an usable option using a browser)
Actions #1

Updated by gstrauss 4 months ago

This optimization will be part of the next lighttpd release (lighttpd 1.4.66). Does it help?
https://git.lighttpd.net/lighttpd/lighttpd1.4/commit/f86b448799fc7f55274a1d61583ff3d390f58e0b

Also, please check that you are running curl 7.84 or later. (See #3089)

Actions #2

Updated by flynn 4 months ago

The used curl version is 7.84.0.
The problem is also reproducable with firefox browser on Linux and Windows.
Both clients hang after 10% downloaded filesize.

Chromium has the same problem, so it is not an issue of a single client.

I applied the patch above to version 1.4.65, the problem remains: curl hangs at the exact same position.

Actions #3

Updated by flynn 4 months ago

I enabled debug.log-state-handling = "enable", it just stops writing log messages:

2022-08-04 18:08:12: (connections.c.958) fd:7 id:1 state:write
2022-08-04 18:08:12: (connections.c.958) fd:7 id:1 state:write
2022-08-04 18:08:12: (connections.c.958) fd:7 id:1 state:write
2022-08-04 18:08:12: (connections.c.958) fd:7 id:1 state:write
2022-08-04 18:08:12: (connections.c.958) fd:7 id:1 state:write
2022-08-04 18:08:12: (connections.c.958) fd:7 id:1 state:write
2022-08-04 18:08:12: (connections.c.958) fd:7 id:1 state:write
Actions #4

Updated by gstrauss 4 months ago

I see similar behavior, but have not tracked down the issue yet.

Actions #5

Updated by gstrauss 4 months ago

I have not had much time to dig into this, but will do so later today.

The issue appears at exactly 4 GiB in size. Things work with one byte fewer.
This suggests to me that there is a uint32_t calculation somewhere which should be off_t.

Actions #6

Updated by gstrauss 4 months ago

  • Status changed from New to Patch Pending

Thanks for the bug report. This looks to be a regression since lighttpd 1.4.60. Here is a patch.

--- a/src/h2.c
+++ b/src/h2.c
@@ -2636,8 +2636,8 @@ h2_send_cqdata (request_st * const r, connection * const con, chunkqueue * const
     if (h2r->h2_swin < 0) return 0;
     if ((int32_t)dlen > r->h2_swin)   dlen = (uint32_t)r->h2_swin;
     if ((int32_t)dlen > h2r->h2_swin) dlen = (uint32_t)h2r->h2_swin;
-    const uint32_t cqlen = (uint32_t)chunkqueue_length(cq);
-    if (dlen > cqlen) dlen = cqlen;
+    const off_t cqlen = chunkqueue_length(cq);
+    if ((int32_t)dlen > cqlen) dlen = (uint32_t)cqlen;
     /*(note: must temporarily disable next line when running h2spec since
      * some h2spec tests expect 1-byte DATA frame, not a deferred response)*/
     else if (dlen < 2048 && cqlen >= 2048) return 0;

Actions #7

Updated by gstrauss 4 months ago

  • Status changed from Patch Pending to Fixed
Actions #8

Updated by flynn 4 months ago

Tested and works - thank you.

Actions

Also available in: Atom