Bug #3233
closedHTTP/2 PROTOCOL_ERROR with directory listing and external-css or external-js
Description
I am using a virtual host for a download page with directory listing, the config is like this:
$HTTP["host"] == "example.de" {
ssl.pemfile = "/etc/ssl/certs/example.de.pem"
ssl.privkey = "/etc/ssl/private/example.de.key"
server.document-root = "/var/www/"
# ratelimit external ips
$HTTP["remoteip"] !~ "192\.168\.|172\.16\." {
server.kbytes-per-second = 10240
connection.kbytes-per-second = 6144
}
url.access-deny = ("~", ".inc", ".bak", ".swp", ".old", ".dist")
$HTTP["url"] =~ "^/download/" {
dir-listing.activate = "enable"
dir-listing.encoding = "utf-8"
dir-listing.set-footer = "Download"
dir-listing.hide-dotfiles = "enable"
dir-listing.external-css = ".download.css"
dir-listing.show-readme = "README"
dir-listing.hide-readme-file = "enable"
$HTTP["url"] =~ "^/download/protected/" {
# Require LDAP authentication (AD)
auth.backend.ldap.filter = "(&(objectclass=user)(objectclass=person)(objectclass=top)(sAMAccountName=$)(!(UserAccountControl:1.2.840.113556.1.4.803:=2)))"
auth.require = ( "/download/protected" =>
(
"method" => "basic",
"realm" => "Protected Download Area",
"require" => "valid-user"
)
)
}
}
}
I am not able to access https://example.de/download/, I get an error ERR_HTTP2_PROTOCOL_ERROR with Chromium or NS_ERROR_NET_HTTP2_SENT_GOAWAY with Firefox.
This did not happen with version 1.4.67.
Since I updated from version 1.4.67 to 1.4.72 (and now even 1.4.73), I cannot access the "root" directory "download".
I am able to enter a subdirectories like https://example.de/download/example/,
but I am also not able to enter https://example.de/download/protected/.
Even if I disable the authentication requirement (commented out the whole $HTTP["url"] =~ "^/download/" section), it is not working...?
I am able to download files using filename and URL.
If I enable:- debug.log-request-header = "enable"
- debug.log-response-header = "enable"
I get:
2023-11-03 15:22:41: (h2.c.1572) fd:57 id:1 rqst: :method: GET 2023-11-03 15:22:41: (h2.c.1572) fd:57 id:1 rqst: :authority: example.de 2023-11-03 15:22:41: (h2.c.1572) fd:57 id:1 rqst: :scheme: https 2023-11-03 15:22:41: (h2.c.1572) fd:57 id:1 rqst: :path: /download/ 2023-11-03 15:22:41: (h2.c.1572) fd:57 id:1 rqst: cache-control: max-age=0 2023-11-03 15:22:41: (h2.c.1572) fd:57 id:1 rqst: sec-ch-ua: "Chromium";v="118", "Microsoft Edge";v="118", "Not=A?Brand";v="99" 2023-11-03 15:22:41: (h2.c.1572) fd:57 id:1 rqst: sec-ch-ua-mobile: ?0 2023-11-03 15:22:41: (h2.c.1572) fd:57 id:1 rqst: sec-ch-ua-platform: "Windows" 2023-11-03 15:22:41: (h2.c.1572) fd:57 id:1 rqst: upgrade-insecure-requests: 1 2023-11-03 15:22:41: (h2.c.1572) fd:57 id:1 rqst: user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36 Edg/118.0.2088.76 2023-11-03 15:22:41: (h2.c.1572) fd:57 id:1 rqst: accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7 2023-11-03 15:22:41: (h2.c.1572) fd:57 id:1 rqst: sec-fetch-site: none 2023-11-03 15:22:41: (h2.c.1572) fd:57 id:1 rqst: sec-fetch-mode: navigate 2023-11-03 15:22:41: (h2.c.1572) fd:57 id:1 rqst: sec-fetch-user: ?1 2023-11-03 15:22:41: (h2.c.1572) fd:57 id:1 rqst: sec-fetch-dest: document 2023-11-03 15:22:41: (h2.c.1572) fd:57 id:1 rqst: accept-encoding: gzip, deflate, br 2023-11-03 15:22:41: (h2.c.1572) fd:57 id:1 rqst: accept-language: de,de-DE;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6 2023-11-03 15:22:41: (h2.c.2326) fd:57 id:1 resp: :status: 200 2023-11-03 15:22:41: (h2.c.2315) fd:57 id:1 resp: content-type: text/html;charset=utf-8 2023-11-03 15:22:41: (h2.c.2315) fd:57 id:1 resp: vary: Accept-Encoding 2023-11-03 15:22:41: (h2.c.2315) fd:57 id:1 resp: content-encoding: br 2023-11-03 15:22:41: (h2.c.2315) fd:57 id:1 resp: content-length: 3301 2023-11-03 15:22:41: (h2.c.2326) fd:57 id:1 resp: date: Fri, 03 Nov 2023 14:22:41 GMT 2023-11-03 15:22:41: (h2.c.2315) fd:57 id:1 resp: server: WebServer
Isn't that a status 200 - ok?
Updated by gstrauss 6 months ago
Isn't that a status 200 - ok?
Yes. So there must be something else going on, perhaps in subsequent HTTP/2 frames delivering the response.
I see that you're using dir-listing.external-css = ".download.css". Have you tried temporarily disabling that and testing? Yes, the page will look different without your .css, but will Chrome or Firefox throw errors?
I see that you're using mod_deflate and brotli encoding. Have you tried temporarily disabling mod_deflate and testing?
How many entries (file and directories) are in the directory listing that works? How many entries (file and directories) are in the directory listings that do not work?
Have you configured lighttpd to stream the response?
Have you configured mod_dirlisting caching?
Have you configured mod_deflate caching?
Please see How to get support
Your post contains some information, but not enough to properly troubleshoot, as you can see from my questions.
Updated by gstrauss 6 months ago
- Category set to mod_dirlisting
- Target version changed from 1.4.xx to 1.4.74
For larger directories, and when dir-listing.external-css or dir-listing.external-js are set, lighttpd mod_dirlisting in lighttpd 1.4.72 or later may send an intermediate response of 103 Early Hints for HTTP/2 requests (which is not done for HTTP/1.x requests). There appears to be an issue here in the way that lighttpd is doing this. I am looking into this.
As a workaround, you can disable HTTP/2 support in lighttpd: server.feature-flags += ("h2proto" => "disable"), though that is probably not desirable. Another workaround is avoiding dir-listing.external-css, which is probably also a non-starter.
Updated by gstrauss 6 months ago
- Status changed from New to Patch Pending
Unfortunate bug. HTTP/2 headers must be lowercase. Here is a minimal patch. I may implement a more general sanity check in mod_h2.
--- a/src/mod_dirlisting.c
+++ b/src/mod_dirlisting.c
@@ -664,7 +664,7 @@ static void http_dirlist_link (request_st * const r, const buffer *b, const char
buffer_clear(tb);
buffer_append_str3(tb, CONST_STR_LEN("<"), BUF_PTR_LEN(b), params, plen);
http_header_response_insert(r, HTTP_HEADER_LINK,
- CONST_STR_LEN("Link"),
+ CONST_STR_LEN("link"),
BUF_PTR_LEN(tb));
}
With lighttpd 1.4.73 just released, there is unfortunately no upcoming schedule for the release of lighttpd 1.4.74 with this patch. lighttpd 1.4.74 is likely to be released before the end of Q1 2024.
Updated by mgottinger 6 months ago
Sorry, didn't see your response until one hour ago.
I recompiled the package and it is working now.
Disabling dir-listing.external-css was not something that crossed my mind, so I did not test it, yes.
I cleared the cache because I use mod_deflate, but seeing no issue on other virtual hosts I was pretty sure it had to do with dirlisting and also did not disable mod_deflate.
I don't use the mod_dirlisting cache, because there are only about 50 files in the directory, but they are in daily use.
Just out of curiosity - disabling HTTP2 is only possible on a global level, isn't it?
Because I did think about that but found no documentation pointing otherwise.
Thank you for your quick response and fix.
Updated by gstrauss 6 months ago
Just out of curiosity - disabling HTTP2 is only possible on a global level, isn't it?
Yes, HTTP/2 enable/disable is a global setting in lighttpd.
HTTP/2 is negotiated in ALPN along with TLS. The TLS encryption (including the ALPN extension) is negotiated before an encrypted request is sent.
While it is theoretically possible to write code to dynamically change the TLS ALPN offered based on the TLS SNI or the socket on which lighttpd is listening, nobody has tried to convince me why this would be a terribly useful feature in lighttpd that is worth the effort to code it.
Notwithstanding, the bug in this issue is a bug and will be fixed the next lighttpd release.
BTW, another workaround is to write your own script to monitor the directory, to generate an index.html whenever something in the directory changes, and to serve index.html using mod_indexfile rather than using mod_dirlisting to generate the directory listing.
Updated by gstrauss 6 months ago
- Status changed from Patch Pending to Fixed
Applied in changeset 0a49dde383e6cd0a9a5168639682a459b0a39f7e.
Also available in: Atom