[Solved] mod_proxy: Missing CONTENT-LENGTH header from internal http server on HEAD request

Added by Anonymous 4 months ago

I have an internal http server configured as the following:

$HTTP["url"] =~ "^/upload(/.*){0,1}$" {
        proxy.server = (
            "" => (
                "" => (
                        "host" => "0.0.0.0",
                            "port" => 3301
                )
                    )
            )
    }
}

When I do a HEAD request on the internal server I get the following headers:

HTTP/1.1 204 No Content
Server: tiny-http (Rust)
Date: Sat, 27 Jul 2024 16:36:54 GMT
content-disposition: attachment
content-type: application/octet-stream
content-security-policy: "default-src 'none'" 
x-content-type-options: "default-src 'none'" 
Content-Length: 6512982

But when I do the request from outside through the proxy, the content-length is missing:

server: tiny-http (Rust)
date: Sat, 27 Jul 2024 16:37:41 GMT
content-disposition: attachment
content-type: application/octet-stream
content-security-policy: "default-src 'none'" 
x-content-type-options: "default-src 'none'"

Could it be a difference in the protocols between HTTP 1.1 and 2? Is lighttpd or the internal server doing something wrong when returning the content-length?

Replies (4)

RE: mod_proxy: Missing CONTENT-LENGTH header from internal http server on HEAD request - Added by gstrauss 4 months ago

Could it be a difference in the protocols between HTTP 1.1 and 2?

Possibly, but unlikely since lighttpd mod_proxy makes an HTTP/1.1 request to the proxy.server backend.

Is lighttpd or the internal server doing something wrong when returning the content-length?

You are jumping to conclusions with "wrong". Is something "different" happening? Seems like it might be and I suggest looking at tiny-http.

You can strace the lighttpd process to see the what is being sent from lighttpd to tiny-http and how tiny-http responds to lighttpd.

For testing purposes, you can have lighttpd make an HTTP/1.0 connection to tiny-http. Try "force-http10" => "enable" in proxy.header in lighttpd.conf. See mod_proxy

Are you using the same client on the internal server to access tiny-http directly and to access lighttpd? What client and what are the request headers?

RE: mod_proxy: Missing CONTENT-LENGTH header from internal http server on HEAD request - Added by gstrauss 4 months ago

Took another look.

HTTP/1.1 204 No Content signifies that there is no content, so lighttpd ignores Content-Length sent from the backend.

tiny-http should return 200 OK with Content-Length for a HEAD request if the same request as GET (instead of HEAD) would return 200 OK.

If the GET request would return 204 No Content, then sending Content-Length is incorrect since there is no content when returning 204 No Content. That's what "No Content" means. ;)

RE: [Solved] mod_proxy: Missing CONTENT-LENGTH header from internal http server on HEAD request - Added by Anonymous 4 months ago

If I let it return 200 instead of 204 the server library will not return the content-length and instead return Transfer-Encoding: chunked. Which would then be the proper return code for applications to peek the headers and size of the resource?

RE: [Solved] mod_proxy: Missing CONTENT-LENGTH header from internal http server on HEAD request - Added by gstrauss 4 months ago

If I let it return 200 instead of 204 the server library will not return the content-length and instead return Transfer-Encoding: chunked. Which would then be the proper return code for applications to peek the headers and size of the resource?

YOU are doing something incorrectly, not lighttpd or tiny-http.

https://www.rfc-editor.org/rfc/rfc9110#name-content-length

A server MUST NOT send a Content-Length header field in any response with a status code of 1xx (Informational) or 204 (No Content).

Your question about tiny-http and Transfer-Encoding: chunked sounds like a question for the tiny-http developers. If tiny-http has the content-length for response to a specific HEAD request, then tiny-http should have the content-length for response to that GET request, too. In that case, the only reason I can think of to use Transfer-Encoding: chunked is if there are trailers that will be included in the response by tiny-http. This seems like a limitation in tiny-http. Alternatively, maybe you can tell tiny-http to respond with HTTP/1.0 instead of HTTP/1.1.

Since lighttpd mod_proxy does not currently support HTTP/2 connections to the backend server -- and I do not know if tiny-http supports HTTP/2 -- your current workaround is to tell lighttpd to make an HTTP/1.0 request to your backend. See "force-http10" => "enable" posted earlier.

References:

RFC 9110 HTTP Semantics
https://www.rfc-editor.org/rfc/rfc9110

https://www.rfc-editor.org/rfc/rfc9110#name-content-semantics

All 1xx (Informational), 204 (No Content), and 304 (Not Modified) responses do not include content. All other responses do include content, although that content might be of zero length.

RFC 9111 HTTP Caching
https://www.rfc-editor.org/rfc/rfc9111

https://www.rfc-editor.org/rfc/rfc9111#name-freshening-responses-with-h

(1-4/4)

Project

General

Profile

Lighttpd