Lighttpd

lighttpd code rejects POST requests without Content-Length since the first revision of lighttpd 1.4 back in Feb 2005. It seems unlikely to me that you are the first person to use Wget to POST an empty request body to lighttpd in 20 years, so you might want to look into what changed in Wget and why, and might file an issue with Wget.

Lighttpd MUST NOT emit error code 411 when the Content-Length request header is not present, but MUST treat it as zero.

BTW, the second part of your statement is false. Absense of Content-Length does not require "MUST treat it as zero". That is demonstrably wrong for a request with a non-zero request body using HTTP/1.1 Transfer-Encoding: chunked or using HTTP/2 or later framing.

In 2014, someone submitted a patch to Wget to send Content-Length: 0 for POST requests with empty request body. https://lists.gnu.org/archive/html/bug-wget/2014-10/msg00073.html I have not looked into what changed in Wget since.

gstrauss wrote in #note-3:

Lighttpd MUST NOT emit error code 411 when the Content-Length request header is not present, but MUST treat it as zero.

BTW, the second part of your statement is false. Absense of Content-Length does not require "MUST treat it as zero". That is demonstrably wrong for a request with a non-zero request body using HTTP/1.1 Transfer-Encoding: chunked or using HTTP/2 or later framing.

I'm sorry, I was oversimplifying that statement. What I meant was: When the content length is derived from the Content-Length header, the Content-Length header should be treated as 0 when absent.

gstrauss wrote in #note-4:

In 2014, someone submitted a patch to Wget to send Content-Length: 0 for POST requests with empty request body. https://lists.gnu.org/archive/html/bug-wget/2014-10/msg00073.html I have not looked into what changed in Wget since.

The observation was with Wget2, which is an other user agent. Other user agents have similar behavior, and should be fixed too, following the RFC1958 robustness principle of "Be strict when sending [...]".
Unfortunately, I don't know which user agents are going to access our web servers, so it's not possible to fix bugs in the other end.

However, it can be fixed in the server end.
Which is why I suggest lighttpd should follow the RFC1958 robustness principle of "[...] and tolerant when receiving" regarding the absent Content-Length request header.

We can fix it locally, but I thought the lighttpd community would like to know about this interoperability issue, and possibly fix it.

PS: Thank you for the quick and qualified responses. And sorry about abusing RFC terminology. And thank you for a great web server!

Postel's principle, the Robustness Principle, is important and yet should be applied with care and balance.
https://medium.com/@mesw1/understanding-the-robustness-principle-postels-law-c1199ea79210

https://www.rfc-editor.org/rfc/rfc9110.html#name-content-length

Because Content-Length is used for message delimitation in HTTP/1.1, its field value can impact how the message is parsed by downstream recipients even when the immediate connection is not using HTTP/1.1. If the message is forwarded by a downstream intermediary, a Content-Length field value that is inconsistent with the received message framing might cause a security failure due to request smuggling or response splitting.

The scenario you are reporting in this issue is only present in HTTP/1.x, as the request body length is handled in HTTP/2 and later with request framing in the protocol.

With HTTP/1.0, absense of Content-Length may be handled by some servers as "read until EOF", or may be handled as Content-Length: 0. This inconsistency is dangerous for request smuggling, and is one of the reasons why your request that lighttpd treat HTTP/1.x POST absense of Content-Length (and absense of Transfer-Encoding: chunked) as Content-Length: 0 is not as straightforward and as safe as you might first assume.

Clients should, of course, prefer HTTP/1.1 or later. So why is HTTP/1.0 still used and supported? Well, one reason is that HTTP/1.1 requires that the client be able to handled Transfer-Encoding: chunked responses, and overly simplistic clients which do not support chunked encoding are overly simplistic and continue to send HTTP/1.0.

lighttpd behavior in all versions of lighttpd 1.4 (since 2005) has been strict in rejecting HTTP POST requests without Content-Length or HTTP/1.1 Transfer-Encoding: chunked or HTTP/2+ message framing. While this may be overly strict for RFC requirements, the behavior is widespread and reasonable, and thwarts request smuggling attempts using HTTP POST without Content-Length (and without HTTP/1.1 Transfer-Encoding: chunked, etc).

If you look in https://git.lighttpd.net/lighttpd/lighttpd1.4/src/branch/master/src/request.c#L1225 and down to the else block which follows, you'll see that lighttpd rejects GET and HEAD requests which contain a request body. All methods are permitted by RFCs to contain request bodies, but GET and HEAD generally do not contain request bodies. Rejecting non-zero request bodies with GET and HEAD thwart request smuggling attempting to use this channel. However, since some newer legitimate usage of GET contains request bodies, I added a configurable flag in lighttpd to allow GET/HEAD to have a non-zero request body, but this flag is disabled by default.

The observation was with Wget2, which is an other user agent. Other user agents have similar behavior, and should be fixed too, following the RFC1958 robustness principle of "Be strict when sending [...]".
Unfortunately, I don't know which user agents are going to access our web servers, so it's not possible to fix bugs in the other end.

If this is a hardship for you, then I will consider adding an option for POST without Content-Length, similar to HTTP_PARSEOPT_METHOD_GET_BODY for allowing request body with GET/HEAD. However, even if you "don't know which user agents are going to access our web servers", you should be able to demonstrate that this is more than a one-off problem for you by looking in your access logs for the frequency of 411 responses from lighttpd, and seeing how often it occurs and for which user-agents.

It sounds to me like you plan to patch lighttpd for your use. That is fine. I also encourage you to file bugs with Wget2 and any other clients which you find relying on this questionable behavior, even if the RFCs do not explicitly forbid it.

gstrauss wrote in #note-6:

If this is a hardship for you, then I will consider adding an option for POST without Content-Length, similar to HTTP_PARSEOPT_METHOD_GET_BODY for allowing request body with GET/HEAD.

That would be great!
We are not intimately familiar with the HTTP protocols or lighttpd source code, and all the details in your reply have made me aware that "fixing" it ourselves would come with a high risk of introducing bugs or security holes.

However, even if you "don't know which user agents are going to access our web servers", you should be able to demonstrate that this is more than a one-off problem for you by looking in your access logs for the frequency of 411 responses from lighttpd, and seeing how often it occurs and for which user-agents.

Unfortunately, we don't have access to logs on appliances deployed by our customers.
Perhaps someone else in the lighttpd community can grep their logs for 411 responses.

After further review of the Wget2 output, I think this issue could be related to HTTP2.
This is the debug output from Wget2, accessing a server running lighttpd version 1.4.67:

$ wget -d --content-on-error --no-check-certificate --auth-no-challenge -O - -q --user=REDACTED --password=REDACTED --method=POST --body-file=/dev/null https://REDACTED/api/sniffer/start
12.113000.386 Local URI encoding = 'UTF-8'
12.113000.386 Input URI encoding = 'UTF-8'
12.113000.386 Fetched HSTS data from '/home/REDACTED/.local/share/wget/.wget-hsts'
12.113000.386 Fetched HPKP data from '/home/REDACTED/.local/share/wget/.wget-hpkp'
12.113000.386 set_exit_status(0)
12.113000.386 path api/sniffer/start ->
12.113000.386      api/sniffer/start
12.113000.386 host_add_job: job fname (null)
12.113000.386 host_add_job: 0x55f5bb6bcd10 https://REDACTED/api/sniffer/start
12.113000.386 host_add_job: qsize 1 host-qsize=1
12.113000.386 queue_size: qsize=1
12.113000.386 queue_size: qsize=1
12.113000.386 queue_size: qsize=1
12.113000.386 [0] action=1 pending=0 host=0x0 goaway=n/a
12.113000.386 dequeue job https://REDACTED/api/sniffer/start
12.113000.386 resolving REDACTED:443...
12.113000.386 has REDACTED:443
12.113000.386 has REDACTED:443
12.113000.386 trying REDACTED:443...
12.113000.423 GnuTLS init
12.113000.424 Certificates loaded: -1
12.113000.424 GnuTLS init done
12.113000.424 TLS False Start requested
12.113000.424 SNI REDACTED
12.113000.424 OCSP stapling requested for REDACTED
12.113000.424 ALPN offering h2
12.113000.424 ALPN offering http/1.1
12.113000.456 TLS False Start: off
12.113000.456 ALPN: Server accepted protocol 'h2'
12.113000.456 Handshake completed
12.113000.456 established connection REDACTED
12.113000.456 cookie_create_request_header for host=REDACTED path=api/sniffer/start
12.113000.456 HTTP2 stream id 1
12.113000.456 [0] action=1 pending=1 host=0x55f5bb6bc890 goaway=false
12.113000.456 [0] action=2 pending=1 host=0x55f5bb6bc890 goaway=false
12.113000.456   ##  pending_requests = 1
12.113000.456 [FRAME 0] > SETTINGS
12.113000.456 [FRAME 0] > WINDOW_UPDATE
12.113000.456 [FRAME 1] > HEADERS
12.113000.456 [FRAME 1] > :method: POST
12.113000.456 [FRAME 1] > :path: /api/sniffer/start
12.113000.456 [FRAME 1] > :scheme: https
12.113000.456 [FRAME 1] > :authority: REDACTED
12.113000.456 [FRAME 1] > accept-encoding: gzip, deflate, br, zstd
12.113000.456 [FRAME 1] > accept: */*
12.113000.456 [FRAME 1] > user-agent: Wget/2.2.0
12.113000.456 [FRAME 1] > authorization: Basic REDACTED
12.113000.456 [FRAME 1] > content-type: application/x-www-form-urlencoded
12.113000.485 [FRAME 0] < SETTINGS
12.113000.485 [FRAME 0] < WINDOW_UPDATE
12.113000.485 [FRAME 0] > SETTINGS
12.113000.485 [FRAME 0] < SETTINGS
12.113000.491 :status: 411
12.113000.491 content-type: text/html
12.113000.491 content-length: 353
12.113000.491 date: Sun, 12 Jan 2025 13:30:00 GMT
12.113000.491 [FRAME 1] < HEADERS
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" 
         "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
 <head>
  <title>411 Length Required</title>
 </head>
 <body>
  <h1>411 Length Required</h1>
 </body>
</html>
12.113000.491 closing stream 1
12.113000.491   ##  response status 411
12.113000.491 keep_alive=1
12.113000.491 _host_remove_job: 0x55f5bb6bcd10
12.113000.491 host_remove_job: qsize=0 host->qsize=0
12.113000.491 [0] action=1 pending=0 host=0x55f5bb6bc890 goaway=false
12.113000.491 closing connection
12.113000.491 [0] action=1 pending=0 host=0x0 goaway=n/a
12.113000.491 main: wake up
12.113000.491 main: done
12.113000.491 blacklist https://REDACTED/api/sniffer/start

I wrote:

If this is a hardship for you, then I will consider adding an option for POST without Content-Length, similar to HTTP_PARSEOPT_METHOD_GET_BODY for allowing request body with GET/HEAD. However, even if you "don't know which user agents are going to access our web servers", you should be able to demonstrate that this is more than a one-off problem for you by looking in your access logs for the frequency of 411 responses from lighttpd, and seeing how often it occurs and for which user-agents.

You wrote the following, conveniently ignoring my second sentence and suggesting someone else should do what I asked of you to help quantify your hardship:

That would be great!

...

Perhaps someone else in the lighttpd community can grep their logs for 411 responses.

We are not intimately familiar with the HTTP protocols or lighttpd source code, and all the details in your reply have made me aware that "fixing" it ourselves would come with a high risk of introducing bugs or security holes.

Commenting out the block of code below https://git.lighttpd.net/lighttpd/lighttpd1.4/src/branch/master/src/request.c#L1225 which sends 411 for POST without Content-Length (or Transfer-Encoding: chunked etc) removes the commonly true expectation that POST is sent with a defined body size, and therefore removes some protection against request smuggling attacks which might try to abuse that avenue. However, commenting that out would also make lighttpd behavior to backends inconsistent, and a better approach would be to replace the 411 with an injection of Content-Length: 0 into the request headers.

Here is an UNTESTED patch, which is unlikely to become part of an official lighttpd release without additional and sufficient supporting data.
This adds an server.http-parseopts option "method-post-inject-0cl"

--- a/src/base.h
+++ b/src/base.h
@@ -98,6 +98,7 @@ typedef struct {
        unsigned char http_host_strict;
        unsigned char http_host_normalize;
        unsigned char http_method_get_body;
+       unsigned char http_method_post_0cl;
        unsigned char high_precision_timestamps;
        unsigned char h2proto;
        unsigned char absolute_dir_redirect;
--- a/src/burl.h
+++ b/src/burl.h
@@ -28,6 +28,7 @@ enum burl_opts_e {
  ,HTTP_PARSEOPT_URL_NORMALIZE_QUERY_20_PLUS       =0x1000
  ,HTTP_PARSEOPT_URL_NORMALIZE_INVALID_UTF8_REJECT =0x2000
  ,HTTP_PARSEOPT_METHOD_GET_BODY                   =0x8000
+ ,HTTP_PARSEOPT_METHOD_POST_0CL                   =0x10000
 };

 int burl_normalize (buffer *b, buffer *t, int flags);
--- a/src/configfile.c
+++ b/src/configfile.c
@@ -606,6 +606,10 @@ static int config_http_parseopts (server *srv, const array *a) {
             srv->srvconf.http_method_get_body = val;
             continue;
         }
+        else if (buffer_eq_slen(k, CONST_STR_LEN("method-post-inject-0cl"))) {
+            srv->srvconf.http_method_post_0cl = val;
+            continue;
+        }
         else {
             log_error(srv->errh, __FILE__, __LINE__,
               "unrecognized key for server.http-parseopts: %s", k->ptr);
@@ -1331,6 +1335,7 @@ static int config_insert(server *srv) {
       | (srv->srvconf.http_host_strict     ? (HTTP_PARSEOPT_HOST_STRICT
                                              |HTTP_PARSEOPT_HOST_NORMALIZE) :0)
       | (srv->srvconf.http_host_normalize  ?  HTTP_PARSEOPT_HOST_NORMALIZE  :0)
+      | (srv->srvconf.http_method_post_0cl ?  HTTP_PARSEOPT_METHOD_POST_0CL :0)
       | (srv->srvconf.http_method_get_body ?  HTTP_PARSEOPT_METHOD_GET_BODY :0);
     p->defaults.http_parseopts |= srv->srvconf.http_url_normalize;
     p->defaults.mimetypes = &srv->srvconf.mimetypes_default;/*must not be NULL*/
--- a/src/request.c
+++ b/src/request.c
@@ -1222,11 +1222,16 @@ http_request_parse (request_st * const restrict r, const int scheme_port)
     }

     if (0 == r->reqbody_length) {
-        /* POST requires Content-Length (or Transfer-Encoding)
+        /* POST generally expects Content-Length (or Transfer-Encoding)
          * (-1 == r->reqbody_length when Transfer-Encoding: chunked)*/
         if (HTTP_METHOD_POST == r->http_method
             && !light_btst(r->rqst_htags, HTTP_HEADER_CONTENT_LENGTH)) {
-            return http_request_header_line_invalid(r, 411, "POST-request, but content-length missing -> 411");
+            if (http_parseopts & HTTP_PARSEOPT_METHOD_POST_0CL)
+                http_header_request_set(r, HTTP_HEADER_CONTENT_LENGTH,
+                                        CONST_STR_LEN("Content-Length"),
+                                        CONST_STR_LEN("0"));
+            else
+                return http_request_header_line_invalid(r, 411, "POST-request, but content-length missing -> 411");
         }
     }
     else {

Regarding your new data about HTTP/2, my original post already told you why lighttpd might send 411:

lighttpd can return 411 as lighttpd deems fit and in fact does so when a client does not set Content-Length (e.g. sends Transfer-Encoding: chunked) but lighttpd is configured to stream the request body to a backend which requires a content-length, such as CGI, FastCGI, SCGI, and some others. See https://git.lighttpd.net/lighttpd/lighttpd1.4/src/branch/master/src/gw_backend.c#L2302

Wget is being sloppy if Wget knows the request body length but does not follow the RFC "SHOULD" and send Content-Length.

You or your client probably enabled request body streaming with lighttpd.conf server.stream-request-body, and you hopefully already read How to get support before posting.

The untested patch I posted above will not affect HTTP/2. Your company might consider sponsoring a feature in lighttpd, such as a feature to delay contacting the backend until first data bytes arrive. Doing so might add a little bit of latency, but is likely to be unnoticed since round-trip-time is quick on a local LAN, and when round-trip-time to a client is long, the time it takes to connect to a backend will likely be much shorter than the latency of the connection to the client. Delaying lighttpd contacting a backend until first data byte arrive implies that lighttpd would find out from HTTP/2 framing that there is 0-length body before contacting the backend, and eliding the lighttpd backend setup sending 411 Length Required when the length of the request body is unknown (at that point) and where the protocol to the backend requires a content length.

Your patch fixed the issue with the sloppy Wget2. Thank you.

gstrauss wrote in #note-9:

Wget is being sloppy if Wget knows the request body length but does not follow the RFC "SHOULD" and send Content-Length.

Agree. (And it's Wget2.) I have now filed an issue there: https://gitlab.com/gnuwget/wget2/-/issues/691

You or your client probably enabled request body streaming with lighttpd.conf server.stream-request-body [...]

Request body streaming is not enabled. It's an "embedded" server in a network appliance, where the config file is part of the firmware image, and not modifiable by customers deploying the appliance.

This is the config file:

server.modules              = (
  "mod_cgi",
  "mod_expire",
  "mod_wstunnel",
  "mod_proxy",
  "mod_openssl",
)
server.document-root        = "/REDACTED/htdocs" 
server.error-handler-404    = "/index.html" 
server.tag                  = "" 
index-file.names            = ( "index.html" )
# mimetype mapping
mimetype.assign             = (
  ".gif"          =>      "image/gif",
  ".css"          =>      "text/css",
  ".html"         =>      "text/html",
  ".js"           =>      "text/javascript",
  # default mime type
  ""              =>      "application/octet-stream",
)
static-file.exclude-extensions = ( "/REDACTED/htdocs/api" )
cgi.assign                  = (
  "/REDACTED/htdocs/api" => "/REDACTED/REDACTED",
)
# Do not buffer the complete response body from CGI backends, but stream it directly to the client. (Only important for very large responses.)
server.stream-response-body = 2
# websocket support.
$HTTP["url"] == "/websocket/gui" {
  proxy.server = ( "" => ( ( "host" => "REDACTED", "port" => "REDACTED" ) ) )
  proxy.header = ( "upgrade" => "enable" )
  proxy.forwarded = ( "for"   => 1,
                      "proto" => 1,
                    )
}
# Browser cache control.
expire.url = (
  "/" => "access plus 0 hours",
)
# Error logging is disabled by default. Uncomment these to log both lighttpd and CGI errors to stderr.
server.errorlog = "/dev/stderr" 
server.breakagelog = "/dev/stderr" 
# Listening sockets
$SERVER["socket"] == ":80" {}
$SERVER["socket"] == ":443" {
    ssl.engine = "enable" 
    ssl.pemfile = "REDACTED" 
}
# Enable use of older TLS protocols.
ssl.openssl.ssl-conf-cmd = ( REDACTED )

We don't have access to customer's production systems, so we cannot grep any logs to determine if Wget2 is a one-off interoperability issue or other user agents are sloppy too. If anybody cares, they can grep their log files.

Correction: it is possible for the patch to affect HTTP/2 if HTTP/2 framing indicates end of request body with the HEADERS frame, since then lighttpd request headers processing will find r->reqbody_length == 0 for the request.

I reviewed lighttpd code
http_cgi.c sets CGI variable CONTENT_LENGTH for cgi-like backends (CGI, FastCGI, SCGI, etc) (whether or not request header content-length was part of request, as request header content-length would end up as CGI variable HTTP_CONTENT_LENGTH).
mod_proxy.c sets content-length, if appropriate, for HTTP/1.1 requests to backends
Therefore, the following patch is what I will probably add to lighttpd since HTTP/2 and later framing handle delimiting of request headers, body, and trailers, and so no request smuggling via POST without Content-Length for HTTP/2 and later.

--- a/src/request.c
+++ b/src/request.c
@@ -1225,6 +1225,7 @@ http_request_parse (request_st * const restrict r, const int scheme_port)
         /* POST requires Content-Length (or Transfer-Encoding)
          * (-1 == r->reqbody_length when Transfer-Encoding: chunked)*/
         if (HTTP_METHOD_POST == r->http_method
+            && r->http_version <= HTTP_VERSION_1_1
             && !light_btst(r->rqst_htags, HTTP_HEADER_CONTENT_LENGTH)) {
             return http_request_header_line_invalid(r, 411, "POST-request, but content-length missing -> 411");
         }

The earlier patch posted could still be used (in addition to this patch) if you wanted that feature for HTTP/1.x, but probably won't be included in the official lighttpd source code without additional data justifying its need.

• Modern versions of lighttpd include a common web types built-in, so you can omit mimetype.assign from your config above and get the same behavior as your config since lighttpd 1.4.75.
• The config does not appear to use mod_wstunnel, and so omitting it would avoid loading it into memory (and you may also omit it from the on-disk image).

Tangential to lighttpd, I would be interested if you can provide an overview of usage surrounding # Enable use of older TLS protocols., as I am working with Mozilla to update https://wiki.mozilla.org/Security/Server_Side_TLS TLS recommendations. What are these environments and why are they unable to be upgraded to support at least TLSv1.2 (for which spec https://www.rfc-editor.org/rfc/rfc5246 was published in Aug 2008, over 16 years ago)? Propriety TLS libraries? Never-upgraded devices, or devices upgraded only at end-of-life? What is the expected lifetime before replacement? Other reasons?

gstrauss wrote in #note-12:

+ && r->http_version <= HTTP_VERSION_1_1

I can confirm that this solves the problem with wget2 (when using HTTP/2).
As a precaution for improved interoperability, we will probably apply the other patch too.

Some comments on the posted config

Thank you for the comments; we will look into it. Keeping it tight reduces the attack surface. :-)

Tangential to lighttpd, I would be interested if you can provide an overview of usage surrounding # Enable use of older TLS protocols.

It is an interoperability precaution for very old user agents and firewalls/proxies. We have seen ancient equipment not being upgraded in some industrial environments.
Our use case is probably an exception, so here is the background for it:
The main purpose of the SmartShare appliances is to improve the network experience for end users (regarding latency and bandwidth), so we generally treat annoyances as bugs. Bending over backwards to ensure interoperability - even in rare cases - is part of our DNA. If some customer wants to keep using TLS 1.0 in their production environment, our equipment is not going to prevent it. (We don't support SSLv3, though.)
Since there is no reliable information about which obsolete encryption protocols are no longer in use "in the wild", we don't want to take the risk of phasing out support for them in our products.