Project

General

Profile

HTTPS POST upload hangs when i reach maximum supported request size

Added by martin.schwarzbauer about 5 years ago

Hi *!

I've the following problem:
When i upload a large file via POST AND HTTPS the 'request-size' maximum detections prints the following line:

lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413 


I also have seen that in code the data is read up to 5 seconds ... but now the transfer is NOT stopped!

If i do one POST upload my messages file looks like:

Feb 18 10:07:27 falcon lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413 
Feb 18 10:07:33 falcon lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413 
Feb 18 10:07:39 falcon lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413 
Feb 18 10:07:45 falcon lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413 
Feb 18 10:07:51 falcon lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413 
Feb 18 10:07:57 falcon lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413 
Feb 18 10:08:03 falcon lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413 
Feb 18 10:08:09 falcon lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413 
Feb 18 10:08:15 falcon lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413 
Feb 18 10:08:21 falcon lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413 
Feb 18 10:08:27 falcon lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413 
Feb 18 10:08:33 falcon lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413 
Feb 18 10:08:39 falcon lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413 
Feb 18 10:08:45 falcon lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413 
Feb 18 10:08:51 falcon lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413 
Feb 18 10:08:57 falcon lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413 
Feb 18 10:09:03 falcon lighttpd[4764]: (response.c.441) request-size too long: 110100927 -> 413
Configuration:
  • HTTPS enabled
  • server.max-request-size=100000
  • server.stream-response-body = 2
  • server.stream-request-body = 2

Does anyone know a solution? I expect the browser to finish the upload, but in chrome i just see that after 15% it jumps back to 0% and start upload again, and again, and again, ....
If i do this using HTTP it works as expected and i see the error 413 in browser !

Hope anyone can help me!

Thank you very much,
Martin


Replies (6)

RE: HTTP POST upload hangs when i reach maximum supported request size - Added by martin.schwarzbauer about 5 years ago

I am using 1.4.45 but also the same troubles with 1.4.53

RE: HTTPS POST upload hangs when i reach maximum supported request size - Added by gstrauss about 5 years ago

You have described behavior that you have observed in Chrome. Why do you think this is a problem in lighttpd?

I quickly tested lighttpd with 'openssl s_client' and an oversized HTTP POST request is properly rejected with a "413 Request Entity Too Large"

Have you tested with something other than Chrome?

RE: HTTPS POST upload hangs when i reach maximum supported request size - Added by gstrauss about 5 years ago

Please try the following patch. I am going to guess that Chrome is seeing a TLS error before it sees the HTTP response.

--- a/src/connections.c
+++ b/src/connections.c
@@ -150,7 +150,7 @@ static int connection_close(server *srv, connection *con) {
        return 0;
 }

-static void connection_read_for_eos(server *srv, connection *con) {
+static void connection_read_for_eos_plain(server *srv, connection *con) {
        /* we have to do the linger_on_close stuff regardless
         * of con->keep_alive; even non-keepalive sockets may
         * still have unread data, and closing before reading
@@ -173,6 +173,18 @@ static void connection_read_for_eos(server *srv, connection *con) {
                con->close_timeout_ts = srv->cur_ts - (HTTP_LINGER_TIMEOUT+1);
 }

+static void connection_read_for_eos_ssl(server *srv, connection *con) {
+       if (con->network_read(srv, con, con->read_queue, MAX_READ_LIMIT) < 0)
+               con->close_timeout_ts = srv->cur_ts - (HTTP_LINGER_TIMEOUT+1);
+       chunkqueue_reset(con->read_queue);
+}
+
+static void connection_read_for_eos(server *srv, connection *con) {
+       !con->srv_socket->is_ssl
+         ? connection_read_for_eos_plain(srv, con)
+         : connection_read_for_eos_ssl(srv, con);
+}
+
 static void connection_handle_close_state(server *srv, connection *con) {
        connection_read_for_eos(srv, con);

RE: HTTPS POST upload hangs when i reach maximum supported request size - Added by martin.schwarzbauer about 5 years ago

Hi!

I've tried your patch - but still the same troubles/behaviour.

The behaviour is as follows:
1.) The HTTPS POST Form-Upload (multipart) is uploaded to target.
2.) In messages file on target I see the following message

lighttpd[23527]: (response.c.441) request-size too long: 110100927 -> 413 

3.) From now the browser doesn't stop sending - in chrome (developer mode) i don't see a response or an error of POST request - it's still "pending".
The percent is jumping from 15% back to 0% => i get the next entry in messgae log ... and from now this continues until EOL ;)

It seems, that after the 5 seconds 'read' the connection is not closed instead it is reset and beginn from 0%!
When i force a call to connection_close everything is ok (instead of reading the 5seconds) - it seems, that for HTTPS the timeout of 5 seconds doesn't trigger the close of connection? HTTP is working as expected.

The same behaviour occurs with firefox (60.5) and chrome (72.0.3626.81).

Thanks,
Martin

RE: HTTPS POST upload hangs when i reach maximum supported request size - Added by gstrauss about 5 years ago

My previous patch above is not sufficient since shutdown(con->fd, SHUT_WR) is called on the socket before that code is executed.

I wasn't able to (quickly) reproduce Firefox spinning while submitting, but I was testing on localhost, with both Firefox and browser running on the same machine.

However, I did see the browser reporting that the SSL connection was reset instead of showing the 413 error page.

There is code in lighttpd mod_openssl.c which reads encrypted data after attempting SSL_shutdown(), and then tries SSL_shutdown() again, but there is currently a 16KB limit to what is read. The limit was put in place because on slow, embedded systems where the CPU is slower decoding encrypted traffic than arriving traffic, the read loop might never end.

I pushed a slightly larger patch to git.lighttpd.net on the unstable branch personal/gstrauss/master. Please check if it works for you. See DevelGit

The updated patch takes into account that I see the state of SSL_is_init_finished() changing. I wonder if that is new behavior or a regression (I am using openssl 1.1.1a)

RE: HTTPS POST upload hangs when i reach maximum supported request size - Added by gstrauss almost 5 years ago

martin.schwarzbauer have you had a chance to test against lighttpd git master since my last post?

    (1-6/6)