Project

General

Profile

[Solved] How to stop large file receiving?

Added by shortreceiver about 7 years ago

Can lighttpd stop receiving large file data as soon as the received data size exceeds a preset threshold? I can get "413 - Request Entity Too Large" of my lighttpd response by setting "server.max-request-size" in the lighttpd config. But, the lighttpd was receiving a large file even after the received data size exceeds "server.max-request-size".

For example, when I set "server.max-request-size = 500000" (= 500MB), if a client tries to upload 1GB file, I want my lighttpd to stop receiving the client uploading data as soon as the received data size exceeds 500MB of 1GB.

Solved: It is reasonable that lighttpd continued receiving to show 413.

My environment
  • OS: Linux, debian jessie (stable release)
  • Version of Lighttpd: 1.4.45 compiled from source code
  • Client: Firefox 45.7.0, curl 7.38.0

Replies (4)

RE: How to stop large file receiving? - Added by carpii about 7 years ago

Once the HTTP 413 response has been sent (due to max-request-size), how are you verifying that lighttpd is still accepting more POST data from the client?

I would have expected lighty to just terminate the connection at this point (although maybe not exactly on 500MB)

If you are just using a normal HTML file upload control, dont forget to add a <input type="hidden" name="MAX_FILE_SIZE" value="xxxxx_bytes" />

This is only client side though, so obviously cannot be relied on.
But if someone does try to upload a 1GB file from a browser, it would at least save you 500MB of bandwidth before its rejected

RE: How to stop large file receiving? - Added by shortreceiver about 7 years ago

how are you verifying that lighttpd is still accepting more POST data from the client?

I verified it by counting received data size in OS side as below:
$ /sbin/ifconfig eth0
...
RX bytes:3373821941 (1.0 GiB) TX bytes:1770941 (1.5 MiB)
...

I would have expected lighty to just terminate the connection at this point (although maybe not exactly on 500MB)

I could make the size 500MB when I killed lighttpd process manually. But I couldn't when I didn't kill the process. lighty seems not to terminate the connection until lighty has got ALL data even when size of the data exceeds "server.max-request-size".

If you are just using a normal HTML file upload control, dont forget to add a <input type="hidden" name="MAX_FILE_SIZE" value="xxxxx_bytes" />
This is only client side though, ... it would at least save you ...

Thanks for your info. This method seems effective if we use it in PHP, I think. I found below articles about your suggestion. I'm trying them.
http://php.net/manual/en/features.file-upload.post-method.php
http://stackoverflow.com/questions/1381364/max-file-size-in-php-whats-the-point


Server side trying (please help)

I'm also trying to resolve my problem in server side. lighttpd checks all connections for timeouts at line 1600 in "server.c" of lighttpd source code, as below:

for (ndx = 0; ndx < conns->used; ndx++) {
    connection * const con = conns->ptr[ndx];
...

The member "con->request_content_queue->bytes_in" seems to represent size of the received data. I could stop receiving data by inserting below code at line 1602 in the "server.c".
off_t bytes_in = con->request_content_queue->bytes_in;
if (bytes_in > 5*1024*1024) { /* checking if the size exceeds 500MB */
    connection_set_state(srv, con, CON_STATE_ERROR);
    changed = 1;
    connection_close(srv, con);
}

Please note that I also rewrote "connection_close()" as a non-static function in "connection.c" and "connection.h".

But the inserting may cause inconsistency of inner state of the variables "srv" and "con". Is there stable method like as using a module or a lighttpd's original non-static function to stop receiving data as soon as the received data size exceeds a preset threshold?

RE: How to stop large file receiving? - Added by gstrauss about 7 years ago

After lighttpd sends an error response, it continues to read and discard what is read from the client for up to about 5 seconds. The reason for this is that if lighttpd simply closed the connection, the client might receive a TCP RST and might lose the error response. If that happened, the client application might not be able to report error 413 Request Entity Too Large to the user, instead reporting that the connection was reset.

RE: How to stop large file receiving? - Added by shortreceiver about 7 years ago

Thanks for your info. I understood the reason.

    (1-4/4)