Project

General

Profile

Bug #2660

[bisected] 1.4.36 regression: uploading 4 MiB+ files results in 413 - Request Entity Too Large

Added by rmilecki over 1 year ago. Updated over 1 year ago.

Status:
Fixed
Priority:
Normal
Assignee:
-
Category:
-
Target version:
Start date:
2015-08-13
Due date:
% Done:

100%

Missing in 1.5.x:

Description

I use a static HTML page with a trivial file upload form:

<form enctype="multipart/form-data" action="" method="POST">
Send this file: <input name="userfile" type="file" />
<input type="submit" value="Send File" />
</form>

Please note it's only about lighttpd handling POST request. Icurrently don't run PHP or anything.

Unfortunately after upgrade from 1.4.35 to 1.4.36 it doesn't workanymore for 4 MiB+ files. There are some file sizes I tried and the results (using 1.4.36):

4210000 Didn't fail once in 50 tries
4220000 Fails sometimes
4225000 Fails with 413 or 500 (randomly)
4230000 Fails with 413

Seeing error 413 results in following log message:

(chunk.c.461) denying upload as opening temp-file for upload failed:
Too many open files

Seeing error 500 results in following log message:

(mod_indexfile.c.181) file not found ... or so:  Too many open files / -> /www/

I was able to find out that this issue can be workarounded be bumping max-fds. When adding:

server.max-fds = 2048

I can successfully upload 8 MiB files.

I decided to check if default value of max-fds has changed between the releases so I hacked lighttpd code to do:

log_error_write(srv, __FILE__, __LINE__, "sd", "server started, srv->max_fds:", srv->max_fds);

for both releases I'm getting the same max-fds value: 1024.

FYI I'm running OpenWrt distribution with:

# cat /proc/sys/fs/file-max
12561

So my next thought was it has to be some code change. I decided to bisect commits between 1.4.35 and 1.4.36 release and I found following one:
http://redmine.lighttpd.net/projects/lighttpd/repository/revisions/2976 Remove chunkqueue_get_{append,prepend}* API

Above commit introduced this problem I'm hitting.

My guess is that having 1024 descriptors limit should be enough to upload big files, especially when having only 1 user connecting to the server. Can there be some issue in lighttpd related to the descriptors usage?

If I'm wrong and this is not the case, maybe lighttpd should pick a bigger max-fds value by default? I guess uploading files bigger than 4 MiB may be a common use case and it may make sense to support it with the default configuration.

Associated revisions

Revision 3009 (diff)
Added by stbuehler over 1 year ago

fix out-of-filedescriptors when uploading "large" files (fixes #2660, thx rmilecki)

From: Stefan Bühler <>

Revision 1371c87c (diff)
Added by stbuehler over 1 year ago

fix out-of-filedescriptors when uploading "large" files (fixes #2660, thx rmilecki)

From: Stefan Bühler <>

git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@3009 152afb58-edef-0310-8abb-c4023f1b3aa9

History

#1 Updated by stbuehler over 1 year ago

  • Description updated (diff)
  • Target version set to 1.4.37

#2 Updated by stbuehler over 1 year ago

  • Status changed from New to Fixed
  • % Done changed from 0 to 100

Applied in changeset r3009.

#3 Updated by petar over 1 year ago

Upgraded from 1.4.35 to 1.4.37 and my text upload form (POST, CGI) stopped working with the same 413 error.

/var/log/messages said:

(chunk.c.471) denying upload as opening temp-file for upload failed: No such file or directory

lighttpd and the CGI script that receives the POST-data are both in a chroot without a tmp directory. That worked fine for many patch-releases so I didn't want to touch anything before asking here.

Are POST-uploads now written into /tmp or is the issue something else?

#4 Updated by petar over 1 year ago

Yes, the chroot was missing /var/tmp and for some reason I was able to get away without it before the recent rewrite of chunk.c.

Thanks

#5 Updated by stbuehler over 1 year ago

@petar: r3046 should "fix" your problem too, which is unrelated to the main issue here.

Also available in: Atom