Bug #2588

Problem when uploading large files

Added by dirk4000 about 3 years ago. Updated over 1 year ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:
Missing in 1.5.x:


The lighttpd webserver has a nice feature making it possible to specify several upload directories. If one directory is full the next directory will be taken. But there is a small implementation issue that has to be fixed to make this feature fully operational. If you look at the implementation in chunkqueue_get_append_tempfile (chunk.c) you will see that the check is only done at file creation time (calling of mkstemp), but what would happen if the file can be created but it is not enough storage available (e.g. 2 bytes left on that storage directory) to write the chunk ? .... an 413 error will occur. For the user this behavior is not understandable. Imagine that you work on an embedded system (like in our project) where the ram disk is very fast (100 MB/sec) but the persistent storage is very slow (1 MB/s) then you would love to use this feature. First write the data to the very fast ram disk, if the storage is full use the next storage directory (and so on). But there has to be done a small fix. I would propose to ensure that the storage space is really available by calling the allocate-function (posix_allocate on posix systems / fallocate on linux system), see the patch attached

chunk.c.patch (1.36 KB) chunk.c.patch Patch to ensure that the space to store the chunk data is available dirk4000, 2014-08-08 08:53
connections.c.patch (413 Bytes) connections.c.patch Patch to prevent that a chunk overrun occurs dirk4000, 2014-08-08 08:54

Associated revisions

Revision 77bd4512 (diff)
Added by gstrauss over 1 year ago

[core] retry tempdirs on partial write, ENOSPC (fixes #2588)

Previous code would fail on partial write, EINTR, and ENOSPC.
Upon any of the above errors, this patch tries next tempdir in list,
if list of tempdirs provided by config option server.upload-dirs

"Problem when uploading large files"

Closes #54


#1 Updated by stbuehler about 2 years ago

  • Target version changed from 1.4.36 to 1.4.37

#2 Updated by stbuehler about 2 years ago

  • Assignee deleted (stbuehler)

I'd rather handle the issue when it occurs instead of trying to prevent it through some magic (pre)allocate functions which might work or not.

#3 Updated by stbuehler almost 2 years ago

  • Target version changed from 1.4.37 to 1.4.x

#4 Updated by gstrauss over 1 year ago

  • Status changed from New to Patch Pending
  • Target version changed from 1.4.x to 1.4.40

dirk4000: would you give this patch a try and let us know if it resolves your issue? Thanks.

#5 Updated by gstrauss over 1 year ago

  • Status changed from Patch Pending to Fixed

Also available in: Atom