Feature #1814
closedsupport for accelerated upload support (REQUEST_BODY_FILE)
Description
please consider adding support for accelerated upload support:
(in russian):
http://php-fpm.anight.org/extra_features.html#accelerated_upload
(in english):
http://translate.google.com/translate?u=http://php-fpm.anight.org/current_php_fastcgi_problems.html#accelerated_upload
according to that document nginx 0.5.9+ already supports this feature.
as i see it, is to add request-body-file => "enable" or something like to fastcgi.server definition,
and when option is enabled, keep uploads to temp file instead of feeding to backend fcgi script
and remove the temp file when request is served.
as afaik lighttpd buffers uploads to different temp files, that should be saved to single temp file.
also this support would require that lighttpd and fcgi server use shared storage (either running on same machine, or using nfs for shared storage)
Updated by darix over 15 years ago
i think lighttpd cant delete the file. the fastcgi backend needs to do that.
we planned a feature like that since quite some time already.
Updated by peto over 15 years ago
From what I can tell, this stores the entire, unprocessed request body to a file. How does this improve performance? Reading the data out of the temporary files and writing it over the pipe to an FCGI handler is going to be just about as fast as having the FCGI handler read the file directly. (There's a performance difference, but it's the same difference as read/write vs. sendfile--for file uploads, this isn't likely to be a factor.)
It would be a clear improvement if multipart/form-data could be decoded within lighttpd and split out into separate parts, and the content parts of the request replaced with placeholders pointing to the location of the files. The first thing many CGI handlers are going to do is exactly that, so it'd often save a whole pass over the files; and if it's outputting to the same partition where the files are going to end up stored (this would take extra work), it could avoid a second write completely as the file could just be moved. But that's significantly more complicated (streaming multipart/form-data decode, maintaining its structure, encoding filenames, doing so securely, and then supporting it on the FCGI side). I doubt it's worth it within the reasonable limits of unresumable POST uploads (maybe a hundred megs or so).
Updated by glen over 15 years ago
darix: if lighttpd created it, surely it can delete it later too
Updated by gstrauss almost 8 years ago
- Description updated (diff)
- Status changed from New to Need Feedback
- Priority changed from Normal to Low
Anybody have links to current documentation? Judging by how little I found in search engines, I am not sure this request is still relevant.
This is the best explanation I found: https://blog.martinfjordvald.com/2010/08/file-uploading-with-php-and-nginx/
Anyway, now that mod_fastcgi can receive response asynchronously from sending request body, enabling "accelerated upload support" might be possible simply by collecting input into a single temporary file (e.g. by allowing configurable max size of tempfile beyond that currently allowed by chunkqueue_set_tempdirs()), and then (somehow) passing the path to the temporary file to PHP via fastcgi. Is there a convention for how to do this? Maybe add a header to the request?
For compatibility, lighttpd could still send the request body via fastcgi. If the PHP never reads the request body via fastcgi, then the temporary file will be cleaned up at the end of the request, which is the desired behavior.
Updated by gstrauss almost 8 years ago
- Status changed from Need Feedback to Missing Feedback
https://manuel.mausz.at/coding/mod_phpfpm_handler/
So PHP-FPM kicks in after the file has successfully been uploaded. This is also known as "accelerated upload support"
If that's the definition of "accelerated upload support", then lighttpd already does this. lighttpd stores request body into temporary files and waits until request body is fully uploaded before it kicks off request to the backend. This avoids tying up expensive backend resources while waiting for the request body to be received from the client.
Now, it appears that the issue requestor in this ticket wants to additionally communicate the path to a single temporary file (containing the (possibly encoded) entire request body) to the backend script, instead of having the contents of the temporary file sent over the socket to the backend. Please re-open this ticket if you can provide pointers to documentation on the conventions used to support this in PHP or other scripting languages.
Updated by gstrauss about 2 months ago
- ASK QUESTIONS IN Forums set to No
Ran across a non-standard nginx plugin for resumable uploads.
https://github.com/fdintino/nginx-upload-module/blob/master/upload-protocol.md
https://github.com/fdintino/nginx-upload-module
https://github.com/vkholodkov/nginx-upload-module
https://github.com/masterzen/nginx-upload-progress-module
https://blog.martinfjordvald.com/file-uploading-with-php-and-nginx/
The feature for resumable uploads could be implemented by the backend, and the backend could be run under PHP-FPM and handle requests fairly quickly as it aggregated file upload segments. Since lighttpd does not stream the request body to the backend by default, instead performing request offloading, the backend would be contacted with the segment only once the segment was completely uploaded to lighttpd.
Overall, this is already possible to do in the backend application, and has been possible for a very long time. There would not be much difference if lighttpd were to do more of the file aggregation versus having an half-decent PHP backend running under PHP-FPM (to reduce PHP startup overhead) do the file aggregation.
Also available in: Atom