https://redmine.lighttpd.net/https://redmine.lighttpd.net/favicon.ico?13667327412010-02-03T13:33:26Zlighty labsLighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=67632010-02-03T13:33:26Zmorgan
<ul><li><strong>Target version</strong> changed from <i>1.4.x</i> to <i>1.4.26</i></li></ul><p>Looks like the 1.4.26 is the place to be, so moving request to 1.4.26 queue.</p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=67642010-02-03T13:37:24Zstbuehler
<ul><li><strong>Target version</strong> changed from <i>1.4.26</i> to <i>1.4.x</i></li></ul><p>Sorry, no. Of course this would be useful, but i don't see it coming in 1.x; the code assumes in too many places to know the content-length of the upload after parsing the request headers.</p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=69122010-04-29T16:00:35Zandreas_stoe
<ul><li><strong>File</strong> <a href="/attachments/1139">lighttpd_1.4.26_chunked_decoding2.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/1139/lighttpd_1.4.26_chunked_decoding2.patch">lighttpd_1.4.26_chunked_decoding2.patch</a> added</li></ul><p>Hi,</p>
<p>the attached "hack" adds support for basic "transfer-encoding: chunked" request handling. Chunked data which is sent to the server will be decoded using a small library (chunk_decoder.c, chunk_decoder.h) and is finally stored in tempfiles. When all chunks have been read, con->request.content_length is set accordingly. So all further processing steps in the webserver will believe data has been sent with a "content-length" header.</p>
<p>I hope this patch might be useful,<br />Andreas</p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=69352010-05-10T09:29:40Zandreas_stoe
<ul><li><strong>File</strong> <a href="/attachments/1140">lighttpd_1.4.26_chunked_decoding3b.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/1140/lighttpd_1.4.26_chunked_decoding3b.patch">lighttpd_1.4.26_chunked_decoding3b.patch</a> added</li></ul><p>My previous patch contained a serious bug which might have caught the web-server to stop responding if a connection got closed abruptly. I also added support for hex numbers written with small letters.</p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=70382010-06-15T11:48:04Zpprkut
<ul></ul><p>I tried your patch, but I couldn't get lighttpd to compile with it. Here's the error I get:</p>
<p>libtool: link: gcc -g -O2 -Wall -W -Wshadow -pedantic -std=gnu99 -o lighttpd server.o response.o connections.o network.o configfile.o configparser.o request.o proc_open.o buffer.o log.o keyvalue.o chunk.o http_chunk.o stream.o fdevent.o stat_cache.o plugin.o joblist.o etag.o array.o data_string.o data_count.o data_array.o data_integer.o md5.o data_fastcgi.o fdevent_select.o fdevent_linux_rtsig.o fdevent_poll.o fdevent_linux_sysepoll.o fdevent_solaris_devpoll.o fdevent_freebsd_kqueue.o data_config.o bitset.o inet_ntop_cache.o crc32.o connections-glue.o configfile-glue.o http-header-glue.o network_write.o network_linux_sendfile.o network_freebsd_sendfile.o network_writev.o network_solaris_sendfilev.o network_openssl.o splaytree.o status_counter.o -Wl,--export-dynamic -L/usr/lib64 /usr/lib64/libpcre.so -ldl /usr/lib64/libfam.so -Wl,-rpath -Wl,/usr/lib64 -Wl,-rpath -Wl,/usr/lib64<br />connections.o: In function `connection_handle_read_state':<br />/mnt/progs/slack/test/lighttpd-1.4.26/src/connections.c:1200: undefined reference to `chunk_decoder_work'<br />/mnt/progs/slack/test/lighttpd-1.4.26/src/connections.c:1336: undefined reference to `chunk_decoder_free'<br />/mnt/progs/slack/test/lighttpd-1.4.26/src/connections.c:1283: undefined reference to `chunk_decoder_free'<br />/mnt/progs/slack/test/lighttpd-1.4.26/src/connections.c:1186: undefined reference to `chunk_decoder_create'<br />connections.o: In function `connection_reset':<br />/mnt/progs/slack/test/lighttpd-1.4.26/src/connections.c:806: undefined reference to `chunk_decoder_free'<br />connections.o: In function `connection_close':<br />/mnt/progs/slack/test/lighttpd-1.4.26/src/connections.c:127: undefined reference to `chunk_decoder_free'<br />collect2: ld returned 1 exit status<br />make<sup><a href="#fn3">3</a></sup>: *** [lighttpd] Error 1</p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=79242012-08-17T12:43:39Zlvs
<ul><li><strong>File</strong> <a href="/attachments/1391">lighttpd_1.4.29_chunked_decoding.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/1391/lighttpd_1.4.29_chunked_decoding.patch">lighttpd_1.4.29_chunked_decoding.patch</a> added</li></ul><p>I attach the current version of the patch. It applies to 1.4.31, please don't mind the name.</p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=92732016-03-31T01:08:30Zgstrauss
<ul></ul><p>FYI: recent patch to lighttpd will communicate to client 411 Length Required, so while Transfer-Encoding: chunked is still not supported, the server and client should behave better when the server sends 411 response.<br /><pre>
commit 06d3c754403b2df99775641e05999ea9b2081618
Author: Glenn Strauss <gstrauss@gluelogic.com>
Date: Sat Mar 26 12:58:33 2016 +0000
[core] respond 411 Length Required if request has Transfer-Encoding: chunked (fixes #631)
lighttpd does not currently support request body transfer-codings
From: Glenn Strauss <gstrauss@gluelogic.com>
git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@3128 152afb58-edef-0310-8abb-c4023f1b3aa9
</pre></p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=94412016-04-21T18:10:48Zgstrauss
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Need Feedback</i></li></ul><p><a class="external" href="https://github.com/lighttpd/lighttpd1.4/pull/53">https://github.com/lighttpd/lighttpd1.4/pull/53</a> moves request body reading to connection_handle_read_post_state(). This location might be enhanced to support chunking (and then request.c modified to not reject chunked POST requests with 411 Length Required)</p>
<p>However, this is low priority and might not be worth the effort if clients are responding properly to 411 Length Required.</p>
<p>Is anyone having issues with clients that do not properly handle 411 Length Required?</p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=94952016-05-01T06:00:26Zgstrauss
<ul><li><strong>Status</strong> changed from <i>Need Feedback</i> to <i>Missing Feedback</i></li><li><strong>Priority</strong> changed from <i>Normal</i> to <i>Low</i></li></ul><p>While supporting Transfer-Encoding: chunked for request body would be a nice feature to have, to increase the priority of this feature request, someone would need to present a (non-contrived) use case where this was really needed.</p>
<p>Well-behaved clients should handle PUTs properly now that lighttpd responds with 411 Length Required (which is compliant with the RFCs) if client sends Transfer-Encoding: chunked.</p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=102532016-07-16T12:50:50Zstbuehler
<ul><li><strong>Target version</strong> deleted (<del><i>1.4.x</i></del>)</li></ul> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=106232016-11-24T12:47:33Zflynn
<ul></ul><p>Any progress on this issue?</p>
The relevant usecase is:
<ul>
<li>webdav with lighttpd/php/sabredav</li>
<li>OSX finder as client</li>
</ul>
<p>This combinations results in PUT request with chunked encoding: <a class="external" href="http://sabre.io/dav/clients/finder/">http://sabre.io/dav/clients/finder/</a></p>
<p>I would offer extended testing, if a patch is provided.</p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=106252016-11-24T13:28:22Zgstrauss
<ul><li><strong>Status</strong> changed from <i>Missing Feedback</i> to <i>Reopened</i></li></ul><p>Thanks for noting your interest and the offer to help test.</p>
<p>Modifying connection_handle_read_post_state() to handle chunked encoding is the central place to implement it. However, con->request.content_length is used in more than a few places to flag the presence of content or not. Those locations would need to be updated to handle delayed content, including sometimes handling delayed content of 0 octets.</p>
<p>While none of this is very difficult, it will take some effort. I'll re-open this feature request, but the priority is still "low" until someone finds some time to do it.</p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=106662016-12-12T00:30:02Zgstrauss
<ul><li><strong>File</strong> <a href="/attachments/1753">lighttpd_1.4.43_chunked_prep.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/1753/lighttpd_1.4.43_chunked_prep.patch">lighttpd_1.4.43_chunked_prep.patch</a> added</li></ul><p>Uploading patch to be applied against lighttpd git master HEAD.</p>
<p>This patch changes existing modules to be aware that Transfer-Encoding: chunked may be used, resulting in an unknown Content-Length of request body, where previously the length of request body was known up-front. For most modules, this means returning 411 Length Requiredt to preserve existing behavior.</p>
<p>For modules handling requests that are not configured to stream the request body to the backend, which collect all content prior to starting the backend, it is now possible (but not yet coded) for request body to be collected from client and de-chunked.</p>
<p><strong>Before this patch is committed, it needs some wider testing to ensure that existing behavior has not been broken.</strong> Specific behavior that needs testing is POST of request body to various dynamic handlers (mod_cgi, mod_fastcgi, mod_scgi, mod_proxy, mod_webdav).</p>
<p>Future work is largely isolated in connections-glue.c:connection_handle_read_post_state(), which needs to keep chunked state as it reads and decodes input from client from con->read_queue to con->request_content_queue.</p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=106742016-12-16T22:04:28Zgstrauss
<ul><li><strong>Status</strong> changed from <i>Reopened</i> to <i>Patch Pending</i></li><li><strong>Target version</strong> set to <i>1.4.44</i></li></ul><p>flynn wrote:</p>
<blockquote>
<p>I would offer extended testing, if a patch is provided.</p>
</blockquote>
<p>flynn, please test Transfer-Encoding: chunked support in using personal/gstrauss/master branch in lighttpd git repo:<br />(<a class="wiki-page" href="https://redmine.lighttpd.net/projects/lighttpd/wiki/DevelGit">DevelGit</a>)<br />$ git clone <a class="external" href="https://git.lighttpd.net/lighttpd/lighttpd1.4.git">https://git.lighttpd.net/lighttpd/lighttpd1.4.git</a></p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=106782016-12-19T18:55:04Zgstrauss
<ul><li><strong>Status</strong> changed from <i>Patch Pending</i> to <i>Fixed</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li></ul><p>Applied in changeset <a class="changeset" title="[core] support Transfer-Encoding: chunked req body (fixes #2156) support Transfer-Encoding: chun..." href="https://redmine.lighttpd.net/projects/lighttpd/repository/14/revisions/4d7f5737f150c4bab8d3832bab9da62eddfea7d2">4d7f5737f150c4bab8d3832bab9da62eddfea7d2</a>.</p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=107392017-01-08T20:54:54Zgstrauss
<ul></ul><p>FYI: a patch was provided for this feature a month ago, but no feedback was received. I added tests to the test suite and released the feature (Transfer-Encoding: chunked for HTTP request body, e.g. HTTP PUT)</p>
<p>Separately, a regression occurred in mod_webdav in lighttpd 1.4.43 which led to error trace in the log, and a different change turned this into a crashing bug in lighttpd 1.4.44. Now then, the bug is my fault, not yours, and I'll be adding tests to the test suite to attempt to avoid such mod_webdav issues in the future. Still, I am disappointed that I received no such feedback from you for either lighttpd 1.4.43 or lighttpd 1.4.44 after you seemed so eager for the new feature.</p>
<p>x-ref:<br />Lighttpd 1.4.43/44 dies - missing cleanup in webdav<br /><a class="external" href="https://redmine.lighttpd.net/boards/2/topics/7034">https://redmine.lighttpd.net/boards/2/topics/7034</a></p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=110832017-09-21T16:03:33Zflynn
<ul></ul><p>Sorry for not testing your patch, but OSX in newer versions does not use chunked PUT-request anymore.</p>
<p>I just tested some uploads, all PUT-requests are now with Content-Length. Tested with</p>
<pre>
server.stream-response-body = 1
server.stream-request-body = 0
server.reject-expect-100-with-417 = "disable"
</pre>
<p>I do not see any 411 Length Required response.</p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=113032018-02-19T10:07:20Zflynn
<ul></ul><pre>
server.stream-request-body = 0
</pre><br />is necessary for chunked PUT requests!
<p>Verified now with OSX finder and <br /><pre>
debug.log-request-header = "enable".
</pre><br />Works perfect now.</p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=113072018-02-19T15:28:49Zgstrauss
<ul></ul><p>That should not be necessary after the patch in <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: chunked transfer encoding in request body only works for tiny chunks (Fixed)" href="https://redmine.lighttpd.net/issues/2854">#2854</a> is released in lighttpd 1.4.49.</p>
<p>[Edit]: Correction: <code>server.stream-request-body = 0</code> is needed with requests that send "Transfer-Encoding: chunked" since lighttpd will return "411 Length Required" before passing to backends, as the CGI spec requires setting CONTENT_LENGTH. mod_proxy is not a full HTTP/1.1 reverse proxy, and also returns 411 Length Required.</p>
<p>Applications like <code>curl</code> correctly handle 411 Length Required, and resend the request with Content-Length, e.g.<br /><code>curl -T sample.txt -H "Transfer-Encoding: chunked" http://localhost:8080/cgi-bin/upload.pl</code></p> Lighttpd - Feature #2156: request: support Chunked Transfer Coding for HTTP PUThttps://redmine.lighttpd.net/issues/2156?journal_id=113452018-03-07T03:46:52Zgstrauss
<ul></ul><p>If you want <code>server.stream-request-body</code> enabled when "Content-Length" is set, but want to workaround when "Transfer-Encoding: chunked" is set, then use this in lighttpd.conf:<br /><pre>
$REQUEST_HEADER["Content-Length"] == "" { server.stream-request-body = 0 }
</pre></p>