Project

General

Profile

WebDAV: support for PATCH / partial uploads?

Added by rk over 4 years ago

Is there any plan to implement WebDAV PATCH + Content-Range (for partial uploads), e.g. in a way PUT worked before 1.4.54?

Or will

webdav.opts = ( "deprecated-unsafe-partial-put" => "enable" )

be the permanent workarount?


Replies (4)

RE: WebDAV: support for PATCH / partial uploads? - Added by gstrauss over 4 years ago

Please read the commit message in commit d334ee5c
The words "deprecated" and "unsafe" should be strong deterrents to the use of deprecated-unsafe-partial-put

(reposted from #2994)

RE: WebDAV: support for PATCH / partial uploads? - Added by gstrauss over 4 years ago

When I rewrote lighttpd mod_webdav, I endeavored to make filesystem operations atomic wherever possible. Files uploaded via PUT are written into temporary files and atomically moved into place. Any existing downloads in progress proceed unaffected using the prior version of the file, by holding an open file descriptor.

The deprecated-unsafe-partial-put is not implemented atomically, and like the historical behavior, writes directly into the existing file, which can corrupt any downloads in progress. lighttpd mod_webdav does support If-Match and If-Unmodified-Since, so that can help protect against accidentally corrupting a file with a partial PUT when the file has changed from the copy on the client from which the partial PUT was generated.

I could have re-implemented the partial PUT to first copy the existing file, then patch it, and finally rename it back into place. Doing so on a very large file might be an expensive operation. The current implementation of lighttpd mod_webdav uses atomic filesystem operations (except for partial PUT) and therefore does not need to take locks to prevent reads while modifications are occurring in temporary files. Similarly, the writer does not need to wait for existing downloads to complete before being able to get a write lock to modify the file in-place. While lighttpd does support WebDAV exclusive locking, lighttpd does not track existing readers who may already be in the process of downloading a very large file.

Partial PUT support is not safe, and is explicitly forbidden by RFC 7231. lighttpd mod_webdav deprecated-unsafe-partial-put is named in an attempt to communicate that partial PUT is both "deprecated" and "unsafe". The option is not enabled by default.

RE: WebDAV: support for PATCH / partial uploads? - Added by rk over 4 years ago

I agree, that atomic partial/resumable uploads are difficult.
But partial/resumable uploads are really useful, even if they are not atomic. And I would like to use them with WebDAV, instead of having to run a separate server/CGI/whatever.
(I personally would prefer non-atomic-uploads over expensive copy-patch-operations; and I don't find it problematic that downloads are corrupted, when the downloaded file is currently uploaded.)

I'm currently using mod_webdav + PUT + Content-Range (+deprecated-unsafe-partial-put) to upload data via WebDAV from a browser.
In this case, I need PUT+Content-Range, since (a) the browser cannot handle a single large PUT, but needs to break it down into smaller chunks and (b) very large HTTP packets are ugly and problematic. In addition, PUT+Content-Range makes it possible to resume uploads, which is especially useful for large files.
To prevent corrupted downloads (and know when a upload is complete), I upload a file as FILENAME.part, and rename it to FILENAME after the upload has completed. This works quite well. (But this is the task of the WebDAV-client, not the server.)

Is there any alternative to the (deprecated) PUT+Content-Range with lighttpd to achieve this?
Would it be possible/allowed to add the PATCH-method with Contant-Range to mod_webdav?

About RFC 7231: As far as I understand it, they forbid PUT+Content-Range because they think that it is used accidentally, and not because they dislike the behaviour. If the "old" mod_webdav PUT+Content-Range would be renamed to PATCH+Content-Range, this would be completely RFC 7231-compliant, right?

RE: WebDAV: support for PATCH / partial uploads? - Added by gstrauss over 4 years ago

that atomic partial/resumable uploads are difficult.

Not difficult. Potentially expensive. Have you considered rsync? Have you considered uploading stupidly large files to an upload directory via a CGI script which supports resumable uploads, and then renaming such files into place?

Is there any alternative to the (deprecated) PUT+Content-Range with lighttpd to achieve this?

You seem to have misunderstood. "deprecated-unsafe-partial-put" => "enable" is the prior behavior, and it is both deprecated and unsafe. I added the option for people to be able to enable the prior UNSAFE behavior.

I don't find it problematic that downloads are corrupted

That's nice. I do find it problematic. That's why the option is labelled deprecated and unsafe.

Would it be possible/allowed to add the PATCH-method with Contant-Range [sic] to mod_webdav?

Possible, yes, but I have little interest in spending the large amount of time that would be needed to do so without some compelling evidence that it would be used by more than one person. Sorry.

    (1-4/4)