Lighttpd

I came across the same issue today. I have not yet got a chance to investigate the issue. I will probably do so in the next few weeks. lameventanas did you look into the problem?

jpc wrote:

I came across the same issue today. I have not yet got a chance to investigate the issue. I will probably do so in the next few weeks. lameventanas did you look into the problem?

I didn't look into this, instead I implemented it in PHP.
But please feel free to investigate, fix and post a patch here :)

stbuehler wrote:

It is your job to generate the appropriate headers; this ensures the solution can be used in other places too (for example pseudo seeking which uses the query string instead of the range http header)

This is the header I am generating:
X-Sendfile2: %2Fvar%2Fwww%2Fuploads%2Fi7b35952163f845f4%2Fasterisk.pdf 4428831-

Why is this incorrect?

stbuehler wrote:

Where did we ever claim that X-Sendfile2 implements the HTTP range spec? It is named "X-Sendfile2", not "Range" or "Content-Range".

The lighttpd documentation is not extensive on the topic and while it never claims to implement the HTTP spec for resumable download, it never claims the opposite! Hence, my assumption that I could leverage X-Sendfile2 to support the wget -c command.

Now, thanks to your last comment, I was able to make this work (sort of). In case someone comes to this issue: a possible workaround is to ignore the HTTP spec and use a GET parameter instead.

Take the following PHP snippet:

$file = '/tmp/big-file.bin';
if (isset($_GET['restart-at']) && (int)$_GET['restart-at']>0) {
header('X-Sendfile2: '.$file.' '.$_GET['restart-at'].'-');
} else {
header('X-Sendfile: '.$file);
}

If you run the command wget http://localhost/test1.php?restart-at=500000 it will start the download at byte 500000.

Now, the problem with not following the HTTP spec, means that doing a wget -c will not work properly. I used the following PHP script:

$file = '/tmp/big-file.bin';
if (isset($_SERVER['HTTP_RANGE']) && preg_match('/\Abytes=[0-9]+-[0-9]*\z/', $_SERVER['HTTP_RANGE'])) {
header('X-Sendfile2: '.$file.' '.substr($_SERVER['HTTP_RANGE'],6));
} else {
header('X-Sendfile: '.$file);
}

• I can see that only the second part of the file is downloaded the 2nd time I run wget (which is what I wanted)
• But at the end, it's not properly attached to the first part of the file (which was retrieved by the first wget)

Try sending a 206 status code and the Content-Range header from your backend.

Ok thanks to the clarification I was able to make my code work.

I assumed that lighttpd took care of the status and content-range headers because so many things could go wrong with the sendfile() call (ie: file doesn't exist, permission denied, range not satisfiable, etc).
What happens in these cases? Does lighttpd overwrite the headers in these cases? Or is there any way for the backend to know about these error conditions?

Lighttpd definitely needs better documentation.

icy wrote:

Try sending a 206 status code and the Content-Range header from your backend.

Thank you for the pointer!

lameventanas wrote:

Ok thanks to the clarification I was able to make my code work.

Me too. I generate the following headers and it works when I run wget -c:

header('Content-Range: bytes 2375680-12103815/12103816');
header('HTTP/1.1 206 Partial content');
header('X-Sendfile2: /tmp/big-file.bin 2375680-');


Lighttpd definitely needs better documentation.

I added an example on the wiki. Hopefully, it'll help the next person having the problem :)