Bug #1548: range requests not working - Lighttpd - lighty labs

Actions

Copy link

Bug #1548

closed

range requests not working

Added by Anonymous almost 17 years ago. Updated over 8 years ago.

Status:

Obsolete

Priority:

High

Category:

core

Target version:

1.5.0

ASK QUESTIONS IN Forums:

Description

Test case I use:
wget -t1 --debug http://site.com/content/movies/pretty_big.wmv --header="Range:bytes=100-200" -O wget.out

Result: wget.out remains empty.


---request begin---
GET /content/movies/pretty_big.wmv HTTP/1.0
User-Agent: Mozilla/5.0 (compatible; Panther)
Accept: */*
Host: site.com
Connection: Keep-Alive
Range: bytes=100-200
X-Panther: Validate

---request end---
HTTP request sent, awaiting response... 
---response begin---
HTTP/1.0 206 Partial Content
Cache-Control: max-age=31536000
Cache-Control: max-age=31536000
Cache-Control: private
X-Powered-By: PHP/5.2.0-8+etch10
Pragma: public
Expires: 0
Content-Type: application/force-download
Content-Disposition: attachment; filename="pretty_big.wmv";
Content-Transfer-Encoding: binary
ETag: "1615866504" 
Accept-Ranges: bytes
Last-Modified: Fri, 14 Dec 2007 12:53:52 GMT
Content-Range: bytes 100-200/2553951618
Content-Length: 101
Connection: keep-alive
Date: Fri, 01 Feb 2008 08:57:00 GMT
Server: lighttpd/1.5.0

---response end---
206 Partial Content
Registered socket 5 for persistent reuse.
Disabling further reuse of socket 5.
Closed fd 5
Giving up.

strace: attached

Nothing I can see in the logs.
The requests are rewritten to go through a small php script to facilitate authentication.
I tried the test with large files, small files, with authorizer and without,
network-backends linux-sendfile and gthread-aio. Same result.

The code was checked out of trunk on 2008_01_22
I added a patch from this site that enables setting the basis of etag generation. I also use this build in clusters. I'll attach that patch too.

I enabled request debugging and will attach the output of that as well.

Logs have been sanitized to change paths and camouflage host names and such. I did not intentionally remove debugging info. If the logs look consistent I am happy to provide the originals. (in a less open channel)

If you need more information don't hesitate to contact me.

Thanks in advance for looking at the ticket.

-- oscar

Files

Download all files

light.trace (9.59 KB) light.trace	strace from a byte-ranged get request	Anonymous, 2008-02-01 09:28
actualized_etags.patch (5.53 KB) actualized_etags.patch	etags patch I applied	Anonymous, 2008-02-01 09:29
lighty.log (3.73 KB) lighty.log	debug log output	Anonymous, 2008-02-01 09:30

Actions

Copy link

Updated by Anonymous almost 17 years ago

huh... I just tested with a 1.4.18 server. I don't ever get anything in the wget.out file.

I wonder if my test is broken...

-- oscar

Actions

Copy link

Updated by Anonymous almost 17 years ago

ughh. I did the test using curl and I am getting the partial transfers ok.
Let's put this on hold while I figure out what's going on.

Actions

Copy link

Updated by Anonymous almost 17 years ago

Found one possible culprit:

doing the test the following way:
--header "Range:bytes=2147483650-2553951618"
fails. Any address range starting above 2G fails!

This time however I found some noise in the log:


network_gthread_aio.c.280: (error) thread returned: 2
connections.c.1381: (trace) (network-subsys sent us a fatal-error)
network_gthread_aio.c.186: (error) reading file failed: 22 (Invalid argument)
network_gthread_aio.c.280: (error) thread returned: 2
connections.c.1381: (trace) (network-subsys sent us a fatal-error)
network_gthread_aio.c.186: (error) reading file failed: 22 (Invalid argument)
network_gthread_aio.c.280: (error) thread returned: 2
connections.c.1381: (trace) (network-subsys sent us a fatal-error)

Good news is: If I switch to linux-sendfile backend the problem disappears.
gthread-aio didn't appreciate this particular task.

So I have a workaround, how can I help creating a fix?

-- oscar

Actions

Copy link

Updated by Anonymous almost 17 years ago

confirmed this is a > 2gb issue on my debian etch box w/ libaio.so.1.0.1, kernel 2.6.18-4-486, lighty 1.5.0 r1992:


network_gthread_aio.c.186: (error) reading file failed: 22 (Invalid argument)
network_gthread_aio.c.280: (error) thread returned: 2
connections.c.1370: (trace) (network-subsys sent us a fatal-error)

when requesting range beyond 2 gig.

this is high priority for me as i need aio to avoid constant i/o bottleneck ...

-- njahnke

Actions

Copy link

Updated by Anonymous almost 17 years ago

edit network_gthread_aio.c

change occurrence of pread() to pread64()

recompile

-- grenola

Actions

Copy link

Updated by Anonymous almost 17 years ago

thanks grenola. i've confirmed this works on my debian box. please integrate into svn (we're too lazy) :P

-- njahnke

Actions

Copy link

Updated by stbuehler over 15 years ago

If you compile with large-file-support (enabled by default), pread should be the right function. I don't think we should use pread64 directly (pread with lfs will probably use a "pread64" syscall).

Actions

Copy link

Updated by darix over 15 years ago

Status changed from New to Need Feedback

can any of the commenters please mention if they had their glib2 and lighttpd compiled with largefile support (LFS) and if the bug also triggered with LFS enabled?

Actions

Copy link