Bug #2299
closedFragmentation is broken when using SSL
Description
Hello,
when trying to use SSL and sending one line per packet, lighttpd does not send a reply.
Sending
"GET / HTTP/1.0\r\n"
"Host: foo.org\r\n"
"User-Agent: foo\r\n"
"\r\n"
as 4 packets, there is no answer from lighttpd.
However, doing this works:
"GET / HTTP/1.0\r\n"
"Host: foo.org\r\n"
"User-Agent: foo\r\n\r\n"
(3 packets).
Updated by js about 13 years ago
Oh, forgot to mention: When using HTTP, it works. The problem only happens with HTTPS. I send exactly the same data using HTTP and HTTPS, but only HTTPS does not work.
Updated by js about 13 years ago
Not sure how similar it actually is. As said, it works when using HTTP, the problem only exists with HTTPS.
Updated by darix about 13 years ago
- Status changed from New to Need Feedback
can you try with 1.4.28?
Updated by js about 13 years ago
Unfortunately not as .26 is the newest version in pkgsrc.
Updated by js about 13 years ago
I updated lighttpd in pkgsrc to .28 now and sent in a patch. Still the same.
Updated by Olaf-van-der-Spek about 13 years ago
- Target version set to 1.4.29
What tool do you use to send the request?
Updated by js about 13 years ago
I wrote something myself (which sent each parameter in a separate packet, this is how I noticed it), but you can easily reproduce it this way:
Use openssl s_client -connect webkeks.org:443
Wait until it connected, type:
GET / HTTP/1.1
Host: webkeks.org
User-Agent: Foo
Now press enter twice. Nothing will happen.
Updated by Olaf-van-der-Spek about 13 years ago
You're right, I can confirm it on 1.4.28.
Updated by andreas_stoe about 13 years ago
As far as I see it, this is not really a bug. The HTTP 1.1 protocol requires the "connection" field to be set in every message:
HTTP/1.1 applications that do not support persistent connections MUST include the "close" connection option in every message.
If you add this to your request it will work in 1.4.28.
However, this bug existed in 1.4.26, but has been fixed. See http://redmine.lighttpd.net/issues/2197
Updated by stbuehler about 13 years ago
Testing with openssl s_client is dangerous, as your terminal probably doesn't send \r\n newlines. try it like this:
(printf 'GET / HTTP/1.1\r\nHost: webkeks.org\r\n\r\n'; cat) | openssl s_client -connect webkeks.org:44
Updated by js about 13 years ago
Sorry, but you both are wrong:
@andreas_stoebe: As you can see above, the request is HTTP/1.0, so this is not the issue. And even then, the problem is fragmentation and not that.
@stbuehler: This is exactly what you DON'T want to demonstrate the bug, as this does not do fragmentation. Actually typing it creates fragmentation because telnet sends as soon as it can. CRLF can be enabled in telnet, read the manpage. It's the default in mine.
Updated by stbuehler almost 13 years ago
js wrote:
Sorry, but you both are wrong:
[...]
@stbuehler: This is exactly what you DON'T want to demonstrate the bug, as this does not do fragmentation. Actually typing it creates fragmentation because telnet sends as soon as it can. CRLF can be enabled in telnet, read the manpage. It's the default in mine.
- That wasn't my point, my point was how to create valid requests
- I am pretty sure your telnet (and openssl s_client) is line buffered (and does NOT send characters as soon as it can)
- I doubt your telnet CRLF setting has any impact on openssl s_client
So here the complete command line to test how a server responds to line-splitted requests:
(printf 'GET / HTTP/1.1\r\n'; sleep 1; printf 'Host: webkeks.org\r\n'; sleep 1; printf '\r\n'; cat) | openssl s_client -connect webkeks.org:443
(this one works fine btw, so i don't see any bug here)
And before anyone asks what the
cat
is for: without the cat
, the stream gets closed, which means "cancel request" to a web server (if the web server happens to check for it).
Or pipe the request data through a shell script like this; choose a high enough sleep value so that the ACK for one character comes back before you send the next one:
IFS="" while read -N 1 c; do printf "%s" "$c" sleep 0.1 done
Updated by js almost 13 years ago
This will still send one packet as pipes are buffered.
As I already said above, I did not trigger the problem with s_client, but also with ObjC code I wrote. The problem is fragmentation, try it yourself using libssl. Do multiple SSL_send calls and you'll see what I mean.
Updated by js almost 13 years ago
Oh, and your example has another bug: As I said before, if only specifying Host:, it works. The problem happens as soon as you add one more field.
Guess next time I will just attach a .c file that generates the packets that cause the bug…
Updated by stbuehler almost 13 years ago
- Status changed from Need Feedback to Invalid
- Target version deleted (
1.4.29)
# (printf 'GET / HTTP/1.1\r\n'; sleep 1; printf 'Host: webkeks.org\r\n'; sleep 1; printf 'User-Agent: foo\r\n'; sleep 1; printf '\r\n'; sleep 5;) | ltrace -tt -s 256 -e read,write,SSL_read,SSL_write openssl s_client -connect webkeks.org:443 > /dev/null 15:17:43.308231 SSL_write(0xc1e0a0, 0xc13d60, 0, 8, 0depth=2 /C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority verify error:num=19:self signed certificate in certificate chain verify return:0 ) = 0 15:17:43.588224 read(0, "GET / HTTP/1.1\r\n", 8192) = 16 15:17:43.588590 SSL_write(0xc1e0a0, 0xc13d60, 16, 8, 0) = 16 15:17:44.229070 read(0, "Host: webkeks.org\r\n", 8192) = 19 15:17:44.229324 SSL_write(0xc1e0a0, 0xc13d60, 19, 8, 0) = 19 15:17:45.231389 read(0, "User-Agent: foo\r\n", 8192) = 17 15:17:45.231686 SSL_write(0xc1e0a0, 0xc13d60, 17, 8, 0) = 17 15:17:46.233613 read(0, "\r\n", 8192) = 2 15:17:46.233834 SSL_write(0xc1e0a0, 0xc13d60, 2, 8, 0) = 2 15:17:46.296820 SSL_read(0xc1e0a0, 0xc15d70, 1024, 8, 0) = 209 15:17:46.297504 write(1, "HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nAccept-Ranges: bytes\r\nETag: "3100938754"\r\nLast-Modified: Sun, 15 Mar 2009 18:50:51 GMT\r\nContent-Length: 1757\r\nDate: Fri, 01 Apr 2011 13:17:44 GMT\r\nServer: lighttpd\r\n\r\n", 209) = 209 15:17:46.308163 SSL_read(0xc1e0a0, 0xc15d70, 1024, 8, 0) = 1024 15:17:46.367021 write(1, "<?xml version="1.0" encoding="UTF-8"?>\n<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"\n "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">\n<html xmlns="http://www.w3.org/1999/xhtml">\n<head>\n<title>webkeks.org</title>\n<style type="text/css">\n<!--\nbody {\n\tbac"..., 1024) = 1024 15:17:46.367514 SSL_read(0xc1e0a0, 0xc15d70, 1024, 0, 0) = 733 15:17:46.367648 write(1, "-size: 48px;\n\tfont-weight: bold;\n}\n//-->\n</style>\n</head>\n<body>\n<!--\n<div id='menubar'>\n<span class='menu'><a href='/blog'>Blog</a>\n</span><span class='menu'><a href='/jabber'>Jabber</a>\n</span><span class='menu'><a href='/jwchat'>JWChat</a>\n</span><span "..., 733) = 733 15:17:51.236407 read(0, "", 8192) = 0 DONE 15:17:51.238166 +++ exited (status 0) +++
"pipes are buffered" - doesn't sound like you have any idea what this is about.
Cannot reproduce, closed.
Updated by js almost 13 years ago
- Target version set to 1.4.29
Ok, indeed, the fragmentation bug seems to be fixed. When I upgraded, I just tried with OpenSSL, which seems to actually only send \n, unlike telnet. So, the problem was that I did not correctly verify that the bug from .26 still exists. I just wrote a C program which tests it and indeed it works since .29.
About the pipes are buffered stuff: Use for example find | grep and you will see that not each time find outputs a line that would match it is displayed, this is due to pipe buffering. It seems that in this case it was not buffered, though.
Updated by icy almost 13 years ago
Pipes might get buffered but they don't wait 1 second :)
Updated by stbuehler almost 13 years ago
- Target version deleted (
1.4.29)
.29 is not released. perhaps your bug was related to #2197. target version without a commit that fixed it doesn't make sense.
and that is why i think you have no idea what "pipes are buffered" means. It means that a sender can finish write to it without a client reading from it at the same time (unless the buffer is full). but if there is data in the pipe and a client wants to read it will always get the data; it just has to wait to get some time on the cpu of course, which is why you sometimes will get more than one write() block in a read() when the cpu was too busy.
Also available in: Atom