Project

General

Profile

Bug #2594

Accept LF instead of CRLF as header line end from proxy backends

Added by omegasteffy over 2 years ago. Updated about 1 year ago.

Status:
Fixed
Priority:
Normal
Assignee:
-
Category:
mod_proxy
Target version:
Start date:
2014-09-12
Due date:
% Done:

100%

Missing in 1.5.x:
No

Description

I was trying to setup a simple reverse proxy forward port 82 on the lighttpd server to port 80 on an backend server.

The welcome page works fine, but after i post a few variables i get 0 byte back (resulting in a download file dialog)
A wireshark dump shows that lighttpd get a fine html response from the backend server.
I also tried the url-rewrite trick, (proxy-ip/serverA/ instead of proxy-ip:82)

I tried with 1.4.13 and 1.4.35.
I suspected configuration, but after several attempts i guess it might be a known bug.
Lighttpd 1.5 works but is not current a possibility for the proxy server platform.

I know this bug is not specific, but please guide me
I would really appreciate patches or work-around.

Attachments
Config (IPs might have been changed since)
a-Response which get forwarded
b-Response which is NOT.

NOTE: It seem the two proxy responses get concatenated by wireshark
Your should be able to determine how it fits together.

proxy_test.conf View - minimum configuration for a proxy (1.12 KB) omegasteffy, 2014-09-12 08:42

b_not_forwarded_backend_resp - Reponse sent to lighttpd, but not forwarded (3.76 KB) omegasteffy, 2014-09-12 08:43

a_forwarded_proxy_resp - OK - response given by proxy to client (1.52 KB) omegasteffy, 2014-09-12 08:43

b_not_forwarded_proxy_resp - The "b_not_forwarded_backend_resp" is not sent (1.52 KB) omegasteffy, 2014-09-12 08:43

a_forwarded_backend_resp - ok response for first paeg (1.01 KB) omegasteffy, 2014-09-12 08:47

Associated revisions

Revision 3126 (diff)
Added by stbuehler about 1 year ago

[mod_proxy] accept LF delimited headers, not just CRLF (fixes #2594)

From: Glenn Strauss <>

Revision 0aa2ea74 (diff)
Added by gstrauss about 1 year ago

[mod_proxy] accept LF delimited headers, not just CRLF (fixes #2594)

From: Glenn Strauss <>

git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@3126 152afb58-edef-0310-8abb-c4023f1b3aa9

History

#1 Updated by gstrauss over 1 year ago

POST /cgi-bin/web_response.exe HTTP/1.1

The above request line suggests you are on a Windows machine.

Considering that some proxying (to thttpd) works for you, while other proxying (to web_response.exe) does not work for you, have you taken a closer look at web_response.exe? How does it handle connections? Does it send a response and then immediately closesocket() on the connection? Are you using closesocket() instead of close() in web_response.exe? What would happen if web_response.exe writes the response, forces a flush of any buffering, and then sleeps for a second before calling closesocket()? Or how about if web_response.exe writes the response, calls shutdown(fd, SHUT_WR), and waits for client (the lighttpd reverse proxy) to close the connection?

#2 Updated by omegasteffy over 1 year ago

I have not looked inside web_response.exe yet. Do not be fooled by ".exe" it is running on a embedded linux machine. It is a web/control site build upon cgiC. This is legacy code maintained by other developers, and there is little change i will be allowed to modify, but i can look at the code though.
I works perfectly if i use HAproxy instead.

If you are interested i can create a wireshark trace.

gstrauss wrote:

[...]
The above request line suggests you are on a Windows machine.

Considering that some proxying (to thttpd) works for you, while other proxying (to web_response.exe) does not work for you, have you taken a closer look at web_response.exe? How does it handle connections? Does it send a response and then immediately closesocket() on the connection? Are you using closesocket() instead of close() in web_response.exe? What would happen if web_response.exe writes the response, forces a flush of any buffering, and then sleeps for a second before calling closesocket()? Or how about if web_response.exe writes the response, calls shutdown(fd, SHUT_WR), and waits for client (the lighttpd reverse proxy) to close the connection?

#3 Updated by stbuehler over 1 year ago

  • Target version set to 1.4.x

I think the difference is whether the backend terminates the response header with "\r\n\r\n" (works) or with "\n\n" (doesn't work)

#4 Updated by omegasteffy over 1 year ago

Sounds as good explanation (looking at the original responses)
This actually means we suck and do not comply with the HTTP protocol.
http://stackoverflow.com/a/11254057/2986991
Will you still make a praqmatic solution and accept \n\n ?
(i am not certain if this is implies by you target version)

stbuehler wrote:

I think the difference is whether the backend terminates the response header with "\r\n\r\n" (works) or with "\n\n" (doesn't work)

#5 Updated by stbuehler over 1 year ago

omegasteffy wrote:

Will you still make a praqmatic solution and accept \n\n ?
(i am not certain if this is implies by you target version)

We probably should, yes.

#6 Updated by gstrauss over 1 year ago

Untested patch:

diff --git a/src/mod_proxy.c b/src/mod_proxy.c
index b7d4b43..2e47436 100644
--- a/src/mod_proxy.c
+++ b/src/mod_proxy.c
@@ -534,14 +534,15 @@ static int proxy_response_parse(server *srv, connection *con, plugin_data *p, bu

        buffer_copy_buffer(p->parse_response, in);

-       for (s = p->parse_response->ptr; NULL != (ns = strstr(s, "\r\n")); s = ns + 2) {
+       for (s = p->parse_response->ptr; NULL != (ns = strchr(s, '\n')); s = ns + 1) {
                char *key, *value;
                int key_len;
                data_string *ds;
                int copy_header;

                ns[0] = '\0';
-               ns[1] = '\0';
+               if (s != ns && ns[-1] == '\r')
+                       ns[-1] = '\0';

                if (-1 == http_response_status) {
                        /* The first line of a Response message is the Status-Line */

#7 Updated by gstrauss over 1 year ago

Submitted pull request https://github.com/lighttpd/lighttpd1.4/pull/29 with the above patch

#8 Updated by stbuehler about 1 year ago

  • Target version changed from 1.4.x to 1.4.40

#9 Updated by stbuehler about 1 year ago

  • Subject changed from Reverse proxy return 0 byte for some pages to Accept LF instead of CRLF as header line end from proxy backends

#10 Updated by stbuehler about 1 year ago

  • Status changed from New to Fixed
  • % Done changed from 0 to 100

Applied in changeset r3126.

Also available in: Atom