Project

General

Profile

Bug #2594

Accept LF instead of CRLF as header line end from proxy backends

Added by omegasteffy about 3 years ago. Updated over 1 year ago.

Status:
Fixed
Priority:
Normal
Assignee:
-
Category:
mod_proxy
Target version:
Start date:
2014-09-12
Due date:
% Done:

100%

Estimated time:
Missing in 1.5.x:
No

Description

I was trying to setup a simple reverse proxy forward port 82 on the lighttpd server to port 80 on an backend server.

The welcome page works fine, but after i post a few variables i get 0 byte back (resulting in a download file dialog)
A wireshark dump shows that lighttpd get a fine html response from the backend server.
I also tried the url-rewrite trick, (proxy-ip/serverA/ instead of proxy-ip:82)

I tried with 1.4.13 and 1.4.35.
I suspected configuration, but after several attempts i guess it might be a known bug.
Lighttpd 1.5 works but is not current a possibility for the proxy server platform.

I know this bug is not specific, but please guide me
I would really appreciate patches or work-around.

Attachments
Config (IPs might have been changed since)
a-Response which get forwarded
b-Response which is NOT.

NOTE: It seem the two proxy responses get concatenated by wireshark
Your should be able to determine how it fits together.

proxy_test.conf (1.12 KB) proxy_test.conf minimum configuration for a proxy omegasteffy, 2014-09-12 08:42
b_not_forwarded_backend_resp (3.76 KB) b_not_forwarded_backend_resp Reponse sent to lighttpd, but not forwarded omegasteffy, 2014-09-12 08:43
a_forwarded_proxy_resp (1.52 KB) a_forwarded_proxy_resp OK - response given by proxy to client omegasteffy, 2014-09-12 08:43
b_not_forwarded_proxy_resp (1.52 KB) b_not_forwarded_proxy_resp The "b_not_forwarded_backend_resp" is not sent omegasteffy, 2014-09-12 08:43
a_forwarded_backend_resp (1.01 KB) a_forwarded_backend_resp ok response for first paeg omegasteffy, 2014-09-12 08:47

Associated revisions

Revision 3126 (diff)
Added by stbuehler over 1 year ago

[mod_proxy] accept LF delimited headers, not just CRLF (fixes #2594)

From: Glenn Strauss <>

Revision 0aa2ea74 (diff)
Added by gstrauss over 1 year ago

[mod_proxy] accept LF delimited headers, not just CRLF (fixes #2594)

From: Glenn Strauss <>

git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@3126 152afb58-edef-0310-8abb-c4023f1b3aa9

History

#1

Updated by gstrauss over 1 year ago

POST /cgi-bin/web_response.exe HTTP/1.1

The above request line suggests you are on a Windows machine.

Considering that some proxying (to thttpd) works for you, while other proxying (to web_response.exe) does not work for you, have you taken a closer look at web_response.exe? How does it handle connections? Does it send a response and then immediately closesocket() on the connection? Are you using closesocket() instead of close() in web_response.exe? What would happen if web_response.exe writes the response, forces a flush of any buffering, and then sleeps for a second before calling closesocket()? Or how about if web_response.exe writes the response, calls shutdown(fd, SHUT_WR), and waits for client (the lighttpd reverse proxy) to close the connection?

#2

Updated by omegasteffy over 1 year ago

I have not looked inside web_response.exe yet. Do not be fooled by ".exe" it is running on a embedded linux machine. It is a web/control site build upon cgiC. This is legacy code maintained by other developers, and there is little change i will be allowed to modify, but i can look at the code though.
I works perfectly if i use HAproxy instead.

If you are interested i can create a wireshark trace.

gstrauss wrote:

[...]
The above request line suggests you are on a Windows machine.

Considering that some proxying (to thttpd) works for you, while other proxying (to web_response.exe) does not work for you, have you taken a closer look at web_response.exe? How does it handle connections? Does it send a response and then immediately closesocket() on the connection? Are you using closesocket() instead of close() in web_response.exe? What would happen if web_response.exe writes the response, forces a flush of any buffering, and then sleeps for a second before calling closesocket()? Or how about if web_response.exe writes the response, calls shutdown(fd, SHUT_WR), and waits for client (the lighttpd reverse proxy) to close the connection?

#3

Updated by stbuehler over 1 year ago

  • Target version set to 1.4.x

I think the difference is whether the backend terminates the response header with "\r\n\r\n" (works) or with "\n\n" (doesn't work)

#4

Updated by omegasteffy over 1 year ago

Sounds as good explanation (looking at the original responses)
This actually means we suck and do not comply with the HTTP protocol.
http://stackoverflow.com/a/11254057/2986991
Will you still make a praqmatic solution and accept \n\n ?
(i am not certain if this is implies by you target version)

stbuehler wrote:

I think the difference is whether the backend terminates the response header with "\r\n\r\n" (works) or with "\n\n" (doesn't work)

#5

Updated by stbuehler over 1 year ago

omegasteffy wrote:

Will you still make a praqmatic solution and accept \n\n ?
(i am not certain if this is implies by you target version)

We probably should, yes.

#6

Updated by gstrauss over 1 year ago

Untested patch:

diff --git a/src/mod_proxy.c b/src/mod_proxy.c
index b7d4b43..2e47436 100644
--- a/src/mod_proxy.c
+++ b/src/mod_proxy.c
@@ -534,14 +534,15 @@ static int proxy_response_parse(server *srv, connection *con, plugin_data *p, bu

        buffer_copy_buffer(p->parse_response, in);

-       for (s = p->parse_response->ptr; NULL != (ns = strstr(s, "\r\n")); s = ns + 2) {
+       for (s = p->parse_response->ptr; NULL != (ns = strchr(s, '\n')); s = ns + 1) {
                char *key, *value;
                int key_len;
                data_string *ds;
                int copy_header;

                ns[0] = '\0';
-               ns[1] = '\0';
+               if (s != ns && ns[-1] == '\r')
+                       ns[-1] = '\0';

                if (-1 == http_response_status) {
                        /* The first line of a Response message is the Status-Line */

#7

Updated by gstrauss over 1 year ago

Submitted pull request https://github.com/lighttpd/lighttpd1.4/pull/29 with the above patch

#8

Updated by stbuehler over 1 year ago

  • Target version changed from 1.4.x to 1.4.40
#9

Updated by stbuehler over 1 year ago

  • Subject changed from Reverse proxy return 0 byte for some pages to Accept LF instead of CRLF as header line end from proxy backends
#10

Updated by stbuehler over 1 year ago

  • Status changed from New to Fixed
  • % Done changed from 0 to 100

Applied in changeset r3126.

Also available in: Atom