Lighttpd

lighttpd src/http-header-glue.c defines

/**     
 * max size of the HTTP response header from backends
 * (differs from server.max-request-field-size for max request field size)
 */     
#define MAX_HTTP_RESPONSE_FIELD_SIZE 65535

If you are abusing HTTP headers for your private API and are sending response header >= 64k, please consider alternatives to your abuse of HTTP response headers. The HTTP response body can be whatever you like.

I would guess the script does not output the Content-Type header properly. Your script should output something like this:

Content-Type: application/json

{ text: "Hello world" }

Try running the python script manually from the command line to see if it, for example, outputs some error which confuses lighttpd.

I would guess the script is sending pure JSON and is neglecting to send any response headers before sending JSON.

A minimal response header before the JSON could be Status: 200\n\n

gstrauss wrote in RE: Pls help: 502 bad gateway - lighttpd response headers...:

I would guess the script is sending pure JSON and is neglecting to send any response headers before sending JSON.

A minimal response header before the JSON could be Status: 200\n\n

this seems did the trick. TKKS!

avij wrote in RE: Pls help: 502 bad gateway - lighttpd response headers...:

I would guess the script does not output the Content-Type header properly. Your script should output something like this:

[...]

Try running the python script manually from the command line to see if it, for example, outputs some error which confuses lighttpd.

surely I did include this, otherwise it should not have worked for days. but I do not know why it worked why it worked when I did not print "Status: 200\n\n".

gstrauss wrote in RE: Pls help: 502 bad gateway - lighttpd response headers...:

lighttpd src/http-header-glue.c defines
[...]

If you are abusing HTTP headers for your private API and are sending response header >= 64k, please consider alternatives to your abuse of HTTP response headers. The HTTP response body can be whatever you like.

Sorry I forgot to mention that my requests are just several short strings.

gstrauss wrote in RE: Pls help: 502 bad gateway - lighttpd response headers...:

I would guess the script is sending pure JSON and is neglecting to send any response headers before sending JSON.

A minimal response header before the JSON could be Status: 200\n\n

I'm closing this thread. But, could you pls explain a bit further more, why it worked for days when I did not do this? TKS anyway bud

Just noticed that, normally my response is of xxK. Today it reached xxxK, which would cost more time. Does this ring a bell?

surely I did include this, otherwise it should not have worked for days. but I do not know why it worked why it worked when I did not print "Status: 200\n\n".

Please use a search engine to look up the definitions of a CGI request and response. RFC3875 from 2004 documents the practices from the 1990s. Responses have response headers followed by a blank line (LF or CRLF), followed by the (optional) response body. Some servers (including lighttpd) will parse HTTP-formatted response lines as CGI NPH (non-parsed headers), though lighttpd intentionally does parse them. FastCGI and other non-HTTP backends typically define a Status line, or might send a response with an HTTP-formatted response line, which lighttpd also supports similar to CGI NPH.

There are some examples in mod_fastcgi which demonstrate sending a Status line.

Just about every python framework has examples demonstrating sending a Status line.

The reason lighttpd worked with your invalid python responses for smaller responses is that there are more than a few people like yourself who write code without first looking at the basic specifications for with what your code is interfacing. If lighttpd receives a small response from a CGI backend and does not detect proper format for response headers, then lighttpd will treat the response as Status: 200 and send the entire response as the response body. If the backend is something else, like FastCGI, and the first line of the response does not look like a header (i.e. contains a :), and the response is <= 2 lines, then lighttpd tries to treat the response as the response body. These are heuristics. You should prefer to always provide a Status line in your FastCGI responses.

I should have done more homework. Thanks for the perfect explanation.