Project

General

Profile

[Solved] Intermittent HTTP 500 on CGI

Added by Davide_Baldini about 10 years ago

Setup:

OS: Debian 6.0.4
Lighttpd: lighttpd/1.4.28 (ssl) Build-Date: Mar 15 2013 20:08:27
Hardware: VPS OpenVZ, w/ abundant free RAM (1GB +), free disk space (90GB), and low CPU load (in light of eventual race conditions)
lighttpd.conf: file attached; also at http://pastebin.mozilla.org/4701923
User agent: Iceweasel 8.0, Debian

Description of problem:

I get intermittent HTTP 500 responses when requesting a CGI (no FastCGI) script.
About 80% of the requests return HTTP 200, the rest HTTP 500, for no particular reason.
This is a shell script, named cloudGenerator.js.sh, with ending .sh extension. As per the attached lighttpd.conf, lighttpd is instructed to handle .sh files via the default system shell - in this case Bash.

The script, cloudGenerator.js.sh, is a "self-extracting" javascript file: at line 1, its sha-bang points to the extractor program; by line 2, it becomes regular JavaScript:

#!/usr/local/bin/jsMinCacher.sh
var genericJsFunction = function() { return []; };

The extractor program, /usr/local/bin/jsMinCacher.sh, is a small (20 lines) regular Bash script (#!/bin/bash) which outputs to stdout a shrunk version of the original file. Its execution is consistent in that it's not influenced by volatile runtime factors, like environment variables or date; it always executed the same algorithms and produces the same output:

#!/bin/bash

source=${1##*/}
cache_dir=/tmp/jsMinCacher/
compressor=/usr/bin/yui-compressor

To summarize, executing "bash cloudGenerator.js.sh" launches "bash /usr/local/bin/jsMinCacher.sh", which in turn produces the stdout to feed thru CGI.

Running "bash cloudGenerator.js.sh" directly via command line never fails. This is tested with 2000 iterations and with PWD environment variable set to the path of cloudGenerator.js.sh.
Instead, when "bash cloudGenerator.js.sh" is called by CGI, 20% of the times I get HTTP 500. When this happens, the script still doesn't encounter any error by its own, and correctly completes its execution from start to end, producing the expected stdout, with stderr empty, and returning the exit status 0.

Apparently, I'm getting HTTP 500 in vain.
How can I debug this further?

lighttpd.conf (3.53 KB) lighttpd.conf Same file, with syntax highlight: pastebin.mozilla.org/4701923

Replies (6)

RE: Intermittent HTTP 500 on CGI - Added by Davide_Baldini about 10 years ago

I found a similar report on this support forum, dating 2009-05: Random 500 Internal Server Error with Perl

That report doesn't specify whether the script correctly executes thru the end. Still, dating a similar report might help tracking an eventual bug.

RE: Intermittent HTTP 500 on CGI - Added by stbuehler about 10 years ago

Printing a proper HTTP/CGI response header before the result should fix it (assuming it is missing).

printf "Status: 200\r\n\r\n"

RE: Intermittent HTTP 500 on CGI - Added by Davide_Baldini about 10 years ago

Is this a feature or a bug, for lighttpd, to generate an apparently "random" response header in case the CGI misses to specify it?

Am I missing a piece of the puzzle?

RE: Intermittent HTTP 500 on CGI - Added by jwmiller over 9 years ago

Glad I found this. I only saw this when banging on the cgi in rapid succession. Adding the status header fixed it. Oddly enough the fcgi did not have the same problem. Thank you for lighttpd!

RE: Intermittent HTTP 500 on CGI - Added by Davide_Baldini about 9 years ago

Thank you stbuehler for pointing out a workaround.
I still remain with a doubt. Why Lighty responds HTTP 500 after the CGI script completes with exit status 0 and non-empty stdout?
Is the HTTP/CGI response header mandatory by specification?

RE: [Solved] Intermittent HTTP 500 on CGI - Added by gstrauss about 7 years ago

Is the HTTP/CGI response header mandatory by specification?

Yes, the response header (even if just a blank line) is required by the specification: https://www.ietf.org/rfc/rfc3875

lighttpd supports lack of a CGI response header for backwards compatibility with bad scripts, but this behavior is deprecated, since there are dangers of non-CGI scripts being executed, and dangers of CGI output being misinterpreted as CGI response headers, corrupting the response returned to the client. Still, lighttpd 1.4.46 will attempt to reduce the occurrence of HTTP 500 for these bad CGI responses.

    (1-6/6)