Lighttpd

The problem persists after several restarts and config changes.
My config:

# lighttpd configuration file
#

## modules to load
# at least mod_access and mod_accesslog should be loaded
server.modules = (
#       "mod_rewrite",
#       "mod_redirect",
#       "mod_alias",
        "mod_access",
#       "mod_cml",
#       "mod_trigger_b4_dl",
#       "mod_auth",
#       "mod_status",
        "mod_setenv",
#       "mod_fastcgi",
#       "mod_proxy",
#       "mod_simple_vhost",
#       "mod_evhost",
#       "mod_userdir",
        "mod_cgi",
#       "mod_compress",
#       "mod_ssi",
#       "mod_usertrack",
#       "mod_expire",
#       "mod_secdownload",
#       "mod_rrdtool",
        "mod_accesslog" )

server.tag              = "lighttpd" 
server.port             = 8081
server.bind             = "localhost" 

server.document-root    = "/Users/jimmy/Work/devbox-www-backup" 
server.pid-file         = "/Users/jimmy/UNIX/var/run/lighttpd.pid" 
server.errorlog         = "/Users/jimmy/UNIX/var/log/lighttpd.error.log" 
accesslog.filename      = "/Users/jimmy/UNIX/var/log/lighttpd.access.log" 

server.event-handler    = "freebsd-kqueue" 

index-file.names        = ( "index.pl", "index.cgi",
                            "index.html", "index.htm" )

url.access-deny         = ( "~", ".pm" )
static-file.exclude-extensions  = ( ".pl", ".cgi" )

cgi.assign              = ( ".pl"  => "",
                            ".cgi" => "" )

setenv.add-environment  = (
        "PERLLIB" => "/Users/jimmy/UNIX/lib/perl5/site_perl" 
)

#dir-listing.activate   = "enable" 

mimetype.assign = (
        ".pdf"          =>      "application/pdf",
        ".sig"          =>      "application/pgp-signature",
        ".spl"          =>      "application/futuresplash",
        ".class"        =>      "application/octet-stream",
        ".ps"           =>      "application/postscript",
        ".torrent"      =>      "application/x-bittorrent",
        ".dvi"          =>      "application/x-dvi",
        ".gz"           =>      "application/x-gzip",
        ".pac"          =>      "application/x-ns-proxy-autoconfig",
        ".swf"          =>      "application/x-shockwave-flash",
        ".tar.gz"       =>      "application/x-tgz",
        ".tgz"          =>      "application/x-tgz",
        ".tar"          =>      "application/x-tar",
        ".zip"          =>      "application/zip",
        ".mp3"          =>      "audio/mpeg",
        ".m3u"          =>      "audio/x-mpegurl",
        ".wma"          =>      "audio/x-ms-wma",
        ".wax"          =>      "audio/x-ms-wax",
        ".ogg"          =>      "application/ogg",
        ".wav"          =>      "audio/x-wav",
        ".gif"          =>      "image/gif",
        ".jpg"          =>      "image/jpeg",
        ".jpeg"         =>      "image/jpeg",
        ".png"          =>      "image/png",
        ".xbm"          =>      "image/x-xbitmap",
        ".xpm"          =>      "image/x-xpixmap",
        ".xwd"          =>      "image/x-xwindowdump",
        ".css"          =>      "text/css",
        ".html"         =>      "text/html",
        ".htm"          =>      "text/html",
        ".js"           =>      "text/javascript",
        ".asc"          =>      "text/plain",
        ".c"            =>      "text/plain",
        ".cpp"          =>      "text/plain",
        ".log"          =>      "text/plain",
        ".conf"         =>      "text/plain",
        ".text"         =>      "text/plain",
        ".txt"          =>      "text/plain",
        ".dtd"          =>      "text/xml",
        ".xml"          =>      "text/xml",
        ".mpeg"         =>      "video/mpeg",
        ".mpg"          =>      "video/mpeg",
        ".mov"          =>      "video/quicktime",
        ".qt"           =>      "video/quicktime",
        ".avi"          =>      "video/x-msvideo",
        ".asf"          =>      "video/x-ms-asf",
        ".asx"          =>      "video/x-ms-asf",
        ".wmv"          =>      "video/x-ms-wmv",
        ".bz2"          =>      "application/x-bzip",
        ".tbz"          =>      "application/x-bzip-compressed-tar",
        ".tar.bz2"      =>      "application/x-bzip-compressed-tar" 
)

-- jimmy

after a few minutes, i also get this in the log:

2006-07-17 15:29:12: (server.c.1126) NOTE: a request for /test.pl timed out after writing 863 bytes. We waited 360 seconds. If this a problem increase server.max-write-idle 

-- jimmy

I tried the version from DarwinPorts also without success, other event handlers, even compiled an older C compiler without success. I'm trying to find out why it stays in CON_STATE_WRITE .. if I use a very simple request using telnet and waiting a few seconds before I finish the headers, it sometimes works, but that makes no sense at all, but it should be related to the at least one other issue i reported using lighttpd on OSX.

-- jimmy

I came to the point that HTTP/1.0 events also hang and not finish.

The times it works, is with concurrent tests imho, because it seems that, WHEN it works, there is an EAGAIN after reading all bytes, an FDEVENT_IN afterwards, and then 0 bytes are read.

When it doesn't work, there is no FDEVENT_IN after reading ALL bytes returned by the CGI.

If I disable the piece of code to close stdout, and put any printf statement in fdevent_select_event_get_revent(), it works (i'm testing now with the select handler instead of freebsd-kqueue). But this is not a solution, and know it's getting to complicated for me.

-- jimmy

now i'm to the point, that adding any printf() statement to the main polling loop in server.c "fixes" the problem, BUT I also found that removing the fcntl call in mod_cgi.c also fixes the issue, so it has got something to do with it!

I really could use some help over here, but I have a feeling that sometimes (most of the time), using any event handler, an event isn't detected when the cgi dies (SIGPIPE related?!). If you have any clues, or short C program just to test the fcntl/o_nonblock/sigpipe behaviour like it happens here, it would be really appreciated, because my C is quite poor and solving this issue is taking a LOT of time from me, but i will continue trying...

-- jimmy

I could not reproduce the problem with a small C program, but I am on a completely wrong track there. I was looking for a bug in OSX's handling of events, while the problem seems to be somewhere else, it seems that the printf() statement previously mentioned, somehow resulted that read() from the CGI's filedescriptor, not returing EAGAIN, while otherwise it would after the first read (with my test CGI). If EAGAIN happens, the problem occurs (always?), otherwise it doesn't (duh). BUT, it seems that somehow, the CGI's filedescriptor is gone from the fd_set being polled...

-- jimmy

Now it's very clear, if EAGAIN happens while reading from the CGI's fd (which does never happen on my OpenBSD test machine), the file descriptor is still removed from the fd_set, and thus, there will never be polled for an event on it again.

-- jimmy

it get closed in SUBREQUEST_FUNC(mod_cgi_handle_subrequest), because of :

if (!con->file_started) return HANDLER_WAIT_FOR_EVENT;

this will remove the event and close the fd because the response started, and thus the return is skipped, and the code below removes it from the event list, and closes the cgi's fd because it thinks it exited after sending all data, but this does not imply that the response to the client was finished, because read() still has to return 0 before the request is marked as handled!

if this is put right below it, the problem is fixed:

if (!con->file_finished) return HANDLER_FINISHED;

I am not sure, if the file_started line still has to be there or not as i do not fully understand the internals yet, and I'm also not sure that everything in that block based on file_started is correctly assumed! (also do not slap me if HANDLER_FINISHED is not the approprate return response, but this is the one that makes it work here)

-- jimmy

Patch available in https://github.com/lighttpd/lighttpd1.4/pull/31