Lighttpd

Me again...
Some additional info:

• I back rev'd to 1.4.19 and this issue disappears.
• After upgrading to 1.4.20 - issue returns.
• I simplified things a bit on our dev machine - installing a vanilla lighttpd.conf that has no fastcgi in it. This conf is from the sample provided on the lighttpd wiki. I just added mod_alias support, and mod_cgi support with cgi.assign settings. Still no warns go to error.log. (see attached)
• lightty start stop messages do go to error.log
• when installing I used the following configure command:
  ./configure --with-openssl --with-pcre --disable-ipv6 --with-PACKAGE=mod_redirect --with-rewrite --with-redirect --with-ssi --enable-fastcgi --enable-discard-path --enable-force-cgi-redirect --without-bzip2

Here's the conf I used in this simplified test:

# lighttpd configuration file
#
# use it as a base for lighttpd 1.0.0 and above
#
# $Id: lighttpd.conf,v 1.7 2004/11/03 22:26:05 weigon Exp $

############ Options you really have to take care of ####################

## modules to load
# at least mod_access and mod_accesslog should be loaded
# all other module should only be loaded if really neccesary
# - saves some time
# - saves memory
server.modules              = (
#                               "mod_rewrite",
#                               "mod_redirect",
                               "mod_alias",
                                "mod_access",
#                               "mod_cml",
#                               "mod_trigger_b4_dl",
#                               "mod_auth",
#                               "mod_status",
#                               "mod_setenv",
#                               "mod_fastcgi",
#                               "mod_proxy",
#                               "mod_simple_vhost",
#                               "mod_evhost",
#                               "mod_userdir",
                               "mod_cgi",
#                               "mod_compress",
#                               "mod_ssi",
#                               "mod_usertrack",
#                               "mod_expire",
#                               "mod_secdownload",
#                               "mod_rrdtool",
                                "mod_accesslog" )

## a static document-root, for virtual-hosting take look at the
## server.virtual-* options
server.document-root        = "/srv/www/htdocs/" 

## where to send error-messages to
server.errorlog             = "/var/log/lighttpd/error.log" 

# files to check for if .../ is requested
index-file.names            = ( "index.php", "index.html",
                                "index.htm", "default.htm" )

## set the event-handler (read the performance section in the manual)
# server.event-handler = "freebsd-kqueue" # needed on OS X

# mimetype mapping
mimetype.assign             = (
  ".pdf"          =>      "application/pdf",
  ".sig"          =>      "application/pgp-signature",
  ".spl"          =>      "application/futuresplash",
  ".class"        =>      "application/octet-stream",
  ".ps"           =>      "application/postscript",
  ".torrent"      =>      "application/x-bittorrent",
  ".dvi"          =>      "application/x-dvi",
  ".gz"           =>      "application/x-gzip",
  ".pac"          =>      "application/x-ns-proxy-autoconfig",
  ".swf"          =>      "application/x-shockwave-flash",
  ".tar.gz"       =>      "application/x-tgz",
  ".tgz"          =>      "application/x-tgz",
  ".tar"          =>      "application/x-tar",
  ".zip"          =>      "application/zip",
  ".mp3"          =>      "audio/mpeg",
  ".m3u"          =>      "audio/x-mpegurl",
  ".wma"          =>      "audio/x-ms-wma",
  ".wax"          =>      "audio/x-ms-wax",
  ".ogg"          =>      "application/ogg",
  ".wav"          =>      "audio/x-wav",
  ".gif"          =>      "image/gif",
  ".jar"          =>      "application/x-java-archive",
  ".jpg"          =>      "image/jpeg",
  ".jpeg"         =>      "image/jpeg",
  ".png"          =>      "image/png",
  ".xbm"          =>      "image/x-xbitmap",
  ".xpm"          =>      "image/x-xpixmap",
  ".xwd"          =>      "image/x-xwindowdump",
  ".css"          =>      "text/css",
  ".html"         =>      "text/html",
  ".htm"          =>      "text/html",
  ".js"           =>      "text/javascript",
  ".asc"          =>      "text/plain",
  ".c"            =>      "text/plain",
  ".cpp"          =>      "text/plain",
  ".log"          =>      "text/plain",
  ".conf"         =>      "text/plain",
  ".text"         =>      "text/plain",
  ".txt"          =>      "text/plain",
  ".dtd"          =>      "text/xml",
  ".xml"          =>      "text/xml",
  ".mpeg"         =>      "video/mpeg",
  ".mpg"          =>      "video/mpeg",
  ".mov"          =>      "video/quicktime",
  ".qt"           =>      "video/quicktime",
  ".avi"          =>      "video/x-msvideo",
  ".asf"          =>      "video/x-ms-asf",
  ".asx"          =>      "video/x-ms-asf",
  ".wmv"          =>      "video/x-ms-wmv",
  ".bz2"          =>      "application/x-bzip",
  ".tbz"          =>      "application/x-bzip-compressed-tar",
  ".tar.bz2"      =>      "application/x-bzip-compressed-tar",
  # default mime type
  ""              =>      "application/octet-stream",
 )

# Use the "Content-Type" extended attribute to obtain mime type if possible
#mimetype.use-xattr        = "enable" 

## send a different Server: header
## be nice and keep it at lighttpd
# server.tag                 = "lighttpd" 

#### accesslog module
accesslog.filename          = "/var/log/lighttpd/access.log" 

## deny access the file-extensions
#
# ~    is for backupfiles from vi, emacs, joe, ...
# .inc is often used for code includes which should in general not be part
#      of the document-root
url.access-deny             = ( "~", ".inc" )

$HTTP["url"] =~ "\.pdf$" {
  server.range-requests = "disable" 
}

##
# which extensions should not be handle via static-file transfer
#
# .php, .pl, .fcgi are most often handled by mod_fastcgi or mod_cgi
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )

######### Options that are good to be but not neccesary to be changed #######

## bind to port (default: 80)
server.port                = 8080

## bind to localhost (default: all interfaces)
#server.bind                = "127.0.0.1" 

## error-handler for status 404
#server.error-handler-404   = "/error-handler.html" 
#server.error-handler-404   = "/error-handler.php" 

## to help the rc.scripts
#server.pid-file            = "/var/run/lighttpd.pid" 

###### virtual hosts
##
##  If you want name-based virtual hosting add the next three settings and load
##  mod_simple_vhost
##
## document-root =
##   virtual-server-root + virtual-server-default-host + virtual-server-docroot
## or
##   virtual-server-root + http-host + virtual-server-docroot
##
#simple-vhost.server-root   = "/srv/www/vhosts/" 
#simple-vhost.default-host  = "www.example.org" 
#simple-vhost.document-root = "/htdocs/" 

##
## Format: <errorfile-prefix><status-code>.html
## -> ..../status-404.html for 'File not found'
#server.errorfile-prefix    = "/usr/share/lighttpd/errors/status-" 
#server.errorfile-prefix    = "/srv/www/errors/status-" 

## virtual directory listings
#dir-listing.activate       = "enable" 
## select encoding for directory listings
#dir-listing.encoding        = "utf-8" 

## enable debugging
#debug.log-request-header   = "enable" 
#debug.log-response-header  = "enable" 
#debug.log-request-handling = "enable" 
#debug.log-file-not-found   = "enable" 

### only root can use these options
#
# chroot() to directory (default: no chroot() )
#server.chroot              = "/" 

## change uid to <uid> (default: don't care)
#server.username            = "wwwrun" 

## change uid to <uid> (default: don't care)
#server.groupname           = "wwwrun" 

#### compress module
#compress.cache-dir         = "/var/cache/lighttpd/compress/" 
#compress.filetype          = ("text/plain", "text/html")

#### proxy module
## read proxy.txt for more info
#proxy.server               = ( ".php" =>
#                               ( "localhost" =>
#                                 (
#                                   "host" => "192.168.0.101",
#                                   "port" => 80
#                                 )
#                               )
#                             )

#### fastcgi module
## read fastcgi.txt for more info
## for PHP don't forget to set cgi.fix_pathinfo = 1 in the php.ini
#fastcgi.server             = ( ".php" =>
#                               ( "localhost" =>
#                                 (
#                                   "socket" => "/var/run/lighttpd/php-fastcgi.socket",
#                                   "bin-path" => "/usr/local/bin/php-cgi" 
#                                 )
#                               )
#                            )

#### CGI module
cgi.assign                 = ( ".pl"  => "/usr/bin/perl",
                               ".cgi" => "/usr/bin/perl",
                               ".cpx" => "/usr/bin/perl")

alias.url = ( "/udm/" => "/srv/www/cgi-bin/")

#### SSL engine
#ssl.engine                 = "enable" 
#ssl.pemfile                = "/etc/ssl/private/lighttpd.pem" 

#### status module
#status.status-url          = "/server-status" 
#status.config-url          = "/server-config" 

#### auth module
## read authentication.txt for more info
#auth.backend               = "plain" 
#auth.backend.plain.userfile = "lighttpd.user" 
#auth.backend.plain.groupfile = "lighttpd.group" 

#auth.backend.ldap.hostname = "localhost" 
#auth.backend.ldap.base-dn  = "dc=my-domain,dc=com" 
#auth.backend.ldap.filter   = "(uid=$)" 

#auth.require               = ( "/server-status" =>
#                               (
#                                 "method"  => "digest",
#                                 "realm"   => "download archiv",
#                                 "require" => "user=jan" 
#                               ),
#                               "/server-config" =>
#                               (
#                                 "method"  => "digest",
#                                 "realm"   => "download archiv",
#                                 "require" => "valid-user" 
#                               )
#                             )

#### url handling modules (rewrite, redirect, access)
#url.rewrite                = ( "^/$"             => "/server-status" )
#url.redirect               = ( "^/wishlist/(.+)" => "http://www.123.org/$1" )
#### both rewrite/redirect support back reference to regex conditional using %n
#$HTTP["host"] =~ "^www\.(.*)" {
#  url.redirect            = ( "^/(.*)" => "http://%1/$1" )
#}

#
# define a pattern for the host url finding
# %% => % sign
# %0 => domain name + tld
# %1 => tld
# %2 => domain name without tld
# %3 => subdomain 1 name
# %4 => subdomain 2 name
#
#evhost.path-pattern        = "/srv/www/vhosts/%3/htdocs/" 

#### expire module
#expire.url                 = ( "/buggy/" => "access 2 hours", "/asdhas/" => "access plus 1 seconds 2 minutes")

#### ssi
#ssi.extension              = ( ".shtml" )

#### rrdtool
#rrdtool.binary             = "/usr/bin/rrdtool" 
#rrdtool.db-name            = "/var/lib/lighttpd/lighttpd.rrd" 

#### setenv
#setenv.add-request-header  = ( "TRAV_ENV" => "mysql://user@host/db" )
#setenv.add-response-header = ( "X-Secret-Message" => "42" )

## for mod_trigger_b4_dl
# trigger-before-download.gdbm-filename = "/var/lib/lighttpd/trigger.db" 
# trigger-before-download.memcache-hosts = ( "127.0.0.1:11211" )
# trigger-before-download.trigger-url = "^/trigger/" 
# trigger-before-download.download-url = "^/download/" 
# trigger-before-download.deny-url = "http://127.0.0.1/index.html" 
# trigger-before-download.trigger-timeout = 10

## for mod_cml
## don't forget to add index.cml to server.indexfiles
# cml.extension               = ".cml" 
# cml.memcache-hosts          = ( "127.0.0.1:11211" )

#### variable usage:
## variable name without "." is auto prefixed by "var." and becomes "var.bar" 
#bar = 1
#var.mystring = "foo" 

## integer add
#bar += 1
## string concat, with integer cast as string, result: "www.foo1.com" 
#server.name = "www." + mystring + var.bar + ".com" 
## array merge
#index-file.names = (foo + ".php") + index-file.names
#index-file.names += (foo + ".php")

#### include
#include /etc/lighttpd/lighttpd-inc.conf
## same as above if you run: "lighttpd -f /etc/lighttpd/lighttpd.conf" 
#include "lighttpd-inc.conf" 

#### include_shell
#include_shell "echo var.a=1" 
## the above is same as:
#var.a=1

Only one process should write to a file at a time, and therefore we now close stderr for all forked applications. I recommend using a wrapper script which redirects the stderr output to an extra logfile, i.e. (not tested):

#!/bin/sh

exec /path/to/real/cgi -perhaps -extra -params $@ 2> /your/log/file

Thanks, stbuehler, for the quick reply!
However, I have to say that I don't understand it.

First off, why did this work in 1.4.19? We had eight lightty fastcgi perl instances (processes)
which logged to the error log as well as a host of other standard cgis.
Was this ability for all cgis to log to the errorlog a broken "feature" that was fixed
in version 1.4.20?

I also don't understand why I can't get the vanilla config with the new version to
log to the error log. You say "Only one process should write to a file at a time". In my
second approach I've removed the fastcgi server from the config and have a super simple
config and I'm just running one standard cgi, yet I still can't get warnings or errors to log
to the errorlog. However, the accesslog works fine.
Server start/stops are logged to the error log. I originally thought it has something to do
with the interaction with fastcgi. But this config doesn't have that... it should work.
I must be doing something wrong.

1. we only want the webserver to write to the errorlog, nobody else (if you are using server.max-worker you may get problems with this)
2. fastcgi has an "error" stream; the errors from this stream are logged by lighty
3. in 1.4.19 cgi apps wrote directly to the webservers error log. this is bad and was therefore fixed (2> /dev/null)
   atm i am working on a patch to read the cgi error stream over a pipe in lighty and log it from there, but there are some problems with the mod_cgi handling.
4. in 1.5 the cgi stderr is already handled correctly with pipes

Thanks for the explanation - I guess I was so used to the way apache the the older version of lighty handled this.

I solved my issue by using perl's CGI::Carp module, redirecting STDERR to a separate error.log.

However, it appears I still have the initial issue that caused me to upgrade, where cookie data is occasionally getting written partially. I'll write a separate support request addressing this, in case you have any ideas.

Thanks again. -Lisa