Lighttpd

I compiled lighttpd-1.5.0_pre1857 and it works on my develop machine. But when I deploy it on the production server (>200req/s), it crashes with a segmentation fault after 2 or 3 seconds of running. lighttpd-1.4.15 has no such problem (we have used 1.4.15 for months). FYI, we are using scgi for the proxy-core.protocol.

Backtrace produced by valgrind followed. If you need any thing more (e.g. config file), please contact me.

==22638== Memcheck, a memory error detector.
==22638== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et al.
==22638== Using LibVEX rev 1732, a library for dynamic binary translation.
==22638== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP.
==22638== Using valgrind-3.2.3, a dynamic binary instrumentation framework.
==22638== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et al.
==22638== 
==22638== My PID = 22638, parent PID = 22622.  Prog and args are:
==22638==    /usr/local/sbin/lighttpd
==22638==    -D
==22638==    -f
==22638==    ./lighttpd.conf
==22638== 
--22638-- 
--22638-- Command line
--22638--    /usr/local/sbin/lighttpd
--22638--    -D
--22638--    -f
--22638--    ./lighttpd.conf
--22638-- Startup, with flags:
--22638--    --tool=memcheck
--22638--    -v
--22638--    --log-file=lighttpd
--22638--    --num-callers=8
--22638-- Contents of /proc/version:
--22638--   Linux version 2.6.16-gentoo-r13 (root@meriadoc) (gcc version 3.4.6 (Gentoo 3.4.6-r1, ssp-3.4.5-1.0, pie-8.7.9)) #3 SMP Fri Aug 18 16:00:37 CST 2006
--22638-- Arch and hwcaps: X86, x86-sse1-sse2
--22638-- Page sizes: currently 4096, max supported 4096
--22638-- Valgrind library directory: /usr/lib/valgrind
--22638-- Reading syms from /lib/ld-2.3.6.so (0x4000000)
--22638-- Reading syms from /usr/local/sbin/lighttpd (0x8048000)
--22638-- Reading syms from /usr/lib/valgrind/x86-linux/memcheck (0x38000000)
--22638--    object doesn't have a symbol table
--22638--    object doesn't have a dynamic symbol table
--22638-- Reading suppressions file: /usr/lib/valgrind/default.supp
--22638-- REDIR: 0x4010B10 (index) redirected to 0x3802F3B3 (???)
--22638-- Reading syms from /usr/lib/valgrind/x86-linux/vgpreload_core.so (0x4017000)
--22638--    object doesn't have a symbol table
--22638-- Reading syms from /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so (0x401A000)
--22638--    object doesn't have a symbol table
==22638== WARNING: new redirection conflicts with existing -- ignoring it
--22638--     new: 0x04010B10 (index     ) R-> 0x0401E150 index
--22638-- REDIR: 0x4010CB0 (strlen) redirected to 0x401E3D0 (strlen)
--22638-- Reading syms from /usr/lib/libpcre.so.0.0.1 (0x4025000)
--22638--    object doesn't have a symbol table
--22638-- Reading syms from /lib/libdl-2.3.6.so (0x4053000)
--22638--    object doesn't have a symbol table
--22638-- Reading syms from /usr/lib/libgthread-2.0.so.0.1200.11 (0x4057000)
--22638--    object doesn't have a symbol table
--22638-- Reading syms from /lib/libpthread-2.3.6.so (0x405C000)
--22638-- Reading syms from /usr/lib/libglib-2.0.so.0.1200.11 (0x406F000)
--22638--    object doesn't have a symbol table
--22638-- Reading syms from /lib/librt-2.3.6.so (0x4111000)
--22638--    object doesn't have a symbol table
--22638-- Reading syms from /lib/libc-2.3.6.so (0x411A000)
--22638--    object doesn't have a symbol table
--22638-- REDIR: 0x41827A0 (memset) redirected to 0x401EED0 (memset)
--22638-- REDIR: 0x4182CB0 (memcpy) redirected to 0x401E780 (memcpy)
--22638-- REDIR: 0x41819D0 (rindex) redirected to 0x401E030 (rindex)
--22638-- REDIR: 0x417D880 (calloc) redirected to 0x401D790 (calloc)
--22638-- REDIR: 0x417DB50 (malloc) redirected to 0x401B710 (malloc)
--22638-- REDIR: 0x4181130 (strcmp) redirected to 0x401E690 (strcmp)
--22638-- REDIR: 0x4181670 (strlen) redirected to 0x401E3B0 (strlen)
--22638-- REDIR: 0x4180FC0 (index) redirected to 0x401E120 (index)
--22638-- REDIR: 0x4182730 (memmove) redirected to 0x401EEF0 (memmove)
--22638-- REDIR: 0x417C060 (free) redirected to 0x401C980 (free)
--22638-- REDIR: 0x4181840 (strncmp) redirected to 0x401E630 (strncmp)
--22638-- REDIR: 0x417E0F0 (realloc) redirected to 0x401D8C0 (realloc)
--22638-- Reading syms from /usr/local/lib/mod_indexfile.so (0x4021000)
--22638-- Reading syms from /usr/local/lib/mod_rewrite.so (0x4333000)
--22638-- Reading syms from /usr/local/lib/mod_redirect.so (0x4336000)
--22638-- Reading syms from /usr/local/lib/mod_access.so (0x4339000)
--22638-- Reading syms from /usr/local/lib/mod_auth.so (0x433C000)
--22638-- Reading syms from /lib/libcrypt-2.3.6.so (0x4347000)
--22638--    object doesn't have a symbol table
--22638-- Reading syms from /usr/local/lib/mod_status.so (0x4375000)
--22638-- Reading syms from /usr/local/lib/mod_compress.so (0x437B000)
--22638-- Reading syms from /lib/libz.so.1.2.3 (0x4380000)
--22638--    object doesn't have a symbol table
--22638-- Reading syms from /lib/libbz2.so.1.0.3 (0x4393000)
--22638--    object doesn't have a symbol table
--22638-- Reading syms from /usr/local/lib/mod_expire.so (0x4343000)
--22638-- Reading syms from /usr/local/lib/mod_accesslog.so (0x43A3000)
--22638-- Reading syms from /usr/local/lib/mod_proxy_core.so (0x43A7000)
--22638-- Reading syms from /usr/local/lib/mod_proxy_backend_scgi.so (0x43B3000)
--22638-- Reading syms from /usr/local/lib/mod_dirlisting.so (0x43B7000)
--22638-- Reading syms from /usr/local/lib/mod_staticfile.so (0x43BC000)
--22638-- Reading syms from /usr/local/lib/mod_chunked.so (0x43C0000)
--22638-- REDIR: 0x41822A0 (memchr) redirected to 0x401E750 (memchr)
--22638-- REDIR: 0x41835C0 (strchrnul) redirected to 0x401EF50 (strchrnul)
--22638-- REDIR: 0x4182800 (mempcpy) redirected to 0x401F080 (mempcpy)
--22638-- REDIR: 0x4182980 (stpcpy) redirected to 0x401EC30 (stpcpy)
--22638-- Reading syms from /lib/libnss_compat-2.3.6.so (0x43C7000)
--22638--    object doesn't have a symbol table
--22638-- Reading syms from /lib/libnsl-2.3.6.so (0x43CF000)
--22638--    object doesn't have a symbol table
--22638-- REDIR: 0x41811A0 (strcpy) redirected to 0x401E410 (strcpy)
--22638-- Reading syms from /lib/libnss_nis-2.3.6.so (0x43E4000)
--22638--    object doesn't have a symbol table
--22638-- Reading syms from /lib/libnss_files-2.3.6.so (0x43EE000)
--22638--    object doesn't have a symbol table
--22638-- REDIR: 0x41834F0 (rawmemchr) redirected to 0x401EF80 (rawmemchr)
--22638-- REDIR: 0x4181940 (strncpy) redirected to 0x401E4E0 (strncpy)
--22638-- REDIR: 0x417F540 (posix_memalign) redirected to 0x401DBE0 (posix_memalign)
==22638== Invalid read of size 4
==22638==    at 0x80633A5: joblist_append (joblist.c:9)
==22638==    by 0x43AEF38: mod_proxy_trigger (mod_proxy_core.c:2505)
==22638==    by 0x806279F: plugins_call_handle_trigger (plugin.c:432)
==22638==    by 0x804EE6A: lighty_mainloop (server.c:664)
==22638==    by 0x8050404: main (server.c:1705)
==22638==  Address 0x170 is not stack'd, malloc'd or (recently) free'd
==22638== 
==22638== Process terminating with default action of signal 11 (SIGSEGV)
==22638==  Access not within mapped region at address 0x170
==22638==    at 0x80633A5: joblist_append (joblist.c:9)
==22638==    by 0x43AEF38: mod_proxy_trigger (mod_proxy_core.c:2505)
==22638==    by 0x806279F: plugins_call_handle_trigger (plugin.c:432)
==22638==    by 0x804EE6A: lighty_mainloop (server.c:664)
==22638==    by 0x8050404: main (server.c:1705)
--22638-- Discarding syms at 0x43C7000-0x43CF000 in /lib/libnss_compat-2.3.6.so due to munmap()
--22638-- Discarding syms at 0x43E4000-0x43EE000 in /lib/libnss_nis-2.3.6.so due to munmap()
--22638-- Discarding syms at 0x43CF000-0x43E4000 in /lib/libnsl-2.3.6.so due to munmap()
--22638-- Discarding syms at 0x43EE000-0x43F8000 in /lib/libnss_files-2.3.6.so due to munmap()
==22638== 
==22638== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 101 from 2)
==22638== 
==22638== 1 errors in context 1 of 1:
==22638== Invalid read of size 4
==22638==    at 0x80633A5: joblist_append (joblist.c:9)
==22638==    by 0x43AEF38: mod_proxy_trigger (mod_proxy_core.c:2505)
==22638==    by 0x806279F: plugins_call_handle_trigger (plugin.c:432)
==22638==    by 0x804EE6A: lighty_mainloop (server.c:664)
==22638==    by 0x8050404: main (server.c:1705)
==22638==  Address 0x170 is not stack'd, malloc'd or (recently) free'd
--22638-- 
--22638-- supp:   25 Ubuntu-stripped-ld.so
--22638-- supp:   76 dl_relocate_object
==22638== 
==22638== IN SUMMARY: 1 errors from 1 contexts (suppressed: 101 from 2)
==22638== 
==22638== malloc/free: in use at exit: 4,938,437 bytes in 84,755 blocks.
==22638== malloc/free: 177,953 allocs, 93,198 frees, 110,781,851 bytes allocated.
==22638== 
==22638== searching for pointers to 84,755 not-freed blocks.
==22638== checked 37,440,740 bytes.
==22638== 
==22638== LEAK SUMMARY:
==22638==    definitely lost: 2,532 bytes in 36 blocks.
==22638==      possibly lost: 1,084 bytes in 8 blocks.
==22638==    still reachable: 4,934,821 bytes in 84,711 blocks.
==22638==         suppressed: 0 bytes in 0 blocks.
==22638== Rerun with --leak-check=full to see details of leaked memory.
--22638--  memcheck: sanity checks: 405 cheap, 17 expensive
--22638--  memcheck: auxmaps: 0 auxmap entries (0k, 0M) in use
--22638--  memcheck: auxmaps: 0 searches, 0 comparisons
--22638--  memcheck: SMs: n_issued      = 317 (5072k, 4M)
--22638--  memcheck: SMs: n_deissued    = 38 (608k, 0M)
--22638--  memcheck: SMs: max_noaccess  = 65535 (1048560k, 1023M)
--22638--  memcheck: SMs: max_undefined = 0 (0k, 0M)
--22638--  memcheck: SMs: max_defined   = 545 (8720k, 8M)
--22638--  memcheck: SMs: max_non_DSM   = 279 (4464k, 4M)
--22638--  memcheck: max sec V bit nodes:    21 (1k, 0M)
--22638--  memcheck: set_sec_vbits8 calls: 72 (new: 21, updates: 51)
--22638--  memcheck: max shadow mem size:   4769k, 4M
--22638-- translate:            fast SP updates identified: 13,084 ( 85.9%)
--22638-- translate:   generic_known SP updates identified: 1,414 (  9.2%)
--22638-- translate: generic_unknown SP updates identified: 718 (  4.7%)
--22638--     tt/tc: 288,806 tt lookups requiring 337,365 probes
--22638--     tt/tc: 288,806 fast-cache updates, 7 flushes
--22638--  transtab: new        11,593 (263,077 -> 4,169,545; ratio 158:10) [0 scs]
--22638--  transtab: dumped     0 (0 -> ??)
--22638--  transtab: discarded  372 (6,915 -> ??)
--22638-- scheduler: 39,728,134 jumps (bb entries).
--22638-- scheduler: 405/573,514 major/minor sched events.
--22638--    sanity: 406 cheap, 17 expensive checks.
--22638--    exectx: 30,011 lists, 1,704 contexts (avg 0 per list)
--22638--    exectx: 271,253 searches, 270,789 full compares (998 per 1000)
--22638--    exectx: 0 cmp2, 373 cmp4, 0 cmpAll