DevelProblemAndSolution » History » Revision 12
Revision 11 (moo, 2007-08-15 12:42) → Revision 12/19 (moo, 2007-08-15 12:42)
[[PageOutline]] h1. = Developmenet Problem And Solution = This page is meant to list Problem And Solutions implemented in, or analyzed for lighttpd. Any simple feature requests without solution should goes to [[tracnewticket|ticket system]]. h1. [/trac/newticket ticket system]. = Redundant FastCGI/SCGI etc h2. = == Problem == In lighttpd 1.4, most of the source code of mod_cgi, mod_scgi and mod_proxy were cut'n'pasted from mod_fastcgi. Why problem: Redundant code make it hard to maintain. h2. == Solution == Combine them into mod_proxy_core and add protocols around it ** * the core provides ***** * config handling ***** * connect/retry on failure ***** * fork/restart worker child on dead. (easier to improve native win32 support) ***** * balancing ***** * x-sendfile ** * the protocol backends take care of ***** * preparing the environment (most cgi env code can be shared) ***** * encode/decode data ***** * handle io Why: scgi/fastcgi/ejb/http or whatever, are just protocols that we can use to send requests to and get response from backend. h2. == Status == Implemented in 1.5.0 h1. = Redundant and simple config handling h2. = == Problem == The current config handling is done in core and plugin. The core parse the config, convert it into a conditional tree, and the plugin pick up some of the values (use PATCH) and use it. Why problem: It's sure easier to fix the 1 core instead of all the plugins, the plugin is still handling too much config handling logical and i't hard to improve. h2. == Solution == Combine most of config handling into core, including: * alloc/free plugin data * init default values * insert values from config (into plugin data) * patch(pick) values from plugin data for connection. this is done by calling a function ptr, but we can kill the string comparisons, lower or higher performance? h2. == Status == Scheduled but not done in 1.5.0, and is likely to be post pound to next major release. h1. = .htaccess support h2. = == Problem == Many users want .htaccess or whatever named config file(s) sit(s) in document root or its sub dir. Neets: # 1. vhost users can do no harm creating such file # 2. limited config settings # 3. has to be dynamic loaded. i.e., it makes affect when the file is updated without restarting lighttpd Improvements: # 1. high performance, e.g.: "cached" # 2. robust, e.g.: syntax error won't stop the server h2. == Solution == No full solution yet from the view of the current source state, but anyway the following precondition should be supported first: # 1. clean config parsing with no memory leak (already) # 2. dynamic config updating # 3. versioned config: multiple clients may still referenced to current config nodes while the new one is being loaded Cautions: * avoid blocking operation (e.g.: disk io) reloading the file h2. == Status == Feature requested. h1. = c10k problem h2. = == Problem == See "c10k problem":http://www.kegel.com/c10k.html. [http://www.kegel.com/c10k.html c10k problem]. TODO: add more notes here h2. == Solution == See "c10k problem":http://www.kegel.com/c10k.html. [http://www.kegel.com/c10k.html c10k problem]. TODO: add more notes here h2. == Status == Implemented since the first release of lighttpd by jan. (While other projects may use "libevent":http://monkey.org/~provos/libevent/) h1. [http://monkey.org/~provos/libevent/ libevent]) = Difficult coding for single threaded h2. = == Problem == Lighttpd was a proof of concept for "c10k problem":http://www.kegel.com/c10k.html, [http://www.kegel.com/c10k.html c10k problem], for which reason it was made single processed/threaded. Why problem: Single threaded is hard to code because everywhere u come to an syscall that would block, u gotta make it async. h2. == Solution == Move some complex jobs: # 1. into a new thread # 2. into a forked child process # 3. as a backend (e.g. fcgi backend), which mean it is built into standalone executable. Rules And things can be done in another thread or process: # 1. Static file request/response handling should stay in the main io thread # 2. Leave most of the current implemention as is unless it cause problem. # 3. For auto reload config from file, vhosts settings from mysql etc, should we do them in another thread or just use fam? # "fastcgized 4. [http://jan.kneschke.de/projects/lua fastcgized magnet, lua + FastCGI":http://jan.kneschke.de/projects/lua # FastCGI] 5. Move out mod_rrdtool (was deprecated by jan in 1.5, but should be moved instead of removed imho) # 6. mod_webdav TODO: more notes here Why: Multi thread/process is not evil, fastcgi/php were make into standalone/backend process already. Neither multi-thread/process was the key problem nor single threaded was the key solution to c10k. The problem was 1:1 model (I'm not talking about thread scheduling model): every 1 connection have to be taken care of by 1 thread or process, and similarly 1 php instance opens up at least 1 persistent connection to mysql. This consume lots of memory and requires lots of processes/threads untill the OS can't handle. With the current implementation of lighttpd, because of http have to be tcp, you still have to 1 vs 1 for connection vs file handler, it's far more better than too many threades/processes, and it's lucky that we have O(1) OS impl for it like linux epoll etc. h2. == Status == Unknown h1. = Buffer whole POST request or not h2. = == Problem h2. == == Solution h2. == == Status == both 1.4 and 1.5 uses buffer all post data for each request currently.