DevelProblemAndSolution » History » Revision 16
Revision 15 (gstrauss, 2016-09-11 00:44) → Revision 16/19 (gstrauss, 2018-10-31 11:09)
[[PageOutline]] h1. Development Problems Developmenet Problem And Solutions Solution This page is meant to list Problems and Problem And Solutions implemented in, or analyzed for lighttpd. Any simple feature requests without solution should go goes to "ticket system":http://redmine.lighttpd.net/projects/lighttpd/issues/new. h1. Redundant FastCGI/SCGI etc h2. Problem In lighttpd 1.4, most of the source code of mod_cgi, mod_scgi and mod_proxy were cut'n'pasted from mod_fastcgi. Why problem: Redundant code make it hard to maintain. h2. Solution Combine them into mod_proxy_core and add protocols around it ** the core provides ***** config handling ***** connect/retry on failure ***** fork/restart worker child on dead. (easier to improve native win32 support) ***** balancing ***** x-sendfile ** the protocol backends take care of ***** preparing the environment (most cgi env code can be shared) ***** encode/decode data ***** handle io Why: scgi/fastcgi/ejb/http or whatever, are just protocols that we can use to send requests to and get response from backend. h2. Status Implemented in 1.4.46 Implemented in 1.5.0 (abandoned) h1. Redundant and simple config handling h2. Problem The current config handling is done in core and plugin. The core parse the config, convert it into a conditional tree, and the plugin pick up some of the values (use PATCH) and use it. Why problem: It's sure easier to fix the 1 core instead of all the plugins, the plugin is still handling too much config handling logical and it is hard to improve. h2. Solution Combine most of config handling into core, including: * alloc/free plugin data * init default values * insert values from config (into plugin data) * patch(pick) values from plugin data for connection. this is done by calling a function ptr, but we can kill the string comparisons, lower or higher performance? h2. Status not yet implemented h1. .htaccess support h2. Problem Many users want .htaccess or whatever named config file(s) sit(s) in document root or its sub dir. Neets: # vhost users can do no harm creating such file # limited config settings # has to be dynamic loaded. i.e., it makes affect when the file is updated without restarting lighttpd Improvements: # high performance, e.g.: "cached" # robust, e.g.: syntax error won't stop the server h2. Solution No full solution yet from the view of the current source state, but anyway the following precondition should be supported first: # clean config parsing with no memory leak (already) # dynamic config updating # versioned config: multiple clients may still referenced to current config nodes while the new one is being loaded Cautions: * avoid blocking operation (e.g.: disk io) reloading the file h2. Status Feature requested. h1. c10k problem h2. Problem See "c10k problem":http://www.kegel.com/c10k.html. TODO: add more notes here h2. Solution See "c10k problem":http://www.kegel.com/c10k.html. TODO: add more notes here h2. Status Implemented since the first release of lighttpd by jan. (While other projects may use "libevent":http://monkey.org/~provos/libevent/) h1. Difficult coding for single threaded h2. Problem Lighttpd was a proof of concept for "c10k problem":http://www.kegel.com/c10k.html, for which reason it was made single processed/threaded. Why problem: Single threaded is hard to code because everywhere u come to an syscall that would block, u gotta make it async. h2. Solution Move some complex jobs: # into a new thread # into a forked child process # as a backend (e.g. fcgi backend), which mean it is built into standalone executable. Rules And things can be done in another thread or process: # Static file request/response handling should stay in the main io thread # Leave most of the current implemention as is unless it cause problem. # For auto reload config from file, vhosts settings from mysql etc, should we do them in another thread or just use fam? # "fastcgized magnet, lua + FastCGI":http://jan.kneschke.de/projects/lua # Move out mod_rrdtool (was deprecated by jan in 1.5 (abandoned), but should be moved instead of removed imho) # mod_webdav TODO: more notes here Why: Multi thread/process is not evil, fastcgi/php were make into standalone/backend process already. Neither multi-thread/process was the key problem nor single threaded was the key solution to c10k. The problem was 1:1 model (I'm not talking about thread scheduling model): every 1 connection have to be taken care of by 1 thread or process, and similarly 1 php instance opens up at least 1 persistent connection to mysql. This consume lots of memory and requires lots of processes/threads untill the OS can't handle. With the current implementation of lighttpd, because of http have to be tcp, you still have to 1 vs 1 for connection vs file handler, it's far more better than too many threades/processes, and it's lucky that we have O(1) OS impl for it like linux epoll etc. h2. Status Unknown h1. Buffer whole POST request or not h2. Problem h2. Solution h2. Status lighttpd buffers all POST data in memory for each request *fixed* in 1.4.40 (not fixed in 1.5 (abandoned))