DevelProblemAndSolution » History » Revision 9
Revision 8 (moo, 2007-08-15 07:42) → Revision 9/19 (moo, 2007-08-15 08:02)
[[PageOutline]]
= Developmenet Problem And Solution =
This page is meant to list Problem And Solutions implemented in, or analyzed for lighttpd. Any simple feature requests without solution should goes to [/trac/newticket ticket system].
= Redundant FastCGI/SCGI etc =
== Problem ==
In lighttpd 1.4, most of the source code of mod_cgi, mod_scgi and mod_proxy were cut'n'pasted from mod_fastcgi.
Why problem: Redundant code make it hard to maintain.
== Solution ==
Combine them into mod_proxy_core and add protocols around it
* the core provides
* config handling
* connect/retry on failure
* fork/restart worker child on dead. (easier to improve native win32 support)
* balancing
* x-sendfile
* the protocol backends take care of
* preparing the environment (most cgi env code can be shared)
* encode/decode data
* handle io
Why: scgi/fastcgi/ejb/http or whatever, are just protocols that we can use to send requests to and get response from backend.
== Status ==
Implemented in 1.5.0
= Redundant and simple config handling =
== Problem ==
The current config handling is done in core and plugin. The core parse it, convert the config, convert it config into a conditional tree, and the plugin pick up some on of the values (use PATCH) and use it.
Why problem: It's sure easier to fix the 1 core instead of all the plugins, the plugin is still handling too much config handling logical and i't hard to improve.
== Solution ==
Combine most of config handling into core, including:
* alloc/free plugin data
* init default values
* insert values from config (into plugin data)
* patch(pick) values from plugin data for connection. this is done by calling a function ptr, but we can kill the string comparisons, lower or higher performance?
== Status ==
Scheduled but not done in 1.5.0, done, and is likely to be post pound to next major release.
= .htaccess support =
== Problem ==
Many users want .htaccess or whatever named config file(s) sit(s) in document root or its sub dir.
Neets:
1. vhost users can do no harm creating such file
2. limited config settings
3. has to be dynamic loaded. i.e., it makes affect when the file is updated without restarting lighttpd
Improvements:
1. high performance, e.g.: "cached"
2. robust, e.g.: syntax error won't stop the server
== Solution ==
No full solution yet from the view of the current source state, but anyway source, as the following precondition should condition must (or may) be supported first:
1. clean config parsing with no memory leak (already)
2. dynamic config updating
3. versioned config: multiple clients may still referenced to current config nodes while the new one is being loaded
Cautions:
* avoid blocking operation (e.g.: disk io) reloading the file
== Status ==
Feature requested.
= c10k problem =
== Problem ==
See [http://www.kegel.com/c10k.html c10k problem].
TODO: add more notes here
== Solution ==
See [http://www.kegel.com/c10k.html c10k problem].
TODO: add more notes here
== Status ==
Implemented since the first release of lighttpd by jan. (While other projects may use [http://monkey.org/~provos/libevent/ libevent])
= Difficult coding for single threaded =
== Problem ==
Lighttpd was a proof of concept for [http://www.kegel.com/c10k.html c10k problem], for which reason it was made single processed/threaded.
Why problem: Single threaded is hard to code because everywhere u come to an syscall that would block, u gotta make it async.
== Solution ==
Move some complex jobs:
1. into a new thread
2. into a forked child process
3. as a backend (e.g. fcgi backend), which mean it is built into standalone executable. backend)
Rules And things can be done in another thread or process:
1. Static file request/response handling should stay in the main io thread
2. Leave most of the current implemention as is unless it cause problem.
3. For auto reload config from file, vhosts settings from mysql etc, should we do them in another thread or just use fam?
4. [http://jan.kneschke.de/projects/lua fastcgized magnet, lua + FastCGI]
5. Move out mod_rrdtool (was deprecated by jan in 1.5, but should be moved instead of removed imho)
6. mod_webdav
TODO: more notes here
Why: Multi thread/process is not evil, fastcgi/php were make into standalone/backend in another process already.
Neither multi-thread/process was the key problem nor single threaded was the key solution to c10k.
The problem was 1:1 model (I'm not talking about thread scheduling model): every 1 connection have to be taken care of by 1 thread or processe, and similarly 1 php instance opens up at least 1 persistent mysql connection.
This consume lots of memory and require lots of processes/threads untill the OS can't handle.
With the current implementation of lighttpd, because of http have to be tcp, you still have to 1 vs 1 for connection vs file handler, it's far more better than too many threades/processes, and it's lucky that we have O(1) OS impl for it like linux epoll etc.
== Status ==
Unknown