DevelProblemAndSolution » History » Revision 18
Revision 17 (gstrauss, 2021-08-25 03:17) → Revision 18/19 (gstrauss, 2021-08-27 17:43)
h1. Development Problems And Solutions
{{>toc}}
This page is meant to list development challenges, problems, and (perhaps) solutions analyzed for lighttpd.
Feature requests should be posted in the "lighttpd Forums":https://redmine.lighttpd.net/projects/lighttpd/boards for discussion.
h2. .htaccess-like functionality
h3. Problem
support .htaccess-like functionality
In a monolithic Apache server: Needs:
* robust, e.g. syntax error will not stop the server # vhost users can do no harm creating such file
* dynamically # limited config settings
# has to be dynamic loaded, i.e. takes affect when the file is updated (without restarting lighttpd)
* limited config settings (restricted options) Improvements:
* limited access to config outside request scope; vhost users can not harm others # high performance, e.g.: "cached"
# robust, e.g.: syntax error won't stop the server
h3. Discussion
lighttpd supports graceful server restarts and can reload its configuration without pausing or waiting for existing requests No full solution yet from the view of the current source state, but anyway the following precondition should be supported first:
# clean config parsing with no memory leak (already)
# dynamic config updating
# versioned config: multiple clients may still referenced to finish. Restarting lighttpd after configuration changes current config nodes while the new one is a rare event and being loaded
Cautions:
* avoid blocking operation (e.g.: disk io) reloading the results are much higher performance than dynamically reloading and reprocessing .htaccess at runtime for each and every HTTP request (Apache).
lighttpd is lightweight and there are better options to isolate user configs, such as using separate lighttpd instances running under separate user accounts. file
h3. Solutions
* Option:
** [[Docs_ModRewrite|mod_rewrite]] @url.rewrite-if-not-file@ is a simple solution for many frameworks
e.g. have lighttpd serve static files (such as .js, .css, .jpg, etc) if file exists, or else rewrite to send request to @/index.php@
@url.rewrite-if-not-file = ( "" => "/index.php?path=${url.path}${qsa}" )@
* Option:
** configure standalone application server and have the application implement the .htaccess policy
e.g. via PHP-FPM and per-application php.ini [[TutorialLighttpdAndPHP#Per-directory-PHP-Config|PHP htscanner]]
* Option:
** convert .htaccess into file included in lighttpd.conf
** user-triggered immediate graceful restart (reload config while continuing to handling existing requests in a background process)
@server.feature-flags += ("server.graceful-restart-bg" => "enable")@ ([[Server_feature-flagsDetails|server.feature-flags]])
@server.systemd-socket-activation = "enable"@
* Option:
** convert .htaccess into custom lua script run with [[Docs_ModMagnet|mod_magnet]]
Common use of .htaccess for url-rewriting and access control can be written into a custom lua script.
@$HTTP["url"] =~ "^/app(?:/|$)" { magnet.attract-raw-url-to = "/path/to/app/.htaccess.lua" }@
[[Docs_ModMagnet|mod_magnet]] scripts are cached and are dynamically reloaded when modified.
If non-admin users are writing lighttpd.conf include files or lua scripts, then a lighttpd frontend should be configured with [[Docs_ModProxy|mod_proxy]] to reverse proxy to a per-application (or per-user) lighttpd backend instance, e.g. running [[Docs_ModMagnet|mod_magnet]], and potentially permissioned to let the application owner (or user) configure lighttpd.conf and trigger restart of the per-application (or per-user) lighttpd instance.
h2. Difficulties Difficult coding for single threaded
h3. Problem
Lighttpd was a proof of concept for "c10k problem":http://www.kegel.com/c10k.html, for which reason it was made single processed/threaded.
Why problem: Single threaded is hard to code because everywhere you come to a syscall that would block, you have to make it async.
h3. Discussion
Move some complex jobs:
# into a new thread
# into a forked child process
# as a backend (e.g. fcgi backend), which mean it is built into standalone executable.
Rules and And things can be done in another thread or process:
# Static file request/response handling should stay in the main io thread
# Leave most of the current implemention as-is unless it cause problem.
# For auto reload config from file, vhosts settings from mysql etc, should we do them in another thread or just use fam?
# "fastcgized magnet, lua + FastCGI":http://jan.kneschke.de/projects/lua
# Move out mod_rrdtool (was deprecated by jan in 1.5 (abandoned), but should be moved instead of removed imho)
TODO: more notes here
Why: Multi thread/process is not evil, fastcgi/php were make into standalone/backend process already.
Neither multi-thread/process was the key problem nor single threaded was the key solution to c10k.
The problem was 1:1 model (I'm not talking about thread scheduling model): every 1 connection have to be taken care of by 1 thread or process, and similarly 1 php instance opens up at least 1 persistent connection to mysql.
This consume lots of memory and requires lots of processes/threads until the OS can't handle.
With the current implementation of lighttpd, using TCP for HTTP, you still have to 1 vs 1 for connection vs file handler, it's far better than too many threads/processes, and it's lucky that we have O(1) OS impl for it like linux epoll etc.