Development Problems And Solutions¶

• Table of contents
• Development Problems And Solutions
  • .htaccess-like functionality
    • Problem
    • Discussion
    • Solutions
  • Difficulties coding for single threaded
    • Problem
    • Discussion

This page is meant to list development challenges, problems, and (perhaps) solutions analyzed for lighttpd.
Feature requests should be posted in the lighttpd Forums for discussion.

.htaccess-like functionality¶

Problem¶

support .htaccess-like functionality

In a monolithic Apache server, .htaccess is:

• robust, e.g. syntax error will not stop the server
• dynamically loaded, i.e. takes affect when the file is updated (without restarting Apache)
• limited config settings (restricted options)
• limited access to config outside request scope; vhost users can not harm others

Discussion¶

lighttpd supports graceful server restarts and can reload its configuration without pausing or waiting for existing requests to finish. Restarting lighttpd after configuration changes is a rare event and the results are much higher performance than dynamically reloading and reprocessing .htaccess at runtime for each and every HTTP request (Apache).

lighttpd is lightweight and there are better options to isolate user configs, such as using separate lighttpd instances running under separate user accounts.

Solutions¶

• Option:
  • mod_rewrite url.rewrite-if-not-file is a simple solution for many frameworks
    e.g. have lighttpd serve static files (such as .js, .css, .jpg, etc) if file exists, or else rewrite to send request to /index.php
url.rewrite-if-not-file = ( "" => "/index.php?path=${url.path}${qsa}" )

• Option:
  • configure standalone application server and have the application implement the .htaccess policy
    e.g. via PHP-FPM and per-application php.ini PHP htscanner

• Option:
  • convert .htaccess into file included in lighttpd.conf
  • user-triggered immediate graceful restart (reload config while continuing to handling existing requests in a background process)
    server.feature-flags += ("server.graceful-restart-bg" => "enable") (server.feature-flags)
    server.systemd-socket-activation = "enable"

• Option:
  • convert .htaccess into custom lua script run with mod_magnet
    Common use of .htaccess for url-rewriting and access control can be written into a custom lua script.
    $HTTP["url"] =~ "^/app(?:/|$)" { magnet.attract-raw-url-to = "/path/to/app/.htaccess.lua" }
    mod_magnet scripts are cached and are dynamically reloaded when modified.

If non-admin users are writing lighttpd.conf include files or lua scripts, then a lighttpd frontend should be configured with mod_proxy to reverse proxy to a per-application (or per-user) lighttpd backend instance, e.g. running mod_magnet, and potentially permissioned to let the application owner (or user) configure lighttpd.conf and trigger restart of the per-application (or per-user) lighttpd instance.

Difficulties coding for single threaded¶

Problem¶

Lighttpd was a proof of concept for c10k problem, for which reason it was made single processed/threaded.
Why problem: Single threaded is hard to code because everywhere you come to a syscall that would block, you have to make it async.

Discussion¶

Move some complex jobs:

1. into a new thread
2. into a forked child process
3. as a backend (e.g. fcgi backend), which mean it is built into standalone executable.

Rules and things can be done in another thread or process:

1. Static file request/response handling should stay in the main io thread
2. Leave most of the current implemention as-is unless it cause problem.
3. For auto reload config from file, vhosts settings from mysql etc, should we do them in another thread or just use fam?
4. fastcgized magnet, lua + FastCGI
5. Move out mod_rrdtool (was deprecated by jan in 1.5 (abandoned), but should be moved instead of removed imho)
   TODO: more notes here

Why: Multi thread/process is not evil, fastcgi/php were make into standalone/backend process already.
Neither multi-thread/process was the key problem nor single threaded was the key solution to c10k.
The problem was 1:1 model (I'm not talking about thread scheduling model): every 1 connection have to be taken care of by 1 thread or process, and similarly 1 php instance opens up at least 1 persistent connection to mysql.
This consume lots of memory and requires lots of processes/threads until the OS can't handle.
With the current implementation of lighttpd, using TCP for HTTP, you still have to 1 vs 1 for connection vs file handler, it's far better than too many threads/processes, and it's lucky that we have O(1) OS impl for it like linux epoll etc.