Mod deflate » History » Revision 43
Revision 42 (da, 2022-08-11 07:04) → Revision 43/44 (gstrauss, 2022-08-11 09:34)
h2. mod_deflate
mod_deflate (since lighttpd 1.4.42) enables output compression of responses. (Content-Encoding)
Output compression reduces the network load and can improve the overall throughput of the webserver. All major http-clients support compression by announcing it in the Accept-Encoding header. This is used to negotiate the most suitable compression method. We support deflate, gzip, bzip2, brotli (since 1.4.56), and zstd (since 1.4.59).
deflate (RFC1950, RFC1951) and gzip (RFC1952) depend on zlib.
brotli (RFC7932) is supported in most popular browsers.
zstd (RFC8478)
bzip2 is only supported by lynx and some other console text-browsers, and is no longer recommended; brotli or gzip should be preferred.
Since lighttpd 1.4.56, mod_deflate subsumes and replaces mod_compress. mod_deflate can compress static and dynamic responses, while mod_compress could compress only static files.
h3. Module options:
<pre>
#deflate.mimetypes = ("text/") # prefix matches all text/* Content-Type responses
deflate.mimetypes = ("text/html", "text/plain", "text/css", "text/javascript", "text/xml")
deflate.allowed-encodings = ( "br", "gzip", "deflate" ) # "bzip2" and "zstd" also supported
## optional
deflate.cache-dir = "/path/to/compress/cache"
#deflate.max-compress-size = 131072 # measured in kilobytes, so 131072 indicates 128 MB
#deflate.min-compress-size = 256 # measured in bytes
#deflate.max-loadavg = "3.50"
#deflate.params = ( # since lighttpd 1.4.60 (recommended: leave defaults unless you test your changes for positive effect)
# "BROTLI_PARAM_QUALITY" => 4, # 0..11 (default is 4 (not 11 (BROTLI_DEFAULT_QUALITY) due to time costs of BROTLI_DEFAULT_QUALITY))
# "BROTLI_PARAM_LGWIN" => 22, # 10..24 (default is 22 (BROTLI_DEFAULT_WINDOW))
# "BROTLI_PARAM_MODE" => 0, # 0..2 (default is 0 (BROTLI_MODE_GENERIC)) (1: BROTLI_MODE_TEXT) (2: BROTLI_MODE_FONT)
# "ZSTD_c_compressionLevel" => 3, # -3..22 (default is 3 (ZSTD_CLEVEL_DEFAULT)
# "ZSTD_c_strategy" => 0, # 1..9 (default is 0 (use zstd default strategy))
# "ZSTD_c_windowLog" => 0, # 10..31 (default is 0 (use zstd default windowLog))
# "gzip.level" => 6, # 1..9 (default is 6 (Z_DEFAULT_COMPRESSION))
# "gzip.windowBits" => 6, # 9..15 (default is 15 (MAX_WBITS))
# "gzip.memLevel" => 8, # 1..9 (default is 8 )
# "gzip.strategy" => 0, # 0..4 (default is 0 (Z_DEFAULT_STRATEGY)) (1: Z_FILTERED 2: Z_HUFFMAN_ONLY 3: Z_RLE 4: Z_FIXED)
# "bzip2.blockSize100k" => 9, # 1..9 (default is 9 (block size 900k))
#)
#deflate.compression-level = 9 # deprecated; overloaded; use deflate.params (since 1.4.60) for encoder-specific values)
#deflate.work-block-size = 2048 # deprecated; gzip-specific; currently has no effect (unused) in code
#deflate.output-buffer-size = 8192 # deprecated; gzip- and bzip2-specific; currently has little effect in code (due to mod_deflate not streaming partial responses)
</pre>
@deflate.cache-dir@ (since 1.4.56) is the location under which to store cache of compressed files. (See "Cache revalidation" below for more details.)
@deflate.allowed-encodings@ lists the encodings from which the server will select. Since lighttpd 1.4.60, the ordering of this list is preserved and the server selects the first encoding from this list which matches a value in the client request Accept header.
@deflate.max-compress-size@ is the largest response size that will be compressed.
@deflate.min-compress-size@ is the smallest response size that will be compressed.
@deflate.max-loadavg@ is max system loadavg before bypassing compression (since 1.4.43)
@deflate.params@ configures encoder-specific tunables. It is strongly recommended to omit @deflate.params@ and to use lighttpd defaults unless you test your changes for positive effect. (FYI: the parameters are named to directly map to the names in the C headers of the encoder libraries)
@deflate.compression-level@ (deprecated) is compression level or quality tuning for the underlying compressor. (deprecated; use deflate.params (since 1.4.60) for encoder-specific values)
(more info: @man gzip@ (1..9); @man bzip2@ (1..9); @man -s 3brotli encode.h@ (0..11)
h3. Cache revalidation
The cache is maintained and updated based on the combination of the url-path and the @ETag@ response header.
For static files served by [[Docs_ModStaticFile|mod_staticfile]], @ETag@ response header is by default generated from file inode, mtime, and size ([[Docs_ModStaticFile#Options|ETag data sources]]).
lighttpd Lighttpd does not automatically purge old or stale old/stale cache entries. An external process such as You are required to handle this through, for example, a periodic scheduled cron job may delete stale files. For example, to delete that deletes files older than 10 days: @find /path/to/deflate/cache -type f -mtime +10 | xargs -r rm@. The @deflate.cache-dir@ directory is organized using your URL structure, so you can purge specific paths on different schedules according to your needs.
For [[Docs_ModStaticFile|static files]], revalidation is handled automatically when changes to the source files’ [[Docs_ModStaticFile#Options|ETag data sources]] is detected. Freshness is limited by the [[Server_stat-cache-engineDetails|server.stat-cache-engine]] handling.
For dynamic responses, you must purge cached responses from the cache according to your freshness criteria.
h3. Known Limitations
mod_deflate currently does not stream compressed content in chunks. This affects very large dynamic responses, or dynamic responses sent in chunks with large time lapses between chunks. If the entire response is not ready when the response header is sent, mod_deflate does not compress the response. (e.g. dynamic response from backend and @server.stream-response-body@ is set > 0) While mod_deflate does not handle this use case, the backend producing the streaming response is able to apply an appropriate Content-Encoding itself (instead of mod_deflate).
mod_deflate @deflate.cache-dir@, if set, contains cached output of static files, but does not cache dynamic responses unless the response contains an ETag response header. (dynamic responses, if eligible, are still compressed by mod_deflate before the response is sent to the client.) The same limitation applies to static files if you have disabled ETags (@static-file.etags = "false"@).
mod_deflate must be listed after mod_setenv in @server.modules@ if mod_setenv might be used to force setting "Content-Encoding" response header.
h3. Content-Negotiation
For selection/negotiation of pre-compressed files in the document root, e.g. main.html.br or main.html.gz when main.html is requested, use [[Docs_ModMagnet|mod_magnet]] with custom lua code for "Content-Negotiation":https://redmine.lighttpd.net/projects/lighttpd/wiki/AbsoLUAtion#Content-Negotiation