Project

General

Profile

Feature #2678

New mod_precompress feature

Added by plantuml over 1 year ago. Updated about 1 month ago.

Status:
Fixed
Priority:
Low
Assignee:
-
Category:
mod_magnet
Target version:
Start date:
2015-10-18
Due date:
% Done:

0%

Missing in 1.5.x:

Description

The existing mod_compress is a nice feature.
However, it would be usefull to let lighttpd serve pre_compress data.

When serving some file (example /foobar.html), rather than compressing foobar.html file in some dedicated directory (compress.cache-dir ), lighttpd could check if (by chance), an existing file foobar.html.gz in the same folder as foobar.html and serve this file instead (with the right HTTP content-encoding header).

Pros:
- you won't have to clean the compress directory
- since data are precompressed, you save some CPU on first request
- you can use Zopfli (https://github.com/google/zopfli) compression algorithm (which gives better compression ratio).

Cons:
- You have to precompress the data

Depending on their need, people could choose to use either mod_compress, mod_defate or mod_precompress.

Any chance to have such feature ?

Thanks!

content-negotiation.lua View (7.58 KB) gstrauss, 2017-02-14 06:20


Related issues

Related to Feature #2736: Test $HTTP["language"] for preferred language? Fixed 2016-07-18
Related to Feature #1259: mod_autoext - Apache-like support for automatically adding extensions Fixed

History

#1 Updated by stbuehler over 1 year ago

  • Category deleted (mod_compress)

should be possible to implement with mod_magnet.

plantuml wrote:

Pros:
- you won't have to clean the compress directory

instead you have to make sure to update both (redundancy is usually bad)

- since data are precompressed, you save some CPU on first request

you don't save CPU time, instead you might possibly waste it. you save some latency though.

- you can use Zopfli (https://github.com/google/zopfli) compression algorithm (which gives better compression ratio).

completely unrelated afaics.

Depending on their need, people could choose to use either mod_compress, mod_deflate or mod_precompress.

There is no mod_deflate in 1.4.x.

#2 Updated by plantuml over 1 year ago

Yes, you are right about redundancy.

About Zopfli, I wanted to do something similar to http://www.cambus.net/serving-precompressed-content-with-nginx-and-zopfli/

But I've understood that you won't go into that direction.

Thank for your answer.

#3 Updated by darix over 1 year ago

it can actually be faster/more efficient to read the compressed file from disk and decompress it on the fly when sending it out to clients which do not understand compression. no idea how well it fits with the event driven model though.

#4 Updated by stbuehler over 1 year ago

plantuml wrote:

About Zopfli, I wanted to do something similar to http://www.cambus.net/serving-precompressed-content-with-nginx-and-zopfli/

Ah right, your point it that one shouldn't run zopfli on the fly, but it would be a good idea for precompress.

But I've understood that you won't go into that direction.

I probably won't do it myself, but I'm open to contributions. There is for example this one by darix: https://gist.github.com/darix/5025010490bb0ea70099, but I'm not happy with the substring match for Accept-Encoding; I think validator.w3.org (and probably others) send x-gzip and expect x-gzip (and not gzip) in return.

darix wrote:

it can actually be faster/more efficient to read the compressed file from disk and decompress it on the fly when sending it out to clients which do not understand compression. no idea how well it fits with the event driven model though.

A "mod_inflate" or "mod_decompress" would indeed solve the redundancy problem, and should probably also take care of the "mod_precompress" part; I see no problem with the event driven model, although we'd probably want to limit the size somehow.

#5 Updated by gstrauss 11 months ago

  • Category set to 3rd party

#6 Updated by gstrauss 6 months ago

#7 Updated by gstrauss about 1 month ago

  • Related to Feature #2736: Test $HTTP["language"] for preferred language? added

#8 Updated by gstrauss about 1 month ago

  • Category changed from 3rd party to mod_magnet
  • Target version set to 1.4.46

This is achievable using some custom lua code with lighttpd mod_magnet

#9 Updated by gstrauss about 1 month ago

Attached content-negotiation.lua with code to handle Accept-Language and Accept-Encoding. Lightly tested. YMMV.

server.modules += ( "mod_magnet" )
magnet.attract-physical-path-to = ( "/path/to/content-negotiation.lua" )

#10 Updated by gstrauss about 1 month ago

  • Related to Feature #1259: mod_autoext - Apache-like support for automatically adding extensions added

Also available in: Atom