Project

General

Profile

Feature #2678

New mod_precompress feature

Added by plantuml almost 2 years ago. Updated 7 months ago.

Status:
Fixed
Priority:
Low
Assignee:
-
Category:
mod_magnet
Target version:
Start date:
2015-10-18
Due date:
% Done:

0%

Estimated time:
Missing in 1.5.x:

Description

The existing mod_compress is a nice feature.
However, it would be usefull to let lighttpd serve pre_compress data.

When serving some file (example /foobar.html), rather than compressing foobar.html file in some dedicated directory (compress.cache-dir ), lighttpd could check if (by chance), an existing file foobar.html.gz in the same folder as foobar.html and serve this file instead (with the right HTTP content-encoding header).

Pros:
- you won't have to clean the compress directory
- since data are precompressed, you save some CPU on first request
- you can use Zopfli (https://github.com/google/zopfli) compression algorithm (which gives better compression ratio).

Cons:
- You have to precompress the data

Depending on their need, people could choose to use either mod_compress, mod_defate or mod_precompress.

Any chance to have such feature ?

Thanks!

content-negotiation.lua (7.58 KB) content-negotiation.lua gstrauss, 2017-02-14 06:20

Related issues

Related to Feature #2736: Test $HTTP["language"] for preferred language?Fixed2016-07-18

Related to Feature #1259: mod_autoext - Apache-like support for automatically adding extensionsFixed

History

#1 Updated by stbuehler almost 2 years ago

  • Category deleted (mod_compress)

should be possible to implement with mod_magnet.

plantuml wrote:

Pros:
- you won't have to clean the compress directory

instead you have to make sure to update both (redundancy is usually bad)

- since data are precompressed, you save some CPU on first request

you don't save CPU time, instead you might possibly waste it. you save some latency though.

- you can use Zopfli (https://github.com/google/zopfli) compression algorithm (which gives better compression ratio).

completely unrelated afaics.

Depending on their need, people could choose to use either mod_compress, mod_deflate or mod_precompress.

There is no mod_deflate in 1.4.x.

#2 Updated by plantuml almost 2 years ago

Yes, you are right about redundancy.

About Zopfli, I wanted to do something similar to http://www.cambus.net/serving-precompressed-content-with-nginx-and-zopfli/

But I've understood that you won't go into that direction.

Thank for your answer.

#3 Updated by darix almost 2 years ago

it can actually be faster/more efficient to read the compressed file from disk and decompress it on the fly when sending it out to clients which do not understand compression. no idea how well it fits with the event driven model though.

#4 Updated by stbuehler almost 2 years ago

plantuml wrote:

About Zopfli, I wanted to do something similar to http://www.cambus.net/serving-precompressed-content-with-nginx-and-zopfli/

Ah right, your point it that one shouldn't run zopfli on the fly, but it would be a good idea for precompress.

But I've understood that you won't go into that direction.

I probably won't do it myself, but I'm open to contributions. There is for example this one by darix: https://gist.github.com/darix/5025010490bb0ea70099, but I'm not happy with the substring match for Accept-Encoding; I think validator.w3.org (and probably others) send x-gzip and expect x-gzip (and not gzip) in return.

darix wrote:

it can actually be faster/more efficient to read the compressed file from disk and decompress it on the fly when sending it out to clients which do not understand compression. no idea how well it fits with the event driven model though.

A "mod_inflate" or "mod_decompress" would indeed solve the redundancy problem, and should probably also take care of the "mod_precompress" part; I see no problem with the event driven model, although we'd probably want to limit the size somehow.

#5 Updated by gstrauss over 1 year ago

  • Category set to 3rd party

#6 Updated by gstrauss about 1 year ago

#7 Updated by gstrauss 7 months ago

  • Related to Feature #2736: Test $HTTP["language"] for preferred language? added

#8 Updated by gstrauss 7 months ago

  • Category changed from 3rd party to mod_magnet
  • Target version set to 1.4.46

This is achievable using some custom lua code with lighttpd mod_magnet

#9 Updated by gstrauss 7 months ago

Attached content-negotiation.lua with code to handle Accept-Language and Accept-Encoding. Lightly tested. YMMV.

server.modules += ( "mod_magnet" )
magnet.attract-physical-path-to = ( "/path/to/content-negotiation.lua" )

#10 Updated by gstrauss 7 months ago

  • Related to Feature #1259: mod_autoext - Apache-like support for automatically adding extensions added

Also available in: Atom