Feature #2361
closedadd support for pcre JIT (patch included)
Description
With PCRE version 8.20 just-in-time compiling is introduced that significantly speeds up the processing of regular expressions in many cases. As regular expressions are an important part of lighttpd, this speedup might be of high importance.
The attached patch adds simple JIT support to pcre_study() functions in lighttpd and replaces the pcre_free() for pcre_free_study() in relevant places. If JIT does not succeed (incompatible regexp, too small stack) PCRE automatically falls back to old style processing so there should be no problem with this.
The default JIT stack size is 32kb (as in the patch). Should this be not enough for usual lighttpd regular expressions, a customly sized stack may be allocated with pcre_jit_stack_alloc(), assigned with pcre_assign_jit_stack() and freed with pcre_jit_stack_free().
More information about the API in pcrejit(3).
Files
Updated by mm about 13 years ago
In the patch, I forgot to replace the last
#ifdef PCRE_CONFIG_JITwith
#ifdef PCRE_STUDY_JIT_COMPILEto be on the safe side.
PCRE_CONFIG_JIT is used to test the availability of JIT support, as the following example shows:
int rc, jit_support;
(void) pcre_config(PCRE_CONFIG_JIT, &rc);
if (rc)
jit_support=1;
else
jit_support=0;
Updated by mm about 13 years ago
Here is a more complete patch that also adds pcre_study() to mod_rewrite and other relevant modules.
Next step would be not using the machine stack but assigning an individual pcre_jit stack e.g. per module.
Updated by mm about 13 years ago
This third version also includes a per-server common PCRE JIT stack of size 32kB-1024kB
Updated by darix about 13 years ago
- Status changed from New to Need Feedback
a few comments
- get rid of the ugly static ints
- make a server wide setting server.use-pcre-jit (see server.errorlog as example) to make it possible to disable the feature.
the setting should warn the user if the he tries to enable it without support in their PCRE library. this also should make it possible to have all the #ifdef fun for this thing in one place. - depending on your benchmarks, you can enable the setting by default.
Updated by mm about 13 years ago
Ok, I have updated your patch according to your description. You cannot actually have the #ifdefs removed.
the macro PCRE_STUDY_JIT_COMPILE, and the functions pcre_assign_jit_stack(), pcre_jit_stack_alloc(), pcre_jit_stack_free, pcre_free_study() are otherwise not available and code won't compile (that's why I check for PCRE_CONFIG_JIT).
One problem that has raised is configparser.y - can you use srv.srvconf inside configparser.y? I guess not.
For now I have removed the code from there, but that is the last part where it is missing.
For starters the option is disabled by default.
Updated by stbuehler about 13 years ago
Imho this patch should only be added, if you can provide benchmarks which show this is actually useful (as 1.4.x is a stable branch, i don't want to add features - and risk breaking something - without a good reason).
Then:- imho the config option should be always available, but it should give a warning if the user enabled the option and pcre does not support it (it looks like even if the new api is present, jit still can be disabled)
- as the patch should only be added if we know it provides better performance, i suggest enabling the option by default if the new api is present and jit support is present.
- the regular expressions in the config file should be jit compiled too; so perhaps we should "re"-study the expressions after the config has been parsed (or move the study)
- darix did not ask you to get rid of all the #ifdef magic, but to put it in one place, which is certainly possible.
- the current src/configparser.y changes are useless
- the jit stack size(s) should be configurable (whatever those values mean, they look too large for embedded systems)
Updated by stbuehler over 12 years ago
- Status changed from Need Feedback to Missing Feedback
- Target version deleted (
1.4.x)
Also available in: Atom