Project

General

Profile

[Solved] How to exclude "PRI * HTTP/2.0" lines from the access.log

Added by cadence 10 months ago

Hi,

I'm seeing thousands of lines like this in my access.log:
xx.xx.xx.xx YYYYYYY.com - [02/Jul/2023:22:55:33 -0400] "PRI * HTTP/2.0" 100 116 "-" "-"

I know it is part of HTTP/2 protocol, but Lighttpd shows me this line with every client request. So I'm getting thousands of useless lines like this. How can I exclude this from showing up in access.log?

I tried the following:

$HTTP["request-method"] == "PRI" { accesslog.filename = "" }
$HTTP["url"] == "\*" { accesslog.filename = "" }
$HTTP["url"] == "*" { accesslog.filename = "" }

But none of these work. Any suggestions on how to get rid of these logs? Thanks.


Replies (6)

RE: How to exclude "PRI * HTTP/2.0" lines from the access.log - Added by gstrauss 10 months ago

I know it is part of HTTP/2 protocol, but Lighttpd shows me this line with every client request. So I'm getting thousands of useless lines like this

lighttpd mod_accesslog reports individual requests on an HTTP/2 connection as separate log entries, AND mod_accesslog reports the overhead of the entire HTTP/2 connection on the "PRI * HTTP/2.0" line. This is by design and is not useless to everyone, even if you think it useless to you. The 100 status is a filler and can be ignored, but other information might be relevant to the admin, such as bytes sent or timestamps, or other connection-level info that the admin has configured to be part of accesslog.format

How can I exclude this from showing up in access.log?

Use a piped logger in accesslog.filename (mod_accesslog) and filter whatever you like before outputting to a file or database.

RE: [Solved] How to exclude "PRI * HTTP/2.0" lines from the access.log - Added by cadence 10 months ago

Thanks for the answer. The piped logger method seems to require executing a separate process with each line that Lighttpd wants to log. This sounds very expensive in terms of CPU usage. I will probably be better off just letting it log it all.

I was hoping that it would be possible to filter these out using lighttpd's native filtering. $HTTP["request-method"] sounded like it should work.

RE: [Solved] How to exclude "PRI * HTTP/2.0" lines from the access.log - Added by gstrauss 10 months ago

The piped logger method seems to require executing a separate process with each line that Lighttpd wants to log.

no no no.

A piped logger is a separate process, but it is started by the lighttpd daemon and the piped logger lifetime is the same as that of the lighttpd daemon. A piped logger is a filter, written in any language you like, which takes log lines on standard input, filters, and prints log lines to standard output (or elsewhere, as desired). The piped logger should exit when it receives reads EOF for end of stream.


Another option is to let lighttpd continue to do the logging, which is very efficient, and you can post-process the logs after log rotation to filter the logs.

RE: [Solved] How to exclude "PRI * HTTP/2.0" lines from the access.log - Added by cadence 9 months ago

While piped logger might be a viable solution for some, I'm certainly not going to be developing a piece of software, testing, and maintaining it just to filter out one particular line from the log files. I think this PRI stuff should be hidden behind a DEBUG flag ( or log-noise ).

In the meantime, my solution is to use this line to get rid of the noise:

$HTTP["request-method"] =~ "^(GET|PUT|POST|HEAD|OPTIONS|CONNECT|TRACE|PATCH|DELETE)$" { accesslog.filename = "/var/log/lighttpd/access.log" }

So far, it looks like it continues to log all the information I care about, and I don't see the PRI stuff anymore. I will come back here and update this post if it turns out that it filters out some legitimate information.

RE: [Solved] How to exclude "PRI * HTTP/2.0" lines from the access.log - Added by gstrauss 9 months ago

I suppose you could also use $HTTP["request-method"] != "" { accesslog.filename = "/var/log/lighttpd/access.log" }

The next version of lighttpd will allow you to set $HTTP["request-method"] == "PRI" { accesslog.filename = "" }, though I have intentionally not enabled other config conditions on HTTP/2 command channel (id 0), e.g for the URL *. I may reconsider that choice in the future.

--- a/src/connections.c
+++ b/src/connections.c
@@ -466,6 +466,11 @@ connection_transition_h2 (request_st * const h2r, connection * const con)
     buffer_copy_string_len(&h2r->target_orig, CONST_STR_LEN("*"));
     buffer_copy_string_len(&h2r->uri.path,    CONST_STR_LEN("*"));
     h2r->http_method = HTTP_METHOD_PRI;
+    /*(setting all bits might break existing lighttpd.conf,
+     * which e.g. might make assumptions in configs for "OPTIONS *",
+     * so probably better to leave other conditions unset)*/
+    /*h2r->conditional_is_valid = ~0u;*/
+    h2r->conditional_is_valid |= (1 << COMP_HTTP_REQUEST_METHOD);
     h2r->reqbody_length = -1; /*(unnecessary for h2r?)*/
     h2r->conf.stream_request_body |= FDEVENT_STREAM_REQUEST_POLLIN;

RE: [Solved] How to exclude "PRI * HTTP/2.0" lines from the access.log - Added by cadence 9 months ago

This looks awesome! It will definitely help manage the size of log files.

Thanks!

    (1-6/6)