Lighttpd

do you have an interest in a new plugin that aims to log in a manner similar to accesslog, but aimed at the transport layer data.

lighttpd has multiple modules for collecting statistics (mod_accesslog, mod_rrdtool) and display (mod_status, lua mod_status)

Additional instrumentation and statistics collection (or display) is welcome.

However, to be accepted into the core lighttpd distribution, the overhead of the module should be limited almost entirely to the module, and should not detrimentally affect the performance or resource usage of lighttpd when the module is not loaded. Expanding core data structures used per-request or per-connection is less likely to be accepted into the core.

Even if not accepted into the core lighttpd distribution, it could be linked at Docs_UserWrittenMods

https://github.com/mux99/lighttpd1.4/tree/mptcplog

this implementation is a test and is quite crude

understood. so please keep that in mind when reading these comments.

The module is not yet ready for inclusion in lighttpd, but you already knew that and have not presented it as ready.
Perhaps we should not be discussing inclusion in lighttpd yet.
Still, here is a very quick review.

lighttpd is very portable and your new module is directed at Linux.
lighttpd supports build systems autoconf, CMake, SCONS, and meson. Your patch includes only autoconf and meson.
Your patch also includes your own custom test.sh for your personal use. That's fine for your development.

You should #include "sys-socket.h" to make it more clear from where SOL_MPTCP and others come, instead of getting it through #include "base.h" inclusion of "sock_addr.h"

This assumes a modern gcc or clang or compile which supports attribute aligned.
typedef uint64_t __attribute__((aligned(8))) __aligned_u64;
See "first.h" for some basic logic for portability. You can borrow a preprocessor block from first.h and do something similar for __aligned__ in mod_mptcplog.c. Similarly most of mod_mptcplog.c should be wrapped in #ifdef __linux__

At first glance, there appears to be some mistakes in SETDEFAULTS_FUNC(mod_mptcplog_set_defaults). You might want to compare to the SETDEFAULTS_FUNC in mod_accesslog.c. mod_mptcplog_merge_config(&p->defaults, cpv); looks out of place to me. Did you mean for it to be inside the for loop?

The information you are logging is connection-level. Do you want to log it per-request on HTTP/1.1 keep-alive or HTTP/2 connections, which may have many requests? p->handle_uri_clean = mod_mptcplog_uri_handler; might consider using p->handle_connection_accept or p->handle_connection_close.

The log format is questionable. You might consider unconditionally logging structured data, such as json, or even a binary format to make parsing and using this information easier. Consider if high resolution timestamps are needed or useful, or conversely, if strftime() is needed or useful. You could log a netstring of info_len and info (struct mptcp_info) binary records, along with binary remote IP and binary timestamp. Related, this module should operate only on TCP sockets, not on unix domain sockets.

BTW, an alternative to a custom lighttpd module is to write a lua script (or C module for lua) for a lua script run by mod_magnet. With a small change to lighttpd mod_magnet to add an accessor function, I could theoretically expose the connection file descriptor to lua. Then, you could getsockopt() and log the results from the lua script.

Thx a lot, those are really helpful comments!
I will keep them i mind while pursuing development.

Concerning the per request/ per connection, it is by design to allow the gathering of stats over time with-out resorting to packet capture. (since as you said, multiple request can occur on the same connection)

Concerning the per request/ per connection, it is by design to allow the gathering of stats over time with-out resorting to packet capture.

Perhaps you want to look at BPF (Berkeley Packet Filter) and run your code as a network filter rather than in lighttpd.
https://www.kernel.org/doc/html/latest/networking/filter.html

First, thank you for the helpful review.
second, after looking more into it, the usefulness of such a plugin seems to be minimal. So for the moment, development is halted in favor of other projects.

This discussion will be linked to internally, so as not to loose the review.

Status: lighttpd 1.4.76 has a server.feature-flags option to enable MPTCP on listening sockets (and is not enabled by default). However, please note that MPTCP is not compatible with KTLS, and so KTLS will fail and fallback to user-space encryption if MPTCP is enabled on listening sockets.

The rest of the discussion above makes some suggestions regarding statistics collection of MPTCP use.