Lighttpd

This means that there is no possibility to access/read the URL after the hashmark inside mod_magnet?

Background: I want to authenticate against authentik using openid and the callback of authentik looks like this:

/callback/openid#access_token=...&id_token=...&...

lighty.r.req_attr["request.orig-uri"]

lighty.r.req_attr["request.orig-uri"]
Note that is the original request sent by client and remains the same (unchanged) by rewrites or other internal redirects.

Note: I tested this with HTTP/1.0. I'll see what curl sends for HTTP/2.0 in a sec.

As I wrote in my initial request, I already tried lighty.r.req_attr["request.orig-uri"] with no success: wget -SqO- 'http://localhost/bla#blabla&test=1'

2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_header: key Host, value: localhost
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_header: key User-Agent, value: Wget/1.21.4
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_header: key Accept, value: */*
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_header: key Accept-Encoding, value: identity
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_header: key Connection, value: Keep-Alive
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key physical.path, value: /var/www/bla
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key physical.rel-path, value: /bla
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key physical.doc-root, value: /var/www/
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key physical.basedir, value: /var/www/
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key uri.path, value: /bla
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key uri.path-raw, value: /bla
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key uri.scheme, value: http
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key uri.authority, value: localhost
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key uri.query, value: nil
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key request.method, value: GET
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key request.uri, value: /bla
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key request.orig-uri, value: /bla
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key request.path-info, value: nil
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key request.remote-ip, value: ::1
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key request.remote-addr, value: ::1
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key request.remote-port, value: 45630
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key request.server-addr, value: ::1
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key request.server-port, value: 80
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key request.protocol, value: HTTP/1.1
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key request.server-name, value: localhost
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_attr: key request.stage, value: handle-req
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_item: key http_status, value: 0
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_item: key req_header_len, value: 127
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_item: key resp_header_len, value: 0
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_item: key bytes_in, value: 127
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_item: key bytes_out, value: 0
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_item: key stream_id, value: 0
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_item: key req_count, value: 1
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) r.req_item: key keep_alive, value: 1
2024-02-14 07:16:41: (mod_magnet.c.2945) (lua-print) lighty.r.req_item: key start_time, value: 1707891401

Configuration: magnet.attract-physical-path-to = ( "/etc/lighttpd/lua/debug.lua" )

I do not think curl or wget are sending the fragment in the request. lighttpd is not receiving the fragment.

I do not think curl or wget are sending the fragment in the request. lighttpd is not receiving the fragment.
==> confirmed with strace: curl and wget strip the fragment before sending the request.

lighttpd will include the fragment in lighty.r.req_attr["request.orig-uri"] if received in the request, but I want to emphasize that sending fragment to the server is fragile at best, and many clients (or intermediary proxies) will strip the fragment when sending the request.

Background: I want to authenticate against authentik using openid and the callback of authentik looks like this:

/callback/openid#access_token=...&id_token=...&...

I am not familiar with authentik.
You may have to rewrite the fragment into the query string or path-info if you're going to forward it to a server, or you need to find a tool which does not strip the fragment from the URI before sending the request.

I used wget as client only for this ticket, orignally I found this issue in the browser (firefox), which has the same problem.

But: thank you a lot for pointing me to the client issue. I'll try to change the way how authentik generates the redirect url...

FYI: see https://datatracker.ietf.org/doc/html/rfc3986#section-3.5 Fragment for discussion of security implications and why fragment should not be sent in URIs

2014 discussion: https://stackoverflow.com/questions/26011050/what-part-omits-the-hash-fragment-from-the-url-or-why-crawlers-do-not-simply-sen

To be more explicit: for security reasons, you might consider sending the access_token, id_token, and additional info in a POST body rather than including sensitive info in a URI, which may go through proxies and may be logged, including the sensitive information.

I suspect that authentik uses the hashmark in the GET request to prevent the logging of sensitive data such as tokens.

I could change the request type of the redirect uri to POST, which solves the problem.

Note: I tried to read the POST-body in lua with lighty.r.req_body.*, but it failed with

unable to collect request body (handler already set); (perhaps load mod_magnet earlier in server.modules, before mod_fastcgi; or require r.req_env['REMOTE_USER'] before attempting r.req_body.collect?)

Neither loading mod_magnet before mod_fastcgi nor require r.req_env['REMOTE_USER'] helped.

The fix was to exclude this url from url.rewrite. Maybe this can be added to the error message: the url must not be rewritten ??

The error message includes the module that was already set to handle the request, in your case mod_fastcgi.

Are you using magnet.attract-raw-url-to to collect the request body? That should work as long as mod_magnet is listed before mod_fastcgi in server.modules. I have a hunch that you are using mod_fastcgi with "check-local" => "disable", and so mod_fastcgi is grabbing the request in plugin_call_handle_uri_clean(), and you are using mod_magnet magnet.attract-physical-path-to which runs later in request processing in plugin_call_handle_physical().

I'll also guess that modifying your URL rewrite rules helped since the url.rewrite rule result was a path that is handled by mod_fastcgi in your config.

I changed the loading order of modules and verified with lighttpd -p that mod_magnet is listed before mode_fastcgi, but it still failed with the same error message.

Even if mod_magnet ist the first module, it fails. Only when I disable the url-rewriting, I can read the body. The whole configuration is inside a $HTTP["url"] conditional, maybe that is the problem ...

You must run the script in an earlier lighttpd request hook if something like mod_fastcgi is handling the request sooner.
magnet.attract-raw-url-to = ( "/etc/lighttpd/lua/debug.lua" )

Maybe try debug.log-request-handling = "enable" or debug.log-condition-handling = "enable" to get some insight?

--- a/src/mod_magnet.c
+++ b/src/mod_magnet.c
@@ -2510,6 +2510,7 @@ static int magnet_reqbody(lua_State *L) {
             else {
                 log_error(r->conf.errh, __FILE__, __LINE__,
                   "unable to collect request body (handler already set); " 
+                  "(prefer to collect in magnet.attract-raw-url-to config) " 
                   "(perhaps load mod_magnet earlier in server.modules, " 
                   "before mod_%s; or require r.req_env['REMOTE_USER'] before " 
                   "attempting r.req_body.collect?)", r->handler_module->name);

@flynn did you get it working by following my instructions?

As suspected I'm using magnet.attract-physical-path-to with "check-local" => "disable" in the fastcgi-section.

Changing magnet.attract-raw-url-to works and solves the problem.

Using debug.log-request-handling = "enable" and debug.log-condition-handling = "enable" the output is quite large and not easy to diff between different configurations. While magnet.attract-raw-url-to works I did not inevestigate this ouput further here.