Project

General

Profile

Actions
Copy link

Feature #2880

closed

Option to turn off response handling in map-urlpath

Added by ganto over 6 years ago. Updated over 6 years ago.

Status:
Wontfix
Priority:
Low
Category:
mod_proxy
Target version:
1.4.x
ASK QUESTIONS IN Forums:

Description

I'm still on my quest of trying to make Lighttpd play nice when proxying requests for Gitea via sub-URL. As #2879 is not in my way anymore, the next issue arises.

Here my current configuration:

$HTTP["url"] =~ "^/gitea" {
    proxy.server = (
        "" => (
            "gitea" => (
                "host" => "127.0.0.1",
                "port" => "3000",
            )
        )
    )
    proxy.header = (
        "map-urlpath" => (
            "/gitea/" => "/",
        ),
    )
}

Unfortunately Gitea always expects requests to be relative to / that's why the map-urlpath is required on first sight. However, Gitea (and I guess Gogs too) is aware of the sub-URL by the ROOT_URL configuration value. It will use this knowledge to already generate the correct "external" URLs for the client what obviously conflicts with the map-urlpath mapping of the response in Lighttpd:

1. External request from /gitea/ to /gitea/user/login?redirect_to=/gitea
2. Lighttpd requests /user/login?redirect_to=%2fgitea from Gitea:

[Macaron] 2018-03-21 21:46:52: Started GET /user/login?redirect_to=%2fgitea for 192.168.10.12
[Macaron] 2018-03-21 21:46:52: Completed /user/login?redirect_to=%2fgitea 200 OK in 2.263991ms

3. Gitea returns a cookie for Path=/gitea to Lighttpd:
21:46:52.770761 read(10, "HTTP/1.0 200 OK\r\nContent-Type: text/html; charset=UTF-8\r\nSet-Cookie: redirect_to=%2Fgitea; Path=/gitea\r\nX-Frame-Options: SAMEORIGIN\r\nDate: Wed, 21 Mar 2018 21:46:52 GMT\r\n\r\n<!DOCTYPE html>\n<html>\n<head data-suburl=\"/gitea\">\n\t<meta charset=\"utf-8\">\n\t<meta http-equiv=\"x-ua-compatible\" content=\"ie=edge\">\n\t<title>Sign In - Gitea: Git with a cup of tea</title>\n\t<meta name=\"theme-color\" content=\"#6cc644\">\n\t<meta name=\"author\" content=\"Gitea - Git with a cup of tea\" />\n\t<meta name=\"description\" content=\"Gitea (Git with a cup of tea) is a painless self-hosted Git service written in Go\" />\n\t<meta name=\"keywords\" content=\"go,git,self-hosted,gitea\">\n\t<meta name=\"referrer\" content=\"no-referrer\" />\n\t<meta name=\"_csrf\" content=\"MWdIKIoM_4OpSiuRlGQPyBYs8rc6MTUyMTY2NzM1NTUwMTY1NzA3Nw==\" />\n\t<meta name=\"_suburl\" content=\"/gitea\" />\n\t\n\t\n\t\n\n\n\t<script>\n\t/*\n\t@licstart  The following is the entire license notice for the\n        JavaScript code in this page.\n\n\tCopyright (c) 2016 The Gitea Authors\n\tCopyright (c) 2015 The Gogs Authors"..., 9055) = 9055 <0.000042>

4. The remap in Lighttpd then updates the URL to an invalid value of Path=/gitea/gitea which is sent to the client:
21:46:52.772043 writev(9, [{iov_base="HTTP/1.1 200 OK\r\nContent-Type: text/html; charset=UTF-8\r\nSet-Cookie: redirect_to=%2Fgitea; Path=/gitea/gitea\r\nX-Frame-Options: SAMEORIGIN\r\nDate: Wed, 21 Mar 2018 21:46:52 GMT\r\nContent-Length: 8883\r\nServer: lighttpd/1.4.49\r\n\r\n", iov_len=225}, {iov_base="<!DOCTYPE html>\n<html>\n<head data-suburl=\"/gitea\">\n\t<meta charset=\"utf-8\">\n\t<meta http-equiv=\"x-ua-compatible\" content=\"ie=edge\">\n\t<title>Sign In - Gitea: Git with a cup of tea</title>\n\t<meta name=\"theme-color\" content=\"#6cc644\">\n\t<meta name=\"author\" content=\"Gitea - Git with a cup of tea\" />\n\t<meta name=\"description\" content=\"Gitea (Git with a cup of tea) is a painless self-hosted Git service written in Go\" />\n\t<meta name=\"keywords\" content=\"go,git,self-hosted,gitea\">\n\t<meta name=\"referrer\" content=\"no-referrer\" />\n\t<meta name=\"_csrf\" content=\"MWdIKIoM_4OpSiuRlGQPyBYs8rc6MTUyMTY2NzM1NTUwMTY1NzA3Nw==\" />\n\t<meta name=\"_suburl\" content=\"/gitea\" />\n\t\n\t\n\t\n\n\n\t<script>\n\t/*\n\t@licstart  The following is the entire license notice for the\n        JavaScript code in this page.\n\n\tCopyright (c) 2016 The Gitea Authors\n\tCopyright (c) 2015 The Gogs Authors\n\n\tPermission is hereby granted, free of charge, to any person obtaining a copy\n\tof this software and associated documentation files (the \"Software\"), to deal\n\tin the Softw"..., iov_len=8883}], 2) = 9108 <0.001270>
21:46:52.773445 write(7, "192.168.10.12 example.com - [21/Mar/2018:21:46:52 +0000] \"GET /gitea/user/login?redirect_to=%2fgitea HTTP/1.1\" 200 8883 \"-\" \"Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0\"\n", 201) = 201 <0.000025>

5. Obviously this makes the cookie invalid and although the login succeeds, there are various issues with the session afterwards.

Is there any chance to add a toggle for suppressing the urlpath mapping in the response? Somehow it's not obvious to me why this is working with other reverse proxies. I can't find such an option there either.

Actions
Copy link
 #1

Updated by gstrauss over 6 years ago

  • Category set to mod_proxy
  • Priority changed from Normal to Low

What you're asking for seems to be an option to make things inconsistent.

Why does Gitea need lighttpd to map the url-path in the first place? Either Gitea should work without the url mapping, or it should work with the url mapping. It is unbalanced behavior if Gitea needs request url mapping in one direction, but requires that lighttpd skip the reverse mapping in Set-Cookie path=... in the other direction.

.

My first hunch would be that this behavior in Gitea is to workaround other proxies which did not reverse map the paths in Set-Cookie.

Maybe Gitea needs an option to disable the "?redirect_to=/gitea" logic? Or maybe you can just set "?redirect_to=/" ?

If none of this is an option (and I'd like some evidence, please, if you think that's the case), then I might consider adding some logic to skip the remap if in the replacement string to be used is already in the url-path, and that replacement string is more than simply "/". This would potentially cause mysterious problems for other backends that happen to have repeated strings in the url-path and that repetition is significant to the url-path.

Actions
Copy link
 #2

Updated by ganto over 6 years ago

gstrauss wrote:

What you're asking for seems to be an option to make things inconsistent.

Why does Gitea need lighttpd to map the url-path in the first place? Either Gitea should work without the url mapping, or it should work with the url mapping. It is unbalanced behavior if Gitea needs request url mapping in one direction, but requires that lighttpd skip the reverse mapping in Set-Cookie path=... in the other direction.

I fully agree with you on this. I guess I'll also need to open an issue regarding this behaviour at Gitea, but I first want to find out, what's the exact problem and how could it be solved.

There are three possible ways how it could work:
(1) Gitea is fully working with / only and the url-path remap in Lighttpd would handle the sub-URL transparently. This would mean that Gitea is configured with ROOT_URL=http://example.com/ but accessed via http://example.com/gitea/
(2) Gitea handles the sub-URL itself so that Lighttpd would forward the request to /gitea unaltered. ROOT_URL would then be http://example.com/gitea and Gitea must be able to properly handle requests to /gitea.
(3) Follow the upstream (Gogs) documentation (which is currently not functional) and set ROOT_URL=http://example.com/gitea while also setting a Lighttpd url-path map. This is what seems to work for (most) other HTTP proxies.

Let's setup and try (1):
  • I'll use again the lighttpd.conf setting as posted initially. I restrict HTTP["url"] on purpose to not accidentally leak any requests without /gitea sub-URL:
  • Gitea app.ini:
    ROOT_URL=http://example.com/
  • Gitea Log (/gitea/ was properly mapped to / by Lighttpd):
    [Macaron] 2018-03-22 19:44:59: Started GET / for 192.168.10.12
[Macaron] 2018-03-22 19:44:59: Completed / 200 OK in 11.447484ms
  • However, the response contains all resources without /gitea url-path:
    $ curl -v http://example.com/gitea/
*   Trying 192.168.10.13...                                                                                                                                                                                                                      
* TCP_NODELAY set                                                                                                                                                                                                                             
* Connected to example.com (192.168.10.13) port 80 (#0)                                                                                                                                                                                 
> GET /gitea/ HTTP/1.1                                                                                                                                                                                                                        
> Host: example.com                                                                    
> User-Agent: curl/7.59.0                                                                               
> Accept: */*                                                                                                                
>                                                                                                                                              
< HTTP/1.1 200 OK                                                                                                              
< Content-Type: text/html; charset=UTF-8                                                                                                                                                                                                      
< Set-Cookie: lang=en-US; Path=/gitea/; Max-Age=2147483647                                                                                                                                                                                    
< Set-Cookie: i_like_gitea=e0ae809ed746bb71; Path=/gitea/; HttpOnly                                                                             
< Set-Cookie: _csrf=N3RX3w8i9NjjbzBeHTvrKllvLPU6MTUyMTc0Nzg5OTY3ODkxNjAzMg%3D%3D; Path=/gitea/; Expires=Fri, 23 Mar 2018 19:44:59 GMT; HttpOnly
< X-Frame-Options: SAMEORIGIN                                                                                                       
< Date: Thu, 22 Mar 2018 19:44:59 GMT                                               
< Content-Length: 9350                                                                                     
< Server: lighttpd/1.4.49
<!DOCTYPE html>                                                                                                                   
<html>
<head data-suburl="">
        <meta charset="utf-8">
        <meta http-equiv="x-ua-compatible" content="ie=edge">
        <title>Gitea: Git with a cup of tea</title>
        <meta name="theme-color" content="#6cc644">
        <meta name="author" content="Gitea - Git with a cup of tea" />
        <meta name="description" content="Gitea (Git with a cup of tea) is a painless self-hosted Git service written in Go" />
        <meta name="keywords" content="go,git,self-hosted,gitea">
        <meta name="referrer" content="no-referrer" />
        <meta name="_csrf" content="N3RX3w8i9NjjbzBeHTvrKllvLPU6MTUyMTc0Nzg5OTY3ODkxNjAzMg==" />
        <meta name="_suburl" content="" />
[...]
        <link rel="shortcut icon" href="/img/favicon.png" />
        <link rel="mask-icon" href="/img/gitea-safari.svg" color="#609926">
        <link rel="preload" href="/vendor/assets/font-awesome/css/font-awesome.min.css" as="style" onload="this.rel='stylesheet'">
        <noscript><link rel="stylesheet" href="/vendor/assets/font-awesome/css/font-awesome.min.css"></noscript>
        <link rel="stylesheet" href="/vendor/assets/octicons/octicons.min.css">
[...]
  • Gitea has no chance to get this links correctly, as both, the client request as well as the ROOT_URL say / is the url-path.
  • Lighttpd would need to map all possible page content back to what "should" point to /gitea (e.g. data-suburl="" or _suburl content=""). Not really an option...

Let's setup and try (2):

  • lighttpd.conf now doesn't contain any proxy.header statements
  • Gitea app.ini:
    ROOT_URL=http://example.com/gitea/
  • Gitea Log:
    [Macaron] 2018-03-22 20:16:16: Started GET /gitea/ for 192.168.10.12
[Macaron] 2018-03-22 20:16:16: Completed /gitea/ 404 Not Found in 10.019546ms
  • Here we get a simple 404 from Gitea as it doesn't expect the request to contain /gitea although we said so in the ROOT_URL:
    $ curl -v http://example.com/gitea/
*   Trying 192.168.10.13...                                                                                                                                                                                                                      
* TCP_NODELAY set                                                                                                                                                                                                                             
* Connected to example.com (192.168.10.13) port 80 (#0)                                                                                                                                                                                 
> GET /gitea/ HTTP/1.1                                                                                                                                                                                                                        
> Host: example.com                                                                                                                                                                                                                 
> User-Agent: curl/7.59.0                                                                                                                                                                                                                     
> Accept: */*                                                                                                                                                                                                                                 
>                                                                                                                                                                                                                                             
< HTTP/1.1 404 Not Found                                                                                                                                                                                                                      
< Content-Type: text/html; charset=UTF-8                                                                                                                                                                                                      
< Set-Cookie: lang=en-US; Path=/gitea; Max-Age=2147483647                                                                                             
< Set-Cookie: i_like_gitea=20f4ff6fc5243d33; Path=/gitea; HttpOnly                                                                      
< Set-Cookie: _csrf=O9cccehIEjhHDTc7Vp0fDt8ws2M6MTUyMTc0OTc3NjA2MzQ4ODg0Mw%3D%3D; Path=/gitea; Expires=Fri, 23 Mar 2018 20:16:16 GMT; HttpOnly
< X-Frame-Options: SAMEORIGIN                                                                                                       
< Date: Thu, 22 Mar 2018 20:16:16 GMT                                               
< Content-Length: 7839                                                                                                      
< Server: lighttpd/1.4.49

[...]
                                                                                <a class="item" href="/gitea/user/login?redirect_to=%2fgitea%2fgitea">
                                                                                        <i class="octicon octicon-sign-in"></i> Sign In 
                                                                                </a>
  • A possible fix here would be, that Gitea would (optionally) need to accept requests to a sub-URL as it is already able to generate proper replies containing the sub-URL. See e.g. the Path=/gitea in the cookie.
  • A funny thing here that it adds the sub-URL from ROOT_URL to the sub-URL from the request which results in a completely wrong redirect URL after an imaginary login. Seems that various things would need a fix here...

Number (3) is the one I described initially:

  • lighttpd.conf from the initial description
  • Gitea app.ini:
    ROOT_URL=http://example.com/gitea/
  • Gitea Log:
    [Macaron] 2018-03-22 20:25:24: Started GET / for 192.168.10.12
[Macaron] 2018-03-22 20:25:24: Completed / 200 OK in 5.798316ms
  • Now we get the wrong cookie path:
    $ curl -v http://example.com/gitea/
*   Trying 192.168.10.13...                                                                                                                                                                                                                      
* TCP_NODELAY set                                                                                                                                                                                                                             
* Connected to example.com (192.168.10.13) port 80 (#0)                                                                                                                                                                                 
> GET /gitea/ HTTP/1.1                                                                                                                                                                                                                        
> Host: example.com
> User-Agent: curl/7.59.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=UTF-8
< Set-Cookie: lang=en-US; Path=/gitea/gitea; Max-Age=2147483647
< Set-Cookie: i_like_gitea=083a01d63cd2dd51; Path=/gitea/gitea; HttpOnly
< Set-Cookie: _csrf=Q1qYjqqOz7eyuONkti580G1WQZ46MTUyMTc1MDMyNDM5ODYwNzUzNw%3D%3D; Path=/gitea/gitea; Expires=Fri, 23 Mar 2018 20:25:24 GMT; HttpOnly
< X-Frame-Options: SAMEORIGIN
< Date: Thu, 22 Mar 2018 20:25:24 GMT
< Content-Length: 9658
< Server: lighttpd/1.4.49
<
<!DOCTYPE html>
<html>
<head data-suburl="/gitea">
        <meta charset="utf-8">
        <meta http-equiv="x-ua-compatible" content="ie=edge">
        <title>Gitea: Git with a cup of tea</title>
        <meta name="theme-color" content="#6cc644">
        <meta name="author" content="Gitea - Git with a cup of tea" />
        <meta name="description" content="Gitea (Git with a cup of tea) is a painless self-hosted Git service written in Go" />
        <meta name="keywords" content="go,git,self-hosted,gitea">
        <meta name="referrer" content="no-referrer" />
        <meta name="_csrf" content="Q1qYjqqOz7eyuONkti580G1WQZ46MTUyMTc1MDMyNDM5ODYwNzUzNw==" />
        <meta name="_suburl" content="/gitea" />
[...]
        <link rel="shortcut icon" href="/gitea/img/favicon.png" />
        <link rel="mask-icon" href="/gitea/img/gitea-safari.svg" color="#609926">
        <link rel="preload" href="/gitea/vendor/assets/font-awesome/css/font-awesome.min.css" as="style" onload="this.rel='stylesheet'">
        <noscript><link rel="stylesheet" href="/gitea/vendor/assets/font-awesome/css/font-awesome.min.css"></noscript>
        <link rel="stylesheet" href="/gitea/vendor/assets/octicons/octicons.min.css">
[...]
                                                                                <a class="item" href="/gitea/user/login?redirect_to=%2fgitea">
                                                                                        <i class="octicon octicon-sign-in"></i> Sign In
                                                                                </a>
  • Other than the cookie Path= the page looks alright on first sight

I will search the Gogs/Gitea issues for any hints on how they think about (2) or raise an issue there if I can't find anything. Still I also think that (3) should somehow be possible with Lighttpd as that's what people know from other proxies.

Regarding the response remapping:
My current understanding is that you remap the path in HTTP headers and Cookies correct? Somehow the documentation about the cookie handling is missing...
I understand that auto-detecting if a string should be replaced or not how you describe it might result in unexpected behaviour. But an intentional configuration knob to skip response mapping shouldn't be too scary, or do I miss something?

Actions
Copy link
 #3

Updated by gstrauss over 6 years ago

What you're asking for seems to be an option to make things inconsistent.

Why does Gitea need lighttpd to map the url-path in the first place? Either Gitea should work without the url mapping, or it should work with the url mapping. It is unbalanced behavior if Gitea needs request url mapping in one direction, but requires that lighttpd skip the reverse mapping in Set-Cookie path=... in the other direction.

I fully agree with you on this. I guess I'll also need to open an issue regarding this behaviour at Gitea

Yes. Add some config to Gitea to disable it's obtuse behavior. Why shouldn't any webapp either require that it be rooted at '/' or be able to handle being located under a configured url-path? (without requiring contortions from the web server)

Given the above, your suggestion that "an intentional configuration knob [in lighttpd] to skip response mapping shouldn't be too scary" is entirely unconvincing. lighttpd should not add sloppy code with inconsistent behavior to work around limitations in a poorly written webapp.

.

My current understanding is that you remap the path in HTTP headers and Cookies correct? Somehow the documentation about the cookie handling is missing...

Docs_ModProxy

proxy.header (since 1.4.46) is a list of options to perform simple prefix matching to remap host and URL paths in proxied HTTP headers (commit 036d3d3d)

Set-Cookie is an HTTP header.

Actions
Copy link
 #4

Updated by gstrauss over 6 years ago

  • Status changed from New to Wontfix

I took a very quick look at https://github.com/go-gitea/gitea and might suggest that instead of using setting.AppSubURL all over the place, that the locations which call ctx.SetCookie or assign SessionConfig.CookiePath should use a new value from setting.*, e.g. setting.CookiePath, and that setting.CookiePath default to setting.AppSubURL unless otherwise defined in the config. That might be a quick hack to workaround gitea getting repeated paths in Set-Cookie when used with lighttpd mod_proxy proxy.header map-urlpath.

A better solution would to modify gitea to avoid needing map-urlpath. After all, gitea already needs to be configured to know the "subURL" to generate proper links, since it is specifying URL root-relative links instead of using relative links in body content.

Actions
Copy link

Also available in: Atom