Project

General

Profile

Actions
Copy link

Bug #2879

closed

Segfault with proxy-header map-urlpath

Added by ganto about 6 years ago. Updated about 6 years ago.

Status:
Fixed
Priority:
Normal
Category:
mod_proxy
Target version:
1.4.50
ASK QUESTIONS IN Forums:

Description

I was trying to setup Gitea behind a Lighttpd reverse proxy. As various other applications should be served from the same vhost I wanted to make Gitea accessible at /gitea. As documented for Gogs (fork origin of Gitea), this can be achieved by setting a corresponding map-urlpath in the proxy-header configuration of Lighttpd.

However, if I try to setup such a setup, I get a segfault of lighttpd when trying to access the login form. In strace this looks like that:

23:12:26.679826 write(6, "2018-03-20 23:12:26: (gw_backend.c.234) got proc: pid: 0 socket: tcp:127.0.0.1:3000 load: 1 \n", 93) = 93 <0.000054>
23:12:26.680098 epoll_ctl(10, EPOLL_CTL_MOD, 18, {EPOLLIN|EPOLLOUT|EPOLLERR|EPOLLHUP|EPOLLRDHUP, {u32=18, u64=18}}) = 0 <0.000047>
23:12:26.680401 writev(18, [{iov_base="GET /user/login?redirect_to=%2fgitea HTTP/1.0\r\nHost: example.com\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-US,en;q=0.5\r\nAccept-Encoding: gzip, deflate, br\r\nCookie: language=en_US; storedtemplate=calm\r\nDNT: 1\r\nUpgrade-Insecure-Requests: 1\r\nX-Forwarded-For: 192.168.10.12\r\nX-Host: example.com\r\nX-Forwarded-Host: example.com\r\nX-Forwarded-Proto: https\r\nConnection: close\r\n\r\n", iov_len=507}], 1) = 507 <0.000101>
23:12:26.680916 epoll_ctl(10, EPOLL_CTL_MOD, 18, {EPOLLIN|EPOLLERR|EPOLLHUP|EPOLLRDHUP, {u32=18, u64=18}}) = 0 <0.000044>
23:12:26.681987 epoll_wait(10, [{EPOLLIN, {u32=18, u64=18}}], 1025, 1000) = 1 <0.003011>
23:12:26.685417 ioctl(18, FIONREAD, [9327]) = 0 <0.000042>
23:12:26.685702 read(18, "HTTP/1.0 200 OK\r\nContent-Type: text/html; charset=UTF-8\r\nSet-Cookie: lang=en-US; Path=/gitea; Max-Age=2147483647\r\nSet-Cookie: i_like_gitea=3a6c16e55d495c0c; Path=/gitea; HttpOnly; Secure\r\nSet-Cookie: _csrf=_GI1THJf4HplpExLPOBK4ElJcQQ6MTUyMTU4Mzk0NjY4MTcxNzA2Mw%3D%3D; Path=/gitea; Expires=Wed, 21 Mar 2018 22:12:26 GMT; HttpOnly\r\nSet-Cookie: redirect_to=%2Fgitea; Path=/gitea\r\nX-Frame-Options: SAMEORIGIN\r\nDate: Tue, 20 Mar 2018 22:12:26 GMT\r\n\r\n<!DOCTYPE html>\n<html>\n<head data-suburl=\"/gitea\">\n\t<meta charset=\"utf-8\">\n\t<meta http-equiv=\"x-ua-compatible\" content=\"ie=edge\">\n\t<title>Sign In - Gitea: Git with a cup of tea</title>\n\t<meta name=\"theme-color\" content=\"#6cc644\">\n\t<meta name=\"author\" content=\"Gitea - Git with a cup of tea\" />\n\t<meta name=\"description\" content=\"Gitea (Git with a cup of tea) is a painless self-hosted Git service written in Go\" />\n\t<meta name=\"keywords\" content=\"go,git,self-hosted,gitea\">\n\t<meta name=\"referrer\" content=\"no-referrer\" />\n\t<meta name=\"_csrf\" content=\"_GI1THJf4HplpExLPOBK4ElJcQQ6"..., 9327) = 9327 <0.000069>
23:12:26.687906 --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x557327a61ffa} ---
23:12:26.689694 +++ killed by SIGSEGV +++

Gitea properly responds to the request but lighttpd then fails to serve the response to the client. I quickly tried to have a look at this via gdb which shows the following backtrace:
#0  0x00007ffff6c470b2 in ?? () from /lib64/libc.so.6
#1  0x000055555556d90e in memmove (__len=18446744073709550854, __src=<optimized out>, __dest=<optimized out>) at /usr/include/bits/string3.h:59
#2  buffer_substr_replace (b=b@entry=0x555555808d90, offset=offset@entry=1054, len=len@entry=1, replace=0x5555557b4210) at buffer.c:472
#3  0x00007ffff5e6183e in http_header_remap_urlpath (b=b@entry=0x555555808d90, off=off@entry=1054, remap_hdrs=remap_hdrs@entry=0x555555806008, is_req=is_req@entry=0) at mod_proxy.c:367
#4  0x00007ffff5e61ee8 in http_header_remap_setcookie (remap_hdrs=0x555555806008, off=1054, b=0x555555808d90) at mod_proxy.c:469
#5  proxy_response_headers (srv=<optimized out>, con=<optimized out>, opts=<optimized out>) at mod_proxy.c:975
#6  0x00005555555828cf in http_response_parse_headers (srv=srv@entry=0x55555579b010, con=con@entry=0x5555557bd610, opts=opts@entry=0x555555805f48, b=b@entry=0x555555809770) at http-header-glue.c:1193
#7  0x0000555555582e43 in http_response_read (srv=srv@entry=0x55555579b010, con=0x5555557bd610, opts=opts@entry=0x555555805f48, b=b@entry=0x555555809770, fd=10, fde_ndx=fde_ndx@entry=0x555555805f34) at http-header-glue.c:1282
#8  0x000055555557487d in gw_recv_response (srv=srv@entry=0x55555579b010, hctx=hctx@entry=0x555555805ee0) at gw_backend.c:2016
#9  0x00005555555756ad in gw_handle_fdevent (srv=0x55555579b010, ctx=0x555555805ee0, revents=1) at gw_backend.c:2139
#10 0x0000555555560d70 in server_main (srv=0x55555579b010, argc=<optimized out>, argv=<optimized out>) at server.c:1988
#11 0x0000555555561c60 in main (argc=4, argv=0x7fffffffe3c8) at server.c:2055

My proxy configuration currently looks like this:
$HTTP["url"] =~ "^/gitea" {
    # forward requests to unix socket
    proxy.debug = 512
    proxy.server = (
        "" => (
            "gitea" => (
                "host" => "127.0.0.1",
                "port" => "3000",
#                "host" => "/run/gitea/gitea.sock",
            )
        )
    )
    proxy.header = (
        "map-urlpath" => (
            "/gitea/" => "/",
            "/gitea" => "/",
        ),
    )
}

The segfaults can be triggered via TCP or unix socket. I also played around with various map-urlpath settings which sometimes also triggered another segfault already for the initial request of /gitea like:
28174 21:26:02.699876 read(9, "GET /gitea/ HTTP/1.1\r\nHost: example.com\r\nUser-Agent: curl/7.59.0\r\nAccept: */*\r\n\r\n", 4159) = 90 <0.000039>
28174 21:26:02.700110 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=127, ...}) = 0 <0.000048>
28174 21:26:02.700330 write(5, "2018-03-20 21:26:01: (gw_backend.c.933) gw - found a host 127.0.0.1 3000 \n", 74) = 74 <0.000065>
28174 21:26:02.700515 socket(AF_INET, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 10 <0.000059>
28174 21:26:02.700669 connect(10, {sa_family=AF_INET, sin_port=htons(3000), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress) <0.000173>
28174 21:26:02.701012 write(5, "2018-03-20 21:26:01: (gw_backend.c.972) connect delayed; will continue later: tcp:127.0.0.1:3000 \n", 98) = 98 <0.000095>
28174 21:26:02.701281 epoll_ctl(7, EPOLL_CTL_ADD, 10, {EPOLLOUT|EPOLLERR|EPOLLHUP, {u32=10, u64=10}}) = 0 <0.000050>
28174 21:26:02.701441 accept4(4, 0x7fff8556e480, [112], SOCK_CLOEXEC|SOCK_NONBLOCK) = -1 EAGAIN (Resource temporarily unavailable) <0.000057>
28174 21:26:02.701608 write(5, "2018-03-20 21:26:01: (gw_backend.c.995) proc: tcp:127.0.0.1:3000 0 0 1 0 \n", 74) = 74 <0.000043>
28174 21:26:02.701746 epoll_wait(7, [{EPOLLOUT, {u32=10, u64=10}}], 1025, 1000) = 1 <0.000034>
28174 21:26:02.701878 getsockopt(10, SOL_SOCKET, SO_ERROR, [0], [4]) = 0 <0.000032>
28174 21:26:02.702017 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=127, ...}) = 0 <0.000046>
28174 21:26:02.702189 write(5, "2018-03-20 21:26:02: (gw_backend.c.234) got proc: pid: 0 socket: tcp:127.0.0.1:3000 load: 1 \n", 93) = 93 <0.000044>
28174 21:26:02.702411 epoll_ctl(7, EPOLL_CTL_MOD, 10, {EPOLLIN|EPOLLOUT|EPOLLERR|EPOLLHUP|EPOLLRDHUP, {u32=10, u64=10}}) = 0 <0.000033>
28174 21:26:02.702543 writev(10, [{iov_base="GET / HTTP/1.0\r\nHost: example.com\r\nUser-Agent: curl/7.59.0\r\nAccept: */*\r\nX-Forwarded-For: 192.168.10.12\r\nX-Host: example.com\r\nX-Forwarded-Host: example.com\r\nX-Forwarded-Proto: http\r\nConnection: close\r\n\r\n", iov_len=227}], 1) = 227 <0.000105>
28174 21:26:02.702758 epoll_ctl(7, EPOLL_CTL_MOD, 10, {EPOLLIN|EPOLLERR|EPOLLHUP|EPOLLRDHUP, {u32=10, u64=10}}) = 0 <0.000024>
28174 21:26:02.702854 epoll_wait(7, [{EPOLLIN, {u32=10, u64=10}}], 1025, 1000) = 1 <0.005444>
28174 21:26:02.708365 ioctl(10, FIONREAD, [10048]) = 0 <0.000019>
28174 21:26:02.708458 read(10, "HTTP/1.0 200 OK\r\nContent-Type: text/html; charset=UTF-8\r\nSet-Cookie: lang=en-US; Path=/gitea; Max-Age=2147483647\r\nSet-Cookie: i_like_gitea=200dda277077a69b; Path=/gitea; HttpOnly\r\nSet-Cookie: _csrf=qYuBbwlEShjIzMpAZfcPkZdSgcU6MTUyMTU4MTE2MjcwMzgzNTI1Nw%3D%3D; Path=/gitea; Expires=Wed, 21 Mar 2018 21:26:02 GMT; HttpOnly\r\nX-Frame-Options: SAMEORIGIN\r\nDate: Tue, 20 Mar 2018 21:26:02 GMT\r\n\r\n<!DOCTYPE html>\n<html>\n<head data-suburl=\"/gitea\">\n\t<meta charset=\"utf-8\">\n\t<meta http-equiv=\"x-ua-compatible\" content=\"ie=edge\">\n\t<title>Gitea: Git with a cup of tea</title>\n\t<meta name=\"theme-color\" content=\"#6cc644\">\n\t<meta name=\"author\" content=\"Gitea - Git with a cup of tea\" />\n\t<meta name=\"description\" content=\"Gitea (Git with a cup of tea) is a painless self-hosted Git service written in Go\" />\n\t<meta name=\"keywords\" content=\"go,git,self-hosted,gitea\">\n\t<meta name=\"referrer\" content=\"no-referrer\" />\n\t<meta name=\"_csrf\" content=\"qYuBbwlEShjIzMpAZfcPkZdSgcU6MTUyMTU4MTE2MjcwMzgzNTI1Nw==\" />\n\t<meta name=\"_suburl\" content=\"/"..., 10048) = 10048 <0.000037>
28174 21:26:02.708614 ioctl(10, FIONREAD, [0]) = 0 <0.000018>
28174 21:26:02.708683 read(10, 0x564a952d81f0, 10111) = -1 EAGAIN (Resource temporarily unavailable) <0.000018>
28174 21:26:02.708779 epoll_wait(7, [{EPOLLIN|EPOLLRDHUP, {u32=10, u64=10}}], 1025, 1000) = 1 <0.000216>
28174 21:26:02.709150 ioctl(10, FIONREAD, [0]) = 0 <0.000030>
28174 21:26:02.709357 read(10, "", 10111) = 0 <0.000034>
28174 21:26:02.709484 epoll_ctl(7, EPOLL_CTL_DEL, 10, 0x7fff8556e42c) = 0 <0.000032>
28174 21:26:02.709605 write(5, "2018-03-20 21:26:02: (gw_backend.c.308) released proc: pid: 0 socket: tcp:127.0.0.1:3000 load: 0 \n", 98) = 98 <0.000039>
28174 21:26:02.709743 close(10)         = 0 <0.000081>
28174 21:26:02.709926 write(2, "chunk.c.127: assertion failed: c->offset <= len\n", 48) = 48 <0.000033>
28174 21:26:02.710069 rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0 <0.000040>
28174 21:26:02.710276 rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0 <0.000027>
28174 21:26:02.710401 getpid()          = 28174 <0.000027>
28174 21:26:02.710504 gettid()          = 28174 <0.000029>
28174 21:26:02.710610 tgkill(28174, 28174, SIGABRT) = 0 <0.000028>
28174 21:26:02.710712 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 <0.000026>
28174 21:26:02.710820 --- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=28174, si_uid=102} ---
28176 21:26:02.711468 <... poll resumed> ) = 1 ([{fd=6, revents=POLLIN|POLLHUP}]) <21.278575>
28174 21:26:02.711555 +++ killed by SIGABRT +++
28176 21:26:02.711575 read(6, "", 4106) = 0 <0.000039>
28176 21:26:02.711687 close(6)          = 0 <0.000022>
28176 21:26:02.711792 poll([{fd=3, events=POLLIN}, {fd=4, events=POLLIN}, {fd=5, events=POLLIN}], 3, 30000) = 0 (Timeout) <30.020517>
28176 21:26:32.732557 exit_group(0)     = ?
28176 21:26:32.733330 +++ exited with 0 +++


#0  0x00007ffff6c3d940 in free () from /lib64/libc.so.6
#1  0x000055555556d3db in buffer_reset (b=0x555555807c70) at buffer.c:52
#2  0x000055555556fa64 in chunk_reset (c=c@entry=0x555555807ae0) at chunk.c:80
#3  0x000055555556fb46 in chunkqueue_push_unused_chunk (cq=cq@entry=0x5555557bd980, c=0x555555807ae0) at chunk.c:158
#4  0x000055555556fd46 in chunkqueue_reset (cq=0x5555557bd980) at chunk.c:214
#5  0x000055555557f5eb in connection_response_reset (srv=srv@entry=0x55555579b010, con=con@entry=0x5555557bd3b0) at connections-glue.c:503
#6  0x0000555555564114 in connection_reset (srv=srv@entry=0x55555579b010, con=con@entry=0x5555557bd3b0) at connections.c:642
#7  0x0000555555565483 in connection_handle_shutdown (con=0x5555557bd3b0, srv=0x55555579b010) at connections.c:192
#8  connection_handle_response_end_state (con=0x5555557bd3b0, srv=0x55555579b010) at connections.c:231
#9  connection_state_machine (srv=0x55555579b010, con=0x5555557bd3b0) at connections.c:1317
#10 0x0000555555560da7 in server_main (srv=0x55555579b010, argc=<optimized out>, argv=<optimized out>) at server.c:2001
#11 0x0000555555561c60 in main (argc=4, argv=0x7fffffffe3c8) at server.c:2055

If Gitea is served on / (without any map-urlpath) no segfaults can be observed.

I could reproduce the issue on two machines running Gentoo Linux with lighttpd-1.4.49 compiled with gcc-6.4.0. If you have some questions or you want me to run some tests, please let me know.

Actions
Copy link
 #1

Updated by gstrauss about 6 years ago

ick! I see a few bugs in http_header_remap_setcookie() but things will have to wait until tomorrow evening for me to have more time to work on a proper fix.

Actions
Copy link
 #2

Updated by gstrauss about 6 years ago

  • Priority changed from Normal to High
  • Target version changed from 1.4.x to 1.4.50
Actions
Copy link
 #3

Updated by gstrauss about 6 years ago

!! Not tested !!

--- a/src/mod_proxy.c
+++ b/src/mod_proxy.c
@@ -341,7 +341,7 @@ static size_t http_header_remap_host (buffer *b, size_t off, http_header_remap_o

 /* (future: might move to http-header-glue.c) */
-static void http_header_remap_urlpath (buffer *b, size_t off, http_header_remap_opts *remap_hdrs, int is_req)
+static size_t http_header_remap_urlpath (buffer *b, size_t off, http_header_remap_opts *remap_hdrs, int is_req)
 {
     const array *urlpaths = remap_hdrs->urlpaths;
     if (urlpaths) {
@@ -355,7 +355,7 @@ static void http_header_remap_urlpath (buffer *b, size_t off, http_header_remap_
                     if (NULL == remap_hdrs->forwarded_urlpath)
                         remap_hdrs->forwarded_urlpath = ds;
                     buffer_substr_replace(b, off, mlen, ds->value);
-                    break;
+                    return buffer_string_length(ds->value);/*(replacement len)*/
                 }
             }
         }
@@ -365,7 +365,7 @@ static void http_header_remap_urlpath (buffer *b, size_t off, http_header_remap_
                 const size_t mlen = buffer_string_length(ds->value);
                 if (mlen <= plen && 0 == memcmp(s, ds->value->ptr, mlen)) {
                     buffer_substr_replace(b, off, mlen, ds->key);
-                    return;
+                    return buffer_string_length(ds->key); /*(replacement len)*/
                 }
             }
             for (size_t i = 0, used = urlpaths->used; i < used; ++i) {
@@ -373,11 +373,12 @@ static void http_header_remap_urlpath (buffer *b, size_t off, http_header_remap_
                 const size_t mlen = buffer_string_length(ds->value);
                 if (mlen <= plen && 0 == memcmp(s, ds->value->ptr, mlen)) {
                     buffer_substr_replace(b, off, mlen, ds->key);
-                    break;
+                    return buffer_string_length(ds->key); /*(replacement len)*/
                 }
             }
         }
     }
+    return 0;
 }

@@ -443,22 +444,22 @@ static void http_header_remap_setcookie (buffer *b, size_t off, http_header_rema
      * entire string in b from offset to end of string.  In response headers,
      * lighttpd may concatenate multiple Set-Cookie headers into single entry
      * in con->response.headers, separated by "\r\nSet-Cookie: " */
-    for (char *s, *n = b->ptr+off; (s = n); ) {
+    for (char *s = b->ptr+off, *e; *s; s = e) {
         size_t len;
-        n = strchr(s, '\n');
-        if (NULL == n) {
-            len = (size_t)(b->ptr + buffer_string_length(b) - s);
-        }
-        else {
-            len = (size_t)(n - s);
-            n += sizeof("Set-Cookie: "); /*(include +1 for '\n')*/
-        }
-        for (char *e = s; NULL != (s = memchr(e, ';', len)); ) {
+        {
+            while (*s != ';' && *s != '\n' && *s != '\0') ++s;
+            if (*s == '\n') {
+                /*(include +1 for '\n', but leave ' ' for ++s below)*/
+                s += sizeof("Set-Cookie:");
+            }
+            if ('\0' == *s) return;
             do { ++s; } while (*s == ' ' || *s == '\t');
             if ('\0' == *s) return;
+            e = s+1;
+            if ('=' == *s) continue;
             /*(interested only in Domain and Path attributes)*/
-            e = memchr(s, '=', len - (size_t)(s - e));
-            if (NULL == e) { e = s+1; continue; }
+            while (*e != '=' && *e != '\0') ++e;
+            if ('\0' == *e) return;
             ++e;
             switch ((int)(e - s - 1)) {
               case 4:
@@ -466,8 +467,8 @@ static void http_header_remap_setcookie (buffer *b, size_t off, http_header_rema
                     if (*e == '"') ++e;
                     if (*e != '/') continue;
                     off = (size_t)(e - b->ptr);
-                    http_header_remap_urlpath(b, off, remap_hdrs, 0);
-                    e = b->ptr+off; /*(b may have been reallocated)*/
+                    len = http_header_remap_urlpath(b, off, remap_hdrs, 0);
+                    e = b->ptr+off+len; /*(b may have been reallocated)*/
                     continue;
                 }
                 break;

Actions
Copy link
 #4

Updated by ganto about 6 years ago

Nice, job. With this patch the segfaults cannot be reproduced anymore.

Actions
Copy link
 #5

Updated by gstrauss about 6 years ago

  • Status changed from New to Patch Pending
  • Priority changed from High to Normal

Would have been nicer if I had gotten it right the first time. :/

Thank you for reporting the issue and testing the patch.

Actions
Copy link
 #6

Updated by gstrauss about 6 years ago

  • Status changed from Patch Pending to Fixed
  • % Done changed from 0 to 100
Actions
Copy link

Also available in: Atom