Lighttpd

More details please. A quick modification of t/test_mod_evhost.c to test this scenario succeeds.

Please provide your lighttpd.conf or at least mod_evhost config and show what you expect and what you are getting. What is lighttpd attempting to access instead of your desired path? Is there any trace from mod_evhost in the lighttpd error log?

You marked this as a regression. Just to confirm: what is the previous version of lighttpd with which you have tested and works as you expect? lighttpd 1.4.52?

gstrauss wrote:

More details please. A quick modification of t/test_mod_evhost.c to test this scenario succeeds.

Please provide your lighttpd.conf or at least mod_evhost config and show what you expect and what you are getting. What is lighttpd attempting to access instead of your desired path? Is there any trace from mod_evhost in the lighttpd error log?

evhost.conf:

server.modules += ( "mod_evhost" )
evhost.path-pattern = "/usr/local/www/%0/" 

When requesting http://host1/, I expect to get /usr/local/www/host1/index.html, but I get /usr/local/www/index.html instead.
It works if I replace %0 with %1 pattern.

There is nothing related in the error.log. Should I enable debugging?

You marked this as a regression. Just to confirm: what is the previous version of lighttpd with which you have tested and works as you expect? lighttpd 1.4.52?

Yes, it works as expected with 1.4.52.

For testing I am using the stock configuration installed by FreeBSD port www/lighttpd (lighttpd-1.4.53). The only difference is:

--- modules.conf.sample 2019-01-29 10:05:16.000000000 +0300
+++ modules.conf        2019-02-04 09:39:50.129861000 +0300
@@ -162,7 +162,7 @@
 ##
 ## mod_evhost
 ##
-#include "conf.d/evhost.conf" 
+include "conf.d/evhost.conf" 

 ##
 ## mod_simple_vhost

--- conf.d/evhost.conf.sample   2019-01-29 10:05:16.000000000 +0300
+++ conf.d/evhost.conf  2019-02-04 10:05:47.361329000 +0300
@@ -16,7 +16,7 @@
 ## %3 => subdomain 1 name
 ## %4 => subdomain 2 name
 ##
-evhost.path-pattern        = vhosts_dir + "/%3/htdocs/" 
+evhost.path-pattern        = "/usr/local/www/%0/" 

 ##
 #######################################################################

--- conf.d/debug.conf.sample    2019-01-29 10:05:16.000000000 +0300
+++ conf.d/debug.conf   2019-02-04 09:49:41.928635000 +0300
@@ -19,7 +19,7 @@
 ##
 ## log all request headers.
 ##
-#debug.log-request-header          = "enable" 
+debug.log-request-header          = "enable" 

 ##
 ## similar to log-request-header.

Request headers:

2019-02-04 09:56:25: (request.c.601) fd: 9 request-len: 469 \nGET / HTTP/1.1\r\nHost: host1\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.9) Gecko/20100101 Goanna/4.1 Firefox/60.9 PaleMoon/2
8.3.1\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: ru,en-US;q=0.7,en;q=0.3\r\nAccept-Encoding: gzip, deflate\r\nX-Compress: 1\r\nProxy-Authorization: 659fe3b
4f93eee5f90a698fb46cabc06c7937c0b678d7a9b4ba1002b58d69b3bbd13310d70e42a30\r\nConnection: keep-alive\r\nUpgrade-Insecure-Requests: 1\r\n\r\n

http://host1/

GET / HTTP/1.1
Host: host1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.9) Gecko/20100101 Goanna/4.1 Firefox/60.9 PaleMoon/28.3.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: ru,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
X-Compress: 1
Proxy-Authorization: 659fe3b4f93eee5f90a698fb46cabc06c7937c0b678d7a9b4ba1002b58d69b3bbd13310d70e42a30
Connection: keep-alive
Upgrade-Insecure-Requests: 1

HTTP/1.1 200 OK
Content-Type: text/html
Accept-Ranges: bytes
ETag: "3801695155" 
Last-Modified: Thu, 08 Nov 2018 14:09:07 GMT
Content-Length: 6
Date: Mon, 04 Feb 2019 06:56:25 GMT
Server: lighttpd/1.4.53
----------------------------------------------------------

For testing I am using the stock configuration installed by FreeBSD port www/lighttpd (lighttpd-1.4.53).

That may be pertinent. I don't have a FreeBSD test system set up at the moment. You might want to experiment with building lighttpd from source to see if the issue persists.

I don't see how it helps. Building from port is just an automated way of building from source. When building from source we basically should do the same things but by hand. It is complicated and time-consuming. If you have any idea what can cause the issue, I would experiment with port modifications (change build options, startup script etc.).

Try building from upstream source, as a non-privileged user, and skip the install step. Create a local config file separate from all the existing configs provided by the FreeBSD package and run that test config from the build location. Then see if you can reproduce the problem.

I could not reproduce the problem on a Linux system and I even added tests to the test drivers to validate that things were working as I expected.
https://git.lighttpd.net/lighttpd/lighttpd1.4.git/commit/?h=personal/gstrauss/master&id=142e54b2a877397e64bc6bb658f71f1c3c1923d8

I have no idea how to run lighttpd from the build location.

% ~/lighttpd-1.4.53/src/lighttpd -f ~/config/lighttpd.conf
2019-02-07 20:21:50: (plugin.c.230) dlopen() failed for: /usr/local/lib/mod_indexfile.so Cannot open "/usr/local/lib/mod_indexfile.so" 
2019-02-07 20:21:50: (server.c.1177) loading plugins finally failed

cd ~/lighttpd-1.4.53/src/
./lighttpd -D -f ~/config/lighttpd.conf -m $PWD/.libs

Thank you for the explanation. I've reproduced the problem. The behavior is exactly the same as I described for lighttpd that was built from port.

Modifications I've made in the configuration:

# diff -ruN /home/moiseev/lighttpd-1.4.53/doc/config/ /home/moiseev/config/
diff -ruN /home/moiseev/lighttpd-1.4.53/doc/config/conf.d/evhost.conf /home/moiseev/config/conf.d/evhost.conf
--- /home/moiseev/lighttpd-1.4.53/doc/config/conf.d/evhost.conf 2019-01-27 12:22:20.000000000 +0300
+++ /home/moiseev/config/conf.d/evhost.conf     2019-02-08 09:55:31.982059000 +0300
@@ -16,7 +16,7 @@
 ## %3 => subdomain 1 name
 ## %4 => subdomain 2 name
 ##
-evhost.path-pattern        = vhosts_dir + "/%3/htdocs/" 
+evhost.path-pattern        = "/tmp/www/%0/" 

 ##
 #######################################################################
diff -ruN /home/moiseev/lighttpd-1.4.53/doc/config/lighttpd.conf /home/moiseev/config/lighttpd.conf
--- /home/moiseev/lighttpd-1.4.53/doc/config/lighttpd.conf      2019-01-27 12:22:20.000000000 +0300
+++ /home/moiseev/config/lighttpd.conf  2019-02-08 09:56:12.796802000 +0300
@@ -101,8 +101,8 @@
 ## Run as a different username/groupname.
 ## This requires root permissions during startup.
 ##
-server.username  = "lighttpd" 
-server.groupname = "lighttpd" 
+server.username  = "www" 
+server.groupname = "www" 

 ##
 ## enable core files.
@@ -178,7 +178,7 @@
 ##
 ## linux-sysepoll is recommended on kernel 2.6.
 ##
-server.event-handler = "linux-sysepoll" 
+server.event-handler = "freebsd-kqueue" 

 ##
 ## The basic network interface for all platforms at the syscalls read()
@@ -495,3 +495,4 @@
 #include "/etc/lighttpd/vhosts.d/*.conf" 
 ##
 #######################################################################
+$SERVER["socket"] == "0.0.0.0:80" { }
diff -ruN /home/moiseev/lighttpd-1.4.53/doc/config/modules.conf /home/moiseev/config/modules.conf
--- /home/moiseev/lighttpd-1.4.53/doc/config/modules.conf       2019-01-27 12:22:20.000000000 +0300
+++ /home/moiseev/config/modules.conf   2019-02-08 08:58:28.888771000 +0300
@@ -162,7 +162,7 @@
 ##
 ## mod_evhost
 ##
-#include "conf.d/evhost.conf" 
+include "conf.d/evhost.conf" 

 ##
 ## mod_simple_vhost

# tree /tmp/www
/tmp/www
├── host1
│   └── index.html
├── host2
│   └── index.html
└── index.html

2 directories, 3 files

Please attach the output of

./lighttpd -f ~/config/lighttpd.conf -m $PWD/.libs -p

I downloaded a FreeBSD 12.0 image, cloned lighttpd git DevelGit, and built the personal/gstrauss/master branch. The tests all pass, including the commit at the tip of personal/gstrauss/master which expands the tests for mod_evhost and explicitly exercises %0.

I disabled "mod_access" and "mod_accesslog". Nothing has changed.

# ./lighttpd -D -f /home/moiseev/config/lighttpd.conf -m $PWD/.libs -p | grep server.modules
    server.modules                 = ("mod_evhost")

I built the personal/gstrauss/master branch and tried to run my test config. The result is the same. Can you please give me the instruction on how to run the functional tests?

$ make check

All 10 tests passed

That the tests pass suggest that there is something specific to your system, and perhaps something else going on which manifests itself when mod_evhost tries to use con->uri.authority.

Try this:
debug.log-request-handling = "enable"
and looking in the error log to see what is reported for "URI-authority: ...", which is logged after the request is parsed.

Are you able to reproduce this on a different machine? I ask since I was not able to reproduce this.

2019-02-11 13:06:22: (response.c.429) -- splitting Request-URI
2019-02-11 13:06:22: (response.c.430) Request-URI     :  /
2019-02-11 13:06:22: (response.c.431) URI-scheme      :  http
2019-02-11 13:06:22: (response.c.432) URI-authority   :  host1
2019-02-11 13:06:22: (response.c.433) URI-path (raw)  :  /
2019-02-11 13:06:22: (response.c.434) URI-path (clean):  /
2019-02-11 13:06:22: (response.c.435) URI-query       :
2019-02-11 13:06:22: (response.c.580) -- before doc_root
2019-02-11 13:06:22: (response.c.581) Doc-Root     : /srv/www/htdocs
2019-02-11 13:06:22: (response.c.582) Rel-Path     : /
2019-02-11 13:06:22: (response.c.583) Path         :
2019-02-11 13:06:22: (response.c.625) -- after doc_root
2019-02-11 13:06:22: (response.c.626) Doc-Root     : /tmp/www//
2019-02-11 13:06:22: (response.c.627) Rel-Path     : /
2019-02-11 13:06:22: (response.c.628) Path         : /tmp/www//
2019-02-11 13:06:22: (response.c.652) -- logical -> physical
2019-02-11 13:06:22: (response.c.653) Doc-Root     : /tmp/www//
2019-02-11 13:06:22: (response.c.654) Basedir      : /tmp/www//
2019-02-11 13:06:22: (response.c.655) Rel-Path     : /
2019-02-11 13:06:22: (response.c.656) Path         : /tmp/www//
2019-02-11 13:06:22: (response.c.668) -- handling physical path
2019-02-11 13:06:22: (response.c.669) Path         : /tmp/www//
2019-02-11 13:06:22: (response.c.676) -- handling subrequest
2019-02-11 13:06:22: (response.c.677) Path         : /tmp/www//
2019-02-11 13:06:22: (response.c.678) URI          : /
2019-02-11 13:06:22: (response.c.679) Pathinfo     :
2019-02-11 13:06:22: (mod_indexfile.c.159) -- handling the request as Indexfile
2019-02-11 13:06:22: (mod_indexfile.c.160) URI          : /
2019-02-11 13:06:22: (mod_staticfile.c.197) -- handling file as static file

I reproduced the issue on VMware ESXi 6.5 using clean image FreeBSD-12.0-RELEASE-amd64.vmdk .
I installed just lighttpd build dependencies and git with its dependencies.

# pkg info -q
autoconf-2.69_2
autoconf-wrapper-20131203
automake-1.16.1_1
ca_root_nss-3.42.1
curl-7.64.0
cvsps-2.1_2
expat-2.2.6_1
gettext-runtime-0.19.8.1_2
git-2.20.1
indexinfo-0.3.1
libffi-3.2.1_3
libnghttp2-1.35.1
libtool-2.4.6_1
m4-1.4.18_1,1
p5-Authen-SASL-2.16_1
p5-CGI-4.40
p5-Digest-HMAC-1.03_1
p5-Error-0.17027
p5-GSSAPI-0.28_1
p5-HTML-Parser-3.72
p5-HTML-Tagset-3.20_1
p5-IO-Socket-INET6-2.72_1
p5-IO-Socket-SSL-2.060
p5-Mozilla-CA-20180117
p5-Net-SSLeay-1.85
p5-Socket6-0.29
pcre-8.42_1
perl5-5.28.1
pkg-1.10.5_5
pkgconf-1.5.4,1
python27-2.7.15
readline-7.0.5

Thank you for trying a different system.

Please duplicate with this patch, to get more information. If you get this trace, then mod_evhost is called. From your posts above, it looks like it is. The trace will validate what has been parsed in the request and may help point to a problem elsewhere.

--- a/src/mod_evhost.c
+++ b/src/mod_evhost.c
@@ -347,6 +347,14 @@ static handler_t mod_evhost_uri_handler(server *srv, connection *con, void *p_d)
                return HANDLER_GO_ON;
        }

+       {
+               data_string *ds;
+               array_reset_data_strings(srv->split_vals);
+               mod_evhost_parse_host(p->tmp_buf, srv->split_vals, con->uri.authority);
+               ds = (data_string *)array_get_element_klen(srv->split_vals, CONST_STR_LEN("%0"));
+               log_error_write(srv, __FILE__, __LINE__, "sbssd", "mod_evhost: authority:", con->uri.authority, "%0", ds && ds->value ? ds->value->ptr : "NULL", p->conf.len);
+       }
+
        mod_evhost_build_doc_root_path(p->tmp_buf, srv->split_vals, con->uri.authority, p->conf.path_pieces, p->conf.len);

        if (HANDLER_ERROR == stat_cache_get_entry(srv, con, p->tmp_buf, &sce)) {

2019-02-12 16:25:05: (mod_evhost.c.355) mod_evhost: authority: host1 %0 NULL 3 

I am unable to reproduce this using a fresh FreeBSD-12.0-RELEASE-amd64.raw with only packages added so that I can build and run lighttpd.
I imagine that you're just as frustrated as I am. What compiler are you using? What does the compilation line for mod_evhost.c look like? You have mentioned that %1 works for you, and it is not immediately apparent to me why that would work but %0 would not, as %0 is created in the array unconditionally, but for you it is not found.

On the off-chance that you have modified the output displayed to hide more sensitive data: are you using literal http://host1/ exactly like that?

# cc -v
FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1)
Target: x86_64-unknown-freebsd12.0
Thread model: posix
InstalledDir: /usr/bin

# pkg install autoconf automake libtool m4 pkgconf
# cd ~/lighttpd1.4
# ./autogen.sh
# ./configure
# make

libtool: compile:  cc -DHAVE_CONFIG_H -DHAVE_VERSIONSTAMP_H -DLIBRARY_DIR=\"/usr/local/lib\" -DSBIN_DIR=\"/usr/local/sbin\" -I. -I.. -D_REENTRANT -I/usr/local/include -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGE_FILES -g -O2 -Wall -W -Wshadow -pedantic -MT mod_evhost.lo -MD -MP -MF .deps/mod_evhost.Tpo -c mod_evhost.c  -fPIC -DPIC -o .libs/mod_evhost.o
/bin/sh ../libtool  --tag=CC    --mode=link cc   -g -O2 -Wall -W -Wshadow -pedantic  -module -export-dynamic -avoid-version  -o mod_evhost.la -rpath /usr/local/lib mod_evhost.lo

It is a test setup, so I am not hiding anything. I am using http://host1/ exactly. I added host1 to the hosts file on the machine from where I am accessing the URI.

I had been compiling with gcc. I compiled clean and then with clang, but still could not reproduce the issue. (clang 6.0.1 on FreeBSD 12.0)
I'll try rebuilding a clean image later this evening.

I am unable to reproduce your issue with clean image. Would you be more precise in how you reproduce this?

# pkg install git autoconf automake libtool m4 pkgconf
# git clone https://github.com/lighttpd/lighttpd1.4
# cd lighttpd1.4
# git checkout lighttpd-1.4.53
# ./autogen.sh
# ./configure
# make
# cd src
# vi t.conf
  server.document-root = "/tmp" 
  server.modules = ("mod_evhost")
  evhost.path-pattern = "/tmp/www/%0/" 
# echo default > /tmp/index.txt
# mkdir -p /tmp/www/host1
# echo host1 > /tmp/www/host1/index.txt
# ./lighttpd -D -f t.conf -m $PWD/.libs &
# telnet localhost 80
  GET /index.txt HTTP/1.0
  Host: host1

#### result 200 OK with response body: "host1", *not* "default" 
# fg
# Ctrl-C

I have done exactly what you did (except I have applied your patch for tracing) and I get exactly the same result.
Previously I used a web browser for testing. I think Connection: keep-alive header is the culprit.
I extended your test as following:

# echo www > /tmp/www/index.txt
# ./lighttpd -D -f t.conf -m $PWD/.libs &

# telnet localhost 80

GET /index.txt HTTP/1.0
Host: host1

2019-02-13 17:40:07: (mod_evhost.c.355) mod_evhost: authority: host1 %0 host1 3
HTTP/1.0 200 OK

host1
Connection closed by foreign host.
# telnet localhost 80

GET /index.txt HTTP/1.0
Host: host1
Connection: keep-alive

2019-02-13 17:40:25: (mod_evhost.c.355) mod_evhost: authority: host1 %0 NULL 3
HTTP/1.0 200 OK

www
Connection closed by foreign host.
# telnet localhost 80

GET /index.txt HTTP/1.0
Host: host1

2019-02-13 17:40:50: (mod_evhost.c.355) mod_evhost: authority: host1 %0 NULL 3
HTTP/1.0 200 OK

www
Connection closed by foreign host.