Lighttpd

Yes, this is a mod_proxy thing. See the first Example in Docs_ModProxy. Basically, create a condition for each "Host" you want to proxy, and in that condition, configure the proxy.server for lighttpd to act as a reverse proxy and forward all requests to (one or more) backend servers. That way, you can configured a different proxy.server for each DNS name for which you want to forward to a different backend server.

Thanks heaps! I take it the first example is:

  $HTTP["host"] == "www.example.org" {
    proxy.balance = "hash" 
    proxy.server  = ( "" => ( ( "host" => "10.0.0.10" ),
                              ( "host" => "10.0.0.11" ),
                              ( "host" => "10.0.0.12" ),
                              ( "host" => "10.0.0.13" ),
                              ( "host" => "10.0.0.14" ),
                              ( "host" => "10.0.0.15" ),
                              ( "host" => "10.0.0.16" ),
                              ( "host" => "10.0.0.17" ) ) )
  }

So I imagine something like this for my example:

  $HTTP["host"] == "sdom1.dom1.tld1" {
    proxy.server  = ( "" => ( ( "host" => "localhost" ) ) )
  }

  $HTTP["host"] == "sdom2.dom1.tld1" {
    proxy.server  = ( "" => ( ( "host" => "serv1" ) ) )
  }

  $HTTP["host"] == "sdom3.dom2.tld2" {
    proxy.server  = ( "" => ( ( "host" => "serv2" ) ) )
  }

Enormously grateful as I can focus my reading and experimenting on these directives now (e.g. understanding the syntax, or structure of the proxy.server property, and whether the host attribute can be a DNS resolvable name or must be an IP address.

Regards,

Bernd.

$HTTP["host"] == "sdom1.dom1.tld1"

"sdom1.dom1.tld1" is the name provided in the HTTP Host header (Host: sdom1.dom1.tld1)

proxy.server = ( "" => ( ( "host" => "serv1" ) ) )

The value (e.g. "serv1") for "host" in each proxy.server definition must be an IP address or a variable (e.g. var.MYNAME) (see Docs_ConfigurationOptions)

If you would like to use DNS names, then have your startup script resolve the IPs and set environment variables (MYNAME=IP-address) before starting up lighttpd.

OK, I've experimented some and failed :-(.

I have mod_proxy installed, and to /etc/lighttpd/lightttpd.conf I add:

$HTTP["host"] == "sub.domain.tld" {
      proxy.server  = ( "" => ( ( "host" => "192.168.0.11", "port" => 80 ) ) )
}

and nothing is redirected. I have logging enabled as follows:

# Configure the access log
accesslog.filename          = "/var/log/lighttpd/access.log" 
accesslog.format            = "%h %V %u %t \"%r\" (%{HOST}i) %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" 

so I can see what's going on notably the host header.

So before adding this line I test in my browser and confirm that:

http://domain.tld
http://sub.domain.tld

both show me the routers default admin interface (not the long term goal, but fine for testing here).

I also test that:

http://192.168.0.11

displays another page, which it does. For now similarly, just a simple default index.html really.

Now I check the access log and see that ${HOST}i in the log is reporting domain.tld and sub.domain.tld respectively on a page request, and moth show the routers admin interface (as they should as both point to the same IP.

Now I add the lines above to lighttpd.conf and restart lighttp and retest.

Alas, same result. No redirection to 192.168.0.11's server for http://sub.domain.tld

So, the real question I have then is how to diagnose further. So I enable debugging:

proxy.debug = 1
$HTTP["host"] == "sub.domain.tld" {
      proxy.server  = ( "" => ( ( "host" => "192.168.0.11", "port" => 80 ) ) )
}

But can't see any debug output anywhere. I even stop lighttpd (/etc/init.d/lighttpd stop) and run it in foreground (lighttpd -D) and all works as above but see no debug output on stdout or stderr I guess.

I wonder if I'm doing anything obviously wrong here?

OK, progressing slightly. Reason to suspect mod_proxy just isn't properly working or running at all.

I found this:

https://redmine.lighttpd.net/projects/1/wiki/DebugVariables

and so I set:

debug.log-request-header = "enable" 

and then:

tail -f /var/log/lighttpd/error.log

and voila I see each request fully presented but no feedback from mod_proxy even though I have:

proxy.debug = 1

So a bit stuck. I can see this:

# cat /etc/lighttpd/modules.d/30-proxy.load 
server.modules += ( "mod_proxy" )
# ll /usr/lib/lighttpd/
-rwxr-xr-x    1 root     root          9348 Apr 24 23:58 mod_access.so*
-rwxr-xr-x    1 root     root         17680 Apr 24 23:58 mod_accesslog.so*
-rwxr-xr-x    1 root     root          9332 Mar 14 05:14 mod_alias.so*
-rwxr-xr-x    1 root     root         21668 Mar 14 05:14 mod_cgi.so*
-rwxr-xr-x    1 root     root         21844 Mar 14 05:14 mod_dirlisting.so*
-rwxr-xr-x    1 root     root         46548 Mar 14 05:14 mod_fastcgi.so*
-rwxr-xr-x    1 root     root          9348 Mar 14 05:14 mod_indexfile.so*
-rwxr-xr-x    1 root     root         21716 Apr 24 23:58 mod_proxy.so*
-rwxr-xr-x    1 root     root          5268 Mar 14 05:14 mod_setenv.so*
-rwxr-xr-x    1 root     root          9364 Mar 14 05:14 mod_staticfile.so*

and in /etc/lighttpd/lighttpd.conf is:

include "/etc/lighttpd/modules.d/*.load" 

which seems to load the other modules just fine.

But I'm not seeing any evidence that mod_proxy is running at all. Aside form the indirect evidence that setting proxy.debug and possible proxy.server does not through and error about an undefined variable.

Perplexed but still researching. If only I could find a way to confirm mod_proxy was running, that the proxy.server variable was set, and that it wants to act on it.

A little more progress. I fonud this neat feature:

http://blog.lighttpd.net/articles/2006/04/02/log-condition-handling-the-hidden-feature-for-debuging/

and low and behold I find in the error log:

2017-05-03 15:33:06: (configfile-glue.c.557) HTTP["host"] ( sub.domain.tld ) compare to  sub.domain.tld 
2017-05-03 15:33:06: (configfile-glue.c.615) 7 (uncached) result: true 

which is awesome. I confirms that the code proxy.server = is reached. But it's having no effect alas. That is still the salient mystery.

In summary again, this:

proxy.debug = 1
$HTTP["host"] == "sub.domain.tld" {
    proxy.server  = ( "" => ( "" => ( "host" => "192.168.0.11", "port" => 80 ) ) )
}

is producing no debug info in error.log nor directing the request to 192.168.0.11 alas.

More progress. This thread:

https://redmine.lighttpd.net/boards/2/topics/5664

proves that with proxy.debug = 1 I should expect something in the error log like:

2013-05-28 14:38:55: (mod_proxy.c.1142) proxy - start

and I see no such thing. I see:

2017-05-03 17:04:25: (server.c.1828) server stopped by UID = 0 PID = 23579 
2017-05-03 17:04:26: (log.c.217) server started

but nothing from mod_proxy.c. Further evidence that mod_proxy simply isn't loading.

No indication in error log as to why alas.

Egads! Hair pulling time. Fixed but not understood. A digression, so I'll document what's been learned here to close it, but there is more to diagnose, understand and learn, with some scope for a lighttpd bug perhaps (though not likely in the first instance as with my level of newbness here, ignorance of some nuance seems more likely).

But here's the deal. If at the top of my lighttpd.conf I put:

server.modules = ("mod_proxy")

then two conflicting things happen (suggesting a bug, but I'm still wary that my newbness is at fault):

1. Restarting lighttpd reports in error log:

2017-05-03 17:26:28: (plugin.c.187) Cannot load plugin mod_proxy more than once, please fix your config (lighttpd may not accept such configs in future releases) 
2017-05-03 17:26:28: (plugin.c.187) Cannot load plugin mod_proxy more than once, please fix your config (lighttpd may not accept such configs in future releases)

Yes, twice, nor once. Without that line at top of my lighttpd.conf I have no evidence of mod_proxy loading in the error log. With this line I get these errors, but also see evidence that it loaded, and here's crunch, the forward then works! Yay. Mission accomplished. But:

2. It's loaded 0 times or twice it seems! Bananas. Total bananas. The second time is the default as configured by openWRT when the package is installed. It uses this method:

In lighttpd.conf:

server.modules = ()
include "/etc/lighttpd/modules.d/*.load" 

and:

# ls -l /etc/lighttpd/modules.d/
-rw-r--r--    1 root     root            35 Apr 24 23:58 30-access.load
-rw-r--r--    1 root     root            38 Apr 24 23:58 30-accesslog.load
-rw-r--r--    1 root     root            34 Mar 14 05:14 30-alias.load
-rw-r--r--    1 root     root            32 Mar 14 05:14 30-cgi.load
-rw-r--r--    1 root     root            36 Mar 14 05:14 30-fastcgi.load
-rw-r--r--    1 root     root            34 Apr 24 23:58 30-proxy.load
-rw-r--r--    1 root     root            35 Mar 14 05:14 30-setenv.load

# cat /etc/lighttpd/modules.d/30-proxy.load 
server.modules += ( "mod_proxy" )

This clearly is a directive to load the module, as evidence by fact if I load it explicitly at top of lighttpd.conf it's loaded twice! Yet, unless I force a load at top of lighttpd.conf it's not loaded at all! Madness.

But off topic and not the original issue, so to explore elsewhere if at all. Just documenting it here for now as it is the learning outcome of a lot of experimentation. I am pleased though that noce it is loaded it works as described and thankful for the tips that got me there!

A little more progress. I fonud this neat feature:
http://blog.lighttpd.net/articles/2006/04/02/log-condition-handling-the-hidden-feature-for-debuging/
and low and behold I find in the error log:

This is documented under the section "Core Debug Info"
https://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_ConfigurationOptions#Core-Debug-Info

If you run lighttpd -h, you'll see the -p option. If you run lighttpd -f /etc/lighttpd/lighttpd.conf -p, then you'll see what lighttpd thinks of your config files. If a module is listed in server.modules, then the module is loaded, or lighttpd exits with an error.

Now, the order in which modules are loaded defines in which order each one gets a chance to service the request. This is in the documentation for at server.modules. If I had to guess, then I would guess that mod_cgi or mod_fastcgi is servicing the request to load the internal router page, and mod_proxy therefore never gets a chance at the request (until you duplicatively load mod_proxy at the top of the modules list).

FYI: while you provided a nice set of observations, you neglected to include the basics: the version of lighttpd you are running, and your complete config (with any sensitive info redacted with xxxxx) (lighttpd -f /etc/lighttpd/lighttpd.conf -p)

You may need to put cgi.assign = () and/or fastcgi.server = () in your vhost configs if the global scope defines the handlers for your router page and mod_proxy is loaded after mod_cgi and mod_fastcgi. This appear to be the way that openwrt is configured to load lighttpd. An alternative is to rename /etc/lighttpd/modules.d/30-proxy.load to 29-proxy.load, as the modules are loaded in sorted order from the glob of files in /etc/lighttpd/modules.d/. Another alternative is to set server.modules = ( "mod_proxy") in /etc/lighttpd/lighttpd.conf before the inclusion of /etc/lighttpd/modules.d/*, and to comment out the server.modules += ( "mod_proxy" ) in /etc/lighttpd/modules.d/30-proxy.load

Thanks, will experiment. The extra explanation is deeply appreciated. And apologies for incomplete. Version is 1.4.45 for the record. But a but late now.

Module loading order is what I'll play with on the morrow. The surprise mysides I guess is that on openWRT, opkg installs the modules all with 30- and pays no head to order, but it may well be that there is no universally applicable plan (i.e. some needs will see one module getting first look-in and other need will see another module getting first look in). I'll enjoy experimenting a little to see if I can isolate which module steals mod_proxy's opportunity to work ... and report here again for completeness (only because such threads show up on web and forum searches others make who have needs as described in subject).

Thanks ever so much for your tips and patience with a newb (who, yes, like most I suspect exercises a decent mix of reading, research and hands on experimentation and didn't catch everything in the docs alas when he maybe should have).

OK, kids faster to put to bed than thought so looked at it now. ;-)

Looking at all the modules I have I identified empirically that it is indeed the fastcgi module and only the fastcgi module which, if loaded before mod_proxy robs it of a look-in. mod_cgi is harmless in this regard.

I admit some surprise at this as fastcgi is only loaded but not used anywhere in the conf. The LuCI interface uses only mod_cgi. To wit, untouched other than loading it still robs mod_proxy of a look-in.

Ever eager to learn as I am, I did juste xperiment with loading mod_fastcgi before mod_proxy (and mod_proxy doesn't get a look-in) then added fastcgi.server = () after loading modules and it makes no difference, mod_proxy still doesn't get a look-in. I am not using fastcgi so may as well not load it (as I'll use mod_proxy to farm out to other servers on the LAN) but I am left wondering if the reverse is true. Namely if loading mod_proxy before mod_fastcgi robs fastcgi of a look-in.

Just musing now. Much learning has been done and is now well noted here. I will probably summarise a short how-to on the openWRT forums too for others wishing to use a gateway router to farm web requests invisibly to LAN servers all through one IP.

I admit some surprise at this as fastcgi is only loaded but not used anywhere in the conf.

More than likely, that is an incorrect conclusion on your part. I'll repeat myself once again:
Review the output of lighttpd -f /etc/lighttpd/lighttpd.conf -p

Please do not pontificate further here until you have reviewed that output of your lighttpd.conf and shared your config here (as an attachment, please)

then added fastcgi.server = () after loading modules

My suggestion was to add that inside the vhost condition

$HTTP["host"] == "abc.tld" {
  fastcgi.server = ()
  proxy.server = ( "" => ( ( "host" => "...", "port" => "..." ) ) )
}

The setting inside the vhost condition will replace any fastcgi.server configured at the global scope for the default vhost/server