[Solved] Route requests to backend HTTP proxy server
Added by brianddk over 6 years ago
I have a very small footprint Ubuntu server that lighttpd is perfect for. The one additional thing I would like to do, to forward requests of the form http://mysite.tld:5678/myonionaddress.onion
to my proxy at localhost:9050
. I was thinking this might be possible in mod_proxy
with map-host-request
and map-urlpath
, but wasn't sure if its possible to add a header such as Proxy-Connection: Keep-Alive
.
Incoming request to myhost.tld:5678
GET /myonionaddress.onion/ HTTP/1.1 Host: myhost.tld:5678 User-Agent: curl/7.58.0 Accept: */*
Desired request to proxy at localhost:9050
GET http://myonionaddress.onion/ HTTP/1.1 Host: myonionaddress.onion User-Agent: curl/7.58.0 Accept: */* Proxy-Connection: Keep-Alive
My best-guess config
$SERVER["socket"] == ":5678" { $HTTP["url"] =~ "^/myonionaddress.onion" { proxy.server = ( "" => ( ( "host" => "127.0.0.1", "port" => "9050" ) ) ) proxy.header = ( "map-host-request" => ( "Host: myhost.tld:5678" => "Host: myonionaddress.onion" ), "map-urlpath" => ( "/myonionaddress.onion/" => "http://myonionaddress.onion/" ) ) } }
Question¶
- Are
map-urlpath
remaps freeform or do they need to start with a/
(backslash)? - Is there any way to add the
Proxy-Connection: Keep-Alive
header? - If this is ridiculous and using the wrong tool, can you recommend any light weight tools to do the same?
Answers (update)¶
- They are freeform and do not need to start with
/
. - Nope... not that I can find.
Working Config (note I did need to proxy through privoxy
to downgrade the SOCKS proxy to a standard HTTP proxy
$SERVER["socket"] == ":5678" { $HTTP["url"] =~ "^/myonionaddress.onion" { proxy.server = ( "" => ( ( "host" => "127.0.0.1", "port" => "8118" ) ) ) proxy.header = ( "map-urlpath" => ( "/myonionaddress.onion/" => "http://myonionaddress.onion/" )) setenv.set-request-header = ( "Host" => "myonionaddress.onion" ) } }
SysInfo¶
- Ubuntu 18.04.1 LTS (Bionic Beaver)
- Kernel 4.15.0-1021-gcp x86_64 (packaged with bionic)
- lighttpd 1.4.45-1ubuntu3 bionic in
/usr/sbin
- Client: curl/7.58.0
Replies (2)
RE: [Solved] Route requests to backend HTTP proxy server - Added by gstrauss over 6 years ago
For URL-rewriting, you should probably read the documentation Docs_ModProxy and see map-host-request
to replace your use of setenv.set-request-header
on the Host header.
"Proxy-Connection: Keep-Alive" is a hop-by-hop header, and setting the header does not magic into existence a whole lot of code that would be necessary to configure and manage connection pools, configure what to consider idempotent requests, and to retry connections, among many other things.
There are probably better tools for at least some of what you're trying to do. You might take a look at HAProxy.
RE: [Solved] Route requests to backend HTTP proxy server - Added by brianddk over 6 years ago
Thx... I had originally tried map-host-request
but couldn't get it to work. Might have just bugged my config, but was satisfied enough when I got setenv.set-request
to work.
Good info on Proxy-Connection
I didn't know it was transient. Turns out the proxies I'm using are forgiving enough to live ok without it. I think I have everything I need at this point without HAProxy
though that would likely be cleaner.
Basically I'm studying distributed networks, so I have a small site that I'm hosting on Internet, Freenet, Tor, I2P. Since it is all running on (effectively) a raspberry Pi, I was hesitant to add any more services. I'm using ModProxy to forward "specific" requests to the Tor, Freenet, and I2P localhost ports. This allows me to use Stackdriver's uptime checks to ensure that the content is reachable (in-theory) on all the networks.
So far so good... Thx again for the suggestions, and putting together such a robust lightweight web server... I've been very impressed I could piece this much together on such an anemic piece of HW.