Project

General

Profile

[Solved] Route requests to backend HTTP proxy server

Added by brianddk over 5 years ago

I have a very small footprint Ubuntu server that lighttpd is perfect for. The one additional thing I would like to do, to forward requests of the form http://mysite.tld:5678/myonionaddress.onion to my proxy at localhost:9050. I was thinking this might be possible in mod_proxy with map-host-request and map-urlpath, but wasn't sure if its possible to add a header such as Proxy-Connection: Keep-Alive.

Incoming request to myhost.tld:5678

GET /myonionaddress.onion/ HTTP/1.1
Host: myhost.tld:5678
User-Agent: curl/7.58.0
Accept: */*

Desired request to proxy at localhost:9050

GET http://myonionaddress.onion/ HTTP/1.1
Host: myonionaddress.onion
User-Agent: curl/7.58.0
Accept: */*
Proxy-Connection: Keep-Alive

My best-guess config

$SERVER["socket"] == ":5678" {
  $HTTP["url"] =~ "^/myonionaddress.onion" {
    proxy.server = ( "" => ( ( "host" => "127.0.0.1", "port" => "9050" ) ) )
    proxy.header =
      ( 
        "map-host-request" => ( "Host: myhost.tld:5678" => "Host: myonionaddress.onion" ),
        "map-urlpath"      => ( "/myonionaddress.onion/" => "http://myonionaddress.onion/" )
      )
  }
}

Question

  1. Are map-urlpath remaps freeform or do they need to start with a / (backslash)?
  2. Is there any way to add the Proxy-Connection: Keep-Alive header?
  3. If this is ridiculous and using the wrong tool, can you recommend any light weight tools to do the same?

Answers (update)

  1. They are freeform and do not need to start with /.
  2. Nope... not that I can find.

Working Config (note I did need to proxy through privoxy to downgrade the SOCKS proxy to a standard HTTP proxy

$SERVER["socket"] == ":5678" {
  $HTTP["url"] =~ "^/myonionaddress.onion" {
    proxy.server = ( "" => ( ( "host" => "127.0.0.1", "port" => "8118" ) ) )
    proxy.header = ( "map-urlpath" => ( "/myonionaddress.onion/" => "http://myonionaddress.onion/" ))
    setenv.set-request-header = ( "Host" => "myonionaddress.onion" )
  }
}

SysInfo

  • Ubuntu 18.04.1 LTS (Bionic Beaver)
  • Kernel 4.15.0-1021-gcp x86_64 (packaged with bionic)
  • lighttpd 1.4.45-1ubuntu3 bionic in /usr/sbin
  • Client: curl/7.58.0

Replies (2)

RE: [Solved] Route requests to backend HTTP proxy server - Added by gstrauss over 5 years ago

For URL-rewriting, you should probably read the documentation Docs_ModProxy and see map-host-request to replace your use of setenv.set-request-header on the Host header.

"Proxy-Connection: Keep-Alive" is a hop-by-hop header, and setting the header does not magic into existence a whole lot of code that would be necessary to configure and manage connection pools, configure what to consider idempotent requests, and to retry connections, among many other things.

There are probably better tools for at least some of what you're trying to do. You might take a look at HAProxy.

RE: [Solved] Route requests to backend HTTP proxy server - Added by brianddk over 5 years ago

Thx... I had originally tried map-host-request but couldn't get it to work. Might have just bugged my config, but was satisfied enough when I got setenv.set-request to work.

Good info on Proxy-Connection I didn't know it was transient. Turns out the proxies I'm using are forgiving enough to live ok without it. I think I have everything I need at this point without HAProxy though that would likely be cleaner.

Basically I'm studying distributed networks, so I have a small site that I'm hosting on Internet, Freenet, Tor, I2P. Since it is all running on (effectively) a raspberry Pi, I was hesitant to add any more services. I'm using ModProxy to forward "specific" requests to the Tor, Freenet, and I2P localhost ports. This allows me to use Stackdriver's uptime checks to ensure that the content is reachable (in-theory) on all the networks.

So far so good... Thx again for the suggestions, and putting together such a robust lightweight web server... I've been very impressed I could piece this much together on such an anemic piece of HW.

    (1-2/2)