Lighttpd

the configuration.

$HTTP["url"] =~ "^/dashboard/daimh/test/" {
proxy.server = ( "" => ( ( "host" => "mengf-s1", "port" => "1024" ) ) )
proxy.header = (
"https-remap" => "enable",
"map-urlpath" => ( "/dashboard/daimh/test/" => "/" ),
"upgrade" => "enable",
)
}

The actual web socket server I used for test is an R shiny demo.

echo "library(shiny); runExample('01_hello', host='0.0.0.0', port = 1024, launch.browser=F)" | R -q --no-save

Please share your lighttpd config: lighttpd -f /etc/lighttpd/lighttpd.conf -p (minus any passwords, if present)

Thanks a lot for getting back to me!

There are two config files. One is /etc/lighttpd/lighttpd.conf ################

server.port        = 80
server.username        = "http" 
server.groupname    = "http" 
server.document-root    = "/srv/http" 
server.errorlog        = "/var/log/lighttpd/error.log" 
dir-listing.activate    = "disable" 
index-file.names    = (
                "index.html",
                "index.py",
                "index.sh",
            )
mimetype.assign        = (
                ".html" => "text/html",
                ".txt" => "text/plain",
                ".css" => "text/css",
                ".js" => "application/x-javascript",
                ".jpg" => "image/jpeg",
                ".jpeg" => "image/jpeg",
                ".gif" => "image/gif",
                ".png" => "image/png",
                ".svg" => "image/svg+xml",
                ".mp4" => "video/mp4",
                "" => "application/octet-stream" 
            )
server.max-read-idle = 1800
server.modules        += (
                "mod_accesslog",
                "mod_alias",
                "mod_auth",
                "mod_authn_file",
                "mod_cgi",
                "mod_openssl",
                "mod_proxy",
                "mod_redirect",
            )
accesslog.filename    = "/var/log/lighttpd/access.log" 
server.breakagelog    = "/var/log/lighttpd/breakage.log" 
cgi.assign        = (
                ".py2" => "/usr/bin/python2",
                ".py" => "/usr/bin/python",
                ".sh" => "/usr/bin/bash",
            )
cgi.upgrade            = "enable" #websocket
static-file.exclude-extensions = ( ".py", ".sh" )
$HTTP["scheme"] == "http" {
    $HTTP["host"] =~ ".*" {
        url.redirect = (
            "^/$" => "https://%0$0",
            "^/[^.].*" => "https://%0$0",
        )
    }
}
$SERVER["socket"] == ":443" {
    ssl.engine = "enable" 
    ssl.pemfile = "/etc/ssl/private/certbot-combined.pem" # Combined Certificate
    ssl.ca-file = "/etc/ssl/certs/certbot-chain.crt" # Root CA
    ssl.cipher-list = "FIPS: +3DES:!aNULL" 
    ssl.openssl.ssl-conf-cmd  = ("MinProtocol" => "TLSv1.2")
}
include "/opt/dashboard/var/*/*.conf" 

###And another file under /opt/dashboard/var/<USER>/<USER>.conf

$HTTP["url"] =~ "^/dashboard/daimh/test/" {
    proxy.server = ( "" => ( ( "host" => "shiny-host", "port" => "1024" ) ) )
    proxy.header = (
        "https-remap" => "enable",
        "map-urlpath" => ( "/dashboard/USER/test/" => "/" ),
        "upgrade" => "enable",
    )
}

A simple websockets test worked for me, but I am continuing to test.

What is the HTTP request being sent? I'd like to confirm if the request is being handled by mod_cgi or mod_proxy.

BTW, do you really want to enable +3DES in ssl.cipher-list? You probably meant -3DES
See lighttpd TLS cipher recommendations for a stronger cipher list.

The http request I tested is an R shiny demo running on another host.
echo "library(shiny); runExample('01_hello', host='0.0.0.0', port = 1024, launch.browser=F)" | R -q --no-save

I just tested it with 1.4.56. If the line mod_proxy is not included in server.modules, the same url is handled by my CGI program with mod_cgi, but if the line mod_proxy is included in server.module, the url request is handled by mod_proxy and sent to R shiny server. This is actually I wanted, and it worked before.

In other words, if the url is /dashboard, it will be handled by my CGI program. But if the url is something like /dashboard/shinydemo, it will be handled by mod_proxy and the request is sent to another host.

Let me know if you have any question, I will check my email tomorrow morning. thanks a million!

echo "library(shiny); runExample('01_hello', host='0.0.0.0', port = 1024, launch.browser=F)" | R -q --no-save

I think you have made an assumption that everyone knows what shiny is and where to get 01_hello, and has R installed. Those assumptions are all incorrect.

I just tested it with 1.4.56. If the line mod_proxy is not included in server.modules, the same url is handled by my CGI program with mod_cgi, but if the line mod_proxy is included in server.module, the url request is handled by mod_proxy and sent to R shiny server. This is actually I wanted, and it worked before.

That is slightly confusing. Would you please try to be more clear about which scenarios work and which scenarios do not? Please do not assume that I know anything about R.

From your initial post, it seems that your client is Firefox. Is that correct? What is the connection string in the javascript which is used to send an HTTP request? (i.e. what is the HTTP request?)

So sorry for the misunderstanding. I picked an R shiny server demo in my test because it is the easiest for me and all my services that need a lighttpd websocket proxy is based on R shiny. Right now I really don't have a quick way to start a web socket server with other language to test the issue, but I will try it tomorrow morning when I get a chance to use my desktop.

And yes, my client is Firefox. But I just tested it with my android chrome, 1.4.56 still works fine, but 1.4.59 still doesn't.

I started a working demo at https://y.mbni.org/dashboard/daimh/test/ , which is running with 1.4.56. You should be able to access the url and see the last request is a websocket GET. If I upgraded lighttpd to 1.4.59, the websocket request still sends the same data, but won't get anything back.

I also did a test by fully disabling mod_cgi. The above demo still works with lighttpd 1.4.56, but not 1.4.59. Thus I believe the problem is not related to mod_cgi.

Feel free to let me know if you have any questions, your help is greatly appreciated!

This is what an HTTP request looks like, which you can see under Firefox "Tools..Web Developer" console.

GET wss://y.mbni.org/dashboard/daimh/test/websocket/ HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.5
Cache-Control: no-cache
Connection: keep-alive, Upgrade
DNT: 1
Host: y.mbni.org
Origin: https://y.mbni.org
Pragma: no-cache
Sec-WebSocket-Extensions: permessage-deflate
Sec-WebSocket-Key: [snip]
Sec-WebSocket-Version: 13
Upgrade: websocket
User-Agent: [snip]

You can use strace -s 2048 -p <pid> on the lighttpd server pid and, separately, on your websocket backend to see what lighttpd is sending to the backend and what the backend is receiving. Look for read write send recv in the strace when you send a single request that fails. Please share the results.

You said that Firefox is sending requests. Is lighttpd sending those requests to the backend? Is the backend receiving those requests? Is the backend responding to those requests? Is lighttpd receiving the response from the backend? Is lighttpd sending the response back to Firefox? (The response to Firefox is likely encrypted, so while you will not be able to read it, you can see whether or not lighttpd is sending something back to the client.)

Thank you for those. I took a quick look at the backend-* and they look similar in syscalls. I'll read more carefully through the lighttpd-* strace later today.

FWIW, I just tested a simple Python websocket client/server program does work fine with both lighttpd versions as websocket proxy.

Please try with the following. Note the addition of server.stream-request-body and server.stream-response-body. Streaming with mode = 2 is automatically enabled in mod_proxy after Connection: upgrade results in 101 Switching Protocols, if streaming is not already enabled. I am curious if changing the streaming mode makes a difference for you, and may help me to track down the issue.

$HTTP["url"] =~ "^/dashboard/daimh/test/" {
    server.stream-response-body = 1
    server.stream-request-body = 1
    proxy.server = ( "" => ( ( "host" => "shiny-host", "port" => "1024" ) ) )
    proxy.header = (
        "https-remap" => "enable",
        "map-urlpath" => ( "/dashboard/USER/test/" => "/" ),
        "upgrade" => "enable",
    )
}

The problem persists after I added the two lines below to the configuration file.

server.stream-response-body = 1
server.stream-request-body = 1

I will report back as soon as Arch Linux package is upgraded with this patch! Thanks a million! @gstrauss

lighttpd 1.4.59 was just released 2 Feb (2 days ago). The above patch is on the lighttpd git master branch and will be part of lighttpd 1.4.60, but the release of lighttpd 1.4.60 is not yet scheduled.

Did you submit a request to Arch package maintainers to pick up this patch in the meantime?

I believe Arch Linux package maintainer won't do so as the 'source' line in its PKGBUILD file at the link below uses your tar.gz file, instead of your git master branch.

I can live with version 1.4.56 for a while, unless you have concern.

I truly appreciate your help! Lighttpd is an excellent software, please keep up the good work! I will report back as soon as 1.4.60 is released.

https://github.com/archlinux/svntogit-packages/blob/packages/lighttpd/trunk/PKGBUILD

FYI: pkg supports patches, e.g. for apache https://github.com/archlinux/svntogit-packages/tree/master/apache/trunk

You could also take the arch repo for lighttpd, add a patch, and build your own arch package of lighttpd.

.

lighttpd 1.4.60 will probably be released in a month or two, depending on feedback.

Just a feedback. I tested this bug is fixed in the current git repo.

I also created an ArchLinux AUR package at

https://aur.archlinux.org/packages/lighttpd1.4-git/

Thanks for the ping. FYI: lighttpd 1.4.60 is gearing up to be released in the next week or so.

FYI: lighttpd 1.4.60 has been released.