WebSockets¶
- Table of contents
- WebSockets
WebSocket echo samples
Overview¶
What are websockets? The WebSocket API
lighttpd can facilitate WebSocket connections in numerous ways, including via mod_proxy, mod_cgi, mod_scgi, mod_fastcgi, mod_wstunnel.
- lighttpd supports HTTP/1.1
Upgrade: websocketfrom the HTTP/1.1 connection. - lighttpd supports HTTP/2 extended CONNECT with
:protocol: websocketon HTTP/2 streams over a single HTTP/2 connection.
For security reasons and supporting the principle of least surprise, lighttpd support for websockets is intentionally disabled by default. lighttpd dynamic modules may be configured to enable connection upgrade to the websocket protocol.
Since lighttpd 1.4.46, lighttpd mod_cgi, mod_proxy, and mod_wstunnel support websockets. Support in additional dynamic modules mod_scgi and mod_fastcgi has been added in lighttpd 1.4.74.
This document demonstrates multiple ways to configure lighttpd modules to run WebSocket "echo" sample applications.
Basic Configuration¶
websockets via mod_wstunnel¶
Configure mod_wstunnel
mod_wstunnel is a WebSocket tunnel endpoint, terminating the websocket tunnel from a client. mod_wstunnel decodes websocket frames and then passes data (without websocket frames) to a backend, and in the opposite direction encodes responses from backend into websocket frames before sending responses to client.
websockets via mod_proxy¶
Configure mod_proxy and enable proxy.header += ("upgrade" => "enable")
With lighttpd 1.4.74 and later, "upgrade" => "enable" is also a host option in proxy.server
mod_proxy can act as a reverse proxy for websocket connections when upgrade is enabled.
websockets via mod_cgi¶
Configure mod_cgi and enable cgi.upgrade = "enable"
mod_cgi allows the target program to upgrade to use the websocket protocol.
websockets via mod_scgi¶
Configure mod_scgi and add "upgrade" => "enable" to options for each host in scgi.server (lighttpd 1.4.74 and later)
mod_scgi allows the target program to upgrade to use the websocket protocol.
websockets via mod_fastcgi¶
Configure mod_fastcgi and add "upgrade" => "enable" to options for each host in fastcgi.server (lighttpd 1.4.74 and later)
mod_fastcgi allows the target program to upgrade to use the websocket protocol.
While lighttpd does not currently multiplex multiple requests on a single FastCGI connection, the FastCGI protocol allows such. lighttpd mod_fastcgi sends and receives FastCGI packets from the backend, even after a request is upgraded to use the websocket protocol. websocket frames are again framed into FastCGI packets. As such, this may be less efficient than using other lighttpd modules which support upgrading to the websocket protocol.
Sample: ws_echo.py¶
ws_echo.py - simple "echo" websocket server CGI sample program
Install files under document root and ws/ subdirectory<docroot>/ws_echo.html<docroot>/ws/ws_echo.py
Modify the URL in ws_echo.html to use wss:// instead of ws:// if using https.var host = "ws://" + window.location.hostname + "/ws/ws_echo.py";
<docroot>/ws_echo.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<!-- favicon: none -->
<link rel="icon" href="data:,">
<!--
<meta
http-equiv="Content-Security-Policy"
content="default-src 'self' 'unsafe-inline'" />
-->
<title>websocket echo client</title>
<style>
#output {
border: solid 1px #000;
}
</style>
</head>
<body>
<form id="form" accept-charset=utf-8>
<input type="text" id="message">
<button type="submit">Send</button>
</form>
<hr>
<div id="output"></div>
<script>
var inputBox = document.getElementById("message");
var output = document.getElementById("output");
var form = document.getElementById("form");
try {
var host = "ws://" + window.location.hostname + "/ws/ws_echo.py";
//var host = "wss://" + window.location.hostname + "/ws/ws_echo.py";
//var host = "ws://" + window.location.hostname + ":8080/ws/ws_echo.py";
//var host = "wss://" + window.location.hostname + ":8443/ws/ws_echo.py";
console.log("Host:", host);
var s = new WebSocket(host);
s.onopen = function (e) {
console.log("Socket opened.");
};
s.onclose = function (e) {
console.log("Socket closed.");
};
s.onmessage = function (e) {
console.log("Socket message:", e.data);
var p = document.createElement("p");
p.innerHTML = e.data;
output.appendChild(p);
};
s.onerror = function (e) {
console.log("Socket error:", e);
};
} catch (ex) {
console.log("Socket exception:", ex);
}
form.addEventListener("submit", function (e) {
e.preventDefault();
s.send(inputBox.value);
inputBox.value = "";
}, false)
</script>
</body>
</html>
<docroot>/ws/ws_echo.py
#!/usr/bin/env python3
#
# ws_echo.py - websocket echo; reflect back to peer the frames sent by peer
#
# Copyright(c) 2023 Glenn Strauss gstrauss()gluelogic.com All rights reserved
# License: BSD 3-clause (same as lighttpd)
import struct
import hashlib
import base64
def ws_upgrade(rd, wr, env):
if env.get('HTTP_UPGRADE', '') != 'websocket':
wr.write("Status: 204\n\n".encode())
return False
# (throws KeyError exception if any of these env vars are not set)
origin = env['HTTP_ORIGIN']
scheme = "wss" if env['REQUEST_SCHEME'] == "https" else "ws"
host = env['HTTP_HOST']
urlpath= env['SCRIPT_NAME']
key = env['HTTP_SEC_WEBSOCKET_KEY'].encode('utf-8')
GUID = b"258EAFA5-E914-47DA-95CA-C5AB0DC85B11"
digest = str(base64.b64encode(hashlib.sha1(key + GUID).digest()), 'utf-8')
wr.write(("Status: 101\n"
+ "Upgrade: WebSocket\n"
+ "Connection: Upgrade\n"
+ f"Sec-WebSocket-Origin: {origin}\n"
+ f"Sec-WebSocket-Location: {scheme}://{host}{urlpath}\n"
+ f"Sec-WebSocket-Accept: {digest}\n\n"
).encode())
return True
## +-+-+-+-+-------++-+-------------+-------------------------------+
## |F|R|R|R| opcode||M| Payload len | Extended payload length |
## |I|S|S|S| (4) ||A| (7) | (16/63) |
## |N|V|V|V| ||S| | (if payload len==126/127) |
## | |1|2|3| ||K| | |
## +-+-+-+-+-------++-+-------------+ - - - - - - - - - - - - - - - +
## +-+-+-+-+--------------------------------------------------------+
## | Extended payload length continued, if payload len == 127 |
## + - - - - - - - - - - - - - - - +--------------------------------+
## + - - - - - - - - - - - - - - - +-------------------------------+
## | |Masking-key, if MASK set to 1 |
## +-------------------------------+-------------------------------+
## | Masking-key (continued) | Payload Data |
## +-------------------------------- - - - - - - - - - - - - - - - +
## : Payload Data continued ... :
## + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
## | Payload Data continued ... |
## +---------------------------------------------------------------+
def ws_recv_and_echo(rd, wr):
# (note: assumes well-formed websocket frame; not validating or paranoid)
# (note: inefficient small reads rather than read/buffering larger blocks)
# (note: no buffering; program exits if partial read of frame header)
# (note: no buffering; program loses state if partial read of frame data
# and subsequently will likely thrown an exception)
# (tl;dr: this toy program is sufficient for a demo of frames with small
# data payloads, but probably nothing more)
data = rd.read(2)
if len(data) == 0: # EOF
ws_send(wr, b'1000', opcode=0x8) # CLOSE "1000 = Normal Closure"
return False
head1, head2 = struct.unpack('!BB', data)
fin = bool(head1 & 0b10000000)
opcode = head1 & 0b00001111
if opcode == 0x8: # CLOSE (0x8)
# (note: not reading and parsing close reason sent by peer)
ws_send(wr, b'1000', opcode=opcode) # "1000 = Normal Closure"
return False
length = head2 & 0b01111111
if length == 126:
data = rd.read(2)
length, = struct.unpack('!H', data)
elif length == 127:
data = rd.read(8)
length, = struct.unpack('!Q', data)
mask_bit = bool(head2 & 0b10000000)
masking_key = rd.read(4) if mask_bit else False
data = bytearray(rd.read(length)) if length else b''
if opcode == 0xA: # PONG
# receive and ignore response to PING
return True
if opcode == 0x9: # PING
# reflect PING back to peer as PONG
opcode = 0xA # PONG
if mask_bit:
# unmask data
for i in range(0, len(data)):
data[i] ^= masking_key[i % 4]
# echo data back to peer using same masking_key
# (note: not validating continuation frame follows frame without fin set)
#if opcode == 0x0: # CONTINUATION
#if opcode == 0x1: # TEXT
#if opcode == 0x2: # BINARY
#if opcode == 0xA: # PONG (from PING above)
ws_send(wr, ''.join(map(chr, data)).encode(),
opcode=opcode, fin=fin, masking_key=masking_key)
return True
def ws_send(wr, data, opcode=1, fin=True, masking_key=False):
if 0x3 <= opcode <= 0x7 or 0xB <= opcode:
raise ValueError('Invalid opcode')
header = struct.pack('!B', ((bool(fin) << 7) | opcode))
mask_bit = (1 << 7) if masking_key else 0
length = len(data)
if length < 126:
header += struct.pack('!B', (mask_bit|length))
elif length < (1 << 16):
header += struct.pack('!B', (mask_bit|126)) + struct.pack('!H', length)
elif length < (1 << 63):
header += struct.pack('!B', (mask_bit|127)) + struct.pack('!Q', length)
else:
raise ValueError('Data too large')
if mask_bit:
# mask data
header += masking_key
data = bytearray(data)
for i in range(0, length):
data[i] ^= masking_key[i % 4]
wr.write(header + data)
def ws_echo_app(rd, wr, env):
if not ws_upgrade(rd, wr, env):
return
while True:
if not ws_recv_and_echo(rd, wr):
break
import os
import sys
#import errno
#import io
#from traceback import format_exc as format
#import logging
#logger = logging.getLogger('mylogger')
#logger.setLevel(logging.WARNING)
if __name__ == '__main__':
try:
sys.stdin = os.fdopen(sys.stdin.fileno(), 'r+b', buffering=0)
sys.stdout = os.fdopen(sys.stdout.fileno(), 'r+b', buffering=0)
ws_echo_app(sys.stdin, sys.stdout, os.environ)
#except IOError as e:
# if e.errno != errno.EPIPE:
# raise e
except KeyboardInterrupt: # Ctrl-C or SIGINT
try:
sys.exit(130)
except SystemExit:
os._exit(130)
except Exception as e:
err = e.args[0]
#logger.warning("err:{}\n".format(str(err)))
Example: configure lighttpd to run CGI programs under /ws/¶
server.modules += ("mod_cgi")
$HTTP["url"] =^ "/ws/" {
cgi.assign = ("" => "")
cgi.upgrade = 1
#cgi.limits = ( "read-timeout" => 60, "write-timeout" => 60 )
}
#server.max-read-idle := 60
#server.max-write-idle := 60
Example: configure lighttpd to run SCGI programs under /ws/¶
Use scgi-cgi to run ws_echo.py
server.modules += ("mod_scgi")
scgi.server = ("/ws/" =>
((
"socket" => "/tmp/scgi-ws.sock", # should use more secure location
"bin-path" => "/usr/local/bin/scgi-cgi" # modify path to scgi-cgi
"check-local" => "disable",
"min-procs" => 1,
"max-procs" => 1,
"upgrade" => 1
#"read-timeout" => 60,
#"write-timeout" => 60,
))
)
#server.max-read-idle := 60
#server.max-write-idle := 60
Example: configure lighttpd to run FastCGI programs under /ws/¶
Use fcgi-cgi to run ws_echo.py
server.modules += ("mod_fastcgi")
fastcgi.server = ("/ws/" =>
((
"socket" => "/tmp/fcgi-ws.sock", # should use more secure location
"bin-path" => "/usr/local/bin/fcgi-cgi" # modify path to fcgi-cgi
"check-local" => "disable",
"min-procs" => 1,
"max-procs" => 1,
"upgrade" => 1
#"read-timeout" => 60,
#"write-timeout" => 60,
))
)
#server.max-read-idle := 60
#server.max-write-idle := 60
Sample: echo.pl¶
echo.pl - trivial "echo" script which reads and echoes line-by-line
This script knows nothing about websockets and could read/write JSON or anything else. This example reads and echoes line-by-line.
Install files under script location and document root/tmp/echo.pl<docroot>/count.html
echo.pl
#!/usr/bin/perl -Tw
$SIG{PIPE} = 'IGNORE';
for (my $FH; accept($FH, STDIN); close $FH) {
select($FH); $|=1; # $FH->autoflush;
print $FH $_ while (<$FH>);
}
<docroot>/count.html
<!DOCTYPE html>
<!-- modified from example in https://github.com/joewalnes/websocketd README.md -->
<pre id="log"></pre>
<script>
// helper function: log message to screen
var logelt = document.getElementById('log');
function log(msg) { logelt.textContent += msg + '\n'; }
// helper function: send websocket msg with count (1 .. 5)
var ll = 0;
function send_msg() { if (++ll <= 5) { log('SEND: '+ll); ws.send(ll+'\n'); } }
// setup websocket with callbacks
var ws = new WebSocket('ws://'+location.host+'/ws/');
ws.onopen = function() { log('CONNECT\n'); send_msg(); };
ws.onclose = function() { log('DISCONNECT'); };
ws.onmessage = function(event) { log('RECV: ' + event.data); send_msg(); };
</script>
echo-incremental.pl
Here is an alternative sample "echo" script which is unbuffered -- instead of the simpler line-buffered example above -- and will sleep 1 second between reads so that you can visualize in the browser that multiple websocket messages are going back and forth to the server. This sample script will run for 5 seconds and then will close the websocket connection. (To run this example, use this script in place of echo.pl and restart lighttpd running lighttpd-wstunnel.conf)
#!/usr/bin/perl -Tw
$SIG{PIPE} = 'IGNORE';
for (my $FH; accept($FH, STDIN); close $FH) {
foreach (1..5) {
sleep 1;
my $foo = "";
sysread($FH, $foo, 1024);
if (length($foo)) { syswrite($FH, $foo); }
}
close $FH;
}
Example: configure lighttpd to act as WebSocket endpoint for URLs under /ws/¶
This is a simple example showing lighttpd mod_wstunnel terminating a websocket tunnel and sending/receiving payloads to backend "echo" script. For this example, run lighttpd with mod_wstunnel on an alternate port, and this example will be reused below in an example using mod_proxy.
lighttpd-wstunnel.conf (listening on port 8081; start with lighttpd -D -f /dev/shm/lighttpd-wstunnel.conf; Ctrl-C to quit)
server.document-root = "/tmp" # place count.html here (better: use a more secure location)
server.bind = "127.0.0.1" # comment out if accessing from remote machine
server.port = 8081
server.modules += ("mod_wstunnel")
wstunnel.server = (
"/ws/" => (
(
"socket" => "/tmp/echo.sock", # should use more secure location
"bin-path" => "/tmp/echo.pl", # should use more secure location
"max-procs" => 1
)
)
)
Load into local browser as http://localhost:8081/count.html and watch it count 1 .. 5. If you are connecting from a remote system, then modify lighttpd-wstunnel.conf to comment out
server.bind = "127.0.0.1" and replace localhost in the URL to count.html. Replace localhost in the URL if accessing from remote machine.
Example: configure lighttpd to reverse proxy URLs under /ws/¶
This is a simple example showing lighttpd mod_proxy supporting websockets to another lighttpd instance using mod_wstunnel to terminate the websocket tunnel and send/receive payloads to backend "echo" script. It uses the mod_wstunnel example above, so be sure that is running, too.
lighttpd-proxy.conf (listening on port 8080; start with lighttpd -D -f /dev/shm/lighttpd-proxy.conf in another shell; Ctrl-C to quit)
server.document-root = "/tmp" # place count.html here (better: use a more secure location)
server.bind = "127.0.0.1" # comment out if accessing from remote machine
server.port = 8080
server.modules += ("mod_proxy")
proxy.server = ( "/" => (( "host" => "127.0.0.1", "port" => "8081" )))
proxy.header = ( "upgrade" => "enable" )
Load into local browser as http://localhost:8080/count.html and watch it count 1 .. 5. If you are connecting from a remote system, then modify lighttpd-proxy.conf to comment out
server.bind = "127.0.0.1" and replace localhost in the URL to count.html. Replace localhost in the URL if accessing from remote machine.
Please note in this example that the count.html target is now http://localhost:8080/count.html, using port 8080 (changed from 8081 in example above).
Updated by gstrauss 11 months ago · 6 revisions