WebSockets » History » Revision 5
Revision 4 (gstrauss, 2023-12-14 08:48) → Revision 5/6 (gstrauss, 2023-12-14 08:56)
h1. WebSockets
{{>toc}}
*WebSocket echo samples*
h2. Overview
What are websockets? "The WebSocket API (WebSockets)":https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API
lighttpd can facilitate WebSocket connections in numerous ways, including via [[mod_proxy]], [[mod_cgi]], [[mod_scgi]], [[mod_fastcgi]], [[mod_wstunnel]].
* lighttpd supports HTTP/1.1 `Upgrade: websocket` from the HTTP/1.1 connection.
* lighttpd supports HTTP/2 extended CONNECT with `:protocol: websocket` on HTTP/2 streams over a single HTTP/2 connection.
For security reasons and supporting the principle of least surprise, lighttpd support for websockets is intentionally disabled by default. lighttpd dynamic modules may be configured to enable connection upgrade to the websocket protocol.
Since lighttpd 1.4.46, lighttpd [[mod_cgi]], [[mod_proxy]], and [[mod_wstunnel]] support websockets. Support in additional dynamic modules [[mod_scgi]] and [[mod_fastcgi]] has been added in lighttpd 1.4.74.
This document demonstrates multiple ways to configure lighttpd modules to run WebSocket "echo" sample applications.
h2. Basic Configuration
h4. websockets via mod_wstunnel
Configure [[mod_wstunnel]]
[[mod_wstunnel]] is a WebSocket tunnel endpoint, terminating the websocket tunnel from a client. [[mod_wstunnel]] decodes websocket frames and then passes data (without websocket frames) to a backend, and in the opposite direction encodes responses from backend into websocket frames before sending responses to client.
h4. websockets via mod_proxy
Configure [[mod_proxy]] and enable @proxy.header += ("upgrade" => "enable")@
With lighttpd 1.4.74 and later, @"upgrade" => "enable"@ is also a host option in @proxy.server@
[[mod_proxy]] can act as a reverse proxy for websocket connections when upgrade is enabled.
h4. websockets via mod_cgi
Configure [[mod_cgi]] and enable @cgi.upgrade = "enable"@
[[mod_cgi]] allows the target program to upgrade to use the websocket protocol.
h4. websockets via mod_scgi
Configure [[mod_scgi]] and add @"upgrade" => "enable"@ to options for each host in @scgi.server@ (lighttpd 1.4.74 and later)
[[mod_scgi]] allows the target program to upgrade to use the websocket protocol.
h4. websockets via mod_fastcgi
Configure [[mod_fastcgi]] and add @"upgrade" => "enable"@ to options for each host in @fastcgi.server@ (lighttpd 1.4.74 and later)
[[mod_fastcgi]] allows the target program to upgrade to use the websocket protocol.
While lighttpd does not currently multiplex multiple requests on a single FastCGI connection, the FastCGI protocol allows such. lighttpd [[mod_fastcgi]] sends and receives FastCGI packets from the backend, even after a request is upgraded to use the websocket protocol. websocket frames are again framed into FastCGI packets. As such, this may be less efficient than using other lighttpd modules which support upgrading to the websocket protocol.
h2. Sample: ws_echo.py
ws_echo.py - simple "echo" websocket server CGI sample program
Install files under document root and ws/ subdirectory
@<docroot>/ws_echo.html@
@<docroot>/ws/ws_echo.py@
Modify the URL in ws_echo.html to use wss:// instead of ws:// if using https.
@var host = "ws://" + window.location.hostname + "/ws/ws_echo.py";@
@<docroot>/ws_echo.html@
<pre>
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<!-- favicon: none -->
<link rel="icon" href="data:,">
<!--
<meta
http-equiv="Content-Security-Policy"
content="default-src 'self' 'unsafe-inline'" />
-->
<title>websocket echo client</title>
<style>
#output {
border: solid 1px #000;
}
</style>
</head>
<body>
<form id="form" accept-charset=utf-8>
<input type="text" id="message">
<button type="submit">Send</button>
</form>
<hr>
<div id="output"></div>
<script>
var inputBox = document.getElementById("message");
var output = document.getElementById("output");
var form = document.getElementById("form");
try {
var host = "ws://" + window.location.hostname + "/ws/ws_echo.py";
//var host = "wss://" + window.location.hostname + "/ws/ws_echo.py";
//var host = "ws://" + window.location.hostname + ":8080/ws/ws_echo.py";
//var host = "wss://" + window.location.hostname + ":8443/ws/ws_echo.py";
console.log("Host:", host);
var s = new WebSocket(host);
s.onopen = function (e) {
console.log("Socket opened.");
};
s.onclose = function (e) {
console.log("Socket closed.");
};
s.onmessage = function (e) {
console.log("Socket message:", e.data);
var p = document.createElement("p");
p.innerHTML = e.data;
output.appendChild(p);
};
s.onerror = function (e) {
console.log("Socket error:", e);
};
} catch (ex) {
console.log("Socket exception:", ex);
}
form.addEventListener("submit", function (e) {
e.preventDefault();
s.send(inputBox.value);
inputBox.value = "";
}, false)
</script>
</body>
</html>
</pre>
@<docroot>/ws/ws_echo.py@
<pre>
#!/usr/bin/env python3
#
# ws_echo.py - websocket echo; reflect back to peer the frames sent by peer
#
# Copyright(c) 2023 Glenn Strauss gstrauss()gluelogic.com All rights reserved
# License: BSD 3-clause (same as lighttpd)
import struct
import hashlib
import base64
def ws_upgrade(rd, wr, env):
if env.get('HTTP_UPGRADE', '') != 'websocket':
wr.write("Status: 204\n\n".encode())
return False
# (throws KeyError exception if any of these env vars are not set)
origin = env['HTTP_ORIGIN']
scheme = "wss" if env['REQUEST_SCHEME'] == "https" else "ws"
host = env['HTTP_HOST']
urlpath= env['SCRIPT_NAME']
key = env['HTTP_SEC_WEBSOCKET_KEY'].encode('utf-8')
GUID = b"258EAFA5-E914-47DA-95CA-C5AB0DC85B11"
digest = str(base64.b64encode(hashlib.sha1(key + GUID).digest()), 'utf-8')
wr.write(("Status: 101\n"
+ "Upgrade: WebSocket\n"
+ "Connection: Upgrade\n"
+ f"Sec-WebSocket-Origin: {origin}\n"
+ f"Sec-WebSocket-Location: {scheme}://{host}{urlpath}\n"
+ f"Sec-WebSocket-Accept: {digest}\n\n"
).encode())
return True
## +-+-+-+-+-------++-+-------------+-------------------------------+
## |F|R|R|R| opcode||M| Payload len | Extended payload length |
## |I|S|S|S| (4) ||A| (7) | (16/63) |
## |N|V|V|V| ||S| | (if payload len==126/127) |
## | |1|2|3| ||K| | |
## +-+-+-+-+-------++-+-------------+ - - - - - - - - - - - - - - - +
## +-+-+-+-+--------------------------------------------------------+
## | Extended payload length continued, if payload len == 127 |
## + - - - - - - - - - - - - - - - +--------------------------------+
## + - - - - - - - - - - - - - - - +-------------------------------+
## | |Masking-key, if MASK set to 1 |
## +-------------------------------+-------------------------------+
## | Masking-key (continued) | Payload Data |
## +-------------------------------- - - - - - - - - - - - - - - - +
## : Payload Data continued ... :
## + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
## | Payload Data continued ... |
## +---------------------------------------------------------------+
def ws_recv_and_echo(rd, wr):
# (note: assumes well-formed websocket frame; not validating or paranoid)
# (note: inefficient small reads rather than read/buffering larger blocks)
# (note: no buffering; program exits if partial read of frame header)
# (note: no buffering; program loses state if partial read of frame data
# and subsequently will likely thrown an exception)
# (tl;dr: this toy program is sufficient for a demo of frames with small
# data payloads, but probably nothing more)
data = rd.read(2)
if len(data) == 0: # EOF
ws_send(wr, b'1000', opcode=0x8) # CLOSE "1000 = Normal Closure"
return False
head1, head2 = struct.unpack('!BB', data)
fin = bool(head1 & 0b10000000)
opcode = head1 & 0b00001111
if opcode == 0x8: # CLOSE (0x8)
# (note: not reading and parsing close reason sent by peer)
ws_send(wr, b'1000', opcode=opcode) # "1000 = Normal Closure"
return False
length = head2 & 0b01111111
if length == 126:
data = rd.read(2)
length, = struct.unpack('!H', data)
elif length == 127:
data = rd.read(8)
length, = struct.unpack('!Q', data)
mask_bit = bool(head2 & 0b10000000)
masking_key = rd.read(4) if mask_bit else False
data = bytearray(rd.read(length)) if length else b''
if opcode == 0xA: # PONG
# receive and ignore response to PING
return True
if opcode == 0x9: # PING
# reflect PING back to peer as PONG
opcode = 0xA # PONG
if mask_bit:
# unmask data
for i in range(0, len(data)):
data[i] ^= masking_key[i % 4]
# echo data back to peer using same masking_key
# (note: not validating continuation frame follows frame without fin set)
#if opcode == 0x0: # CONTINUATION
#if opcode == 0x1: # TEXT
#if opcode == 0x2: # BINARY
#if opcode == 0xA: # PONG (from PING above)
ws_send(wr, ''.join(map(chr, data)).encode(),
opcode=opcode, fin=fin, masking_key=masking_key)
return True
def ws_send(wr, data, opcode=1, fin=True, masking_key=False):
if 0x3 <= opcode <= 0x7 or 0xB <= opcode:
raise ValueError('Invalid opcode')
header = struct.pack('!B', ((bool(fin) << 7) | opcode))
mask_bit = (1 << 7) if masking_key else 0
length = len(data)
if length < 126:
header += struct.pack('!B', (mask_bit|length))
elif length < (1 << 16):
header += struct.pack('!B', (mask_bit|126)) + struct.pack('!H', length)
elif length < (1 << 63):
header += struct.pack('!B', (mask_bit|127)) + struct.pack('!Q', length)
else:
raise ValueError('Data too large')
if mask_bit:
# mask data
header += masking_key
data = bytearray(data)
for i in range(0, length):
data[i] ^= masking_key[i % 4]
wr.write(header + data)
def ws_echo_app(rd, wr, env):
if not ws_upgrade(rd, wr, env):
return
while True:
if not ws_recv_and_echo(rd, wr):
break
import os
import sys
#import errno
#import io
#from traceback import format_exc as format
#import logging
#logger = logging.getLogger('mylogger')
#logger.setLevel(logging.WARNING)
if __name__ == '__main__':
try:
sys.stdin = os.fdopen(sys.stdin.fileno(), 'r+b', buffering=0)
sys.stdout = os.fdopen(sys.stdout.fileno(), 'r+b', buffering=0)
ws_echo_app(sys.stdin, sys.stdout, os.environ)
#except IOError as e:
# if e.errno != errno.EPIPE:
# raise e
except KeyboardInterrupt: # Ctrl-C or SIGINT
try:
sys.exit(130)
except SystemExit:
os._exit(130)
except Exception as e:
err = e.args[0]
#logger.warning("err:{}\n".format(str(err)))
</pre>
h5. Example: configure lighttpd to run CGI programs under /ws/
<pre>
server.modules += ("mod_cgi")
$HTTP["url"] =^ "/ws/" {
cgi.assign = ("" => "")
cgi.upgrade = 1
#cgi.limits = ( "read-timeout" => 60, "write-timeout" => 60 )
}
#server.max-read-idle := 60
#server.max-write-idle := 60
</pre>
h5. Example: configure lighttpd to run SCGI programs under /ws/
Use "scgi-cgi":https://git.lighttpd.net/lighttpd/scgi-cgi to run `ws_echo.py`
<pre>
server.modules += ("mod_scgi")
scgi.server = ("/ws/" =>
((
"socket" => "/tmp/scgi-ws.sock", # should use more secure location
"bin-path" => "/usr/local/bin/scgi-cgi" # modify path to scgi-cgi
"check-local" => "disable",
"min-procs" => 1,
"max-procs" => 1,
"upgrade" => 1
#"read-timeout" => 60,
#"write-timeout" => 60,
))
)
#server.max-read-idle := 60
#server.max-write-idle := 60
</pre>
h5. Example: configure lighttpd to run FastCGI programs under /ws/
Use "fcgi-cgi":https://git.lighttpd.net/lighttpd/fcgi-cgi to run `ws_echo.py`
<pre>
server.modules += ("mod_fastcgi")
fastcgi.server = ("/ws/" =>
((
"socket" => "/tmp/fcgi-ws.sock", # should use more secure location
"bin-path" => "/usr/local/bin/fcgi-cgi" # modify path to fcgi-cgi
"check-local" => "disable",
"min-procs" => 1,
"max-procs" => 1,
"upgrade" => 1
#"read-timeout" => 60,
#"write-timeout" => 60,
))
)
#server.max-read-idle := 60
#server.max-write-idle := 60
</pre>
h2. Sample: echo.pl
echo.pl - trivial "echo" script which reads and echoes line-by-line
This script knows nothing about websockets and could read/write JSON or anything else. This example reads and echoes line-by-line.
Install files under script location and document root
@/tmp/echo.pl@
@<docroot>/count.html@
@echo.pl@
<pre>
#!/usr/bin/perl -Tw
$SIG{PIPE} = 'IGNORE';
for (my $FH; accept($FH, STDIN); close $FH) {
select($FH); $|=1; # $FH->autoflush;
print $FH $_ while (<$FH>);
}
</pre>
@<docroot>/count.html@
<pre>
<!DOCTYPE html>
<!-- modified from example in https://github.com/joewalnes/websocketd README.md -->
<pre id="log"></pre>
<script>
// helper function: log message to screen
var logelt = document.getElementById('log');
function log(msg) { logelt.textContent += msg + '\n'; }
// helper function: send websocket msg with count (1 .. 5)
var ll = 0;
function send_msg() { if (++ll <= 5) { log('SEND: '+ll); ws.send(ll+'\n'); } }
// setup websocket with callbacks
var ws = new WebSocket('ws://'+location.host+'/ws/');
ws.onopen = function() { log('CONNECT\n'); send_msg(); };
ws.onclose = function() { log('DISCONNECT'); };
ws.onmessage = function(event) { log('RECV: ' + event.data); send_msg(); };
</script>
</pre>
@echo-incremental.pl@
Here is an alternative sample "echo" script which is unbuffered -- instead of the simpler line-buffered example above -- and will sleep 1 second between reads so that you can visualize in the browser that multiple websocket messages are going back and forth to the server. This sample script will run for 5 seconds and then will close the websocket connection. (To run this example, use this script in place of @echo.pl@ and restart lighttpd running lighttpd-wstunnel.conf)
<pre>
#!/usr/bin/perl -Tw
$SIG{PIPE} = 'IGNORE';
for (my $FH; accept($FH, STDIN); close $FH) {
foreach (1..5) {
sleep 1;
my $foo = "";
sysread($FH, $foo, 1024);
if (length($foo)) { syswrite($FH, $foo); }
}
close $FH;
}
</pre>
h5. Example: configure lighttpd to act as WebSocket endpoint for URLs under /ws/
This is a simple example showing lighttpd [[mod_wstunnel]] terminating a websocket tunnel and sending/receiving payloads to backend "echo" script. For this example, run lighttpd with [[mod_wstunnel]] on an alternate port, and this example will be reused below in an example using [[mod_proxy]].
@lighttpd-wstunnel.conf@ (listening on port 8081; start with @lighttpd -D -f /dev/shm/lighttpd-wstunnel.conf@; Ctrl-C to quit)
<pre>
server.document-root = "/tmp" # place count.html here (better: use a more secure location)
server.bind = "127.0.0.1" # comment out if accessing from remote machine
server.port = 8081
server.modules += ("mod_wstunnel")
wstunnel.server = (
"/ws/" => (
(
"socket" => "/tmp/echo.sock", # should use more secure location
"bin-path" => "/tmp/echo.pl", # should use more secure location
"max-procs" => 1
)
)
)
</pre>
Load into local browser as http://localhost:8081/count.html and watch it count 1 .. 5. If you are connecting from a remote system, then modify lighttpd-wstunnel.conf to comment out @server.bind = "127.0.0.1"@ and replace @localhost@ in the URL to count.html. Replace @localhost@ in the URL if accessing from remote machine.
h5. Example: configure lighttpd to reverse proxy URLs under /ws/
This is a simple example showing lighttpd [[mod_proxy]] supporting websockets to another lighttpd instance using [[mod_wstunnel]] to terminate the websocket tunnel and send/receive payloads to backend "echo" script. It uses the [[mod_wstunnel]] example above, so be sure that is running, too.
@lighttpd-proxy.conf@ (listening on port 8080; start with @lighttpd -D -f /dev/shm/lighttpd-proxy.conf@ in another shell; Ctrl-C to quit)
<pre>
server.document-root = "/tmp" # place count.html here (better: use a more secure location)
server.bind = "127.0.0.1" # comment out if accessing from remote machine
server.port = 8080
server.modules += ("mod_proxy")
proxy.server = ( "/" => (( "host" => "127.0.0.1", "port" => "8081" )))
proxy.header = ( "upgrade" => "enable" )
</pre>
Load into local browser as http://localhost:8080/count.html and watch it count 1 .. 5. If you are connecting from a remote system, then modify lighttpd-proxy.conf to comment out @server.bind = "127.0.0.1"@ and replace @localhost@ in the URL to count.html. Replace @localhost@ in the URL if accessing from remote machine.
Please note in this example that the count.html target is now http://localhost:8080/count.html, using port 8080 (changed from 8081 in example above).