[Solved] "invalid character in URI" After Moving to Ubuntu 20.04
Added by mattbrown015 over 1 year ago
Hi,
Sorry, about the vague question. A problem has come up with an installation that was done some time ago by someone who has now left the company.
We run lighttpd on our embedded device to provide simple device configuration.
One of our pages uses a textarea control and allows the config file to be edited directly.
This page has been working for ages on our ARM device that runs Ubuntu 18.04:
:~$ lighttpd -v lighttpd/1.4.45 (ssl) - a light and fast webserver Build-Date: Jun 24 2019 22:58:56
We're in the process of porting to an x86_64, Ubuntu 20.04 embedded device and have just discovered the config web page doesn't work,:
:~$ lighttpd -v lighttpd/1.4.55 (ssl) - a light and fast webserver
On Ubuntu 20.04/lighttpd 1.4.55 if the config file contains new lines lighttpd rejects the URI saying it contains invalid characters:
2023-02-17 09:40:01: (response.c.358) invalid character in URI -> 400 /CharmUpdater.cgi?task=saveusrcfg&fname=%2Fvar%2Fcharm%2Fconfigfiles%2Fuser%2Fuserconfig.xml&config=1%0D%0A2
I believe the main lighttpd.conf is the same on both devices. There are differences in conf-available and conf-enabled because the packages have installed difference files.
I've tried with Chrome and Edge.
I've no idea whether this is related to the lighttpd configuration, the OS configuration, architecture or the compiler version.
Any clues or suggestions for debugging greatly appreciated.
Thanks,
Matt
# lighttpd configuration file
#
# use it as a base for lighttpd 1.0.0 and above
#
# $Id: lighttpd.conf,v 1.7 2004/11/03 22:26:05 weigon Exp $
############ Options you really have to take care of ####################
## modules to load
# at least mod_access and mod_accesslog should be loaded
# all other module should only be loaded if really neccesary
# - saves some time
# - saves memory
server.modules = (
# "mod_rewrite",
# "mod_redirect",
# "mod_alias",
"mod_access",
# "mod_cml",
# "mod_trigger_b4_dl",
"mod_auth",
# "mod_status",
# "mod_setenv",
# "mod_fastcgi",
# "mod_proxy",
# "mod_simple_vhost",
# "mod_evhost",
# "mod_userdir",
"mod_cgi",
# "mod_compress",
# "mod_ssi",
# "mod_usertrack",
# "mod_expire",
# "mod_secdownload",
# "mod_rrdtool",
# "mod_webdav",
"mod_accesslog" )
## a static document-root, for virtual-hosting take look at the
## server.virtual-* options
server.document-root = "/var/www"
## where to send error-messages to
server.errorlog = "/tmp/lighttpd.error.log"
# files to check for if .../ is requested
index-file.names = ( "index.html",
"index.htm", "default.htm" )
## set the event-handler (read the performance section in the manual)
# server.event-handler = "freebsd-kqueue" # needed on OS X
# mimetype mapping
mimetype.assign = (
".pdf" => "application/pdf",
".sig" => "application/pgp-signature",
".spl" => "application/futuresplash",
".class" => "application/octet-stream",
".ps" => "application/postscript",
".torrent" => "application/x-bittorrent",
".dvi" => "application/x-dvi",
".gz" => "application/x-gzip",
".pac" => "application/x-ns-proxy-autoconfig",
".swf" => "application/x-shockwave-flash",
".tar.gz" => "application/x-tgz",
".tgz" => "application/x-tgz",
".tar" => "application/x-tar",
".zip" => "application/zip",
".mp3" => "audio/mpeg",
".m3u" => "audio/x-mpegurl",
".wma" => "audio/x-ms-wma",
".wax" => "audio/x-ms-wax",
".ogg" => "application/ogg",
".wav" => "audio/x-wav",
".gif" => "image/gif",
".jpg" => "image/jpeg",
".jpeg" => "image/jpeg",
".png" => "image/png",
".xbm" => "image/x-xbitmap",
".xpm" => "image/x-xpixmap",
".xwd" => "image/x-xwindowdump",
".css" => "text/css",
".html" => "text/html",
".htm" => "text/html",
".js" => "text/javascript",
".asc" => "text/plain",
".c" => "text/plain",
".cpp" => "text/plain",
".log" => "text/plain",
".conf" => "text/plain",
".text" => "text/plain",
".txt" => "text/plain",
".dtd" => "text/xml",
".xml" => "text/xml",
".mpeg" => "video/mpeg",
".mpg" => "video/mpeg",
".mov" => "video/quicktime",
".qt" => "video/quicktime",
".avi" => "video/x-msvideo",
".asf" => "video/x-ms-asf",
".asx" => "video/x-ms-asf",
".wmv" => "video/x-ms-wmv",
".bz2" => "application/x-bzip",
".tbz" => "application/x-bzip-compressed-tar",
".tar.bz2" => "application/x-bzip-compressed-tar"
)
# Use the "Content-Type" extended attribute to obtain mime type if possible
#mimetype.use-xattr = "enable"
## send a different Server: header
## be nice and keep it at lighttpd
# server.tag = "lighttpd"
#### accesslog module
accesslog.filename = "/tmp/access.log"
debug.log-request-handling = "enable"
## deny access the file-extensions
#
# ~ is for backupfiles from vi, emacs, joe, ...
# .inc is often used for code includes which should in general not be part
# of the document-root
url.access-deny = ( "~", ".inc" )
$HTTP["url"] =~ "\.pdf$" {
server.range-requests = "disable"
}
##
# which extensions should not be handle via static-file transfer
#
# .php, .pl, .fcgi are most often handled by mod_fastcgi or mod_cgi
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi", ".cgi", ".sh")
######### Options that are good to be but not neccesary to be changed #######
## bind to port (default: 80)
#server.port = 81
## bind to localhost (default: all interfaces)
#server.bind = "grisu.home.kneschke.de"
## error-handler for status 404
#server.error-handler-404 = "/error-handler.html"
#server.error-handler-404 = "/error-handler.php"
## to help the rc.scripts
#server.pid-file = "/var/run/lighttpd.pid"
###### virtual hosts
##
## If you want name-based virtual hosting add the next three settings and load
## mod_simple_vhost
##
## document-root =
## virtual-server-root + virtual-server-default-host + virtual-server-docroot
## or
## virtual-server-root + http-host + virtual-server-docroot
##
#simple-vhost.server-root = "/home/weigon/wwwroot/servers/"
#simple-vhost.default-host = "grisu.home.kneschke.de"
#simple-vhost.document-root = "/pages/"
##
## Format: <errorfile-prefix><status-code>.html
## -> ..../status-404.html for 'File not found'
#server.errorfile-prefix = "/home/weigon/projects/lighttpd/doc/status-"
## virtual directory listings
#dir-listing.activate = "enable"
## enable debugging
#debug.log-request-header = "enable"
#debug.log-response-header = "enable"
#debug.log-request-handling = "enable"
#debug.log-file-not-found = "enable"
### only root can use these options
#
# chroot() to directory (default: no chroot() )
#server.chroot = "/"
## change uid to <uid> (default: don't care)
#server.username = "wwwrun"
## change uid to <uid> (default: don't care)
#server.groupname = "wwwrun"
#### compress module
#compress.cache-dir = "/tmp/lighttpd/cache/compress/"
#compress.filetype = ("text/plain", "text/html")
#### proxy module
## read proxy.txt for more info
#proxy.server = ( ".php" =>
# ( "localhost" =>
# (
# "host" => "192.168.0.101",
# "port" => 80
# )
# )
# )
#### fastcgi module
## read fastcgi.txt for more info
## for PHP don't forget to set cgi.fix_pathinfo = 1 in the php.ini
#fastcgi.server = ( ".php" =>
# ( "localhost" =>
# (
# "socket" => "/tmp/php-fastcgi.socket",
# "bin-path" => "/usr/local/bin/php"
# )
# )
# )
#### CGI module
cgi.assign = ( ".pl" => "/usr/bin/perl",
".cgi" => "",
".sh" => "/bin/sh" )
#### SSL engine
#ssl.engine = "enable"
#ssl.pemfile = "server.pem"
#### status module
#status.status-url = "/server-status"
#status.config-url = "/server-config"
#### auth module
## read authentication.txt for more info
auth.backend = "plain"
auth.backend.plain.userfile = "/var/www/lighttpd.user"
#auth.backend.plain.groupfile = "lighttpd.group"
#auth.backend.ldap.hostname = "localhost"
#auth.backend.ldap.base-dn = "dc=my-domain,dc=com"
#auth.backend.ldap.filter = "(uid=$)"
auth.require = ( "/" =>
(
"method" => "basic",
"realm" => "CHARM Update Password",
"require" => "user=v4"
)
)
# "/server-config" =>
# (
# "method" => "digest",
# "realm" => "download archiv",
# "require" => "valid-user"
# )
# )
#### url handling modules (rewrite, redirect, access)
#url.rewrite = ( "^/$" => "/server-status" )
#url.redirect = ( "^/wishlist/(.+)" => "http://www.123.org/$1" )
#### both rewrite/redirect support back reference to regex conditional using %n
#$HTTP["host"] =~ "^www\.(.*)" {
# url.redirect = ( "^/(.*)" => "http://%1/$1" )
#}
#
# define a pattern for the host url finding
# %% => % sign
# %0 => domain name + tld
# %1 => tld
# %2 => domain name without tld
# %3 => subdomain 1 name
# %4 => subdomain 2 name
#
#evhost.path-pattern = "/home/storage/dev/www/%3/htdocs/"
#### expire module
#expire.url = ( "/buggy/" => "access 2 hours", "/asdhas/" => "access plus 1 seconds 2 minutes")
#### ssi
#ssi.extension = ( ".shtml" )
#### rrdtool
#rrdtool.binary = "/usr/bin/rrdtool"
#rrdtool.db-name = "/var/www/lighttpd.rrd"
#### setenv
#setenv.add-request-header = ( "TRAV_ENV" => "mysql://user@host/db" )
#setenv.add-response-header = ( "X-Secret-Message" => "42" )
## for mod_trigger_b4_dl
# trigger-before-download.gdbm-filename = "/home/weigon/testbase/trigger.db"
# trigger-before-download.memcache-hosts = ( "127.0.0.1:11211" )
# trigger-before-download.trigger-url = "^/trigger/"
# trigger-before-download.download-url = "^/download/"
# trigger-before-download.deny-url = "http://127.0.0.1/index.html"
# trigger-before-download.trigger-timeout = 10
## for mod_cml
## don't forget to add index.cml to server.indexfiles
# cml.extension = ".cml"
# cml.memcache-hosts = ( "127.0.0.1:11211" )
#### variable usage:
## variable name without "." is auto prefixed by "var." and becomes "var.bar"
#bar = 1
#var.mystring = "foo"
## integer add
#bar += 1
## string concat, with integer cast as string, result: "www.foo1.com"
#server.name = "www." + mystring + var.bar + ".com"
## array merge
#index-file.names = (foo + ".php") + index-file.names
#index-file.names += (foo + ".php")
#### include
#include /etc/lighttpd/lighttpd-inc.conf
## same as above if you run: "lighttpd -f /etc/lighttpd/lighttpd.conf"
#include "lighttpd-inc.conf"
#### include_shell
#include_shell "echo var.a=1"
## the above is same as:
#var.a=1
Replies (3)
RE: "invalid character in URI" After Moving to Ubuntu 20.04 - Added by gstrauss over 1 year ago
URL normalization, including rejecting bad characters, defaults to be enabled since lighttpd 1.4.54. See commit: 1cf68f79
You might consider using POST to submit the config in the request body rather than a GET with the modified config data in the query string.
RE: "invalid character in URI" After Moving to Ubuntu 20.04 - Added by mattbrown015 over 1 year ago
Excellent, thanks for the quick response. I only wish I'd asked this question several days ago! :-)
I've added the following to the config and I'm back on the road again:
server.http-parseopts = (
"url-normalize" => "disable"
)
You might consider using POST to submit the config in the request body rather than a GET with the modified config data in the query string.
Yes, I've been reading/learning about GET and POST and was just about to start investigating POST.
We're always talking about what improvements we'd like to make to the web interface but, unfortunately, it never seems to come to the top of list.
Thanks,
Matt
RE: "invalid character in URI" After Moving to Ubuntu 20.04 - Added by gstrauss over 1 year ago
FYI: The most recent lighttpd stable release is lighttpd 1.4.69. Please consider using the latest stable lighttpd release.