Project

General

Profile

[Solved] Need help to fix SSL_new error: null ssl ctx

Added by Snufkin about 3 years ago

My home web server has been successfully running for over 2 years. It's exposed on the Internet via IPv4/IPv6 dual stack. Data exchange with server is protected by Let's Encrypt certificate.

Server configuration file did not require changes until lighttpd version 1.4.56. Starting with this version, I had to face two errors.
  1. Error building certificate chain
    lighttpd: (mod_openssl.c.1084) SSL: building cert chain for TLS server name cloud.mydomain.tld: error:00000000:lib(0):func(0):reason(0)
    Entering server certificate in .pem file and intermediate certificates via ssl.ca-file option results in an error. The reason is well explained in the chapter Chain Certificates on the lighttpd Wiki page.
    To fix this error I've moved the entire certificate chain in .pem file.
  2. Empty SSL context
    All attempts to connect to the server using http requests result in an empty server response.
    curl -I http://mydomain.tld
    curl: (52) Empty reply from server
    
    curl -I http://www.mydomain.tld
    curl: (52) Empty reply from server
    
    curl -I http://cloud.mydomain.tld
    curl: (52) Empty reply from server
    Each attempt matches the following syslog entry
    lighttpd: (mod_openssl.c.3111) SSL: error:140BA0C3:SSL routines:SSL_new:null ssl ctx
    I would like to ask for help in trying to understand the causes of this error and fix it.

Latest working version of the system without above two errors.

FreeBSD 11.4, OpenSSL 1.0.2u-freebsd, lighttpd/1.4.55 (ssl)
System versions with errors.
FreeBSD 11.4, OpenSSL 1.0.2u-freebsd, lighttpd/1.4.56 (ssl)
...
FreeBSD 11.4, OpenSSL 1.0.2u-freebsd, lighttpd/1.4.59 (ssl)

FreeBSD 12.2, OpenSSL 1.1.1h-freebsd, lighttpd/1.4.58 (ssl)
Here is actual lighttpd config file. Please note that the user-accessible "Additional parameters" are at the end of the file. The rest of options are set by the software (XigmaNAS project) on which web server is running.
# created by websrv
server.port = 443
server.username = "www" 
server.groupname = "www" 
server.document-root = "/mnt/dsk/services/webserver" 
server.modules = (
  "mod_access",
  "mod_auth",
  "mod_expire",
  "mod_cgi",
  "mod_fastcgi",
  "mod_openssl",
  "mod_setenv",
  "mod_rewrite",
  "mod_redirect",
  "mod_alias" 
)
server.errorlog-use-syslog = "enable" 
# debugmode debug.log-request-handling enable/disable
debug.log-request-handling = "disable" 
server.event-handler = "freebsd-kqueue" 
server.max-write-idle = 360
index-file.names   = ( "index.php", "index.html", "index.htm", "index.shtml", "default.htm" )

# set trust path
setenv.add-environment = ( "PATH" => "/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin" )
#$HTTP["url"] =~ "\.(js|css|png|gif|jpg)$" {
#    expire.url = ( "" => "access plus 1 hours" )
#}

# mimetype mapping
mimetype.assign = (
  ".pdf"    =>    "application/pdf",
  ".sig"    =>    "application/pgp-signature",
  ".spl"    =>    "application/futuresplash",
  ".class"    =>    "application/octet-stream",
  ".ps"    =>    "application/postscript",
  ".torrent"    =>    "application/x-bittorrent",
  ".dvi"    =>    "application/x-dvi",
  ".gz"        =>    "application/x-gzip",
  ".pac"    =>    "application/x-ns-proxy-autoconfig",
  ".swf"    =>    "application/x-shockwave-flash",
  ".tar.gz"    =>    "application/x-tgz",
  ".tgz"    =>    "application/x-tgz",
  ".tar"    =>    "application/x-tar",
  ".zip"    =>    "application/zip",
  ".mp3"    =>    "audio/mpeg",
  ".m3u"    =>    "audio/x-mpegurl",
  ".wma"    =>    "audio/x-ms-wma",
  ".wax"    =>    "audio/x-ms-wax",
  ".ogg"    =>    "application/ogg",
  ".wav"    =>    "audio/x-wav",
  ".gif"    =>    "image/gif",
  ".jar"    =>    "application/x-java-archive",
  ".jpg"    =>    "image/jpeg",
  ".jpeg"    =>    "image/jpeg",
  ".png"    =>    "image/png",
  ".svg"    =>    "image/svg+xml",
  ".xbm"    =>    "image/x-xbitmap",
  ".xpm"    =>    "image/x-xpixmap",
  ".xwd"    =>    "image/x-xwindowdump",
  ".css"    =>    "text/css",
  ".html"    =>    "text/html",
  ".htm"    =>    "text/html",
  ".js"        =>    "text/javascript",
  ".asc"    =>    "text/plain",
  ".c"        =>    "text/plain",
  ".cpp"    =>    "text/plain",
  ".log"    =>    "text/plain",
  ".conf"    =>    "text/plain",
  ".text"    =>    "text/plain",
  ".txt"    =>    "text/plain",
  ".spec"    =>    "text/plain",
  ".dtd"    =>    "text/xml",
  ".xml"    =>    "text/xml",
  ".mp4"    =>    "video/mp4",
  ".mpg4"    =>    "video/mp4",
  ".mpeg"    =>    "video/mpeg",
  ".mpg"    =>    "video/mpeg",
  ".mov"    =>    "video/quicktime",
  ".qt"        =>    "video/quicktime",
  ".avi"    =>    "video/x-msvideo",
  ".asf"    =>    "video/x-ms-asf",
  ".asx"    =>    "video/x-ms-asf",
  ".wmv"    =>    "video/x-ms-wmv",
  ".xz"        =>    "application/x-xz",
  ".bz2"    =>    "application/x-bzip",
  ".tbz"    =>    "application/x-bzip-compressed-tar",
  ".tar.bz2"    =>    "application/x-bzip-compressed-tar",
  ".rpm"    =>    "application/x-rpm",
  # make the default mime type application/octet-stream.
  ""        =>    "application/octet-stream",
 )
url.access-deny = ( "~", ".inc", ".websrv_htpasswd" )
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )
server.pid-file = "/var/run/websrv.pid" 
# cgi.assign = (".php" => "/usr/local/bin/php-cgi-websrv")
server.tag = "webserv" 

# FastCGI
fastcgi.server = ( ".php" =>
  (( "socket" => "/var/tmp/php-wsrv.socket",
        "bin-path" => "/usr/local/bin/php-cgi",
        "bin-environment" => (
            "PHP_FCGI_CHILDREN" => "5",
            "PHP_FCGI_MAX_REQUESTS" => "1000" 
        ),
        "bin-copy-environment" => (
            "PATH", "SHELL", "USER" ),
        "broken-scriptfilename" => "enable",
        "min-procs" => 1,
        "max-procs" => 2,
        "max-load-per-proc" => 90,
        "idle-timeout" => 360,
        "x-sendfile" => "enable" 
    )
  )
)
server.upload-dirs = ( "/mnt/dsl/tmp" )
server.max-request-size = 16777216
#
## Start of "Additional parameters" 
#
## Activate access log
# server.modules += ( "mod_accesslog" )
# accesslog.filename = "/mnt/dsk/services/logs/webserv.log" 
#
## Debugging
# debug.log-request-header-on-error = "enable" 
#
## Hardening lighttpd
setenv.set-response-header = (
"Referrer-Policy"        => "no-referrer",
"X-Content-Type-Options" => "nosniff",
"X-Frame-Options"        => "sameorigin",
"X-Robots-Tag"           => "none",
"X-XSS-Protection"       => "1; mode=block" 
)
## Limit total throughput (80 Mbit/s)
server.kbytes-per-second = 10240
#
## Increase request header size (default 8192 bytes)
server.max-request-field-size = 16384
#
## Path prefix to a list of static error files
server.errorfile-prefix = "/mnt/dsk/services/webserver/landpage/" 
#
## Check for Nextcloud subdomains
$HTTP["host"] =~ "^cloud\.mydomain\.tld$" {
    server.document-root = "/mnt/dsk/services/webserver/nextcloud/" 
    url.redirect-code = 307
    url.redirect = ( "^/\.well-known/caldav"  => "/remote.php/dav","^/\.well-known/carddav" => "/remote.php/dav" )
    $HTTP["scheme"] == "http" {
        url.redirect-code = 307
        url.redirect = ( "" => "https://${url.authority}${url.path}${qsa}" )
    }
## Check for Landpage domains and subdomains
} else $HTTP["host"] =~ "^(|www\.)mydomain\.tld$" {
    server.document-root = "/mnt/dsk/services/webserver/landpage/" 
    $HTTP["host"] =~ "^www\.(.*)$" {
        url.redirect-code = 307
        url.redirect = (  "" => "${url.scheme}://%1${url.path}${qsa}" )
    }
## Check for any other subdomains not specified explicitly
} else $HTTP["host"] =~ "" {
    server.document-root = "/mnt/dsk/services/webserver/dummy/" 
}
## Let's encrypt challenge
alias.url += ( "/.well-known/acme-challenge/" => "/mnt/dsk/services/webserver/dummy/.well-known/acme-challenge/" )
#
## Prevent MITM attack
$HTTP["scheme"] == "https" {
    setenv.add-response-header = ( "Strict-Transport-Security" => "max-age=63072000;includeSubdomains;" )
}
## Open HTTP port 80 to serve ACME challenge (and just for fun)
$SERVER["socket"] == "0.0.0.0:80" { }
#
## Listen to IPv6
$SERVER["socket"] == "[::]:80" {  }
#
## Secure HTTP configuration
# https://redmine.lighttpd.net/projects/1/wiki/docs_ssl#Configuration
$SERVER["socket"] == "[::]:443" { ssl.engine = "enable" }
#
## Mozilla SSL Configuration Generator
# https://mozilla.github.io/server-side-tls/ssl-config-generator/
#
## Set of Diffie-Hellman domain parameters
# curl https://ssl-config.mozilla.org/ffdhe2048.txt > /path/to/dhparam
ssl.dh-file = "/mnt/dsk/services/lighttpd/ffdhe2048.pem" 
#
## Intermediate configuration, lighttpd 1.4.55, OpenSSL 1.0.2u
ssl.openssl.ssl-conf-cmd = (
"Protocol" => "ALL, -SSLv2, -SSLv3, -TLSv1, -TLSv1.1",
"Options" => "-ServerPreference",
"CipherString" => "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384" 
)
#
## DEPRECATED!
#
## CA file for chained certificates support
# ssl.ca-file -> Prefer complete certificate chains in ssl.pemfile = "/var/etc/websrvcert.pem" 
#
## Specify openssl config commands in ssl.openssl.ssl-conf-cmd
# ssl.honor-cipher-order = "disable" -> Prefer "Options" => "-ServerPreference" 
# ssl.cipher-list = "..."-> Prefer "CipherString" => "..." 
#
## End of "Additional parameters" 
#
ssl.engine = "enable" 
ssl.pemfile = "/var/etc/websrvcert.pem"


Replies (6)

RE: Need help to fix SSL_new error: null ssl ctx - Added by gstrauss about 3 years ago

I would like to ask for help in trying to understand the causes of this error and fix it.

Some recent history: in lighttpd 1.4.56, mod_openssl migrated to use the cert callback API provided in openssl 1.0.2 and later. See #2842 and #3009 Given the issues you are having with lighttpd 1.4.56 and later, these changes are probably related.

The recommended solution is as you have done, which is to put the certificate chain in the pemfile. Still, it is curious that the code I wrote for backwards compatibility to build the chain is failing. Would you mind sharing the original .pem file and ca-file that you were using? You can attach them by clicking the "Browse" button under "Files" at the bottom of this page. (Please do not include the private key.)

As for the second error, SSL_new:null ssl ctx, you have ssl.engine = "enable" enabled in the global scope, and so it is inherited by the $SERVER["socket"]. Try adding this:

$SERVER["socket"] == "0.0.0.0:80" { ssl.engine = "disable" }
$SERVER["socket"] == "[::]:80"    { ssl.engine = "disable" }

RE: Need help to fix SSL_new error: null ssl ctx - Added by Snufkin about 3 years ago

Would you mind sharing the original .pem file and ca-file that you were using?

Original three files are attached as they are received from Let's Encrypt CA by asme.sh script.

XigmaNAS stores key and certificate web form fields in its config file.
After starting XigmaNAS, but before starting lighttpd, the key and certificate are extracted from the config file and written as a pem file (attached).

As for the second error, SSL_new:null ssl ctx, you have ssl.engine = "enable" enabled in the global scope, and so it is inherited by the $SERVER["socket"]. Try adding this:

$SERVER["socket"] == "0.0.0.0:80" { ssl.engine = "disable" }
$SERVER["socket"] == "[::]:80"    { ssl.engine = "disable" }

The second error is gone.

curl -I http://mydomain.tld
HTTP/1.1 200 OK
Content-Type: text/html
Accept-Ranges: bytes
ETag: "2642143151" 
Last-Modified: Thu, 14 Feb 2019 16:54:16 GMT
Content-Length: 2231
Referrer-Policy: no-referrer
X-Content-Type-Options: nosniff
X-Frame-Options: sameorigin
X-Robots-Tag: none
X-XSS-Protection: 1; mode=block
Date: Sun, 14 Feb 2021 18:33:15 GMT
Server: webserv

curl -I http://www.mydomain.tld
HTTP/1.1 307 Temporary Redirect
Location: http://mydomain.tld/
Referrer-Policy: no-referrer
X-Content-Type-Options: nosniff
X-Frame-Options: sameorigin
X-Robots-Tag: none
X-XSS-Protection: 1; mode=block
Date: Sun, 14 Feb 2021 18:33:31 GMT
Server: webserv

curl -I http://cloud.mydomain.tld
HTTP/1.1 307 Temporary Redirect
Location: https://cloud.mydomain.tld/
Referrer-Policy: no-referrer
X-Content-Type-Options: nosniff
X-Frame-Options: sameorigin
X-Robots-Tag: none
X-XSS-Protection: 1; mode=block
Date: Sun, 14 Feb 2021 18:33:40 GMT
Server: webserv

Thank you very much for your prompt reply!

RE: Need help to fix SSL_new error: null ssl ctx - Added by Snufkin about 3 years ago

Still, it is curious that the code I wrote for backwards compatibility to build the chain is failing.

If I am not mistaken, root cause of the error could be that the XigmaNAS pem file contains Windows <CR><LF> and the ca.cer file uses Unix <LF>.

Now I remember that I had to add a replace function to the deployment script in order to compare the certificates in the XigmaNAS config and those received by acme.sh script.

Hope this information can help you.

RE: Need help to fix SSL_new error: null ssl ctx - Added by gstrauss about 3 years ago

I tried different combinations of stray char at end of file, and differences in line endings ("\n" and "\r\n"), but lighttpd 1.4.59 with openssl 1.1.1i handles them just fine. lighttpd uses openssl routines to read certificate files (with mod_openssl), so I don't think this is an issue with lighttpd, but if lighttpd can detect the issue, maybe lighttpd can issue better error trace.

Perhaps the issue you are having is related to Let's Encrypt changing its root CA certificate? See similar recent issue in OpnSense: https://forum.opnsense.org/index.php?topic=20325.0

https://letsencrypt.org/2020/09/17/new-root-and-intermediates.html
https://letsencrypt.org/2020/11/06/own-two-feet.html
https://letsencrypt.org/2020/12/21/extending-android-compatibility.html

RE: Need help to fix SSL_new error: null ssl ctx - Added by Snufkin about 3 years ago

I tried different combinations of stray char at end of file, and differences in line endings ("\n" and "\r\n"), but lighttpd 1.4.59 with openssl 1.1.1i handles them just fine. lighttpd uses openssl routines to read certificate files (with mod_openssl), so I don't think this is an issue with lighttpd, but if lighttpd can detect the issue, maybe lighttpd can issue better error trace.

I appreciate your attention to my question.
There may be some other unusual things in XigmNAS during data conversion from certificates to configuration file, and then to pem file. Most likely, I should inform the XigmaNAS developers about this.

Perhaps the issue you are having is related to Let's Encrypt changing its root CA certificate?

I am closely following the changes in Let's Encrypt CA. Their latest innovations can confuse anyone. However, I use the certificate Let's Encrypt signed at the end of January, and it works without issues.

I would like to thank you for the explanations and help in fixing the error. Can I make changes to the Wiki and add ssl.engine = "disable" option in Docs_SSL chapter? I noticed that this option is not documented anywhere yet.

RE: [Solved] Need help to fix SSL_new error: null ssl ctx - Added by gstrauss about 3 years ago

There may be some other unusual things in XigmNAS during data conversion from certificates to configuration file, and then to pem file. Most likely, I should inform the XigmaNAS developers about this.

If you do, please link here so that we can follow, too.

[Edit] Please convey to XigmaNAS that lighttpd supports ssl.privkey since lighttpd 1.4.53 and that lighttpd recommends using ssl.pemfile = "/path/to/fullchain.pem". Since lighttpd 1.4.56, the historical use of ssl.ca-file is recommended only for use with client certificate verification (ssl.verifyclient.*) These changes should hopefully make XigmaNAS certificate maintanence easier; XigmaNAS should no longer need to manipulate the certificates received from the upstream issuer. Also, if XigmaNAS has suggestions on how to improve lighttpd, they are also welcome to post here.

Can I make changes to the Wiki and add ssl.engine = "disable" option in Docs_SSL chapter? I noticed that this option is not documented anywhere yet.

Please go right ahead. Please write something (relatively brief) which makes sense to you. It's always nice hear explanations that make sense to an alternate writer.

(Note: A number of us review edits to filter for spam, and we may end up rewording your edit. Your edit is still welcome.)

    (1-6/6)