https://redmine.lighttpd.net/https://redmine.lighttpd.net/favicon.ico?13667327412009-12-27T10:14:22Zlighty labsLighttpd - Bug #2125: Multiple https certs doesn't workhttps://redmine.lighttpd.net/issues/2125?journal_id=66682009-12-27T10:14:22Znitrox
<ul></ul><p>Just a few hours ago we debugged this on our irc-channel, this time it was an old openssl/libssl problem, updated -> worked. Make sure you´ve atleast openssl 0.9.8_j_!</p>
<p>You can test it with openssl´s own ssl-server implementation:<br />openssl s_server -cert cert1.pem -cert2 cert2.pem -servername host2.domain.tld</p>
<p>Check with the following that 2 different certs are presented:<br />openssl s_client -connect domain.tld:4433 -tls1 vs.<br />openssl s_client -connect domain.tld:4433 -tls1 -servername host2.domain.tld</p>
<p>If that works, check if your lighty version is current, atleast 1.4.24, .25 fixed a small problem and is recommended.</p>
<p>Basically you just add a ssl.pemfile statement to your host-conditional, a working ssl setup assumed here.</p> Lighttpd - Bug #2125: Multiple https certs doesn't workhttps://redmine.lighttpd.net/issues/2125?journal_id=66692009-12-27T10:15:00Znitrox
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Need Feedback</i></li></ul> Lighttpd - Bug #2125: Multiple https certs doesn't workhttps://redmine.lighttpd.net/issues/2125?journal_id=66702009-12-27T13:04:26Zdbb
<ul></ul><p>nitrox wrote:</p>
<blockquote>
<p>Just a few hours ago we debugged this on our irc-channel, this time it was an old openssl/libssl problem, updated -> worked. Make sure you´ve atleast openssl 0.9.8_j_!</p>
<p>You can test it with openssl´s own ssl-server implementation:<br />openssl s_server -cert cert1.pem -cert2 cert2.pem -servername host2.domain.tld</p>
<p>Check with the following that 2 different certs are presented:<br />openssl s_client -connect domain.tld:4433 -tls1 vs.<br />openssl s_client -connect domain.tld:4433 -tls1 -servername host2.domain.tld</p>
<p>If that works, check if your lighty version is current, atleast 1.4.24, .25 fixed a small problem and is recommended.</p>
<p>Basically you just add a ssl.pemfile statement to your host-conditional, a working ssl setup assumed here.</p>
</blockquote>
<p>I'm running Arch Linux with OpenSSL 0.9.8l and Lighttpd 1.4.25:</p>
<pre>
dbb@guru ~ $ pacman -Qi openssl lighttpd | egrep "Name|Version"
Name : openssl
Version : 0.9.8l-1
Name : lighttpd
Version : 1.4.25-1
</pre>
<p>In case the problem may be distribution specific, the patches applied to the OpenSSL package for Arch can be found here: <a class="external" href="http://repos.archlinux.org/wsvn/packages/openssl/repos/core-i686/">http://repos.archlinux.org/wsvn/packages/openssl/repos/core-i686/</a>. There are no patches applied to the Lighttpd package.</p>
<p>I manually tested with OpenSSL's server/client as above, which worked. However I'm still seeing the issue as described in the thread, in particular, if you have a primary and secondary host, Lighttpd will always return the proper certificate for the secondary host up until the point where a request is made to the primary host, which from then on out leads to only the primary certificate being returned.</p>
<p>For reference here is the relevant section of my configuration:</p>
<pre>
$SERVER["socket"] == ":443" {
ssl.engine = "enable"
ssl.pemfile = "/etc/lighttpd/certs/liqd.org.pem"
ssl.ca-file = "/etc/lighttpd/certs/liqd.org.ca"
$HTTP["host"] == "liqd.org" {
ssl.pemfile = "/etc/lighttpd/certs/liqd.org.pem"
ssl.ca-file = "/etc/lighttpd/certs/liqd.org.ca"
}
$HTTP["host"] == "code.liqd.org" {
ssl.pemfile = "/etc/lighttpd/certs/code.liqd.org.pem"
ssl.ca-file = "/etc/lighttpd/certs/code.liqd.org.ca"
}
$HTTP["host"] =~ "^www\.(.*)$" {
url.redirect = ( "^/(.*)" => "https://%1/$1" )
}
}
</pre> Lighttpd - Bug #2125: Multiple https certs doesn't workhttps://redmine.lighttpd.net/issues/2125?journal_id=66972010-01-10T13:11:36Zdbb
<ul></ul><p>Any word on this?</p> Lighttpd - Bug #2125: Multiple https certs doesn't workhttps://redmine.lighttpd.net/issues/2125?journal_id=66982010-01-10T13:17:06Znitrox
<ul></ul><p>Well, for some it works without problems, for others it doesn´t. But we don´t have enough informations, debugs, straces, logs to reproduce it. On the other side we had some cases on irc, where updating openssl solved all problems.</p> Lighttpd - Bug #2125: Multiple https certs doesn't workhttps://redmine.lighttpd.net/issues/2125?journal_id=66992010-01-10T17:23:13Zdbb
<ul></ul><p>So I whipped up a script to test this:</p>
<p><a class="external" href="http://pastebin.com/f528c07d">http://pastebin.com/f528c07d</a></p>
<p>Basically it uses gnutls-cli to randomly make connections to a list of hosts and check that the certificate returned matches the host. My observations using it on my host:</p>
<ol>
<li>After I did some rearranging of my certificates yesterday, it appears this error is harder to reproduce. The main thing I did was remove my separate CA certificates and concatenate the entire chain into a single file for each certificate. Therefore I am no longer using ssl.ca-file.</li>
<li>Running the tester for several thousand connections will work fine but once a mismatch occurs (I had to up the number of connections to 10,000 to consistently get a break... eventually), pretty much any subsequent connections will result in a mismatch as well.</li>
<li>The break always occurs on the host that is not the default, i.e. when a break occurs the certificate that is returned is the one defined outside of any conditionals.</li>
<li>When running 4 of the testers in parallel when one broke, 2 others immediately broke as well. However one kept going successfully for significantly longer until breaking later.</li>
</ol>
<p>And like I said earlier, I'm running the latest stable versions of both lighttpd and openssl.</p> Lighttpd - Bug #2125: Multiple https certs doesn't workhttps://redmine.lighttpd.net/issues/2125?journal_id=67002010-01-10T17:48:48Zstbuehler
<ul></ul><p>I'm not sure if it is good to skip ssl.ca-file - i think openssl will ignore the additional certs in the pem-file. (Perhaps you can use the same file for pem-file and ca-file).</p>
I think we need some debug info to find the bug, so if you know how to code c a little bit you can try this:
<ul>
<li>print a debug line for every call to network_ssl_servername_callback (network.c)</li>
<li>enable the "failed to get TLS server name" log line (#if 1 instead of #if 0)</li>
<li>log the TLS server name (copy line 94+95 in network.c) after a successful call to SSL_get_servername</li>
</ul>
<p>It would be interesting what debug output you get if the certs don't match anymore.</p>
<p>--</p>
<p>script from <a class="external" href="http://pastebin.com/f528c07d">http://pastebin.com/f528c07d</a></p>
<pre>
#!/bin/bash
numtries=10000
hosts=(host1.localhost.local host2.localhost.local host3.localhost.local)
checkcert()
{
output=$(gnutls-cli $1 &)
check=$(echo "$output" | grep "hostname")
if [[ "$check" != *NOT* ]]; then
# match
echo -en "\e[32m"
echo "$check"
echo -en "\e[0m"
else
# no match
echo -en "\e[31m"
echo "$output"
echo -en "\e[0m"
exit 1
fi
}
randhost()
{
let "i = $RANDOM % ${#hosts[@]}"
echo ${hosts[$i]}
}
for i in $(seq 1 $numtries)
do
checkcert $(randhost)
done
</pre>
<p>(please always inline/attach files so we have all information in the tracker)</p> Lighttpd - Bug #2125: Multiple https certs doesn't workhttps://redmine.lighttpd.net/issues/2125?journal_id=67012010-01-10T20:13:09Zdbb
<ul><li><strong>File</strong> <a href="/attachments/1081">error.log</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/1081/error.log">error.log</a> added</li></ul><p>stbuehler wrote:</p>
<blockquote>
<p>I'm not sure if it is good to skip ssl.ca-file - i think openssl will ignore the additional certs in the pem-file. (Perhaps you can use the same file for pem-file and ca-file).</p>
I think we need some debug info to find the bug, so if you know how to code c a little bit you can try this:
<ul>
<li>print a debug line for every call to network_ssl_servername_callback (network.c)</li>
<li>enable the "failed to get TLS server name" log line (#if 1 instead of #if 0)</li>
<li>log the TLS server name (copy line 94+95 in network.c) after a successful call to SSL_get_servername</li>
</ul>
</blockquote>
<p>I made the following changes in network.c:</p>
<pre>
--- lighttpd-orig/lighttpd-1.4.25/src/network.c 2009-10-16 18:03:44.000000000 -0400
+++ lighttpd/src/lighttpd-1.4.25/src/network.c 2010-01-10 14:46:59.290717919 -0500
@@ -65,14 +65,16 @@
#if defined USE_OPENSSL && ! defined OPENSSL_NO_TLSEXT
static int network_ssl_servername_callback(SSL *ssl, int *al, server *srv) {
- const char *servername;
+ log_error_write(srv, __FILE__, __LINE__, "ss", "SSL:", "called network_ssl_servername_callback");
+
+ const char *servername;
connection *con = (connection *) SSL_get_app_data(ssl);
UNUSED(al);
buffer_copy_string(con->uri.scheme, "https");
if (NULL == (servername = SSL_get_servername(ssl, TLSEXT_NAMETYPE_host_name))) {
-#if 0
+#if 1
/* this "error" just means the client didn't support it */
log_error_write(srv, __FILE__, __LINE__, "ss", "SSL:",
"failed to get TLS server name");
@@ -103,7 +105,10 @@
return SSL_TLSEXT_ERR_ALERT_FATAL;
}
- return SSL_TLSEXT_ERR_OK;
+ log_error_write(srv, __FILE__, __LINE__, "ssb", "SSL:",
+ "successful call to get servername", con->tlsext_server_name);
+
+ return SSL_TLSEXT_ERR_OK;
}
#endif
</pre>
<p>Then I ran the tester. One thing I noticed is that to cause the break a request needs to come in from an actual browser. I ran through 40000 connections without issue, but then refreshing the pages in Firefox a few times triggers a break. Anyway, the error log from this is attached, although I don't think it provides any additional insight. One additional observation though, is that this time the break went boths ways (connecting to host1 returned the certificate for host2, and connecting o host2 returned the certificate for host1):</p>
<pre>
Resolving 'host2.localhost.local'...
Connecting to '127.0.0.1:443'...
- Certificate type: X.509
- Got a certificate list of 1 certificates.
- Certificate[0] info:
- subject `C=US,ST=NY,L=New York,O=Internet Widgits Pty Ltd,CN=host1.localhost.local', issuer `C=US,ST=NY,L=New York,O=Internet Widgits Pty Ltd,CN=host1.localhost.local', RSA key 1024 bits, signed using RSA-SHA, activated `2010-01-10 13:39:36 UTC', expires `2020-01-08 13:39:36 UTC', SHA-1 fingerprint `bab1ee719387dbab5d7f901bdab695ca613fd266'
- The hostname in the certificate does NOT match 'host2.localhost.local'
</pre>
<pre>
Resolving 'host1.localhost.local'...
Connecting to '127.0.0.1:443'...
- Certificate type: X.509
- Got a certificate list of 1 certificates.
- Certificate[0] info:
- subject `C=US,ST=NY,L=New York,O=Internet Widgits Pty Ltd,CN=host2.localhost.local', issuer `C=US,ST=NY,L=New York,O=Internet Widgits Pty Ltd,CN=host2.localhost.local', RSA key 1024 bits, signed using RSA-SHA, activated `2010-01-10 13:40:02 UTC', expires `2020-01-08 13:40:02 UTC', SHA-1 fingerprint `8e78aeb59a42b3da70826571a40c9f1d6d48c4bc'
- The hostname in the certificate does NOT match 'host1.localhost.local'
</pre>
<p>EDIT: Also, having the entire chain in one file doesn't seem to be an issue. I've tested it and browsers see the entire certificate chain and validate the site.</p> Lighttpd - Bug #2125: Multiple https certs doesn't workhttps://redmine.lighttpd.net/issues/2125?journal_id=68102010-03-06T10:06:29ZKawatoriShinji
<ul></ul><p>For some reason con->uri.authority when network_ssl_servername_callback() gets called. When the callback calls config_patch_connection(srv, con, COMP_HTTP_HOST), it sees uri.authority and never looks at tlsext_server_name.</p>
<p>Since I haven't looked outside of the callback or the COMP_HTTP_HOST patch code I don't know how hacky this is, but I just dropped a fix directly into the callback. So far it's worked for me.</p>
<pre>
diff -Nrup src/network.c src/network.c
--- src/network.c 2010-03-01 16:44:01.000000000 +0000
+++ src/network.c 2010-03-01 21:51:16.000000000 +0000
@@ -82,6 +82,9 @@ static int network_ssl_servername_callba
buffer_copy_string(con->tlsext_server_name, servername);
buffer_to_lower(con->tlsext_server_name);
+ /* Sometimes this is still set, confusing COMP_HTTP_HOST */
+ buffer_reset(con->uri.authority);
+
config_cond_cache_reset(srv, con);
config_setup_connection(srv, con);
</pre> Lighttpd - Bug #2125: Multiple https certs doesn't workhttps://redmine.lighttpd.net/issues/2125?journal_id=68112010-03-06T18:27:52Zdigital
<ul></ul><p>KawatoriShinji- Thank you for the patch; I've just applied it to my lighttpd 1.4.25 branch [FreeBSD 7.x] env, but appears that it had no effect.</p>
<p>KawatoriShinji wrote:</p>
<blockquote>
<p>For some reason con->uri.authority when network_ssl_servername_callback() gets called. When the callback calls config_patch_connection(srv, con, COMP_HTTP_HOST), it sees uri.authority and never looks at tlsext_server_name.</p>
<p>Since I haven't looked outside of the callback or the COMP_HTTP_HOST patch code I don't know how hacky this is, but I just dropped a fix directly into the callback. So far it's worked for me.</p>
<p>[...]</p>
</blockquote> Lighttpd - Bug #2125: Multiple https certs doesn't workhttps://redmine.lighttpd.net/issues/2125?journal_id=68372010-03-15T20:29:51Zdigital
<ul><li><strong>Target version</strong> set to <i>1.4.27</i></li></ul> Lighttpd - Bug #2125: Multiple https certs doesn't workhttps://redmine.lighttpd.net/issues/2125?journal_id=68512010-03-25T23:18:41ZNecoro
<ul></ul><p>This small patch seems to solve the issue on my setup (which is lighttpd-1.4.25 with the additional CVE-2010-0295 patch running on Gentoo Linux). I'll report if things change :)</p> Lighttpd - Bug #2125: Multiple https certs doesn't workhttps://redmine.lighttpd.net/issues/2125?journal_id=69092010-04-28T21:10:03Zstbuehler
<ul><li><strong>Status</strong> changed from <i>Need Feedback</i> to <i>Fixed</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li></ul><p>Applied in changeset r2724.</p> Lighttpd - Bug #2125: Multiple https certs doesn't workhttps://redmine.lighttpd.net/issues/2125?journal_id=69222010-05-05T17:02:27Zdigital
<ul></ul><p>For those that continue to have a problem and are testing under Windows XP:</p>
<p>SNI does not work on Windows XP using various browsers, see:<br /><a class="external" href="http://en.wikipedia.org/wiki/Server_Name_Indication">http://en.wikipedia.org/wiki/Server_Name_Indication</a></p>
<p>(Thus not a Lighttpd issue)</p> Lighttpd - Bug #2125: Multiple https certs doesn't workhttps://redmine.lighttpd.net/issues/2125?journal_id=69262010-05-08T02:36:04Zdbb
<ul></ul><p>Tested 1.4.27rc1 on my own server with 10,000 iterations of my test script and no wrong certificates were returned. Looks like this one has finally be resolved.</p> Lighttpd - Bug #2125: Multiple https certs doesn't workhttps://redmine.lighttpd.net/issues/2125?journal_id=69272010-05-08T08:25:11Zstbuehler
<ul></ul><p>Sounds good - thank you very much for your feedback!</p>