Bug #3244
closedUnconditionally enabling 64-bit time_t breaks armv7hf
Description
Hello,
I noticed this issue using the following setup:
- Yocto Kirkstone (ships with lighttpd-1.4.67)
- i.MX7 platform (ARMv7 hardfloat)
This error is printed each tile the server starts:
(../lighttpd-1.4.67/src/mod_openssl.c.1341) SSL: inactive/expired X509 certificate '/etc/lighttpd/server.pem'
Some Google investigation later I found this:
https://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg1903119.html
Similar hardware environment but different distro. Debian report indicates that this seems to go away when switching to aarch64. Since the i.MX7 isn't 64-bit capable, I looked into this a bit more.
I quickly isolated this problem to something going wrong in mod_openssl_cert_is_active(). The logic itself looks sound. I added some debug prints to check the result codes from ASN1_TIME_cmp_time_t(), and indeed, this call seemed to return bogus values.
So I went and patched openssl to do some debug prints in ASN1_TIME_cmp_time_t() itself. Noticed that the value of the second argument inside the function was different from the value the argument had in the caller.
After adding a sizeof(time_t) print in both openssl and lighttpd the real problem was revealed. openssl was using 4-bytes for it, while lighttpd was using 8-bytes. This was obviously going to break big time.
I very much suspect that this is also the problem in the Debian bug report I linked.
So it seems that Yocto doesn't build the system with 64-bit time_t by default. I'm not sure who really is at fault here, but currently it looks like the lighttpd build system shouldn't unconditionally enable _TIME_BITS=64, but only if the rest of the system is also build like this.
With best wishes,
Tobias
Updated by gstrauss 9 months ago · Edited
Thank you for your detailed sleuthing.
While libc are you using? glibc? musl? What version?
The -D_TIME_BITS=64 was added in commit 92576a2f.
reference: https://www.phoronix.com/news/Glibc-More-Y2038-Work
Maybe I should restrict it to when __GLIBC__
is defined?
I could also bypass openssl and use my internal implementation mod_openssl_asn1_time_to_posix()
, but I should also review if this time_t difference might be a problem in other places, too.
FYI: lighttpd tries very hard to use 64-bit time. See commit 309c1693
FYI: Debian is currently trying to migrate to 64-bit time_t ABI change for all platforms.
https://wiki.debian.org/ReleaseGoals/64bit-time
Updated by gstrauss 9 months ago · Edited
This describes using _TIME_BITS=64 in buildroot Version 11.2-2022.02:
"glibc build with flags _FILE_OFFSET_BITS=64 and _TIME_BITS=64 fails"
https://stackoverflow.com/questions/72801397/glibc-build-with-flags-file-offset-bits-64-and-time-bits-64-fails
Interesting thread from Jan 2022:
https://sourceware.org/pipermail/libc-alpha/2022-January/134985.html
Gentoo dealt with a similar message with GnuTLS
Bug 828001 - net-misc/wget-1.21.2 - Always reports "The certificate has not yet been activated" when using GnuTLS
https://bugs.gentoo.org/828001
I am curious: do you get the error trace for the expired certificate if you use lighttpd mod_gnutls? I think lighttpd mod_gnutls handles this case.
Regarding lighttpd mod_openssl
This error is printed each tile the server starts:
(../lighttpd-1.4.67/src/mod_openssl.c.1341) SSL: inactive/expired X509 certificate '/etc/lighttpd/server.pem'
Is this cause you problems beyond trace at startup? It should not, though I imagine that some startup scripts fail if a preflight check (lighttpd -f /etc/lighttpd/lighttpd.conf -tt
) produces any trace on stderr, even if the preflight check exit value is 0 (success).
I did a quick review of lighttpd's code and did not find other places where lighttpd is passing time_t as an argument to an API from a different library, besides mod_gnutls and mod_wolfssl, also where lighttpd checks and warns if the certificate is not active (within range of notBefore and notAfter dates in the certificate). As I asked above, I wonder if this is an issue or not for lighttpd mod_gnutls on armv7hf. For mod_wolfssl, wolfssl wolfSSL_X509_cmp_time()
takes (time_t *)
as an argument and would have a similar issue to openssl ASN1_TIME_cmp_time_t()
The issue, as you have pointed out, is lighttpd using 64-bit time_t
, but openssl or wolfssl using 32-bit time_t
and exposing API interfaces using time_t
. There are a number of OS distro-level migrations in progress, such as Debian and Gentoo that I linked above, and I am sure there are more.
If you confirm that the following works, I'll modify lighttpd mod_openssl to use my code (mod_openssl_asn1_time_to_posix()
) to parse the ASN1_TIME to make this error trace go away when openssl is built with 32-bit time_t.
--- a/src/mod_openssl.c +++ b/src/mod_openssl.c @@ -1383,6 +1383,7 @@ network_ssl_servername_callback (SSL *ssl, int *al, void *srv) #if OPENSSL_VERSION_NUMBER < 0x10101000L \ + || !(defined(_LP64) || defined(__LP64__) || defined(_WIN64)) \ || defined(BORINGSSL_API_VERSION) \ ||(defined(LIBRESSL_VERSION_NUMBER) && LIBRESSL_VERSION_NUMBER < 0x3060000fL) static unix_time64_t @@ -1402,6 +1403,7 @@ mod_openssl_cert_is_active (const X509 *crt) const ASN1_TIME *notBefore = X509_get0_notBefore(crt); const ASN1_TIME *notAfter = X509_get0_notAfter(crt); #if OPENSSL_VERSION_NUMBER < 0x10101000L \ + || !(defined(_LP64) || defined(__LP64__) || defined(_WIN64)) \ || defined(BORINGSSL_API_VERSION) \ ||(defined(LIBRESSL_VERSION_NUMBER) && LIBRESSL_VERSION_NUMBER < 0x3060000fL) const unix_time64_t before = mod_openssl_asn1_time_to_posix(notBefore);
(I do not plan to copy mod_openssl_asn1_time_to_posix()
from lighttpd mod_openssl to lighttpd mod_wolfssl unless someone complains that this issue affects them with lighttpd mod_wolfssl since the (time_t *)
argument on little-endian systems with 32-bit time_t
in wolfssl might work OK -- that is, work until 2038.)
Updated by tobias.jakobi.compleo 9 months ago
Hi Glenn,
thanks for looking into this!
Yocto Kirkstone is currently using glibc-2.35. But it doesn't seem to build anything using _TIME_BITS=64, including openssl (version 3.0.13).
I know that it's possible to instruct Yocto to build with the required defines added to the CFLAGS, see:
https://git.yoctoproject.org/poky/tree/meta/conf/distro/include/time64.inc
I've chosen against this at the moment, since this would have an impact on more packages than just lighttpd.
Sadly I can't test mod_gnutls that easily (the Yocto recipe doesn't expose this, so I would need to patch the recipe first).
I'll try to check the code using mod_openssl_asn1_time_to_posix() in the next days.
With best wishes,
Tobias
Updated by gstrauss 9 months ago
When testing, please see patch above which I edited to modify it to apply only to 32-bit builds, since 64-bit operating systems already use 64-bit time_t.
Other than the spurious error trace at startup, how are you impacted by this? Does lighttpd still work for you? (I expect that it should.)
Updated by gstrauss 9 months ago
- Status changed from Patch Pending to Fixed
Applied in changeset 7adc0f65fb0459b33b75e2af57b41d27e4853925.
Updated by tobias.jakobi.compleo 9 months ago
gstrauss wrote in #note-4:
When testing, please see patch above which I edited to modify it to apply only to 32-bit builds, since 64-bit operating systems already use 64-bit time_t.
Patch looks good. Error no longer appears!
Other than the spurious error trace at startup, how are you impacted by this? Does lighttpd still work for you? (I expect that it should.)
I'm actually not sure. I guess we have been carrying around that error for some time now. The thing is that the hardware where the server runs on, just carries a self-signed certificate. If something didn't work properly then people have probably ignored it anyway.
I just stumbled upon this because I was going through the system journal for other reasons.
Also available in: Atom