Project

General

Profile

Actions

Bug #3048

closed

Broken LDAP authentication on lighttpd 1.4.56

Added by mgottinger almost 4 years ago. Updated almost 4 years ago.

Status:
Fixed
Priority:
Normal
Category:
mod_auth
Target version:
ASK QUESTIONS IN Forums:
No

Description

I updated lighttpd to version 1.4.56 and LDAP authentication seems to be broken.
Lighttpd will crash immediately and restart if authentication is tried.

Messages in log:

lighttpd: search.c:96: ldap_pvt_search: Assertion `LDAP_VALID( ld )' failed.
lighttpd-angel.c.156: child (pid=487496) exited unexpectedly with signal 6, restarting

It does not matter if HTTP2 is enabled or not.
(I'm connecting to a domain controller (MS 2012R2) with starttls enabled using a lookup user with it's own ca.)

Actions #1

Updated by mgottinger almost 4 years ago

That is another message:

lighttpd: unbind.c:46: ldap_unbind_ext: Assertion `LDAP_VALID( ld )' failed.

Actions #2

Updated by gstrauss almost 4 years ago

From what version did you update from? (i.e. did this previously work?) Thx.

Actions #3

Updated by mgottinger almost 4 years ago

I updated from 1.4.55.3 (Arch Linux).

Actions #4

Updated by mgottinger almost 4 years ago

Forgot - it was working well until 1.4.56. I did not update anything else on this system before the update until now.

Actions #5

Updated by gstrauss almost 4 years ago

I don't have a test environment set up for LDAP at the moment, so this may take a bit of time for me to troubleshoot.

It appears that assert() is being called in the underlying LDAP library. lighttpd is possibly passing in a NULL value, but I don't immediately see where that might be (after a quick review of the code changes since lighttpd 1.4.55).

If this is easily reproducible for you, would you please post a stack trace? Omitting any sensitive info, would you share your lighttpd.conf? lighttpd -f /etc/lighttpd/lighttpd.conf -p

Are there any other possibly relevant changes that have been made to the system since it last worked? Maybe an upgraded LDAP library?

I see an inconsistency in mod_authn_ldap.c, but this inconsistency existed in lighttpd 1.4.55, too, and wouldn't cause a crash.

--- a/src/mod_authn_ldap.c
+++ b/src/mod_authn_ldap.c
@@ -507,7 +507,7 @@ static LDAP * mod_authn_ldap_host_init(log_error_st *errh, plugin_config_ldap *s
         /* if no CA file is given, it is ok, as we will use encryption
          * if the server requires a CAfile it will tell us */
         if (s->auth_ldap_cafile) {
-            ret = ldap_set_option(NULL, LDAP_OPT_X_TLS_CACERTFILE,
+            ret = ldap_set_option(ld, LDAP_OPT_X_TLS_CACERTFILE,
                                   s->auth_ldap_cafile);
             if (LDAP_OPT_SUCCESS != ret) {
                 mod_authn_ldap_err(errh, __FILE__, __LINE__,

Actions #6

Updated by mgottinger almost 4 years ago

Ok I integrated this patch - but it did not solve the problem as you predicted.

I did indeed update libldap on November 29 and I updated lighttpd on December 13.
I guess it was compiled with a previous version before - I'm going to check this first. (pacman log: upgraded libldap (2.4.53-1 -> 2.4.54-1) )

I will try to strace later (I lack experience in this topic), but I need to block traffic first - there are a lot of permanently connected clients producing lots of noise and people just went home - I'm going to wait a little longer.

This part of the config I can share:

    server.tag                       = "********" 
    server.username                  = "http" 
    server.groupname                 = "http" 
    server.document-root             = "/srv/http/default" 
    server.errorlog                  = "/var/log/lighttpd/error.log" 
    server.feature-flags             = (
        "server.h2proto" => "enable",
        "server.h2c"     => "enable",
        # 2
    )
    server.pid-file                  = "/run/lighttpd/lighttpd.pid" 
    dir-listing.activate             = "disable" 
    dir-listing.encoding             = "utf-8" 
    accesslog.filename               = "/var/log/lighttpd/access.log" 
    deflate.allowed-encodings        = ("brotli", "gzip", "deflate")
    deflate.cache-dir                = "/srv/lighttpd-cache" 
    deflate.compression-level        = 9
    deflate.mimetypes                = (
        "text/html",
        "text/plain",
        "text/javascript",
        "application/javascript",
        "text/x-js",
        "text/json",
        "application/json",
        "application/x-javascript",
        "text/css",
        "text/xml",
        "application/xml",
        "text/xml+rss",
        "application/xml+rss",
        # 13
    )
    index-file.names                 = ("index.php", "index.html", "index.lighttpd.html")
    server.modules                   = (
        "mod_auth",
        "mod_authn_ldap",
        "mod_access",
        "mod_accesslog",
        "mod_alias",
        "mod_deflate",
        "mod_fastcgi",
        "mod_openssl",
        "mod_proxy",
        "mod_redirect",
        "mod_rewrite",
        "mod_setenv",
        "mod_status",
        # 13
    )
    static-file.exclude-extensions   = (".php", ".inc")
    fastcgi.server                   = (
        ".php" => (
            "localhost" => (
                "socket"                => "/run/php-fpm/php-fpm.sock",
                "broken-scriptfilename" => "enable",
                # 2
            ),
        ),
    )
    ssl.disable-client-renegotiation = "enable" 
    ssl.cipher-list                  = "HIGH:!TLSv1.1:!TLSv1:!SSLv3:!SSLv2" 
    ssl.honor-cipher-order           = "enable" 
    auth.backend                     = "ldap" 
    auth.backend.ldap.hostname       = "dc.example.local" 
    auth.backend.ldap.starttls       = "enable" 
    auth.backend.ldap.ca-file        = "/etc/ssl/certs/CA.pem" 
    auth.backend.ldap.allow-empty-pw = "disable" 
    auth.backend.ldap.base-dn        = "OU=Users,DC=example,DC=local" 
    auth.backend.ldap.bind-dn        = "CN=lookup,DC=example,DC=local" 
    auth.backend.ldap.bind-pw        = "*******" 

and I filter like this on urls in a few $HTTP["host"]:
    auth.backend.ldap.filter = "(&(objectclass=user)(objectclass=person)(objectclass=top)(sAMAccountName=$)(memberOf:1.2.840.113556.1.4.1941:=cn=XYZ,ou=Groups,dc=example,dc=local)(!(UserAccountControl:1.2.840.113556.1.4.803:=2)))" 
    auth.require             = (
        "/download/protected" => (
            "method"  => "basic",
            "realm"   => "Protected Area",
            "require" => "valid-user",
            # 3
        ),
    )

Actions #7

Updated by mgottinger almost 4 years ago

I tried compiling with older versions - same error...

Actions #8

Updated by gstrauss almost 4 years ago

I tried compiling with older versions - same error...

I presume you mean older versions of libldap

Getting a stack trace is different from strace. To create a basic test environment, take your existing lighttpd.conf and change the port on which lighttpd is listening so that it does not interfere with the existing listening interfaces. Then change all the filesystem paths to logs and pid files so that the test-lighttpd.conf does not overwrite the same paths. In the config that you shared, those are:

    server.errorlog                  = "/var/log/lighttpd/error.log" 
    server.pid-file                  = "/run/lighttpd/lighttpd.pid" 
    accesslog.filename               = "/var/log/lighttpd/access.log" 

You could just comment those three lines out in test-lighttpd.conf.
If you set server.port = 8443 and comment out the other $SERVER["socket"] conditions, the test lighttpd will listen on *:8443
Start it with
gdb --args /usr/sbin/lighttpd -D -f /path/to/test-lighttpd.conf
At the prompt, type run and press Enter.
From your client, access https://your-servername:8443/ to cause the crash when the authentication occurs.
In the debugger, which now reports the crash, type bt for a backtrace, and save the result.
Then, type quit and press Enter. Answer y and Enter if it asks if you want to "Quit anyway?"

Actions #9

Updated by gstrauss almost 4 years ago

Well, this is a start. Now to find out why.

(gdb) where
#0  0x00007ffff7d4a9d5 in raise () from /lib64/libc.so.6
#1  0x00007ffff7d338a4 in abort () from /lib64/libc.so.6
#2  0x00007ffff7d33789 in __assert_fail_base.cold () from /lib64/libc.so.6
#3  0x00007ffff7d43026 in __assert_fail () from /lib64/libc.so.6
#4  0x00007ffff7b3db0e in ldap_pvt_search () from /lib64/libldap-2.4.so.2
#5  0x00007ffff7b4078d in ldap_pvt_search_s () from /lib64/libldap-2.4.so.2
#6  0x00007ffff7b40860 in ldap_search_ext_s () from /lib64/libldap-2.4.so.2
#7  0x00007ffff7b94eab in mod_authn_ldap_search (errh=0x45b490, s=0x46cf10, base=0x45e100 "dc=example,dc=com", filter=0x4b3800 "(&(objectClass=person)(sn=any))") at mod_authn_ldap.c:581
#8  0x00007ffff7b951be in mod_authn_ldap_get_dn (filter=0x4b3800 "(&(objectClass=person)(sn=any))", base=<optimized out>, s=0x46cf10, errh=0x45b490) at mod_authn_ldap.c:630
#9  mod_authn_ldap_basic (r=<optimized out>, p_d=0x46b940, require=0x46c690, username=0x4b3700, pw=0x4b37b4 "any") at mod_authn_ldap.c:737
#10 0x00007ffff7fb78ed in mod_auth_check_basic (r=0x46ca20, p_d=<optimized out>, require=0x46c690, backend=0x45a000 <http_auth_backends>) at mod_auth.c:810
...

Actions #10

Updated by gstrauss almost 4 years ago

lighttpd is not passing NULL for ld. I'll try to bisect the lighttpd git history to see if I can find the change which caused this.

(gdb) up 7
#7  0x00007ffff7b94eab in mod_authn_ldap_search (errh=0x45b490, s=0x46cf10, base=0x45e100 "dc=example,dc=com", filter=0x4b3800 "(&(objectClass=person)(sn=any))") at mod_authn_ldap.c:581
581            ret = ldap_search_ext_s(s->ldap, base, LDAP_SCOPE_SUBTREE, filter,
(gdb) print s->ldap
$1 = (LDAP *) 0x7ffff7ed1a60 <main_arena+96>
(gdb) print base
$2 = 0x45e100 "dc=example,dc=com" 
(gdb) print filter
$3 = 0x4b3800 "(&(objectClass=person)(sn=any))" 
(gdb) print attrs
$4 = {0x7ffff7b960f2 "1.1", 0x0}

Actions #11

Updated by mgottinger almost 4 years ago

Yes, I meant older versions of libldap.

Thanks for explaining and your help - this is new territory.
Do you still want me to stack trace my setup?

Actions #12

Updated by gstrauss almost 4 years ago

Not needed. Your stack would likely show the same.

Actions #13

Updated by mgottinger almost 4 years ago

It does.
(What an easy tool - I will do better next time.)

Actions #14

Updated by gstrauss almost 4 years ago

I have tracked the issue back near a03afc90. I have a hunch that there is a memory error before this point, but the changes here perturb the system enough that it shows up when libldap allocates memory. This may take me a few more hours to find the actual bug. Time to break out valgrind.

Actions #15

Updated by gstrauss almost 4 years ago

  • Status changed from New to Patch Pending
  • Target version changed from 1.4.x to 1.4.57

It is a memory initialization error from 8e713130

--- a/src/mod_authn_ldap.c
+++ b/src/mod_authn_ldap.c
@@ -216,7 +216,7 @@ SETDEFAULTS_FUNC(mod_authn_ldap_set_defaults) {
                     buffer *b;
                     *(const buffer **)&b = cpv->v.b;
                     mod_authn_add_scheme(srv, b);
-                    ldc = malloc(sizeof(plugin_config_ldap));
+                    ldc = calloc(1, sizeof(plugin_config_ldap));
                     force_assert(ldc);
                     ldc->srv = srv;
                     ldc->auth_ldap_hostname = b->ptr;

Actions #16

Updated by mgottinger almost 4 years ago

Compiled and working well - excellent :-)
Thank you very much for this rapid solution!

Actions #17

Updated by gstrauss almost 4 years ago

Sorry about the bug.

Thank you for your quick and helpful responses, and, of course, for reporting the bug.

Actions #18

Updated by gstrauss almost 4 years ago

  • Status changed from Patch Pending to Fixed
Actions

Also available in: Atom