Feature #1743


[PATCH] Add support for CRYPT-MD5-NTLM

Added by Anonymous almost 14 years ago. Updated over 5 years ago.

Target version:


I would like to propose the following patch for inclusion in lighttpd 1.5.x. It adds support for a new type of password hashes in htpasswd files, which I called the CRYPT-MD5-NTLM password hashing algorithm. I am not trying to reinvent the wheel (see below for a tech description of the algorithm). Before, here are a few usage cases:

1. Alice runs a Lighttpd server configured with mod_auth and the LDAP backend to authenticate hundreds of HTTP users. The LDAP server (a Windows box) has to be decommissioned, but Alice wants the change to be transparent to the HTTP users, she wants them to be able to keep using their current passwords. So Alice runs one of the numerous password hash dumping utilities (pwdump, fgdump...) on the Windows box to extract the NTLM hashes, and runs a script re-hash them using a secure, salted algorithm (CRYPT-MD5-NTLM) and hence create an htpasswd file that Lighttpd can interpret.

2. Alice runs a farm of high-performance web servers serving mostly static files where latency is absolutely critical. One day she is asked to authenticate the HTTP users through a Windows domain controller. She experiments with Lighttpd's mod_auth LDAP backend and unfortunately notices that this backend adds 20+ ms of latency. This is unacceptable as most of the content could before often be served with a sub-ms latency. So Alice simply sets a task up on the domain controller to periodically dump the NTLM hashes and re-hash them using CRYPT-MD5-NTLM. The resulting hashes are then distributed to Lighttpd which can now perform much faster local authentication. Password changes on the domain controller are reflected almost immediately on Lighttpd as the task is executed periodically.

3. Alice administers 2 independent Windows domains, each with its own domain controller and she wants Lighttpd to be able to authenticate users from any of the 2 Windows domains. She can't because Lighttpd's mod_auth LDAP backend can only communicate with a single LDAP server. So Alice sets a task up on each of the domain controllers to periodically dump the NTLM hashes and re-hash them using CRYPT-MD5-NTLM. Another script can then process these files, combine them (and perform any other wanted operation) and distribute them to Lighttpd.

Now, onto the tech specs of the CRYPT-MD5-NTLM password hashing algorithm: it is basically the standard MD5-based "$1$" crypt() algorithm applied to the lowercase hexadecimal representation of the NTLM hash of the password, with the "$1$" label replaced with "$1+ntml$". Since the NTLM hash is simply the MD4 hash of the UCS-2LE encoded password, a simple shell script can generate CRYPT-MD5-NTLM hashes very easily:

  $ echo -n test | sed -r 's,(.),\1\x00,g' | openssl md4
  $ openssl passwd -1 0cb6948805f797bf2a82807973b89537 | \
    sed 's,\$1,$1+ntlm,'

The above example shows that the NTLM hash for the password "test" is 0cb6948805f797bf2a82807973b89537 and that its CRYPT-MD5-NTLM hash (with salt ZcoRcQyP) is $1+ntlm$ZcoRcQyP$FgeohMFPzL4Gp3E9KtOBl1

This algorithm allows for the construction of secure, salted password hashes from an environment where only legacy NTLM hashes are available and where it is not feasible to re-hash all the passwords with the MD5-based crypt().


-- m.bevand


lighttpd-crypt-md5-ntlm.patch (4.01 KB) lighttpd-crypt-md5-ntlm.patch Patch to add support for the CRYPT-MD5-NTLM password hashing algorithm Anonymous, 2008-08-05 18:19
Actions #1

Updated by Anonymous almost 14 years ago

I screwed up the formatting of my shell commands in the description. They should read:

$ echo -n test | sed -r 's,(.),\1\x00,g' | openssl md4
$ openssl passwd -1 0cb6948805f797bf2a82807973b89537 | sed 's,\$1,$1+ntlm,'

-- m.bevand

Actions #2

Updated by Anonymous over 13 years ago

It's been 3 weeks since I created this ticket. So, any comment ?

-- m.bevand

Actions #3

Updated by stbuehler over 13 years ago

Patch itself:
- i guess if it works, it is basically ok
- your UCS-2LE encoding assumes ISO-8859-1 as input - i am not sure if this is correct

But why should we use it upstream?
- "secure, salted password hashes": i think we already have this
- "where only legacy NTLM hashes are available": this is way to specific imho - the next one is going to need that for another hash method.
- "lets support as many backends as possible": and i don't want to maintaim them :)

So.. for now i am not convinced we should use it upstream.

Btw: authentication could be done via fastcgi (in 1.4 via responder, in 1.5 via x-rewrite) or mod_magnet, too.

Actions #4

Updated by Anonymous over 13 years ago

First of all, thank you for your feedback.

I assume ISO-8859-1 input because RFC 2617 doesn't define the charset that the browser is supposed to be use when encoding the username and password. From my experience, most browsers seem to default to ISO-8859-1 even when the 401 Unauthorized response has the UTF-8 charset specified in the Content-Type header.

Why should you use it ? It is true Lighttpd already supports secure, salted password hashes, so in theory I could force my users to rehash their password.

I don't think NTLM is too specific. In fact it is so common that even FreeBSD's crypt() supports it (search for "nthash" in r1). NTLM has been supported by Windows for 15 years, from Windows NT 3.1 to Windows 2008, and Microsoft has no plans to replace it. It is the most widespread hashing algorithm in Windows environments (LM is becoming irrelevant).

Maintaining these < 100 lines of code implementing something that hasn't changed in 15 years should be easy. At least the ratio of usefulness to maintenance time is pretty high.

I am currently using the patch in a production environment. It works fine. If you decide to not accept it, well no big deal, I don't mind having to run a custom mod_auth for now. At least I am sharing it and we can see whether others find it useful or not.


-- m.bevand

Actions #5

Updated by gstrauss almost 6 years ago

  • Assignee deleted (jan)
Actions #6

Updated by gstrauss almost 6 years ago

  • Target version deleted (1.5.0)
Actions #7

Updated by gstrauss over 5 years ago

  • Description updated (diff)
  • Status changed from New to Need Feedback

Is there still interest in this?

Actions #8

Updated by gstrauss over 5 years ago

The patch submitted has a mix of tabs alternating with lines which use indent groups of 3 spaces.
sprintf() is used to convert characters to hex one char at a time, which is extremely inefficient, not to mention using sprintf() is generally discouraged.
fprintf(stderr, ...) is used instead of log_error_write()

I lack the words to describe this patch politely.

Actions #9

Updated by gstrauss over 5 years ago

  • Status changed from Need Feedback to Patch Pending
  • Target version set to 1.4.42

prepared patch with < 50 lines, using stack (no memory allocation), no snprintf, no sprintf, and minimizing string copies

minimally tested using the password 'test' example. YMMV.

Actions #10

Updated by gstrauss over 5 years ago

  • Status changed from Patch Pending to Fixed
  • % Done changed from 0 to 100

Also available in: Atom