Project

General

Profile

Bug #1727

mod_access/server-error-404 regression

Added by chris@arachsys.com over 11 years ago. Updated about 11 years ago.

Status:
Invalid
Priority:
High
Assignee:
-
Category:
core
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Missing in 1.5.x:

Description

The behaviour of url.access-deny in a `$HTTPurl conditional has changed between lighttpd 1.4.18 and 1.4.19 in a way that prevents @url.access-deny being used to protect areas of a website with files handled by a @server-error-404` handler outside the protected area. The simplest test case I've been able to cook up is as follows:


server.bind = "127.0.0.1" 
server.port = 8080
server.errorlog = "/dev/null" 
server.modules = (
  "mod_access",
)

server.document-root = "/tmp/www" 
server.error-handler-404 = "/index.html" 
$HTTP["url"] =~ "^/forbidden/" {
    url.access-deny = ("")
}

where /tmp/www/forbidden is empty and /tmp/www/index.html exists. On 1.4.18, this will refuse to return copies of /index.html for URLs in /forbidden/, whereas on 1.4.19, it allows them to be served from /index.html. Lighttpd seems to have started checking the URL of the handler instead of the URL of the original request. (I've also seen breakage the other way around, where the handler was in an otherwise forbidden directory, and stopped working in the change from 1.4.18 -> 1.4.19.)

This is potentially quite a nasty regression as, following an upgrade, people may not immediately notice that protected directories covered by a server-error-404 handler with wider scope are suddenly open when they should be forbidden.

I bisected the changesets between 1.4.18 and 1.4.19 to pinpoint where this behaviour first appeared. The change between subversion revisions 2079 and 2080 (where the cond_cache reset moves from connections.c to response.c) is the culprit: revision 2079 behaves correctly whereas revision 2080 allows the file through. However, I'm afraid I'm not familiar enough with the lighttpd core to supply a patch for this which I'm confident doesn't break any later changes.

History

#1

Updated by stbuehler over 11 years ago

  • Status changed from New to Fixed
  • Resolution set to invalid

The error 404 handler also handles 403 (i don't know why... couldn't find documentation for that) and acts like a rewrite.

I see nothing wrong with the condition cache reset.

#2

Updated by chris@arachsys.com over 11 years ago

  • Status changed from Fixed to Need Feedback
  • Resolution deleted (invalid)

Hang on, this isn't just a report of unexpected behaviour, it's a (reasonably common) configuration that previously worked and has completely stopped working due to a change in a stable release of lighttpd! The error 404 handler didn't previously catch 403 responses generated by url.access-deny in this way.

Anyone who protects an area of their web site covered by a 404-handler (e.g. in one standard configuration of Rails, Django et al.) in the canonical way, with


$HTTP["url"] =~ "^/forbidden/" {
    url.access-deny = ("")
}

will suddenly have gone from a protected area to having access wide open!

The former behaviour, of $HTTPurl matching against the request URI instead of the handler URI is actually useful, whereas testing $HTTPurl against the url of the 404-handler is never useful because the 404-handler location is static so will always fail or succeed unconditionally.

You write that you see nothing wrong with moving the condition cache reset. At a casual glance, not being especially familiar with the code, neither do I, but this is the only change between revisions 2079 and 2080, and does indeed cause the described breakage: I've built and tested both revisions and can definitively confirm that this is the revision where the regression first appears.

#3

Updated by stbuehler over 11 years ago

"The error 404 handler didn't previously catch 403 responses generated by url.access-deny in this way."

You are wrong. You could of course say that it is a bug that the 404-handler also handles 403, but the one who coded it wanted it that way (and if you think it is a bug, please open a new ticket with only that - but i guess you will get a wontfix for that, as it is a stable release).

And the changed behaviour was a bug fix, and the current behaviour is wanted.

#4

Updated by stbuehler over 11 years ago

  • Status changed from Need Feedback to Fixed
  • Resolution set to invalid
#5

Updated by chris@arachsys.com over 11 years ago

I wrote "The error 404 handler didn't previously catch 403 responses generated by url.access-deny in this way." and you responded to this asserting "You are wrong."

Sorry but this behaviour has genuinely changed. The 404-handler did not previously catch 403 responses generated by url.access-deny in a `HOST["url"]` block (prior to revision 2080), and does catch them now. Here is an explicit demonstration of the difference:


$ HOST=127.0.0.1 /tmp/lighttpd-1.4.x-r2079/bin/lighttpd -f /tmp/lighttpd.conf
$ curl http://127.0.0.1:8080/forbidden/index.html 2>/dev/null \
  | grep -iq forbidden && echo forbidden || echo allowed; echo
forbidden

$ HOST=127.0.0.2 /tmp/lighttpd-1.4.x-r2080/bin/lighttpd -f /tmp/lighttpd.conf
$ curl http://127.0.0.2:8080/forbidden/index.html 2>/dev/null \
  | grep -iq forbidden && echo forbidden || echo allowed; echo
allowed

$ cat /tmp/lighttpd.conf
server.bind = env.HOST
server.port = 8080
server.errorlog = "/dev/null" 
server.modules = (
  "mod_access",
)
server.document-root = "/tmp/www" 
server.error-handler-404 = "/index.html" 
$HTTP["url"] =~ "^/forbidden/" {
  url.access-deny = ("")
}
$

I am aware that some other sources of 403 were previously caught by the 404-handler, such as those generated from a static file with no read permissions. These are unaffected by the change in revision 2080, of course.

It's certainly cleaner for the 404-handler to receive every 403 response rather than just some of them and I'm not going to argue that sending them to the 404-handler is wrong per se. However, this is a fairly major change in behaviour which could affect quite a few users. 404-handlers are still the route that the Rails and Django projects suggest for overlaying dynamic content on top of the static tree. For instance, Rails ships with the following default lighttpd.conf:

http://github.com/rails/rails/tree/master/railties/configs/lighttpd.conf

As a result, the change will bite Rails and Django users who have masked off access to areas of their sites in the intuitively obvious way, with url.access-deny inside a `$HTTP'URL'` block. This configuration will have worked up to and including 1.4.18, albeit unintentionally from your point of view! We have a large number of Rails users who've configured their lighttpd instances based on the default Rails lighttpd.conf, and who have masked off areas with url.access-deny in a `$HTTP'url'`-matching block. They weren't expecting the upgrade to 1.4.19 to open up all the areas of their sites to which access had been disabled.

So, given this is the intended behaviour, how should I correctly be advising these users to deny access to areas that would otherwise be covered by a server.error-handler-404? I'd suggest turning off the server.error-handler-404 inside the `$HTTPurl` block, but there doesn't appear to be a way to do this. (Setting it to an empty string isn't the same thing as unsetting it, and doesn't do the right thing.) If there's a canonical recipe, I can document it both for our users and on the lighttpd wiki page for error-handler-404, as I'm sure our users won't be the only people caught by this.

#6

Updated by stbuehler over 11 years ago

Ok, let me explain it (thx for not reopening this again):

Before 1.4.19, the error-handler-404 triggered on your forbidden url, but as the condition cache was not reset, that handler got forbidden too. That is, a part of your config (the condition block) was active when it shouldn't have been.
As the error-handler-404 is only a rewrite and must be treated like a normal request in any other way (except when it generates an error again), as you may have some conditional fastcgi/... handling, this was clearly a bug and was therefore fixed.

The recommended way is to use mod_magnet, but for the forbidden part you could setup a real error-handler-404, which displays a forbidden message (static document would be fine), or just use a document which is forbidden too (like "/forbidden/index.html").

#7

Updated by chris@arachsys.com over 11 years ago

"Before 1.4.19, the error-handler-404 triggered on your forbidden url, but as the condition cache was not reset, that handler got forbidden too. That is, a part of your config (the condition block) was active when it shouldn't have been."

Yes, okay, that's certainly not a valid path for the error-handler-404 to end up being skipped in older lighttpd.

I'll document a warning about this sort of configuration and guide our users in the direction of something like


  $HTTP["url"] =~ "^/forbidden" {
    url.access-deny = ("")
    server.error-handler-404 = "/forbidden/" 
  }

so that the attempt to access the 404-handler generates the expected 403 response. (Another possibility is to use mod_auth/mod_redirect to mask off areas of the site that aren't supposed to be accessible.)

#8

Updated by stbuehler about 11 years ago

  • Status changed from Fixed to Invalid

Also available in: Atom