Project

General

Profile

[Solved] Preclude rewrite-if-not-file precedence over index-file.names

Added by spillkiss about 1 month ago

I see that mod_rewrite takes precedence over mod_indexfile, which causes **/index.php files to fail to load, despite being my desired directory indexer and the file exists.
I'm using lighttpd-1.4.64 on Ubuntu 20.04.4 LTS x86_64.

Here's my config:

server.modules = (
    "mod_indexfile",
    "mod_fastcgi",
    "mod_rewrite",
)
server.document-root        = "/var/www/test/" 
server.pid-file             = "/run/lighttpd.pid" 
server.username             = "www-data" 
server.groupname            = "www-data" 
server.port                 = 80
fastcgi.server = ( ".php" => (( "socket" => "/run/php/php7.4-fpm.sock" )) )
index-file.names            = ( "index.php" )
url.rewrite-if-not-file = ( "" => "/index.php" )

Here's my directory structure:

/var/www/test/
           └─ index.php
           └─ subdirectory/
               └─ index.php

Both index.php files have the following code:

<?php echo __FILE__;

When I browse http://example.com I get the following output:

/var/www/test/index.php

But, when I browse http://example.com/subdirectory I get:

/var/www/test/index.php

I'm most-likely wrong, but shouldn't the subdirectory/index.php take precedence over the rewrite rule since the index.php file exists? If I comment out the rewrite-if-not-file rule, the browser shows what I expected to see:

/var/www/test/subdirectory/index.php

My original goal was to mimick the functionally required .htaccess directives for a typical Wordpress installation, which rewrite-if-not-file does perfectly well. But, if I add a custom subdirectory with an index.php file, my custom code will not load unless I explicitly browse to the **/index.php file.

So, I guess the real question here is: is there any way to achieve this without resorting to mod_magnet (since this appears to require Apache's -f and -d)?


Replies (8)

RE: Preclude rewrite-if-not-file precedence over index-file.names - Added by gstrauss about 1 month ago

url.rewrite-if-not-file rules apply if the target is not a file.
If the target does not exist, then it is not a file.
If the target is a directory, then it is not a file.

http://example.com/subdirectory/ (note the trailing slash) might be used with the following, which does not rewrite url-paths ending in '/' (untested):
url.rewrite-if-not-file = ( "^/[^?]*[^/](\?.*)?$" => "/index.php" )
or

$HTTP["url"] !~ "/$" {
    url.rewrite-if-not-file = ( "" => "/index.php" )
}

RE: Preclude rewrite-if-not-file precedence over index-file.names - Added by spillkiss about 1 month ago

Your suggestion worked quite elegantly; I'm trying to wrap my head around your first regex.

There are a couple of issues here however.

To be clear, this is the directory structure (subdirectory/index.php is custom code):

/var/www/test/
           └─ index.php
           . (wordpress files)
           .
           .
           .
           └─ subdirectory/
               └─ index.php

1. If I change subdirectory to _subdirectory then browsing http://example.com/_subdirectory leads to a Wordpress generated 404, but http://example.com/_subdirectory/ works. So if I were to make a custom directory with an underscore (or with a hyphen) then it would not load properly without the trailing slash.

2. http://example.com/login correctly forwards to http://example.com/wp-login.php as would be expected, but http://example.com/login/ goes to lighttpd generated 404.

RE: Preclude rewrite-if-not-file precedence over index-file.names - Added by gstrauss about 1 month ago

There are a couple of issues here however.

1. PHP
2. As you continue to describe "DWIM" (do what I mean), your answer is mod_magnet so that you can do exactly what you please.
See lua mod_rewrite

RE: Preclude rewrite-if-not-file precedence over index-file.names - Added by spillkiss about 1 month ago

As you continue to describe "DWIM" (do what I mean), your answer is mod_magnet

Thank you very much for the time you've taken to get me here! I'm actually thrilled to trying this out in Lua, which is new territory for me. I've invested a lot of time on lighttpd documentation and forum posts in the last month or so.

I initially wanted this to work for a directory with an underscore in the name, but I tried to isolate the issue with the test scenario described above. So despite appearing like I'm changing what I mean, I wasn't expecting different behavior on a directory with/without an underscore. As well, I'm porting this from openlitespeed to lighttpd, and ols seems to "discover" the index.php in the directory before passing the URL into lsphp land. I don't claim to qualify for giving any idea to the inner workings of either application, nor am I suggesting how they should work. However, correcting the trailing slash on a database of URIs may be trickier than updating a conf.

There are a couple of issues here however.

1. PHP

I know PHP and asking "why" about PHP is a can of worms, but still, I'm curious what you meant here.

RE: Preclude rewrite-if-not-file precedence over index-file.names - Added by gstrauss about 1 month ago

I wasn't expecting different behavior on a directory with/without an underscore

lighttpd does not care about that distinction, either. There must be something specific in your environment or lighttpd.conf.

While mod_magnet will provide you with the most flexibility, an alternative in your case might be to use lighttpd.conf conditions to apply different rules to different parts of your site.

url.rewrite-if-not-file = ( "" => "/index.php" )
$HTTP["url"] =~ "^/subdirectory" {
    url.rewrite-if-not-file = ()
    index-file.names = ( "index.php" )
}


I know PHP and asking "why" about PHP is a can of worms, but still, I'm curious what you meant here.

I must confess that part of my comment is unrelated to your post.
My personal opinion:
  • PHP is not in my top 5 languages for security.
  • PHP is not in my top 5 languages for performance.

tl;dr: I personally avoid PHP

RE: Preclude rewrite-if-not-file precedence over index-file.names - Added by spillkiss about 1 month ago

I wasn't expecting different behavior on a directory with/without an underscore

lighttpd does not care about that distinction, either. There must be something specific in your environment

Yes, you are correct. I have confused the behavior from Wordpress redirection logic (for custom 404 pages). I just tried the same configuration using only .html files and your solution works perfectly.

While mod_magnet will provide you with the most flexibility, an alternative in your case might be to use lighttpd.conf conditions

Yes, you are correct. My case is an edge use case that doesn't require much more than a few lines in the configuration. I guess I'll leave mod_magnet Lua for another day.

Thank you again, really appreciate your help.

RE: [Solved] Preclude rewrite-if-not-file precedence over index-file.names - Added by spillkiss about 1 month ago

So, here's the follow up: It turned out that a Lua magnet was the way to go because there were too many cases of missing/included trailing slash that would fail to reach the application.

I tested this on my Wordpress site port and on a new Wordpress install which covers cases of where Wordpress isn't/is configured to generate a custom 404. I'm including what I came up with in hopes of helping another:

The conf:

server.modules = (
    "mod_indexfile",
    "mod_fastcgi",
    "mod_magnet",
)
server.errorlog             = "/var/log/lighttpd/error.log" 
server.document-root        = "/var/www/test/" 
server.pid-file             = "/run/lighttpd.pid" 
server.username             = "www-data" 
server.groupname            = "www-data" 
server.port                 = 80
include_shell "/usr/local/lighttpd/create-mime.conf.pl" 
fastcgi.server = ( ".php" => (( "socket" => "/run/php/php7.4-fpm.sock" )) )
index-file.names            = ( "index.php" )

magnet.attract-physical-path-to = ( "/path/to/your/wprewrite.lua" )

/path/to/your/wprewrite.lua:

--[[
    rewrite for wordpress
        Wordpress (most-probably) requires the behavior of Apache's mod_rewrite for:
        short-links, custom 404, etc. A uri may or may not have a trailing slash or
        may relate to a directory with or without an index.php file, so the uri
        *most-probably should* be delegated through the wordpress bootstrap
        instead of being handled within a lighttpd conditional rewrite.
        see: https://redmine.lighttpd.net/boards/2/topics/10548
        and: https://wordpress.org/support/article/htaccess/
--]]
local req = lighty.r.req_attr
local docroot = req["physical.doc-root"]
local basedir = req["physical.basedir"]
if docroot ~= basedir then return 0 end -- likely mod_alias affected this request
local stat = lighty.c.stat
-- not sure if double slash ( dirpath//index.php ) may in some scenario present
-- a subtle/difficult to identify problem, so removing it:
local path = req["physical.path"]:gsub( "/$", "" )
local function rewriteto( req, newpath )
    req["physical.path"] = newpath
    return lighty.REQUEST_RESTART
end
local st = stat( path )
if st == nil then
    return rewriteto( req, basedir .. "/index.php" )
end
if not st.is_dir then return 0 end -- probably a readable file
local pathtoindex = path .. "/index.php" 
st = stat( pathtoindex )
if st ~= nil and st.is_file then
    return rewriteto( req, pathtoindex )
end
return rewriteto( req, basedir .. "/index.php" )

Certainly outside the scope of this support ticket, but since I may get more helpful direction I'll ask:
You stated on Docs_ModMagnet:

For performance reasons, mod_magnet caches each compiled script.

Q: Is there some way to maintain Lua table data in-memory (instead of reloading from file,db) so that one could throttle requests based on IPs with a Lua script? Specifically, I mean maintaining a table in memory land of requests by IP addresses and the Lua script may conditionally fail the re-request(s) on some threshold.

RE: [Solved] Preclude rewrite-if-not-file precedence over index-file.names - Added by gstrauss about 1 month ago

Q: Is there some way to maintain Lua table data in-memory (instead of reloading from file,db) so that one could throttle requests based on IPs with a Lua script? Specifically, I mean maintaining a table in memory land of requests by IP addresses and the Lua script may conditionally fail the re-request(s) on some threshold.

See lua mod_evasive (requires lighttpd 1.4.65 or later)
There are other examples on ModMagnetExamples which store global data in _G, and also an example in AbsoLUAtion under "Redirect map" in redirect-map.lua The global data in _G lasts for the lifetime of the lua_State, which is across many requests. However, if you modify the lua script (on disk), lighttpd reloads the lua script into a new lua_State.


Thank you for sharing your working lua script above.

For reference, here is something with slightly different behavior:

-- do nothing if /path refers to a file
-- redirect /path to /path/index.php if path is a dir and /path/index.php exists
-- else redirect to /index.php (target /path is not a file or does not exist)
-- (This script must be called from magnet.attract-physical-path-to hook)
local req_attr = lighty.r.req_attr
local fspath = req_attr["physical.path"]
local st = lighty.c.stat(fspath)
if st then
  if st.is_file then return 0 end
  if st.is_dir and lighty.c.stat(fspath .. "/index.php") then
      req_attr["request.uri"] = req_attr["uri.path-raw"] .. "/index.php" 
      return lighty.REQUEST_RESTART
  end
end

req_attr["request.uri"] = "/index.php" 
return lighty.REQUEST_RESTART

    (1-8/8)