lighttpd request manipulation using Lua¶

Module: mod_magnet

see AbsoLUAtion for additional infos, code-snippets, examples ...

• Table of contents
• lighttpd request manipulation using Lua
  • Requirements
  • Overview
  • Options
  • mod_magnet API before lighttpd 1.4.60
    • Return Codes
    • Tables
    • lighty.env[]
    • lighty.header[]
    • lighty.content[]
    • lighty.status[]
    • Exported Functions
  • Examples
  • Porting mod_cml scripts

Requirements¶

lighttpd 1.4.12 or higher built --with-lua
lua >= 5.1

Overview¶

mod_magnet enables programmatic manipulation of lighttpd request handling via Lua (programming language) scripts. mod_magnet allows you to do more complex URL rewrites and caching than you would otherwise be able to do.

While the Lua language is very powerful, mod_magnet is not meant to be a general replacement for your regular scripting environment. This is because mod_magnet is executed in the core of lighttpd and EVERY long-running operation is blocking for ALL connections in the server. You have been warned. For time-consuming or blocking scripts, use mod_fastcgi and friends.

For performance reasons, mod_magnet caches the compiled script. For each script-run the script itself is checked for freshness and recompiled if necessary.

Options¶

mod_magnet can attract a request in several stages in the request-handling.

• when URL is processed (but after rewrite); this is the same stage that mod_proxy and other handlers (fastcgi in certain modes) use which do not need a physical file.
• when the doc-root is known and the physical-path is already set up
• when response starts, right before response headers are finalized

The stage to intercept depends on the purpose of the each script. Usually you want to use the 2nd stage where the physical-path which relates to your request is known. At this level you can run checks against lighty.env["physical.path"].

magnet.attract-raw-url-to = ( ... )
magnet.attract-physical-path-to = ( "/absolute/path/to/script.lua"  )
magnet.attract-response-start-to = ( ... )  # (since 1.4.56)

You can define multiple scripts when separated by a comma. The scripts are executed in the specified order. If one of them returns a bad status-code, the scripts following will not be executed.

mod_magnet API before lighttpd 1.4.60¶

Return Codes¶

• If script return 0, or returns nothing (or nil), request handling continues.
• If script return lighty.RESTART_REQUEST (currently equal to 99), the request is restarted, reprocessing the request-uri. This is usually used in combination with changing the request.uri in a rewrite.
• If script returns 1xx, a 1xx intermediate response is sent, and request handling continues.
• If script returns >= 200, the value is used as final HTTP status code and the response is finalized. No other modules are executed.

If handling the response in lua script, return an HTTP status code from the lua script. Examples are: Redirected, Input Validation, ...

  if (lighty.env["uri.scheme"] == "http") then
    lighty.header["Location"] = "https://" .. lighty.env["uri.authority"] .. lighty.env["request.uri"]
    return 302
  end

Tables¶

Most of the interaction between mod_magnet and lighttpd is done through tables. Tables in lua are similar to hashes (Perl, Ruby), dictionaries (Java, Python), associative arrays (PHP), ...

• lighty.request[] - certain request headers like Host, Cookie, or User-Agent are available
• lighty.req_env[] - request environment variables
• lighty.env[]
  • physical.path
  • physical.rel-path
  • physical.doc-root
  • uri.path (the URI without the query-string)
  • uri.path-raw
  • uri.scheme (http or https)
  • uri.authority (the server-name)
  • uri.query (the URI after the ? )
  • request.method (e.g. GET)
  • request.uri (uri after rewrite)
  • request.orig-uri (before rewrite)
  • request.path-info
  • request.remote-ip
  • request.protocol (e.g. "HTTP/1.0", "HTTP/1.1", "HTTP/2.0")
  • response.http-status # (since 1.4.56) (read-only value)
  • response.body-length # (since 1.4.56) (read-only value; not nil only if response body is complete)
  • response.body # (since 1.4.56) (read-only value; not nil only if response body is complete)
• lighty.header[] - certain response headers like Location are available
• lighty.content[]
• lighty.status[]

You can loop with "pairs()" through the special tables "lighty.request", "lighty.env", "lighty.req_env" and "lighty.status"; "lighty.header" and and "lighty.content" are normal lua tables, so you can use them with "pairs()" too.

lighty.env[]¶

lighttpd has its internal variables which are exported as read/write to the magnet.

If "http://example.org/search.php?q=lighty" is requested this results in a request like:

  GET /search.php?q=lighty HTTP/1.1
  Host: example.org

When you are using ``attract-raw-url-to`` you can access the following variables:

• parts of the request-line
  • lighty.env["request.uri"] = "/search.php?q=lighty"

• HTTP request-headers
  • lighty.request["Host"] = "example.org"

• parts of the URI
  • lighty.env["uri.path"] = "/search.php"
  • lighty.env["uri.path-raw"] = "/search.php"
  • lighty.env["uri.scheme"] = "http"
  • lighty.env["uri.authority"] = "example.org"
  • lighty.env["uri.query"] = "q=lighty"

Later in the request-handling, the URL is split, cleaned up and turned into a physical path name:

• filenames, pathnames
  • lighty.env["physical.path"] = "/my-docroot/search.php"
  • lighty.env["physical.rel-path"] = "/search.php"
  • lighty.env["physical.doc-root"] = "/my-docroot"

All of them are readable, but not all of them are writable (or have no effect if you write to them).

  -- 1. simple rewriting is done via the request.uri
  lighty.env["request.uri"] = ... 
  return lighty.RESTART_REQUEST

  -- 2. changing the physical-path
  lighty.env["physical.path"] = ...

  -- 3. changing the query-string
  lighty.env["uri.query"] = ...

lighty.header[]¶

If you want to set a response header for your request, you can add a field to the lighty.header[] table:

lighty.header["Content-Type"] = "text/html"

lighty.content[]¶

You can generate your own content and send it out to the clients.

  lighty.content = { "<pre>", { filename = "/etc/passwd" }, "</pre>" }
  lighty.header["Content-Type"] = "text/html" 

  return 200

The lighty.content[] table is executed when the script is finished. The elements of the array are processed left to right and the elements can either be a string or a table. Strings are included AS IS into the output of the request.

• Strings
  • are included as is

• Tables
  • filename = "<absolute-path>" is required
  • offset = <number> [default: 0]
  • length = <number> [default: size of the file]

This results in sending the range [offset, length-1] of the file.
('length' param is misnamed and actually indicates the range offset + 1; kept as such for historical compatibility with existing scripts)

lighty.status[]¶

mod_status support a global statistics page and mod_magnet allows to add and update values in the status page:

Config

status.statistics-url = "/server-counters" 
  magnet.attract-raw-url-to = (server.docroot + "/counter.lua")

counter.lua

lighty.status["core.connections"] = lighty.status["core.connections"] + 1

Result

core.connections: 7
  fastcgi.backend.php-foo.0.connected: 0
  fastcgi.backend.php-foo.0.died: 0
  fastcgi.backend.php-foo.0.disabled: 0
  fastcgi.backend.php-foo.0.load: 0
  fastcgi.backend.php-foo.0.overloaded: 0
  fastcgi.backend.php-foo.1.connected: 0
  fastcgi.backend.php-foo.1.died: 0
  fastcgi.backend.php-foo.1.disabled: 0
  fastcgi.backend.php-foo.1.load: 0
  fastcgi.backend.php-foo.1.overloaded: 0
  fastcgi.backend.php-foo.load: 0

Exported Functions¶

mod_magnet exports a few functions to the script:

• pairs() (extends the default pairs() function)
• print() (writes to the lighttpd error log)
• lighty.stat()

print()¶

print() replaces the lua-default version and redirects the trace to the lighttpd error log. print() is useful for debugging.

print("Host: " .. lighty.request["Host"])
print("Request-URI: " .. lighty.env["request.uri"])

lighty.stat()¶

lighty.stat() checks the existence of a file/dir/socket and returns the stat() information for it.
It uses the lighttpd internal stat-cache.

lighty.stat(path)
- path: (string) absolute path
- returns: table or nil on error

If the call was successful you'll be able to query the following fields from the table:

• is_file
• is_dir
• is_char
• is_block
• is_socket
• is_link
• is_fifo
• st_mode
• st_mtime
• st_ctime
• st_atime
• st_uid
• st_gid
• st_size
• st_ino
• etag
• content-type

Examples¶

see Abso lua tion for additional infos, code-snippets, examples ...

Porting mod_cml scripts¶

mod_cml got replaced by mod_magnet.

• mod_cml functions memcache_get_string() memcache_get_long() memcache_exists() (and lighttpd.conf cml.memcache-hosts) should be replaced with a lua-only solution:
  https://github.com/silentbicycle/lua-memcached

• CACHE_HIT in mod_cml
  output_include = { "file1", "file2" }
return CACHE_HIT
  becomes
  lighty.content = { { filename = "/path/to/file1" }, { filename = "/path/to/file2"} }
return 200

• CACHE_MISS like (CML):
  trigger_handler = "/index.php"
return CACHE_MISS
  becomes (magnet)
  lighty.env["request.uri"] = "/index.php"
return lighty.RESTART_REQUEST

Questions? Post in the lighttpd Forums