Mod fastcgi » History » Revision 73
« Previous |
Revision 73/98
(diff)
| Next »
nitrox, 2011-02-21 12:48
the FastCGI Interface¶
Module: mod_fastcgi
Note
This means you need to include the line
server.modules += ( "mod_fastcgi" )
or
include "conf.d/fastcgi.conf"
(which should have the line server.modules += ( "mod_fastcgi" ) by default)
in your lighttpd.conf file. Without this, you'll get the error
WARNING: unknown config-key: fastcgi.server (ignored)
when trying to use lighttpd with fastcgi and, for example, php.
Description¶
- Table of contents
- the FastCGI Interface
- Description
- Options
- X-Sendfile
- Load-Balancing
- FastCGI and Programming Languages
- Skeleton for remote authorizer
- Troubleshooting
lighttpd provides an interface to external programs that support the FastCGI interface. The FastCGI Interface is defined by http://www.fastcgi.com/ and is a platform-independent and server-independent interface between a web-application and a webserver.
This means that FastCGI programs that run with the Apache webserver will run seamlessly with lighttpd and vice versa.
FastCGI
Warning: As of php 5.2.3, the cgi binary is installed as "/usr/local/bin/php-cgi" by default, instead of "/usr/local/bin/php". Make sure to substitute the relevant parts in the documentation below.
FastCGI removes a lot of the limitations of CGI programs. CGI programs have the problem that they have to be restarted by the webserver for every request which leads to really bad performance values.
FastCGI removes this limitation by keeping the process running and handling the requests by this always running process. This removes the time used for the fork() and the overall startup and cleanup time which is necessary to create and destroy a process.
While CGI programs communicate to the server over pipes, FastCGI processes use Unix-Domain-Sockets or TCP/IP to talk with the webserver. This gives you the second advantage over simple CGI programs: FastCGI don't have to run on the webserver itself but anywhere in the network.
lighttpd takes it a little bit further by providing an internal FastCGI load-balancer which can be used to balance the load over multiple FastCGI Servers. In contrast to other solutions only the FastCGI process has to be on the cluster and not the whole webserver. That gives the FastCGI process more resources than a e.g. load-balancer+apache+mod_php solution.
If you compare FastCGI against an apache+mod_php setup you should note that FastCGI provides additional security as the FastCGI process can be run under different permissions than the webserver and can also live in a chroot which might be different than the one the webserver is running in.
Options¶
lighttpd provides the FastCGI support via the fastcgi-module (mod_fastcgi) which provides 3 options in the config-file:
fastcgi.debug
a value between 0 and 65535 to set the debug-level in the FastCGI module. Currently only 0 and 1 are used. Use 1 to enable some debug output, 0 to disable it.
Example:
fastcgi.debug = 1
fastcgi.map-extensions
map multiple extensions to the same fastcgi server
Example:
fastcgi.map-extensions = ( ".php3" => ".php" )
or for multiple
fastcgi.map-extensions = ( ".php3" => ".php", ".php4" => ".php" )
fastcgi.server
tell the module where to send FastCGI requests to. Every file-extension can have it own handler. Load-Balancing is done by specifying multiple handlers for the same extension.
structure of fastcgi.server section:
( <extension> =>
( [ <name> => ]
( # Be careful: lighty does *not* warn you if it doesn't know a specified option here (make sure you have no typos)
"host" => <string> ,
"port" => <integer> ,
"socket" => <string>, # either socket or host+port
"bin-path" => <string>, # optional
"bin-environment" => <array>, # optional
"bin-copy-environment" => <array>, # optional
"mode" => <string>, # optional
"docroot" => <string> , # optional if "mode" is not "authorizer"
"check-local" => <string>, # optional
"max-procs" => <integer>, # optional - when omitted, default is 4
"broken-scriptfilename" => <boolean>, # optional
"fix-root-scriptname" => <boolean>, # optional, since 1.4.23 (option didn't work before 1.4.23)
"disable-time" => <integer>, # optional
"allow-x-send-file" => <boolean>, # optional
"kill-signal" => <integer>, # optional, default is SIGTERM(15) (v1.4.14+)
),
( "host" => ...
)
)
)
- <extension>: is the file-extension or prefix (if started with "/")
- <name>: optional name that shows up in the generated statistics of mod_status, useful for indicating which backend handler processed this extension (e.g. The name "php" would show as "fastcgi.backend.php.*")
- "host": is ip of the FastCGI process (no support for hostnames right now, see #1417)
- "port": is tcp-port on the "host" used by the FastCGI process
- "bin-path": path to the local FastCGI binary which should be started if no local FastCGI is running
- "socket": path to the unix-domain socket
- "mode": is the FastCGI protocol mode. Default is "responder", also "authorizer" mode is implemented.
- "docroot": is optional and is the docroot on the remote host for default "responder" mode. For "authorizer" mode it is MANDATORY and it points to docroot for authorized requests. For security reasons it is recommended to keep this docroot outside of server.document-root tree.
- "check-local": is optional and may be "enable" (default) or "disable". If enabled the server first checks for a file in local 'server.document-root' tree and returns 404 (Not Found) if no such file, and does not fall back to FastCGI. If disabled, the server forwards a request to the FastCGI interface without this check.
- "broken-scriptfilename": breaks SCRIPT_FILENAME in a way that PHP can extract PATH_INFO from it (default: disabled)
- "fix-root-scriptname": use this for backends with extension "/" (and check-local is disabled).
- "disable-time": time to wait before a disabled backend is checked again (don't set this to 0 for backends you spawn with lighttpd, i.e. bin-path set, as lighty won't restart your backends then)
- "allow-x-send-file": controls if X-LIGHTTPD-send-file, X-Sendfile and X-Sendfile2 (since 1.4.24) headers are used. See X-Sendfile.
If bin-path is set:
- "max-procs": the number of fastcgi processes that will be started
- "bin-environment": put an entry into the environment of the started process
- "bin-copy-environment": clean up the environment and copy only the specified entries into the fresh environment of the spawn process
- "kill-signal": By default lighttpd send SIGTERM to FastCGI processes, which were spawned by lighttpd. Applications, which link libfcgi, need to be killed with SIGUSR1. This applies to php <5.2.1, "lua-magnet":http://jan.kneschke.de/projects/lua and others.
Examples¶
Multiple extensions for the same host:
fastcgi.server = (
".php" =>
(( "host" => "127.0.0.1",
"port" => 1026,
"bin-path" => "/usr/local/bin/php"
)),
".php4" =>
(( "host" => "127.0.0.1",
"port" => 1026
))
)
Example with prefix:
fastcgi.server = (
"/remote_scripts/" =>
(( "host" => "192.168.0.3",
"port" => 9000,
"check-local" => "disable",
"docroot" => "/" # remote server may use
# it's own docroot
))
)
The request "http://my.example.org/remote_scripts/test.cgi" will be forwarded to fastcgi server at "192.168.0.3" and the value "/remote_scripts/test.cgi" will be used for the SCRIPT_NAME variable. Remote server may prepend it with its own document root. The handling of index files is also the responsibility of remote server for this case.
In the case that the prefix is not terminated with a slash, the prefix will be handled as file and "/test.cgi" would become a PATH_INFO instead of part of SCRIPT_NAME.
Example for "authorizer" mode:
fastcgi.server = ( "/remote_scripts/" =>
(( "host" => "10.0.0.2",
"port" => 9000,
"docroot" => "/path_to_private_docs",
"mode" => "authorizer"
))
)
Note that if "docroot" is specified then its value will be used in DOCUMENT_ROOT and SCRIPT_FILENAME variables passed to FastCGI server.
X-Sendfile¶
If the "allow-x-send-file" feature is active, you can use the following headers to send files instead of the generated content from the backend.
As this allows the backend to send any file lighttpd can read to the client, you should only use this for trusted backends. (As an example: your backend probably can't read the SSL key file, but lighttpd can. So although lighty wouldn't deliver the SSL key file in a normal request and the backend can't read it, the backend could use X-Sendfile to get the SSL key file)
- send the "X-Sendfile: /absolute/path/to/file" header ("X-LIGHTTPD-send-file" is an old alias for "X-Sendfile"):
lighttpd will send the specified file and use the file length for the Content-Length header if it can read the file, otherwise it will ignore the header. - send the "X-Sendfile2: filename range" header (available since 1.4.24)
The filename has to be urlencoded (including encoding ',' as '%2c':str_replace(',', '%2c', urlencode($path));)
Range has the form "start-end" (inclusive end) or "start-" (from start until file end); you must always specify the range, use "0-" for the complete file.
Example:X-Sendfile2: /tmp/test.txt 0-499sends the first 500 bytes from/tmp/test.txt.
You can send this header more than once with different parameters (or you can combine single values with ", " between them).
Load-Balancing¶
The FastCGI plugin provides automatically a load-balancing between multiple FastCGI servers.
fastcgi.server = ( ".php" =>
(
( "host" => "10.0.0.2",
"port" => 1030
),
( "host" => "10.0.0.3",
"port" => 1030 )
)
)
To understand how the load-balancing works you can enable the fastcgi.debug option and will get a similar output as here:
proc: 127.0.0.1 1031 1 1 1 31454 proc: 127.0.0.1 1028 1 1 1 31442 proc: 127.0.0.1 1030 1 1 1 31449 proc: 127.0.0.1 1029 1 1 2 31447 proc: 127.0.0.1 1026 1 1 2 31438 got proc: 34 31454 release proc: 40 31438 proc: 127.0.0.1 1026 1 1 1 31438 proc: 127.0.0.1 1028 1 1 1 31442 proc: 127.0.0.1 1030 1 1 1 31449 proc: 127.0.0.1 1031 1 1 2 31454 proc: 127.0.0.1 1029 1 1 2 31447
Even if this for multiple FastCGI children on the local machine the following explanation is valid for remote connections too.
The output shows:
- IP, port, unix-socket (is empty here)
- is-local, state (0 - unset, 1 - running, ... )
- active connections (load)
- PID
As you can see the list is always sorted by the load field.
Whenever a new connection is requested, the first entry (the one with the lowest load) is selected, the load is increased (got proc: ...) and the list is sorted again.
If a FastCGI request is done or the connection is dropped, the load on the FastCGI proc decreases and the list is sorted again (release proc: ...)
This behaviour is very light-weight in code and still very efficient as it keeps the fastcgi-servers equally loaded even if they have different CPUs.
FastCGI and Programming Languages¶
Preparing PHP as a FastCGI program
One of the most important application that has a FastCGI interface is php which can be downloaded from http://www.php.net/ . You have to recompile the php from source to enable the FastCGI interface as it is normally not enabled by default in the distributions.
If you already have a working installation of PHP on a webserver execute a small script which just contains
<?php phpinfo(); ?>
and search for the line in that contains the configure call. You can use it as the base for the compilation.
You have to remove all occurrences of `--with-apxs`, `--with-apxs2` and the like which would build PHP with Apache support. Add the next three switches to compile PHP with FastCGI support.
$ ./configure \
--enable-fastcgi \
--enable-force-cgi-redirect \
...
After compilation and installation check that your PHP binary contains FastCGI support by calling:
$ php -v PHP 4.3.3RC2-dev (cgi-fcgi) (built: Oct 19 2003 23:19:17)
The important part is the (cgi-fcgi).
Starting a FastCGI-PHP
Starting with version 1.3.6 lighttpd can spawn the FastCGI processes locally itself if necessary:
fastcgi.server = ( ".php" =>
(( "socket" => "/tmp/php-fastcgi.socket",
"bin-path" => "/usr/local/bin/php"
))
)
PHP provides 2 special environment variables which control the number of spawned workers under the control of a single watching process (PHP_FCGI_CHILDREN) and the number of requests what a single worker handles before it kills itself.
fastcgi.server = ( ".php" =>
(( "socket" => "/tmp/php-fastcgi.socket",
"bin-path" => "/usr/local/bin/php",
"bin-environment" => (
"PHP_FCGI_CHILDREN" => "16",
"PHP_FCGI_MAX_REQUESTS" => "10000"
)
))
)
To increase the security of the started process you should only pass the necessary environment variables to the FastCGI process.
fastcgi.server = ( ".php" =>
(( "socket" => "/tmp/php-fastcgi.socket",
"bin-path" => "/usr/local/bin/php",
"bin-environment" => (
"PHP_FCGI_CHILDREN" => "16",
"PHP_FCGI_MAX_REQUESTS" => "10000" ),
"bin-copy-environment" => (
"PATH", "SHELL", "USER" )
))
)
Configuring PHP
If you want to use PATH_INFO and PHP_SELF in you PHP scripts you have to configure php and lighttpd. The php.ini needs the option:
cgi.fix_pathinfo = 1
and the option "broken-scriptfilename" in your fastcgi.server config:
fastcgi.server = ( ".php" =>
(( "socket" => "/tmp/php-fastcgi.socket",
"bin-path" => "/usr/local/bin/php",
"bin-environment" => (
"PHP_FCGI_CHILDREN" => "16",
"PHP_FCGI_MAX_REQUESTS" => "10000"
),
"bin-copy-environment" => ( "PATH", "SHELL", "USER" ),
"broken-scriptfilename" => "enable"
))
)
Why this ? the "cgi.fix_pathinfo = 0" would give you a working PATH_INFO but no PHP_SELF. If you enable it, it turns around. To fix the PATH_INFO `--enable-discard-path` needs a SCRIPT_FILENAME which is against the CGI spec, a broken-scriptfilename. With "cgi.fix_pathinfo = 1" in php.ini and "broken-scriptfilename => "enable"" you get both.
Please note that the CGI binary has been renamed to php-cgi in 5.2.3, so you'll probably have a path like /usr/local/bin/php-cgi in your config.
Roadsend PHP -- The Other PHP
Roadsend PHP, freely available from http://code.roadsend.com/pcc, is an alternative implementation of PHP that works with lighttpd. It can run PHP code directly (as Zend PHP does), but it can also compile PHP code directly to native FastCGI binaries.
An example configuration that can be used for interpreting .php files on your server with Roadsend PHP:
fastcgi.server = ( ".php" =>
( "localhost" =>
( "host" => "127.0.0.1",
"port" => 1026,
"bin-path" => "/opt/roadsend/pcc/modules/fastcgi/pcc.fcgi",
)
)
)
To deploy a compiled application, you will need mod_rewrite enabled in your lighttpd server. Assuming your app is named "myapp", copy the compiled app "myapp.fcgi" and associated library "libmyapp_u.so" created by pcc to your cgi-bin directory:
cp myapp.fcgi libmyapp_u.so /usr/lib/cgi-bin/
We use the mod_rewrite module to grab all requests for PHP files in the application's web root directory and re-route them to the FastCGI binary. We also allow for an index.php file
# serve index pages
url.rewrite-once = ( "^/myapp/$" => "/myapp/index.php" )
# main fastcgi entry
$HTTP["url"] =~ "^/myapp/.+\.php$" {
fastcgi.server = ( "/myapp" =>
( "localhost" =>
( "bin-path" => "/var/www/localhost/cgi-bin/myapp.fcgi",
"docroot" => "/var/www/localhost/htdocs/myapp",
"host" => "127.0.0.1",
"port" => 1026,
"check-local" => "disable"
)
)
)
} # HTTP[url]
External Spawning
Spawning FastCGI processes directly in the webserver has some disadvantages like
- FastCGI process can only run locally
- has the same permissions as the webserver
- has the same base-dir as the webserver
As soon as you are using a separate FastCGI Server to take off some load from the webserver you have to control the FastCGI process by a external program like "spawn-fcgi".
"spawn-fcgi" is used to start a FastCGI process in its own environment and set the user-id, group-id and change to another root-directory (chroot).
For convenience a wrapper script should be used which takes care of all the necessary option. Such a script in included in the lighttpd distribution and is call spawn-php.sh.
The script has a set of config variables you should take a look at:
## ABSOLUTE path to the spawn-fcgi binary SPAWNFCGI="/usr/local/sbin/spawn-fcgi" ## ABSOLUTE path to the PHP binary FCGIPROGRAM="/usr/local/bin/php" ## bind to tcp-port on localhost FCGIPORT="1026" ## bind to unix domain socket # FCGISOCKET="/tmp/php.sock" ## number of PHP childs to spawn PHP_FCGI_CHILDREN=10 ## number of request server by a single php-process until ## is will be restarted PHP_FCGI_MAX_REQUESTS=1000 ## IP adresses where PHP should access server connections ## from FCGI_WEB_SERVER_ADDRS="127.0.0.1,192.168.0.1" # allowed environment variables separated by spaces ALLOWED_ENV="ORACLE_HOME PATH USER" ## if this script is run as root switch to the following user USERID=wwwrun GROUPID=wwwrun exec $SPAWNFCGI -a $FCGI_WEB_SERVER_ADDRS -p $FCGIPORT -f $FCGIPROGRAM -u $USERID -g $GROUPID
If you have set the variables to values that fit to your setup you can start it by calling:
$ spawn-php.sh spawn-fcgi.c.136: child spawned successfully: PID: 6925
If you get "child spawned successfully: PID:" the php processes could be started successfully. You should see them in your process list:
$ ps ax | grep php 6925 ? S 0:00 /usr/local/bin/php 6928 ? S 0:00 /usr/local/bin/php ...
The number of processes should be PHP_FCGI_CHILDREN + 1. Here the process 6925 is the master of the slaves which handle the work in parallel. Number of parallel workers can be set by PHP_FCGI_CHILDREN. A worker dies automaticly of handling PHP_FCGI_MAX_REQUESTS requests as PHP might have memory leaks.
If you start the script as user root php processes will be running as the user USERID and group GROUPID to drop the root permissions. Otherwise the php processes will run as the user you started script as.
As the script might be started from a unknown stage or even directly from the command-line it cleans the environment before starting the processes. ALLOWED_ENV contains all the external environement variables that should be available to the php-process.
A patch for "spawn-fcgi" so that it can accept a custom configuration file location can be found here:
http://forum.lighttpd.net/topic/446#1294
The patch must be applied in the source directory of lighttpd's source code. The file to patch is "spawn-fcgi.c" and requires a recompile of lighty to have the binary compiled with it as well.
It is required to add the following in the "spawn-php.sh" file:
## ABSOLUTE path to PHP config FCGI_CONFIG="/path/to/php/config/php5.ini"
After:
## ABSOLUTE path to the PHP binary FCGIPROGRAM="/etc/lighttpd/php"
And change the similar lines at the bottom of the file to:
if test x$UID = x0; then
EX="$SPAWNFCGI -p $FCGIPORT -f $FCGIPROGRAM -u $USERID -g $GROUPID -C $PHP_FCGI_CHILDREN -x $FCGI_CONFIG"
else
EX="$SPAWNFCGI -p $FCGIPORT -f $FCGIPROGRAM -C $PHP_FCGI_CHILDREN -x $FCGI_CONFIG"
fi
Perl
For Perl you have to install the FCGI module from CPAN.
Skeleton for remote authorizer¶
The basic functionality of authorizer is as follows (see http://www.fastcgi.com/devkit/doc/fcgi-spec.html, 6.3 for details).
#include <fcgi_stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main () {
char* p;
while (FCGI_Accept() >= 0) {
/* wait for fastcgi authorizer request */
printf("Content-type: text/html\r\n");
if ((p = getenv("QUERY_STRING")) == NULL) ||
<QUERY_STRING is unauthorized>)
printf("Status: 403 Forbidden\r\n\r\n");
else printf("\r\n");
/* default Status is 200 - allow access */
}
return 0;
}
It is possible to use any other variables provided by FastCGI interface for authorization check. Here is only an example.
Troubleshooting¶
fastcgi.debug should be enabled for troubleshooting.
If you get:
(fcgi.c.274) connect delayed: 8
(fcgi.c.289) connect succeeded: 8
(fcgi.c.745) unexpected end-of-file (perhaps the fastcgi
process died): 8
the fastcgi process accepted the connection but closed it right away. This happens if FCGI_WEB_SERVER_ADDRS doesn't include the host where you are connection from.
If you get:
(fcgi.c.274) connect delayed: 7
(fcgi.c.1107) error: unexpected close of fastcgi connection
for /peterp/seite1.php (no fastcgi process on host/port ?)
(fcgi.c.1015) emergency exit: fastcgi: connection-fd: 5
fcgi-fd: 7
the fastcgi process is not running on the host/port you are connection to. Check your configuration.
If you get ::
(fcgi.c.274) connect delayed: 7 (fcgi.c.289) connect succeeded: 7
everything is fine. The connect() call just was delayed a little bit and is completely normal.
Chrooted lighttpd, evhost and non-chrooted fastcgi
If you're trying to use non-chrooted fastcgi combined with evhost in a chrooted lighttpd, you seem to be out of luck. To properly support this configuration, lighttpd would need to prepend its chroot path before its document root when making fastcgi requests. The "docroot" option can do this statically (ie, when not using evhost), but not for multiple hosts. Also see #1816 about this.
Docs:ModProxyCore will be supporting this configuration in lighttpd 1.5.
C/C++ FastCGI on lightty named socket
Remember that named sockets lookup operations are performed through a /proc entry in Linux so that you need to mount the proc-device inside any chrooted webserver where you wish to use named sockets, in other operation systems also the required /dev entries are needed (most chrooted installations will require a /dev/null atleast). The lightty mod_fastcgi.c will pre-open an application server socket on file descriptor 0 (look for FCGI_LISTENSOCK_FILENO in "fastcgi.h") already going through the socket()/bind()/listen() cycle for AF_UNIX sockets - so in your FastCGI program you only need to run accept(). For the multithreaded case you need to run FCGX_Init one time and to create as many FCGX_Request handles as you wish.
// gcc -I/usr/include/fastcgi -lfcgi testfastcgi.c -o test.fastcgi
#include <stdlib.h>
#include <string.h>
#include <syslog.h>
#include <alloca.h>
#include <fcgiapp.h>
#define LISTENSOCK_FILENO 0
#define LISTENSOCK_FLAGS 0
int main(int argc, char** argv) {
openlog("testfastcgi", LOG_CONS|LOG_NDELAY, LOG_USER);
int err = FCGX_Init(); /* call before Accept in multithreaded apps */
if (err) { syslog (LOG_INFO, "FCGX_Init failed: %d", err); return 1; }
FCGX_Request cgi;
err = FCGX_InitRequest(&cgi, LISTENSOCK_FILENO, LISTENSOCK_FLAGS);
if (err) { syslog(LOG_INFO, "FCGX_InitRequest failed: %d", err); return 2; }
while (1) {
err = FCGX_Accept_r(&cgi);
if (err) { syslog(LOG_INFO, "FCGX_Accept_r stopped: %d", err); break; }
char** envp;
int size = 200;
for (envp = cgi.envp; *envp; ++envp) size += strlen(*envp) + 11;
char* result = (char*) alloca(size);
strcpy(result, "Status: 200 OK\r\nContent-Type: text/html\r\n\r\n");
strcat(result, "<html><head><title>testcgi</title></head><body><ul>\r\n");
for (envp = cgi.envp; *envp; ++envp) {
strcat(result, "<li>");
strcat(result, *envp);
strcat(result, "</li>\r\n");
}
strcat(result, "</ul></body></html>\r\n");
FCGX_PutStr(result, strlen(result), cgi.out);
}
return 0;
}
and the configuration record
fastcgi.server = (
"/login" => (
"test.fastcgi.handler" => (
"socket" => socket_dir + "test.fastcgi.socket",
"check-local" => "disable",
"bin-path" => "/usr/bin/test.fastcgi",
"max-procs" => 30,
)
)
)
Python FastCGI on lighty with flup
Download flup http://trac.saddi.com/flup
Configure:
fastcgi.server = (
".py" =>
(
"python-fcgi" =>
(
"socket" => socket_dir + "fastcgi.python.socket",
"bin-path" => "test.py",
"check-local" => "disable",
"max-procs" => 1,
)
))
test.py:
#!/usr/bin/python2.5
def myapp(environ, start_response):
start_response('200 OK', [('Content-Type', 'text/plain')])
return ['Hello World!\n']
if __name__ == '__main__':
from flup.server.fcgi import WSGIServer
WSGIServer(myapp).run()
Updated by nitrox almost 14 years ago · 73 revisions