HowToSpeedUpStatWithFastcgi » History » Revision 4

« Previous | Revision 4/12 (diff) | Next »
jan, 2007-01-27 02:38

The problem

Blocking operations are real problem for event based servers. Blocking IO is a good example here.
Even with lighttpd 1.5 and AIO you have blocking calls like stat().

Fobax on #lighttpd brought up the idea to use fastcgi to speed up the blocking stat() for us.

The idea

Lighttpd passes the request to a simple fastcgi app that does a stat() call.
Than the app sends back a X-LIGHTTPD-send-file header with the original filename.
If lighttpd now stats the same file, the information is already hot in the kernel cache and we get

On one side the way adds the overhead of fastcgi to static file sending, on the otherside it really speeds up
the stat().

This works best for servers which have a very high stat() count like for all the servers which have to send alot small files (thumbnails, images, ads, ...).

The code

Fobax provided a working example code for us. I just added support to modify the thread count at runtime.
For easier support with spawn-fcgi we abused the PHP_FCGI_CHILDREN variable for that.

compile with:

$ gcc -lfcgi -lpthread fcgi-stat-accel.c -o fcgi-stat-accel
fcgi-stat-accel will use the PHP_FCGI_CHILDREN environment variable to set the thread count.
The default value, if spawned from lighttpd, is 20.

#include "fcgi_config.h"

#include <pthread.h>
#include <sys/types.h>
#include <unistd.h>
#include "fcgiapp.h"
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>

#include <stdlib.h>
#include <stdio.h>

#define THREAD_COUNT 20

#define FORBIDDEN \
FCGX_FPrintF(stream, "Status: 403 Forbidden\r\nContent-Type: text/html\r\n\r\n<h1>403 Forbidden</h1>\n");
#define NOTFOUND \
FCGX_FPrintF(stream, "Status: 404 Not Found\r\nContent-Type: text/html\r\n\r\n<h1>404 Not Found</h1>\r\n%s", filename);
#define SENDFILE \
FCGX_FPrintF(stream, "X-LIGHTTPD-send-file: %s\r\n\r\n", filename);

static void *doit(void *a){
FCGX_Request request;
int rc;
char *filename;
FILE *fd;

FCGX_InitRequest(&request, 0, FCGI_FAIL_ACCEPT_ON_INTR);
//Some platforms require accept() serialization, some don't. The documentation claims it to be thread safe
// static pthread_mutex_t accept_mutex = PTHREAD_MUTEX_INITIALIZER;
// pthread_mutex_lock(&accept_mutex);
rc = FCGX_Accept_r(&request);
// pthread_mutex_unlock(&accept_mutex);
if(rc < 0)
//get the filename
if((filename = FCGX_GetParam("SCRIPT_FILENAME", request.envp)) NULL){
//don't try to open directories
}else if(filename[strlen(filename)-1] '/'){
//open the file
}else if((fd = fopen(filename, "r")) == NULL){
NOTFOUND(request.out, filename);
//no error, serve it
SENDFILE(request.out, filename);
return NULL;

int main(void){
int i,j,thread_count;
pthread_t* id;
char* env_val;

thread_count = THREAD_COUNT;
env_val = getenv("PHP_FCGI_CHILDREN");
if (env_val != NULL) {
j = atoi(env_val);
if (j != 0) {
thread_count = j;
id = malloc(sizeof(*id) * thread_count);
for (i = 0; i < thread_count; i++) {
pthread_create(&id[i], NULL, doit, NULL);
return 0;
lighty config snippet

In your lighttpd.conf you have to pick one of the following snippets.

{{{ ## for all files
fastcgi.server = ( "" =>
( "fastcgi-stat-local" =>
"socket" => "/var/tmp/fcgi-stat-accel.socket",
"bin-path" => "/usr/local/bin/fcgi-stat-accel",
"bin-environment" => (
"max-procs" => 1,
"disable-time" => 2,
"check-local" => "disable",
"allow-x-send-file" => "enable",

Updated by jan almost 13 years ago · 4 revisions