Lighttpd

{{{
#!rst ==============================
Optimizing FastCGI performance ==============================

.. contents::

Overview ========

If you lately ran into the questions

• "How many PHP backends do I need for my load" or
• "Why is my application returning the error 500 from time to time"

you want to read this article very carefully.

How many php-process do I need ? ================================

That's the question why you are here I think and to answer it let me create a small example:

lighty is managing a pipe. On one side are your users with their webbrowsers, on the other side is PHP. In case
you have more incoming requests than your backends can handle, lighty will queue them up and will push the new
requests to the backends when they are free again. If the queue still fills up, it will burst and the next
requests to this backend will be denied. You will see a message in the errorlog like ::

... load = 380 ...

To come up with a formula to calculate the number of backends you need think of:

• you have 100 php-req/s
• the average request-time on PHP side is 0.1s

In the average case you need: ::

100 php-reqs/s * 0.1s/php-req = 10 php-procs

You see ? As you propably can't control the number of incoming PHP-requests, you can only tune the average
request time spent in the PHP process.

• Use a byte-code cache like http://xcache.lighttpd.net/ or http://pecl.php.net/APC
• add caching to your application
• tune your queries to the database

As measuring the average request time is not that easy the (fastcgi.backend.0.load: 22) is your indicator
how many php-process would be used right now.

Measuring the load ==================

Load the status-module and enable the statistics: ::

server.modules = {..., "mod_status", ... }

status.statistics-url = "/server-counters"

The counters page lists serveral counters of the fastcgi module:

• the over all number of requests handle by the module
• currently waiting requests
• currently waiting request per backend

.. note::

If you have more than one backend you should name each backend individually. ::

fastcgi.server = ( 
      ".php" => (
        "backend1" => ( "host" => "php-srv1", ... ),
        "backend2" => ( "host" => "php-srv2", ... ),
      )
    )

You might get this output: ::

fastcgi.active-requests: 22
  fastcgi.backend.0.0.connected: 5639
  fastcgi.backend.0.0.died: 0
  fastcgi.backend.0.0.disabled: 0
  fastcgi.backend.0.0.load: 11
  fastcgi.backend.0.0.overloaded: 0
  fastcgi.backend.0.1.connected: 7724
  fastcgi.backend.0.1.died: 0
  fastcgi.backend.0.1.disabled: 0
  fastcgi.backend.0.1.load: 11
  fastcgi.backend.0.1.overloaded: 0
  fastcgi.backend.0.load: 22
  fastcgi.requests: 13363

We have 2 backends (max-procs = 2) and a current load of 22 (fastcgi.backend.0.load: 22). The load is equally
distributed over the two backends (fastcgi.backend.0.0.load: 11, fastcgi.backend.0.1.load: 11).

Using rrdtool to monitor the load
---------------------------------

(to be added)

Installing XCache =================

XCache is one of the cachers that speed up your fastcgi in case you're using PHP. Pick the lastest version from http://trac.lighttpd.net/xcache/wiki/ReleaseArchive.

To install: ::

~/src $ wget http://... (the release url)
  ~/src $ tar zxf xcache*.tar.gz
  ~/src $ cd xcache
  ~/src/xcache $ phpize
  ~/src/xcache $ ./configure --enable-xcache --enable-xcache-coverager
  ~/src/xcache $ make
  ~/src/xcache $ su
  ~/src/xcache # make install
  ~/src/xcache # cat xcache.ini >> /etc/php.ini
  ~/src/xcache # $EDITOR /etc/php.ini

turn xcache.size=64M, and setup your xcache.admin.pass.

Setting up web interface: ::

alias.url += ("/xcache-admin/" => "/usr/share/xcache/admin/")

Check it out by pointing your browser to http://localhost/xcache-admin/

Tuning the database ===================

This is a very short intro to tuning MySQL. First check your my.cnf:

.. note::

Read up before changing anything on your server especially if you are not on a dedicated MySQL box with spare memory

::

[mysqld]

1. default is 100, might need to raise it

max-connections = 200

1. if you use innodb alot, increase the pool-size
2. default is 8M, far too low.

innodb_buffer_pool_size = 512M

1. for MyISAM it is
   key_buffer_size = 128M

query_cache_size = 32M

1. logs are good

log-slow-queries

long-query-time = 2

log-queries-not-using-indexes

Restart the MySQL server and check for 'hostname-slow.log' in the datadir. It will list all queries which

• take longer than 2 seconds to execute
• not using an index

On all these queries run a EXPLAIN and add a index when neccesary. You want to concentrate on queries which

• are run often
• have a query-time > 2

If the examined rows is several times larger than the sent-rows, add an index.

What you want to check too is: ::

mysql> SHOW GLOBAL STATUS;
  ...
| Created_tmp_disk_tables | 88    |
| Created_tmp_files       | 2     |
| Created_tmp_tables      | 39079 |
  ...
  Created_tmp_disk_tables should as small as 
  possible compared to Created_tmp_tables

| Com_select              | 39004 |
| Select_scan             | 39004 |
  Ooops, all SELECT statements are Table-Scans, very bad.

...
| Table_locks_immediate   | 3     |
| Table_locks_waited      | 0     |
  This is good, we never had to wait for a table-lock.

.. note::

In MySQL before 5.0.x it is SHOW STATUS.

Benchmarking ============

By measuring the response time before and after the optimizations you can get a idea of how a increased load
will effect you in the future.

Several tools are available to measure the response-time:

• ab/ab2 is part of the Apache Server package
  • handles max 1024 connections
  • hammers a single URL only
  • select() based

• flood is a apache project
• siege
  • handles about 100 parallel connections (before it dies here with out-of-memory)
  • threaded
  • can generate random load

• http_load
  • can generate random load
  • allow throttling

• httpperf

When you are using a benchmark tool which only queries a single URL several times, you won't see problems caused by:

• dirty caches (MySQL Query Cache, Byte-Code Cache, ...)
• locking (Table Locks, File Locks, ...)

If you use siege for the random load and ab for the single-URL load, you should get useful results.

(to be continued)

Can I have too many php-processes ? ===================================

Having too many PHP processes can be bad too. If you have more than you need, the PHP processes will eat all the memory
and will hit the swap-space.

If you are using a config like: ::

fastcgi.server = ( ".php" =>
      (( "socket" => "/tmp/php-fastcgi.socket",
          "bin-path" => "/usr/bin/php-cgi",
          "max-procs" => 10,
          "bin-environment" => (
              "PHP_FCGI_CHILDREN" => "16",
              "PHP_FCGI_MAX_REQUESTS" => "1000" 
          ),
          "broken-scriptfilename" => "enable" 
      ))
  )

you will have (following the famous formular)::

num-procs = max-procs * ( 1 + PHP_FCGI_CHILDREN )

10 * (16 + 1) = 170 procs

Together with a PHP 5.1.5 running APC 3.0.11 a single PHP process takes about 13Mb (RSZ) for itself: ::

$ ps axu | grep php
    400 web       16   0  152m  13m 6804 S    1  0.7   0:01.99 php-fcgi

13M * 160 = 2Gb RAM

Not all of them will be using 13m from the start, but you see the problem I think. Running into swap is counter-productive.

Remote Balancing ================

If all this doesn't help you can still take a few servers, install a shared file-system like NFS and run
PHP on this servers. Just add the IPs to the "host" field in the fastcgi.server setting and lighty
will balance of those servers.

I use Perl, Ruby, Python, ... =============================

PHP was only used as an example. The same applies to all languages.

}}}