Project

General

Profile

Bug #1667

reverse proxy consuming memory until lighttpd is killed

Added by aetius over 11 years ago. Updated over 3 years ago.

Status:
Obsolete
Priority:
Normal
Assignee:
-
Category:
mod_proxy_core
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Missing in 1.5.x:
Yes

Description

Tested on 1.5.0-r1992 and 1.5.0-r2169 (current svn as of 13 May 2008). Hardware is an dual-core Athlon64 5000 with 3GB of RAM running Gentoo (2.6.23). I'm http reverse proxying (static) to a set of load-balanced 1.4.19 servers (behind LVS/keepalived) serving large static content (up to 200-250MB, flv video). Proxying a large file through the 1.5.0 machine results in excessive memory usage, far beyond what I would expect from a full memory cache. For example, after proxying one 195MB file, lighttpd is using 1995m of virtual memory, and 683m of resident memory (this from a startup memory usage of about 40m virt/1.5m res).

As I understand it, lighttpd is supposed to re-use this memory once allocated, so having that much allocated memory should be ok. However, the re-use appears to have a problem as well. Additional file requests (other files, or, after another file, the first file again) result in rapid additional memory allocation. Eventually, the server memory usage builds up to the point where the OS cleanly kills the lighttpd process due to lack of available memory. With large files (50-200MB) on a busy site this happens within a few minutes of server startup. Cutting off a connection in the middle on the client side, without finishing the download, seems to aggravate the problem; if a file finishes downloading, it appears that some memory is freed, but not all, and the total allocated still keeps going up until the process is killed.

So I fired up lighttpd under valgrind ... and the problem goes away. Memory allocated is similar to file size and the memory is re-used efficiently - allocated memory does not grow excessively and the server proxies a number of files (20ish) both partial and full without growing significantly in memory usage. I'm attaching the valgrind log, though I couldn't really find anything useful in it. Example config is also attached.

lighttpd.conf (3.22 KB) lighttpd.conf aetius, 2008-05-14 21:20
valgrind.log (10.2 KB) valgrind.log aetius, 2008-05-14 21:21

History

#1

Updated by gstrauss over 3 years ago

  • Description updated (diff)
  • Assignee deleted (jan)
  • Missing in 1.5.x set to Yes

In lighttpd 1.4.40, large files are buffered to tempfiles on disk rather than using gobs of real memory.

#2

Updated by gstrauss over 3 years ago

  • Status changed from New to Obsolete

Also available in: Atom