Bug #1364

lighttpd uses most parts of the physical memory and causes out of memory crashes

Added by Anonymous almost 3 years ago. Updated almost 2 years ago.

Status:Missing Feedback Start:
Priority:Urgent Due date:
Assigned to:- % Done:

0%

Category:core
Target version:1.5.0
Missing in 1.5.x:

Description

Our lighttpd uses around 3gb of memory and causes out of memory crashes from time to time.

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                
6318 www-data 16 0 3983m 2.9g 860 D 6 94.4 0:52.17 lighttpd

The server is running Debian etch x64. I don't think this memory usage is normal? We use mod_redirect and thought the problem is maybe solved with #874, but upgrading to 1.5.0 r1992 didn't help either. What else can we try?

-- mail

graph-month.png - memory usage monthly -- mail (22.3 KB) Anonymous, 09/16/2007 10:15 am

graph-week.png - memory usage weekly -- mail (18.2 KB) Anonymous, 09/16/2007 10:15 am

errors-in-access-log.png - -- mail (53.9 KB) Anonymous, 09/16/2007 05:07 pm

lighttpd.4620 - valgrind trace -- mail (20.8 KB) Anonymous, 11/05/2007 10:48 pm

History

Updated by Anonymous almost 3 years ago

How about sharing config and possible fastcgi info, as well as a possible trace on when it out of memorys? This would help targetting the issue at hand.

-- Lfe

Updated by jrabbit almost 3 years ago

Try running the server under valgrind (http://valgrind.org/) and see if it reports a memory leak.

Updated by Anonymous almost 3 years ago

For now the problem actually stopped as sudden as it started. We are pretty sure we were attacked. Check the two attached ganglia graphs about memory usage.
We could share the config in private to avoid revealing even more information about possible attack scenarios, if you understand.

-- mail

Updated by Anonymous almost 3 years ago

We noticed that the access log during the problems contains strange block of 0x00's displayed in the screenshot from time to time. The logs of today, where we experience no problems, does not contain these. Can this be some kind of overflow?

-- mail

Updated by Anonymous almost 3 years ago

We had no problem for two months now and all of a sudden it started again. Please find a valgrind trace attached! r2019 didn't solve the problem either!

-- mail

Updated by stbuehler over 2 years ago

The valgrind trace is not really helping as long you didn't have a complete one (i.e., terminate lighttpd).

Do you use server.max-worker? See http://trac.lighttpd.net/trac/wiki/Docs%3AMultiProcessor - that may be the source of your access.log problems.

Your config and a little description what your server is doing (serving big files with fastcgi? ssl? ...) would be nice too.

And just as a process is using much memory that doesn't mean it leaks!

Updated by Anonymous over 2 years ago

Valgrind: OK sorry, didn't know that.

max-worker: no!

Our solution was to switch to Apache for a period of time, finished coding of our new site and cleaned up our rewrite rules a lot. Since then the problem is gone. We don't know if it was the new php coding or the config cleanup which solved it in the end.

-- mail

Updated by stbuehler over 2 years ago

  • Status changed from New to Fixed
  • Resolution set to worksforme

Hm k, nice it works for you now.

As you obviously can't provide more info about this bug and it seems to work for you now, i close it with "worksforme"

Updated by stbuehler almost 2 years ago

  • Status changed from Fixed to Missing Feedback

Also available in: Atom