Bug #881


memory usage when ssl.engine used and large data uploaded through CGI

Added by Anonymous almost 17 years ago. Updated about 7 years ago.

Target version:


system specs: Red Hat Enterprise Linux 4, 2.6.9-34.0.2.ELsmp kernel, lighttpd-1.4.12 (built with ssl support), 4Gigs memory, openssl-0.9.7a-43.8

This problem does not happen when not using the ssl.engine.
Below is my lighttpd.conf file, a CGI perl script, and a client perl script to hit the CGI perl script. It simply uploads a file and write is to /tmp.

lighttpd is configured such that the files in the document root are treated as cgi (for example, creating a soap service to be executed through cgi). The certificate being used by lighttpd is a self signed certificate (which should not matter).

The contents of the test file I use (testfile.dat) that gets uploaded should not matter (I've tried different files and got the same behavior). To recreate this, for example, simply use a file around 130 megs.

When the ssl.engine engine is enabled, uploading the 130Meg test file causes lighttpd to eat over 130Megs of memory (over 3% according to top)- and it keeps it and never releases it. When another file is uploaded - larger than the first, more memory is used - and the memory not released. However, if a smaller file is then uploaded the memory usage will drop back down to the 130Megs. It remains there even when lighttpd is idle.

When the ssl.engine is NOT enabled this is not the case. It uses a little memory while the file is uploaded, but then goes back to 0% usage and the memory footprint of lighttpd returns to being normal (about what it is at startup).

I'm sure the memory usage while the file is being uploaded with ssl is due to how ssl works. But, I didn't expect lighttpd to continue to keep the memory after the upload was complete.


lighttpd.conf (678 Bytes) lighttpd.conf Anonymous, 2006-10-03 22:23 (321 Bytes) Anonymous, 2006-10-03 22:24
testapp (808 Bytes) testapp Anonymous, 2006-10-03 22:32

Related issues 3 (0 open3 closed)

Related to Bug #758: memory fragmentation leads to high memory usage after peaksFixedActions
Related to Bug #1265: SSL + file upload = lots of memoryFixedActions
Related to Bug #949: fastcgi, cgi, flush, php5 problem.FixedActions
Actions #1

Updated by Anonymous almost 17 years ago

I had to base64 encode the testapp perl script because Trac kept flagging it as spam. So, you'll need to decode it before using

Actions #2

Updated by stbuehler about 15 years ago

  • Status changed from New to Fixed
  • Resolution set to wontfix

The memory-not-freed is a known issue. and a wontfix for 1.4.x

Actions #3

Updated by stbuehler almost 15 years ago

  • Status changed from Fixed to Wontfix
Actions #4

Updated by gstrauss over 7 years ago

  • Related to Bug #1265: SSL + file upload = lots of memory added
Actions #5

Updated by gstrauss over 7 years ago

  • Description updated (diff)
  • Status changed from Wontfix to Fixed
  • Target version set to 1.4.x

Large request body input is saved in temporary files instead of completely buffered into memory. This was fixed a long time ago.

Actions #6

Updated by gstrauss over 7 years ago

Will additionally soon commit a change to use SSL_MODE_MODE_RELEASE_BUFFERS in OpenSSL >= 1.0.0 to potentially reduce memory footprint of idle SSL connections.

If memory usage with SSL is still an issue for some embedded systems, please reopen this ticket or create a new ticket. #1265 suggests disabling SSL read ahead to trade off reduced memory usage for additional CPU usage. Would be a one line patch to change network.c call to SSL_CTX_set_default_read_ahead(). If there was a demonstrable need where this is the solution, we might make enabling/disabling SSL read ahead into a configuration option. For now, lighttpd continues to enable SSL read ahead.

Actions #7

Updated by gstrauss over 7 years ago

  • Related to Bug #949: fastcgi, cgi, flush, php5 problem. added
Actions #8

Updated by gstrauss over 7 years ago

New: asynchronous, bidirectional streaming support for request and response
Submitted pull request:

included in the pull request are flags to openssl (SSL_MODE_RELEASE_BUFFERS) to release memory buffers when finished with them, instead of holding onto the buffers.

Actions #9

Updated by stbuehler about 7 years ago

  • Target version changed from 1.4.x to 1.4.40

Also available in: Atom