Bug #881
closedmemory usage when ssl.engine used and large data uploaded through CGI
Description
system specs: Red Hat Enterprise Linux 4, 2.6.9-34.0.2.ELsmp kernel, lighttpd-1.4.12 (built with ssl support), 4Gigs memory, openssl-0.9.7a-43.8
This problem does not happen when not using the ssl.engine.
Below is my lighttpd.conf file, a CGI perl script, and a client perl script to hit the CGI perl script. It simply uploads a file and write is to /tmp.
lighttpd is configured such that the files in the document root are treated as cgi (for example, creating a soap service to be executed through cgi). The certificate being used by lighttpd is a self signed certificate (which should not matter).
The contents of the test file I use (testfile.dat) that gets uploaded should not matter (I've tried different files and got the same behavior). To recreate this, for example, simply use a file around 130 megs.
When the ssl.engine engine is enabled, uploading the 130Meg test file causes lighttpd to eat over 130Megs of memory (over 3% according to top)- and it keeps it and never releases it. When another file is uploaded - larger than the first, more memory is used - and the memory not released. However, if a smaller file is then uploaded the memory usage will drop back down to the 130Megs. It remains there even when lighttpd is idle.
When the ssl.engine is NOT enabled this is not the case. It uses a little memory while the file is uploaded, but then goes back to 0% usage and the memory footprint of lighttpd returns to being normal (about what it is at startup).
I'm sure the memory usage while the file is being uploaded with ssl is due to how ssl works. But, I didn't expect lighttpd to continue to keep the memory after the upload was complete.
Files
Updated by Anonymous about 18 years ago
I had to base64 encode the testapp perl script because Trac kept flagging it as spam. So, you'll need to decode it before using
Updated by stbuehler about 16 years ago
- Status changed from New to Fixed
- Resolution set to wontfix
The memory-not-freed is a known issue. and a wontfix for 1.4.x
Updated by gstrauss over 8 years ago
- Related to Bug #1265: SSL + file upload = lots of memory added
Updated by gstrauss over 8 years ago
- Description updated (diff)
- Status changed from Wontfix to Fixed
- Target version set to 1.4.x
Large request body input is saved in temporary files instead of completely buffered into memory. This was fixed a long time ago.
Updated by gstrauss over 8 years ago
Will additionally soon commit a change to use SSL_MODE_MODE_RELEASE_BUFFERS in OpenSSL >= 1.0.0 to potentially reduce memory footprint of idle SSL connections.
If memory usage with SSL is still an issue for some embedded systems, please reopen this ticket or create a new ticket. #1265 suggests disabling SSL read ahead to trade off reduced memory usage for additional CPU usage. Would be a one line patch to change network.c call to SSL_CTX_set_default_read_ahead(). If there was a demonstrable need where this is the solution, we might make enabling/disabling SSL read ahead into a configuration option. For now, lighttpd continues to enable SSL read ahead.
Updated by gstrauss over 8 years ago
- Related to Bug #949: fastcgi, cgi, flush, php5 problem. added
Updated by gstrauss over 8 years ago
New: asynchronous, bidirectional streaming support for request and response
Submitted pull request: https://github.com/lighttpd/lighttpd1.4/pull/66
included in the pull request are flags to openssl (SSL_MODE_RELEASE_BUFFERS) to release memory buffers when finished with them, instead of holding onto the buffers.
Updated by stbuehler about 8 years ago
- Target version changed from 1.4.x to 1.4.40
Also available in: Atom