Bug #2948
closed[regression][Bisected] lighttpd uses way more memory with POST since 1.4.52
Description
Since commit 88ee73d0a216 ("[multiple] perf: simplify chunkqueue_get_memory()")
uploading a big file via POST uses a lot more memory than in v1.4.51
It's actually the next part of #2922.
At the time, I didn't pay attention to the memory used by the lighttpd process.
I did my tests onto a1077d18cb36 ("[tests] more test config cleanup")
and then track down the memory problem on commit 88ee73d0a216, even with patches:
fe3dc1796894 ("[mod_fastcgi] fix NULL ptr deref from bugfix #2922 (fixes #2923)")
a1b527e47374 ("[multiple] reduce initial buffer sz if large POST (fixes #2922)")
I've used the same config as in #2922 with:server.stream-request-body = 2
server.stream-response-body = 2
server.chunkqueue-chunk-sz = 1024
To highlight the high memory usage, here's simple index.php: if(isset($_FILES['image'])){
$errors= array();
$file_name = $_FILES['image']['name'];
$file_size =$_FILES['image']['size'];
$file_tmp =$_FILES['image']['tmp_name'];
$file_type=$_FILES['image']['type'];
$file_ext=strtolower(end(explode('.',$_FILES['image']['name'])));
if(empty($errors)==true){
echo "Success";
}else{
print_r($errors);
}
}
?>
<html>
<body>
<form action="" method="POST" enctype="multipart/form-data">
<input type="file" name="image" />
<input type="submit"/>
</form>
</body>
</html>
and, on the server side :top -b -d 1 | grep lighttpd
Upload a 50MB file, and watch the memory used by lighttpd grow.
at version 1.4.51, the memory used by lighttpd was between 4012K and 4528K during the upload.
since commit fccc7fc607ff ("[core] perf: chunk.c chunk pool") it's between 4012K and 7856K
and since commit 88ee73d0a216 ("[multiple] perf: simplify chunkqueue_get_memory()"), with its 2 fixes, it's between 4012K and 51156K (it's actually the whole file size)
It seems that the whole file is mapped in lighttpd memory.
Updated by gstrauss over 5 years ago
Would you please test with commit c83fff1d (on lighttpd git master branch) which is already slated to be part of lighttpd 1.4.54?
(As an aside, please do not set the target version when filing a bug report. You do not have enough information to accurately set that)
Updated by rgenoud over 5 years ago
Tested with commit c83fff1d, it's the same, lighttpd uses 50MB in memory
(sorry for setting the target version, I don't know why I clicked on that! :) )
Updated by gstrauss over 5 years ago
- Target version changed from 1.4.54 to 1.4.x
Sorry rgenoud, this is not easy to reproduce on a typical system. Please summarize your setup and try to provide more information (e.g. an strace) and the specs and speed of the embedded (or tiny) system on which your are running. 32-bit or 64-bit? architecture? openssl? CGI or FastCGI?
I was unable to reproduce this with and without mod_openssl, with CGI or with FastCGI on a 3 year old x86_64 laptop (64-bit lighttpd)
I created a 50 MB file for upload, a simple target PHP for the POST request, and made requests withcurl -k -d /dev/shm/50M-file https://localhost:8080/index.php
.
server.stream-response-body = 2 server.stream-request-body = 2 server.chunkqueue-chunk-sz = 1024
Please verify that you have not set ssl.read-ahead = "enable". The default is disabled, and should be disabled on slow systems.
Updated by gstrauss over 5 years ago
- Status changed from New to Patch Pending
- Target version changed from 1.4.x to 1.4.54
In reusing buffers, the code looked at the first entry on an (unsorted) linked list and if the chunk at the head of the linked list of reusable chunks was not large enough, a new chunk would be allocated. With small response headers and small chunks being sent to backend combined with large request body which filled kernel socket buffers could lead to new chunks being allocated and added to the reusable linked list, but ending up behind a small chunk at the head of the linked list.
I have commited a fix on my development branch. https://git.lighttpd.net/lighttpd/lighttpd1.4.git/commit/?h=personal/gstrauss/master&id=0e62fb8939ab62617b99ec6b7e44966e026e424b
Updated by rgenoud over 5 years ago
Tested with commit commit:0e62fb893 , the memory usage is back to normal.
Thanks a lot !
Updated by gstrauss over 5 years ago
- Status changed from Patch Pending to Fixed
- % Done changed from 0 to 100
Applied in changeset 599b4f05c8ccebd0c06074972824b930afc9c832.
Also available in: Atom