Project

General

Profile

Bug #2948

[regression][Bisected] lighttpd uses way more memory with POST since 1.4.52

Added by rgenoud 1 day ago. Updated about 8 hours ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
core
Target version:
Start date:
2019-04-17
Due date:
% Done:

0%

Estimated time:
Missing in 1.5.x:

Description

Since commit 88ee73d0a216 ("[multiple] perf: simplify chunkqueue_get_memory()")
uploading a big file via POST uses a lot more memory than in v1.4.51

It's actually the next part of #2922.
At the time, I didn't pay attention to the memory used by the lighttpd process.

I did my tests onto a1077d18cb36 ("[tests] more test config cleanup")
and then track down the memory problem on commit 88ee73d0a216, even with patches:
fe3dc1796894 ("[mod_fastcgi] fix NULL ptr deref from bugfix #2922 (fixes #2923)")
a1b527e47374 ("[multiple] reduce initial buffer sz if large POST (fixes #2922)")

I've used the same config as in #2922 with:
server.stream-request-body = 2
server.stream-response-body = 2
server.chunkqueue-chunk-sz = 1024

To highlight the high memory usage, here's simple index.php:
if(isset($_FILES['image'])){
$errors= array();
$file_name = $_FILES['image']['name'];
$file_size =$_FILES['image']['size'];
$file_tmp =$_FILES['image']['tmp_name'];
$file_type=$_FILES['image']['type'];
$file_ext=strtolower(end(explode('.',$_FILES['image']['name'])));
if(empty($errors)==true){
echo "Success";
}else{
print_r($errors);
}
}
?>
<html>
<body>
<form action="" method="POST" enctype="multipart/form-data">
<input type="file" name="image" />
<input type="submit"/>
</form>
</body>
</html>

and, on the server side :
top -b -d 1 | grep lighttpd

Upload a 50MB file, and watch the memory used by lighttpd grow.

at version 1.4.51, the memory used by lighttpd was between 4012K and 4528K during the upload.

since commit fccc7fc607ff ("[core] perf: chunk.c chunk pool") it's between 4012K and 7856K

and since commit 88ee73d0a216 ("[multiple] perf: simplify chunkqueue_get_memory()"), with its 2 fixes, it's between 4012K and 51156K (it's actually the whole file size)

It seems that the whole file is mapped in lighttpd memory.

History

#1

Updated by gstrauss about 14 hours ago

Would you please test with commit c83fff1d (on lighttpd git master branch) which is already slated to be part of lighttpd 1.4.54?

(As an aside, please do not set the target version when filing a bug report. You do not have enough information to accurately set that)

#2

Updated by rgenoud about 8 hours ago

Tested with commit c83fff1d, it's the same, lighttpd uses 50MB in memory

(sorry for setting the target version, I don't know why I clicked on that! :) )

Also available in: Atom