Project

General

Profile

Actions

Bug #1665

closed

file uploads >= 700mb

Added by Anonymous over 16 years ago. Updated over 8 years ago.

Status:
Obsolete
Priority:
High
Category:
core
Target version:
ASK QUESTIONS IN Forums:

Description

I am trying to upload some files via http forms. A 400mb file worked fine but a 700mb file failed. Lighty splits the data but then doesn't put them together correct. After putting 300 or 400mb together lighty passes the file to a php script, which outputs that the file is only partially uploaded. (http://de2.php.net/manual/en/features.file-upload.errors.php)


Files

error_log.txt (1.43 KB) error_log.txt -- nimda.tm Anonymous, 2008-07-11 12:25
php_strace.txt (718 Bytes) php_strace.txt -- nimda.tm Anonymous, 2008-07-11 12:25
Actions #1

Updated by Anonymous over 16 years ago

Confirmed. 341,57Mb is the limit. 1,5 months passed.

Actions #2

Updated by stbuehler over 16 years ago

perhaps your disk was full?

Actions #3

Updated by Anonymous over 16 years ago

No, disk space is ok. I'm getting "Partially uploaded" error in php very often on big (>400Mb) files. What's interesting is that FF seem to detect the broken upload and tries to reupload again and again in background in which case I see many "Partially uploaded" errors.

Actions #4

Updated by Anonymous over 16 years ago

Can confirm that bug. On IE same error, FF tries more times to upload same file.

#1542 (keepalive off) not helped.

-- nimda.tm

Actions #5

Updated by Anonymous over 16 years ago

on version r1922 seams like php is waiting for additional data 3 times. When not get any data connection closed from php side.

attached php strace log and debug info.

-- nimda.tm

Actions #6

Updated by Anonymous over 16 years ago

I did a little investigation with a tool called tcptunnel which is a simple tunnel software written in pure perl. I have a spawn-fcgi process running on port 10260 and fastcgi backend configured in lighty to connect to port 1026 which is where the tunnel resides. I'm running tcptunnel like this:


./tcptunnel -srv 172.16.10.30 -srvport 10260 -addr localhost -port 1026 -verbose 1 -keepalive > upload.big
The results:
  • for small files everything is ok:

<CLIENT>
./tcptunnel 20611: client closed connection at Mon Jul 14 18:45:20 2008
./tcptunnel 20611: server started on port 1026 at Mon Jul 14 18:45:20 2008
./tcptunnel 20611: connection from localhost [ 127.0.0.1 ] at port 48812 at Mon Jul 14 18:45:29 2008
./tcptunnel 20611: connecting to 172.16.10.30 at Mon Jul 14 18:45:29 2008
./tcptunnel 20611: connected at Mon Jul 14 18:45:29 2008

<CLIENT>
HERE FASTCGI PROTOCOL DATA AND BINARY CRAP

-----------------------------10804146851834620772078975831^M
Content-Disposition: form-data; name="max_file_size"^M
^M
734003200^M
-----------------------------10804146851834620772078975831^M
Content-Disposition: form-data; name="userfile[1]"; filename=""^M
Content-Type: application/octet-stream^M
^M
^M
-----------------------------10804146851834620772078975831^M
Content-Disposition: form-data; name="userfile[0]"; filename=".viminfo"^M
Content-Type: application/octet-stream^M

FILE CONTENTS 
<CLIENT>
^M
-----------------------------10804146851834620772078975831--^M
^A^E^@^A^@^@^@^@
<SERVER>
^A^F^@^A^B?^E^@X-Powered-By: PHP/5.2.6RC4-pl0-gentoo^M
Expires: Thu, 19 Nov 1981 08:52:00 GMT^M
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0^M
Pragma: no-cache^M
Content-type: text/html; charset=utf-8
  • for big files (uploading a movie ~650 Mb) gives me the same annoying "partially uploaded" problem and I see in tcptunnel logs smth like:

HERE FASTCGI PROTOCOL DATA AND BINARY CRAP
BINARY FILE CONTENTS 
<CLIENT>
./tcptunnel 20500: client closed connection at Mon Jul 14 18:31:43 2008
./tcptunnel 20500: server started on port 1026 at Mon Jul 14 18:31:43 2008
./tcptunnel 20500: connection from localhost [ 127.0.0.1 ] at port 45061 at Mon Jul 14 18:32:11 2008
./tcptunnel 20500: connecting to 172.16.10.30 at Mon Jul 14 18:32:11 2008
./tcptunnel 20500: connected at Mon Jul 14 18:32:11 2008

It seems that lighty just drops the connection while sending big files to php thru socket somewhere in the middle. The worst is that it happens often but not always.

Sorry, I can't attach these logs to ticket since they are huge, but I hope I gave you the test case.

Actions #7

Updated by Anonymous over 16 years ago

Trying to deeper debug of the issue, I've found that it depends on "server.network-backend" configuration setting. I've tried "gthread-aio" and "linux-aio-sendfile" with the same success, but when I set it to "write" everything works just fine -- tested with ~700Mb files, none partial upload error anymore. That would be a clue. Thank you.

Actions #8

Updated by stbuehler over 16 years ago

linux-aio* backends are disabled in recent versions. So perhaps you should update?

Actions #9

Updated by Anonymous over 16 years ago

Thank you for response, but what backend do you recommend me to try? I'm using r1992.

Actions #10

Updated by gstrauss over 8 years ago

  • Description updated (diff)
  • Assignee deleted (jan)
  • Priority changed from Urgent to High
  • Missing in 1.5.x set to Yes
Actions #11

Updated by gstrauss over 8 years ago

  • Status changed from New to Obsolete
Actions

Also available in: Atom