https://redmine.lighttpd.net/https://redmine.lighttpd.net/favicon.ico?13667327412016-02-09T12:43:41Zlighty labsLighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=86892016-02-09T12:43:41Zgstrauss
<ul></ul><p>The mmap() man page on Linux states that mmap() will return EINVAL for one of the following:<br /><pre>
EINVAL We don't like addr, length, or offset (e.g., they are too large,
or not aligned on a page boundary).
EINVAL (since Linux 2.6.12) length was 0.
EINVAL flags contained neither MAP_PRIVATE or MAP_SHARED, or contained
both of these values.
</pre></p>
<p>Since you mentioned that you have a custom build ("home rolled buildroot linux distro"), I would recommend the first place to check is config.h at the top of the lighttpd source tree. If HAVE_SYS_MMAN_H <strong>is not</strong> defined, then lighttpd src/sys-mmap.h will #define MAP_SHARED 0, which will result in EINVAL from the mmap() call, since a non-zero value of MAP_PRIVATE or MAP_SHARED was not passed to mmap().</p>
<p>Then again, src/mod_cgi.c does not #include "sys-mmap.h" -- inconsistency, sigh -- so maybe my suggestion is a red herring. You might still choose to temporarily modify line 780 of mod_cgi.c to print out PROT_READ and MAP_SHARED values, since the other values are already included in the error message you provided above.</p>
<pre>
log_error_write(srv, __FILE__, __LINE__, "ssbdoodd", "mmap failed:",
strerror(errno), c->file.name, c->file.fd, c->file.mmap.offset, (off_t) c->file.mmap.length,
PROT_READ, MAP_SHARED);
</pre> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=86922016-02-09T19:25:14ZDan
<ul></ul><p>Hello,</p>
<p>Thank you for looking into this.</p>
<p>In config.h I have:</p>
<pre>
#define HAVE_SYS_MMAN_H 1
</pre>
<p>I added the extra items (PROT_READ and MAP_SHARED) to the error message on line 780 as suggested. I also added the text -new so I could tell for sure my changes made it into the mod_cgi.so file. I was expecting to see something added to the end of the output. Below is a snippet from the log file, it is the same as before, except for my -new text.</p>
<pre>
2016-02-09 11:01:46: (log.c.194) server started
2016-02-09 11:09:30: (mod_cgi.c.780) mmap failed-new: Invalid argument /data/lighttpd-upload-3j2k3x 8 0 80192
</pre>
<p>Let me know if I should try something else.</p>
<p>Thank you,<br />Dan Pattison</p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=86932016-02-09T19:39:52Zpatrickdk
<ul></ul><p>Did you forget to add dd to the "ssbdoodd" string, if so, those extra params are just ignored.</p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=86942016-02-09T20:21:08ZDan
<ul></ul><p>Hello,</p>
<p>Yes, I did not notice that (shame). Below is another snippet from the log file.</p>
<pre>
2016-02-09 12:10:43: (log.c.194) server started
2016-02-09 12:11:15: (mod_cgi.c.780) mmap failed-newest: Invalid argument /data/lighttpd-upload-YXMNQo 8 0 80186 1 1
</pre>
<p>Thank you,<br />Dan Pattison</p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=86952016-02-09T20:33:38Zgstrauss
<ul></ul><p>Those values look correct. The offset must be page-aligned, and 0 is page-aligned. The values for PROT_READ and MAP_SHARED are both 1 on Linux. File-size is non-zero. The fd is 8, and should have failed a few lines above in network_open_file_chunk() if it failed to open the file. Something is weird since those arguments all look correct.</p>
<p>Try adding this directly <strong>above</strong> the call to mmap() on line 779 in mod_cgi.c:<br /><pre>
struct stat st;
if (0 != fstat(c->file.fd, &st))
log_error_write(srv, __FILE__, __LINE__, "ssbd", "fstat failed:",
strerror(errno), c->file.name, c->file.fd);
errno = ENOSYS; /* something to see if the library function mmap() is actually being called */
</pre></p>
<p>As an alternative, would you strace the server pid and post the strace output line for the failing mmap() call?</p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=86962016-02-09T20:45:03Zgstrauss
<ul></ul><p>If the fstat() succceeds, then it is probably worth looking at what fstat() reports as the current file size:<br /><pre>
struct stat st;
if (0 != fstat(c->file.fd, &st))
log_error_write(srv, __FILE__, __LINE__, "ssbd", "fstat failed:",
strerror(errno), c->file.name, c->file.fd);
log_error_write(srv, __FILE__, __LINE__, "so", "fstat succeeded. CGI input file size:", st.st_size);
errno = ENOSYS; /* something to see if the library function mmap() is actually being called */
</pre></p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=86972016-02-09T20:46:56Zstbuehler
<ul></ul><p>Although the <code>mmap</code> man page says the error code would be <code>ENODEV</code> instead of <code>EINVAL</code> I think it's is possible the underlying filesystem doesn't support mmap; and <code>mod_cgi</code> wouldn't handle it anyway.</p>
<p>The (temp) files sent to the cgi backend shouldn't get smaller - they are owned by lighttpd.</p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=86992016-02-09T21:04:19ZDan
<ul></ul><p>Hello,</p>
<p>I added that extra code, but it did not fire.<br /><pre>
2016-02-09 12:58:49: (log.c.194) server started
2016-02-09 13:00:53: (mod_cgi.c.786) mmap failed-newest1: Invalid argument /data/lighttpd-upload-wOgiCO 8 0 80186 1 1
</pre></p>
<p>More info if it helps<br /><pre>
[root@RMS-300 /root]# uname -a
Linux RMS-300 2.6.33 #47 Tue Sep 8 23:45:56 PDT 2015 armv5tejl GNU/Linux
</pre></p>
<p>Thanks,</p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=87002016-02-09T21:37:03ZDan
<ul></ul><p>Hello,</p>
<p>Newest output.</p>
<pre>
2016-02-09 13:32:46: (log.c.194) server started
2016-02-09 13:33:39: (mod_cgi.c.783) fstat succeeded. CGI input file size: 80189
2016-02-09 13:33:39: (mod_cgi.c.787) mmap failed-newest2: Invalid argument /data/lighttpd-upload-I3JN6J 8 0 80189 1 1
</pre>
<p>Thanks,</p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=87012016-02-09T21:42:48Zgstrauss
<ul></ul><p>stbuehler is probably on to something with his comment about the filesystem supporting mmap().</p>
<p>Would you try changing the following in your lighttpd.conf:<br /><pre>
server.upload-dirs = "/dev/shm"
</pre></p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=87022016-02-09T21:55:18ZDan
<ul></ul><p>Hello,</p>
<p>OK, tried that. The webserver will not start with that config file change.</p>
<pre>
Feb 9 13:51:46 RMS-300 respawnd[773]: Child pid 896 exited with status 255.
Feb 9 13:51:49 RMS-300 respawnd[773]: Child pid 897 exited with status 255.
Feb 9 13:51:52 RMS-300 respawnd[773]: Child pid 898 exited with status 255.
Feb 9 13:51:55 RMS-300 respawnd[773]: Child pid 899 exited with status 255.
Feb 9 13:51:58 RMS-300 respawnd[773]: Child pid 900 exited with status 255.
Feb 9 13:52:01 RMS-300 respawnd[773]: Child pid 908 exited with status 255.
Feb 9 13:52:04 RMS-300 respawnd[773]: Child pid 910 exited with status 255.
</pre>
<p>Thanks,</p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=87032016-02-09T21:57:46Zgstrauss
<ul></ul><p>What is the current value of server.upload-dirs? "/data" ? Try:<br /><pre>
server.upload-dirs = "/tmp"
or
server.upload-dirs = "/var/tmp"
</pre></p>
<p>Of course, make sure the target directory exists.</p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=87042016-02-09T21:59:45Zgstrauss
<ul></ul><p>Of course, make sure the target directory exists <strong>and</strong> that the user the server is running under (e.g. 'lighttpd') has write access to that location.</p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=87052016-02-09T22:17:33ZDan
<ul></ul><p>Hello,</p>
<p>I figured out what the issue was.</p>
<pre>
2016-02-09 14:00:55: (configfile-glue.c.75) server.upload-dirs should have been a array of strings like ... = ( "..." )
</pre>
<p>Fixed the above error, and now it works.</p>
<p>Web output from my custom board.</p>
<pre>
RMS-300 Database Restore Operation
Recieved FILENAME: rms300(1).db
SIZE: 79872 bytes
Saving File...
File saved successfully as /tmp/rms300.db.new
Validating uploaded file....
Current Database Version: 2
Imported Database Version: 2
873 831 [OK]
Stopping Dependant Services...
Stopping RMSD...
Stopping snmpD...
Stopping Dependant Services...[OK]
Move new Sqlite3 database... [OK]
Restarting Dependant Services...
Starting RMSD...
Starting snmpD...
Restarting Dependant Services...[OK]
Database Restore Operation Successfull!
</pre>
<p>I need this to work at /data. Is there some file permission or something that I do not have set right? Dev and Data are both set the same though.<br /><pre>
drwxr-xr-x 6 root root 0 Dec 31 1969 data
drwxr-xr-x 8 root root 0 Feb 9 13:53 dev
</pre></p>
<p>Thanks,</p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=87062016-02-09T22:21:53Zgstrauss
<ul></ul><p>Sorry that I gave you incorrect syntax.</p>
<p>What are the underlying filesystem types of /data versus /dev/shm?<br /><pre>
df -h /dev/shm /data
mount | egrep "^(/data|/dev/shm)"
</pre></p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=87072016-02-09T22:27:20Zgstrauss
<ul></ul><p>Also, would you check those permissions on the directories? /data and /dev/shm<br />Note: /dev/shm is a different mount from that of /dev.</p>
<p>What is the output of the following?<br /><pre>
ls -ld /data/ /dev/shm/
</pre></p>
<p>What is the user that lighttpd is running as?</p>
<p>A non-root user won't be able to write to /data with these permissions:<br />drwxr-xr-x 6 root root 0 Dec 31 1969 data<br />drwxr-xr-x 8 root root 0 Feb 9 13:53 dev</p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=87082016-02-09T22:38:26ZDan
<ul></ul><p>Hello,</p>
<p>Lighttpd is running as user root on an embedded arm board. /data is in a separate flash partition.</p>
<pre>
# /etc/fstab: static file system information.
#
# <file system> <mount pt> <type> <options> <dump> <pass>
/dev/mtdblock3 /data jffs2 rw,noatime 0 0
proc /proc proc defaults 0 0
devpts /dev/pts devpts defaults,gid=5,mode=620 0 0
tmpfs /tmp tmpfs defaults 0 0
tmpfs /var tmpfs defaults 0 0
sysfs /sys sysfs defaults 0 0
usbfs /proc/bus/usb usbfs defaults
</pre>
<pre>
[root@RMS-300 /dev]# ls -ld /data/ /dev/shm/
drwxr-xr-x 6 root root 0 Dec 31 1969 /data/
drwxr-xr-x 2 root root 0 Feb 9 14:02 /dev/shm/
</pre>
<p>Thanks,</p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=87092016-02-09T23:15:32Zgstrauss
<ul></ul><p><a class="external" href="https://sourceware.org/bugzilla/show_bug.cgi?id=5033">https://sourceware.org/bugzilla/show_bug.cgi?id=5033</a> notes that JFFS2 does not support mmap() MAP_SHARED. Perhaps you could try MAP_PRIVATE instead of MAP_SHARED? (Sorry, I don't have a system on which to try this myself.)</p>
<p>Internet searches turn up warnings about wearing out flash more quickly if using mmap() for <strong>writable</strong> maps on flash file systems, but the use here in mod_cgi is read-only so if it works with MAP_PRIVATE, that should be fine, as the file is a temporary file specific to the request.</p>
<p><a class="external" href="https://lkml.org/lkml/2015/9/3/66">https://lkml.org/lkml/2015/9/3/66</a> also suggests a possible solution using MAP_PRIVATE with perf.</p>
<p>If this works for you, perhaps mod_cgi.c can be modified to use MAP_PRIVATE.</p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=87102016-02-10T03:29:33ZDan
<ul></ul><p>Hello,</p>
<p>MAP_PRIVATE fixed it. Previous versions of lighttpd worked for me but I notice it was using 0_RDONLY then and not MAP_SHARED. My update program was broken in 1.4.39 but worked in 1.4.35. I can patch future versions using the hack-by-hand method unless you make that a permenant change.</p>
<p>I want to thank you guys (gstrauss and stbuehler) for helping me with this. I believe you both have PHDs in awesomicity! I left a little something in your PayPal tip jar for the effort.</p>
<p>Thank you!<br />Dan Pattison</p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=87132016-02-10T08:06:55Zgstrauss
<ul></ul><p>Glad to hear it works!</p>
<p>I submitted a pull request with a patch: <a class="external" href="https://github.com/lighttpd/lighttpd1.4/pull/16">https://github.com/lighttpd/lighttpd1.4/pull/16</a></p> Lighttpd - Bug #2715: mmap error while uploading file in 1.4.39https://redmine.lighttpd.net/issues/2715?journal_id=87202016-02-10T19:35:05Zstbuehler
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Fixed</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li></ul><p>Applied in changeset r3075.</p>