Lighttpd

The mmap() man page on Linux states that mmap() will return EINVAL for one of the following:

       EINVAL We don't like addr, length, or offset (e.g., they are too large,
              or not aligned on a page boundary).

       EINVAL (since Linux 2.6.12) length was 0.

       EINVAL flags  contained neither MAP_PRIVATE or MAP_SHARED, or contained
              both of these values.

Since you mentioned that you have a custom build ("home rolled buildroot linux distro"), I would recommend the first place to check is config.h at the top of the lighttpd source tree. If HAVE_SYS_MMAN_H is not defined, then lighttpd src/sys-mmap.h will #define MAP_SHARED 0, which will result in EINVAL from the mmap() call, since a non-zero value of MAP_PRIVATE or MAP_SHARED was not passed to mmap().

Then again, src/mod_cgi.c does not #include "sys-mmap.h" -- inconsistency, sigh -- so maybe my suggestion is a red herring. You might still choose to temporarily modify line 780 of mod_cgi.c to print out PROT_READ and MAP_SHARED values, since the other values are already included in the error message you provided above.

    log_error_write(srv, __FILE__, __LINE__, "ssbdoodd", "mmap failed:",
                    strerror(errno), c->file.name, c->file.fd, c->file.mmap.offset, (off_t) c->file.mmap.length,
                    PROT_READ, MAP_SHARED);

Hello,

Thank you for looking into this.

In config.h I have:

#define HAVE_SYS_MMAN_H 1

I added the extra items (PROT_READ and MAP_SHARED) to the error message on line 780 as suggested. I also added the text -new so I could tell for sure my changes made it into the mod_cgi.so file. I was expecting to see something added to the end of the output. Below is a snippet from the log file, it is the same as before, except for my -new text.

2016-02-09 11:01:46: (log.c.194) server started 
2016-02-09 11:09:30: (mod_cgi.c.780) mmap failed-new: Invalid argument /data/lighttpd-upload-3j2k3x 8 0 80192 

Let me know if I should try something else.

Thank you,
Dan Pattison

Did you forget to add dd to the "ssbdoodd" string, if so, those extra params are just ignored.

Hello,

Yes, I did not notice that (shame). Below is another snippet from the log file.

2016-02-09 12:10:43: (log.c.194) server started 
2016-02-09 12:11:15: (mod_cgi.c.780) mmap failed-newest: Invalid argument /data/lighttpd-upload-YXMNQo 8 0 80186 1 1 

Thank you,
Dan Pattison

Those values look correct. The offset must be page-aligned, and 0 is page-aligned. The values for PROT_READ and MAP_SHARED are both 1 on Linux. File-size is non-zero. The fd is 8, and should have failed a few lines above in network_open_file_chunk() if it failed to open the file. Something is weird since those arguments all look correct.

Try adding this directly above the call to mmap() on line 779 in mod_cgi.c:

    struct stat st;
    if (0 != fstat(c->file.fd, &st))
       log_error_write(srv, __FILE__, __LINE__, "ssbd", "fstat failed:",
                       strerror(errno), c->file.name, c->file.fd);
    errno = ENOSYS;  /* something to see if the library function mmap() is actually being called */

As an alternative, would you strace the server pid and post the strace output line for the failing mmap() call?

If the fstat() succceeds, then it is probably worth looking at what fstat() reports as the current file size:

    struct stat st;
    if (0 != fstat(c->file.fd, &st))
       log_error_write(srv, __FILE__, __LINE__, "ssbd", "fstat failed:",
                       strerror(errno), c->file.name, c->file.fd);
    log_error_write(srv, __FILE__, __LINE__, "so", "fstat succeeded.  CGI input file size:", st.st_size);
    errno = ENOSYS;  /* something to see if the library function mmap() is actually being called */

Although the mmap man page says the error code would be ENODEV instead of EINVAL I think it's is possible the underlying filesystem doesn't support mmap; and mod_cgi wouldn't handle it anyway.

The (temp) files sent to the cgi backend shouldn't get smaller - they are owned by lighttpd.

Hello,

I added that extra code, but it did not fire.

2016-02-09 12:58:49: (log.c.194) server started 
2016-02-09 13:00:53: (mod_cgi.c.786) mmap failed-newest1: Invalid argument /data/lighttpd-upload-wOgiCO 8 0 80186 1 1 

More info if it helps

[root@RMS-300 /root]# uname -a
Linux RMS-300 2.6.33 #47 Tue Sep 8 23:45:56 PDT 2015 armv5tejl GNU/Linux

Thanks,

Hello,

Newest output.

2016-02-09 13:32:46: (log.c.194) server started 
2016-02-09 13:33:39: (mod_cgi.c.783) fstat succeeded.  CGI input file size: 80189 
2016-02-09 13:33:39: (mod_cgi.c.787) mmap failed-newest2: Invalid argument /data/lighttpd-upload-I3JN6J 8 0 80189 1 1 

Thanks,

stbuehler is probably on to something with his comment about the filesystem supporting mmap().

Would you try changing the following in your lighttpd.conf:

  server.upload-dirs = "/dev/shm" 

Hello,

OK, tried that. The webserver will not start with that config file change.

Feb  9 13:51:46 RMS-300 respawnd[773]: Child pid 896 exited with status 255. 
Feb  9 13:51:49 RMS-300 respawnd[773]: Child pid 897 exited with status 255. 
Feb  9 13:51:52 RMS-300 respawnd[773]: Child pid 898 exited with status 255. 
Feb  9 13:51:55 RMS-300 respawnd[773]: Child pid 899 exited with status 255. 
Feb  9 13:51:58 RMS-300 respawnd[773]: Child pid 900 exited with status 255. 
Feb  9 13:52:01 RMS-300 respawnd[773]: Child pid 908 exited with status 255. 
Feb  9 13:52:04 RMS-300 respawnd[773]: Child pid 910 exited with status 255. 

Thanks,

What is the current value of server.upload-dirs? "/data" ? Try:

    server.upload-dirs = "/tmp" 
or
    server.upload-dirs = "/var/tmp" 

Of course, make sure the target directory exists.

Of course, make sure the target directory exists and that the user the server is running under (e.g. 'lighttpd') has write access to that location.

Hello,

I figured out what the issue was.

2016-02-09 14:00:55: (configfile-glue.c.75) server.upload-dirs should have been a array of strings like ... = ( "..." ) 

Fixed the above error, and now it works.

Web output from my custom board.

RMS-300 Database Restore Operation
Recieved FILENAME: rms300(1).db
SIZE: 79872 bytes

Saving File...
File saved successfully as /tmp/rms300.db.new
Validating uploaded file....

Current Database Version: 2
Imported Database Version: 2

873 831 [OK]

Stopping Dependant Services...
  Stopping RMSD...
  Stopping snmpD...
Stopping Dependant Services...[OK]

Move new Sqlite3 database... [OK]

Restarting Dependant Services...
  Starting RMSD...
  Starting snmpD...
Restarting Dependant Services...[OK]

Database Restore Operation Successfull!

I need this to work at /data. Is there some file permission or something that I do not have set right? Dev and Data are both set the same though.

drwxr-xr-x    6 root     root            0 Dec 31  1969 data
drwxr-xr-x    8 root     root            0 Feb  9 13:53 dev

Thanks,

Sorry that I gave you incorrect syntax.

What are the underlying filesystem types of /data versus /dev/shm?

    df -h /dev/shm /data
    mount | egrep "^(/data|/dev/shm)" 

Also, would you check those permissions on the directories? /data and /dev/shm
Note: /dev/shm is a different mount from that of /dev.

What is the output of the following?

    ls -ld /data/ /dev/shm/

What is the user that lighttpd is running as?

A non-root user won't be able to write to /data with these permissions:
drwxr-xr-x 6 root root 0 Dec 31 1969 data
drwxr-xr-x 8 root root 0 Feb 9 13:53 dev

Hello,

Lighttpd is running as user root on an embedded arm board. /data is in a separate flash partition.

# /etc/fstab: static file system information.
#
# <file system>    <mount pt>    <type>    <options>               <dump> <pass>
/dev/mtdblock3    /data        jffs2    rw,noatime            0    0
proc        /proc        proc    defaults            0    0
devpts        /dev/pts    devpts    defaults,gid=5,mode=620        0    0
tmpfs        /tmp        tmpfs    defaults            0    0
tmpfs        /var        tmpfs    defaults            0    0
sysfs        /sys        sysfs    defaults            0    0
usbfs        /proc/bus/usb    usbfs    defaults    

[root@RMS-300 /dev]# ls -ld /data/ /dev/shm/
drwxr-xr-x    6 root     root            0 Dec 31  1969 /data/
drwxr-xr-x    2 root     root            0 Feb  9 14:02 /dev/shm/

Thanks,

https://sourceware.org/bugzilla/show_bug.cgi?id=5033 notes that JFFS2 does not support mmap() MAP_SHARED. Perhaps you could try MAP_PRIVATE instead of MAP_SHARED? (Sorry, I don't have a system on which to try this myself.)

Internet searches turn up warnings about wearing out flash more quickly if using mmap() for writable maps on flash file systems, but the use here in mod_cgi is read-only so if it works with MAP_PRIVATE, that should be fine, as the file is a temporary file specific to the request.

https://lkml.org/lkml/2015/9/3/66 also suggests a possible solution using MAP_PRIVATE with perf.

If this works for you, perhaps mod_cgi.c can be modified to use MAP_PRIVATE.

Hello,

MAP_PRIVATE fixed it. Previous versions of lighttpd worked for me but I notice it was using 0_RDONLY then and not MAP_SHARED. My update program was broken in 1.4.39 but worked in 1.4.35. I can patch future versions using the hack-by-hand method unless you make that a permenant change.

I want to thank you guys (gstrauss and stbuehler) for helping me with this. I believe you both have PHDs in awesomicity! I left a little something in your PayPal tip jar for the effort.

Thank you!
Dan Pattison

Glad to hear it works!

I submitted a pull request with a patch: https://github.com/lighttpd/lighttpd1.4/pull/16