Project

General

Profile

[Solved] cross-compilation for blackfin processor

Added by msink over 1 year ago

On a embedded device we use lighttpd for configuration web inteface,
target OS - uClinux for blackfin, host OS - Linux.
Version 1.4.32 works perfectly, but now we were asked to upgrade lighttpd for security reason.

Working Makefile for 1.4.32:

LIB = lighttpd
VER = 1.4.32
DIR = $(LIB)-$(VER)
TGZ = $(LIB)-$(VER).tar.gz
URL = http://download.lighttpd.net/lighttpd/releases-1.4.x

EXTRA_CFLAGS += -g

all: compile

$(TGZ):
    wget $(URL)/$(TGZ)

$(DIR)/.unpacked: $(TGZ) Makefile
    tar xzf $(TGZ)
    patch -d $(DIR) -u -p1 < mmap.patch
    patch -d $(DIR) -u -p1 < lighttpd-angel.patch
    touch $@

$(DIR)/.configured: $(DIR)/.unpacked
    cd $(DIR) && ./configure \
        --host=$(CONFIGURE_HOST) \
        --build=$(CONFIGURE_BUILD) \
        --prefix=$(STAGEDIR)/usr \
        --libdir=$(STAGEDIR)/usr/lib \
        --includedir=$(STAGEDIR)/usr/include \
        --mandir=$(STAGEDIR)/usr/share/man \
          --without-libicu \
          --without-openssl\
        --disable-ipv6 \
        --without-bzip2 \
        --without-pcre
    echo "#define HAVE_LIBPCRE 1" >> $(DIR)/config.h
    echo "#define HAVE_PCRE_H 1" >> $(DIR)/config.h
    echo "#define HAVE_FORK 1" >> $(DIR)/config.h
    echo "#define fork vfork" >> $(DIR)/config.h
    sed -i "s/^LDFLAGS =/LDFLAGS = -lpcre /" $(DIR)/src/Makefile
    touch $@

$(DIR)/src/lighttpd: $(DIR)/.configured
    $(MAKE) -C $(DIR) all

$(STAGEDIR)/usr/bin/lighttpd: $(DIR)/src/lighttpd
    $(MAKE) -C $(DIR) install

compile: $(STAGEDIR)/usr/bin/lighttpd

install romfs:
    $(ROMFSINST) $(STAGEDIR)/usr/sbin/lighttpd /usr/bin/lighttpd
    $(ROMFSINST) $(STAGEDIR)/usr/sbin/lighttpd-angel /usr/bin/lighttpd-angel
    $(ROMFSINST) $(STAGEDIR)/usr/lib/mod_rewrite.so /usr/lib/mod_rewrite.so
    $(ROMFSINST) $(STAGEDIR)/usr/lib/mod_access.so /usr/lib/mod_access.so
    $(ROMFSINST) $(STAGEDIR)/usr/lib/mod_fastcgi.so /usr/lib/mod_fastcgi.so
    $(ROMFSINST) $(STAGEDIR)/usr/lib/mod_cgi.so /usr/lib/mod_cgi.so
    $(ROMFSINST) $(STAGEDIR)/usr/lib/mod_indexfile.so /usr/lib/mod_indexfile.so
    $(ROMFSINST) $(STAGEDIR)/usr/lib/mod_staticfile.so /usr/lib/mod_staticfile.so
    $(ROMFSINST) $(STAGEDIR)/usr/lib/mod_dirlisting.so /usr/lib/mod_dirlisting.so
    $(ROMFSINST) $(STAGEDIR)/usr/lib/mod_expire.so /usr/lib/mod_expire.so
    $(ROMFSINST) $(STAGEDIR)/usr/lib/mod_auth.so /usr/lib/mod_auth.so

diff --git a/libs/lighttpd/Makefile b/libs/lighttpd/Makefile
index c84de18..41413fc 100755
--- a/libs/lighttpd/Makefile
+++ b/libs/lighttpd/Makefile
@@ -1,5 +1,5 @@
 LIB = lighttpd
-VER = 1.4.32
+VER = 1.4.67
 DIR = $(LIB)-$(VER)
 TGZ = $(LIB)-$(VER).tar.gz
 URL = http://download.lighttpd.net/lighttpd/releases-1.4.x
@@ -13,7 +13,7 @@ $(TGZ):

 $(DIR)/.unpacked: $(TGZ) Makefile
     tar xzf $(TGZ)
-    patch -d $(DIR) -u -p1 < mmap.patch
+#    patch -d $(DIR) -u -p1 < mmap.patch
     patch -d $(DIR) -u -p1 < lighttpd-angel.patch
     touch $@

@@ -29,7 +29,8 @@ $(DIR)/.configured: $(DIR)/.unpacked
           --without-openssl\
         --disable-ipv6 \
         --without-bzip2 \
-        --without-pcre
+        --without-pcre \
+        --without-pcre2
     echo "#define HAVE_LIBPCRE 1" >> $(DIR)/config.h
     echo "#define HAVE_PCRE_H 1" >> $(DIR)/config.h
     echo "#define HAVE_FORK 1" >> $(DIR)/config.h

Build failed with message:

t_test_mod-fdevent.o: In function `fdevent_accept_listenfd':
/KD3U-KCP1/libs/lighttpd/lighttpd-1.4.67/src/fdevent.c:293: undefined reference to `_accept4'

Seems that our cross-compilation toolchain does not support relatively new system function accept4
And we cannot upgrade toolchain...

Recomendations?


Replies (18)

RE: cross-compilation for blackfin processor - Added by gstrauss over 1 year ago

Let's be clear: your company is grossly neglectful if you have not upgraded since lighttpd 1.4.32, which was released Nov 2012, over 10 years ago. While lighttpd has not had many security bugs, lighttpd has had some in the past 10 years.

What Linux kernel are you using? What version of uclibc? If you are also running a grossly ancient Linux kernel or grossly ancient uclibc, then accept4() might not be defined and you can use the following patch to disable use of accept4()

--- a/src/fdevent.c
+++ b/src/fdevent.c
@@ -283,7 +283,7 @@ int fdevent_accept_listenfd(int listenfd, struct sockaddr *addr, size_t *addrlen
        int fd;
        socklen_t len = (socklen_t) *addrlen;

-      #if defined(SOCK_CLOEXEC) && defined(SOCK_NONBLOCK)
+      #if defined(SOCK_CLOEXEC) && defined(SOCK_NONBLOCK) && 0
        #if defined(__NetBSD__)
        const int sock_cloexec = 1;
        fd = paccept(listenfd, addr, &len, NULL, SOCK_CLOEXEC | SOCK_NONBLOCK);

RE: cross-compilation for blackfin processor - Added by gstrauss over 1 year ago

According to man accept4 on a modern Linux system:

The accept4() system call is available starting with Linux 2.6.28; support in glibc is available starting with version 2.10.

From a quick search of the source code for uClibc, I found that the git history shows that the accept4 syscall was part of uclibc v0.9.33, released Feb 2012.

RE: cross-compilation for blackfin processor - Added by msink over 1 year ago

Kernel:

$ version
kernel:    Linux release 3.5.7-ADI-2012R2+, build #50 PREEMPT Mon Nov 14 15:30:19 YEKT 2022
toolchain: bfin-linux-uclibc-gcc release gcc version 4.3.5 (ADI-trunk/git-df5aa43)

uClibc:
$ cat /bfin-toolchain/uClibc/.config
#
# Automatically generated make config: don't edit
# Version: 0.9.33.2
# Wed Feb 27 14:08:53 2013
#
...

After applying your path - build passed, but now on each access to any page - instead of rendering page
web browser asks to save it, and saved file is zero size. :(

RE: cross-compilation for blackfin processor - Added by msink over 1 year ago

Mot sure it's related - but for 1.4.32 there was path for disable shared mmap (not availabe on no-mmu processors)

diff --git a/src/network_writev.c b/src/network_writev.c
index 65bb19d..2fa7946 100755
--- a/src/network_writev.c
+++ b/src/network_writev.c
@@ -240,7 +240,7 @@ int network_write_chunkqueue_writev(server *srv, connection *con, int fd, chunkq
 #endif
                 }

-                if (MAP_FAILED == (c->file.mmap.start = mmap(NULL, to_mmap, PROT_READ, MAP_SHARED, c->file.fd, c->file.mmap.offset))) {
+                if (MAP_FAILED == (c->file.mmap.start = mmap(NULL, to_mmap, PROT_READ, MAP_PRIVATE, c->file.fd, c->file.mmap.offset))) {
                     /* close it here, otherwise we'd have to set FD_CLOEXEC */

                     log_error_write(srv, __FILE__, __LINE__, "ssbd", "mmap failed:",
diff --git a/src/stream.c b/src/stream.c
index f91e73c..584a637 100755
--- a/src/stream.c
+++ b/src/stream.c
@@ -34,7 +34,7 @@ int stream_open(stream *f, buffer *fn) {
         return -1;
     }

-    f->start = mmap(NULL, f->size, PROT_READ, MAP_SHARED, fd, 0);
+    f->start = mmap(NULL, f->size, PROT_READ, MAP_PRIVATE, fd, 0);

     close(fd);

RE: cross-compilation for blackfin processor - Added by gstrauss over 1 year ago

If your kernel, uclibc, and lighttpd versions support accept4, then you or someone else must have configured the environment or patched out support for accept4. The error message you reported is
/KD3U-KCP1/libs/lighttpd/lighttpd-1.4.67/src/fdevent.c:293: undefined reference to `_accept4'
and lighttpd did not change accept4 to _accept4, so something else in your environment or toolchain changed that, e.g. headers.

$ version
kernel:    Linux release 3.5.7-ADI-2012R2+, build #50 PREEMPT Mon Nov 14 15:30:19 YEKT 2022
toolchain: bfin-linux-uclibc-gcc release gcc version 4.3.5 (ADI-trunk/git-df5aa43)

Is that the kernel version of your build system or is it the kernel version of the target embedded system?

After applying your path (sic) - build passed, but now on each access to any page - instead of rendering page web browser asks to save it, and saved file is zero size. :(

What steps have you taken to troubleshoot? It sounds like you are guessing. See DebugVariables

Mot sure it's related - but for 1.4.32 there was path for disable shared mmap (not availabe on no-mmu processors)

Regarding mmap, lighttpd does not use mmap if HAVE_MMAP is not defined (config.h), and even then, if mmap returns NULL, lighttpd generally falls back to a non-mmap code path. If you were able to strace a request received by lighttpd, you should be able to see that.

RE: cross-compilation for blackfin processor - Added by msink over 1 year ago

Returning to this issue - so I tried to bisect when my usecase was broken, and failed on first step - compiling 1.4.32 from git:

git clone https://github.com/lighttpd/lighttpd1.4.git && cd lighttpd1.4
git checkout lighttpd-1.4.32
./autogen.sh
./configure \
    --host=$(CONFIGURE_HOST) \
    --build=$(CONFIGURE_BUILD) \
    --prefix=$(STAGEDIR)/usr \
    --libdir=$(STAGEDIR)/usr/lib \
    --includedir=$(STAGEDIR)/usr/include \
    --mandir=$(STAGEDIR)/usr/share/man \
    --without-libicu \
    --without-openssl\
    --disable-ipv6 \
    --without-bzip2
echo "#define HAVE_FORK 1" >> config.h
echo "#define fork vfork" >> config.h
sed -i "s|^PCRE_LIB = -L/usr/local/lib|PCRE_LIB = -L$(STAGEDIR)/usr/lib|" src/Makefile
make all

....
bfin-linux-uclibc-gcc: mod_ssi_exprparser.c: no such file or directory

Why mod_ssi_exprparser.c was not generated here?

RE: cross-compilation for blackfin processor - Added by msink over 1 year ago

Solved, now (since 1.4.64) ./configure requires explicit --with-pcre --without-pcre2 parameters

Works for me (target - uClinux NOMMU):

LIB = lighttpd
VER = 1.4.67
DIR = $(LIB)-$(VER)
TGZ = $(LIB)-$(VER).tar.gz
URL = http://download.lighttpd.net/lighttpd/releases-1.4.x

EXTRA_CFLAGS += -g

all: compile

$(TGZ):
    wget $(URL)/$(TGZ)

$(DIR)/.unpacked: $(TGZ) Makefile
    tar xzf $(TGZ)
    patch -d $(DIR) -u -p1 < no-accept4.patch
    patch -d $(DIR) -u -p1 < lighttpd-angel.patch
    touch $@

$(DIR)/.configured: $(DIR)/.unpacked
    cd $(DIR) && ./configure \
        --host=$(CONFIGURE_HOST) \
        --build=$(CONFIGURE_BUILD) \
        --prefix=$(STAGEDIR)/usr \
        --libdir=$(STAGEDIR)/usr/lib \
        --includedir=$(STAGEDIR)/usr/include \
        --mandir=$(STAGEDIR)/usr/share/man \
        --without-libicu \
        --without-openssl\
        --disable-ipv6 \
        --without-bzip2 \
        --with-pcre \
        --without-pcre2
    echo "#define HAVE_FORK 1" >> $(DIR)/config.h
    echo "#define fork vfork" >> $(DIR)/config.h
    sed -i "s|^PCRE_LIB =|PCRE_LIB = -L$(STAGEDIR)/usr/lib|" $(DIR)/src/Makefile
    touch $@

$(DIR)/src/lighttpd: $(DIR)/.configured
    $(MAKE) -C $(DIR) all

$(STAGEDIR)/usr/bin/lighttpd: $(DIR)/src/lighttpd
    $(MAKE) -C $(DIR) install

compile: $(STAGEDIR)/usr/bin/lighttpd

RE: cross-compilation for blackfin processor - Added by gstrauss over 1 year ago

I hope you learned your lesson that bisecting from an ancient lighttpd 1.4.32 was a waste of time.

PCRE2 should be preferred for all builds.
The original PCRE is end-of-lifed and no longer supported and no longer receiving any security or bug fixes.

This is excess and unnecessary (though otherwise harmless) since you specified --prefix=$(STAGEDIR)/usr \

        --libdir=$(STAGEDIR)/usr/lib \
        --includedir=$(STAGEDIR)/usr/include \
        --mandir=$(STAGEDIR)/usr/share/man \

These are excess and unnecessary (though otherwise mostly harmless) since they are lighttpd build defaults (with modern lighttpd), the exception being if IPv6 is detected by the ./configure script, but you want to disable IPv6 support in your build. (And --without-libicu is meaningless to current lighttpd build config)

        --without-libicu \
        --without-openssl\
        --disable-ipv6 \
        --without-bzip2 \

I don't know your system, but
sed -i "s|^PCRE_LIB =|PCRE_LIB = -L$(STAGEDIR)/usr/lib|" $(DIR)/src/Makefile
might be more portably replaced with
LDFLAGS="-L$(STAGEDIR)/usr/lib" ./configure ...
since it seems that is where you have libraries you want to build against.

RE: cross-compilation for blackfin processor - Added by gstrauss over 1 year ago

The adjustment in fdevent.c for accept4() (3 years ago) in #2998 was apparently not sufficient for the development environment of OP here.

accept4() was (apparently?) added to Linux 2.4.36, even though x86 got accept4() in Linux 2.4.28
https://lwn.net/Articles/789961/

I may add the following patch to lighttpd for kicks

--- a/src/fdevent.c
+++ b/src/fdevent.c
@@ -283,6 +283,20 @@ int fdevent_mkostemp(char *path, int flags) {
 }

+/* accept4() added in Linux x86 in kernel 2.6.28, but not in arm until 2.6.36
+ * https://lwn.net/Articles/789961/ */
+#if defined(__linux__) \
+ && (defined(__arm__) || defined(__thumb__) || defined(__arm64__))
+#ifdef __has_include
+#if __has_include(<sys/syscall.h>)
+#include <sys/syscall.h>
+#endif
+#endif
+#ifndef SYS_accept4
+#define accept4(a,b,c,d) (errno = ENOTSUP, -1)
+#endif
+#endif
+
 int fdevent_accept_listenfd(int listenfd, struct sockaddr *addr, size_t *addrlen) {
        int fd;
        socklen_t len = (socklen_t) *addrlen;

RE: cross-compilation for blackfin processor - Added by msink over 1 year ago

It will not fix situations when function is available in kernel syscalls, but not (yet) available in userspace libc.
As in my case.

RE: cross-compilation for blackfin processor - Added by gstrauss over 1 year ago

It will not fix situations when function is available in kernel syscalls, but not (yet) available in userspace libc.
As in my case.

Nice to know. Some cases are better used examples to others of what not to do.

I do not recommend anyone use your incomplete makefiles with (unshared) patches above.

lighttpd already provides simpler build instructions in InstallFromSource and in the INSTALL file in the source code.

RE: cross-compilation for blackfin processor - Added by msink over 1 year ago

BTW our testers have found new regression when new lighttpd is used - when date/time is changed by child process called from lighttpd fcgi interface - that child process terminates and not restarted.
Not yet resolved.

RE: cross-compilation for blackfin processor - Added by msink over 1 year ago

Correction for above - it happens only when RTC is broken (lithium battery removed).
Old lighttpd works in this situation.

RE: cross-compilation for blackfin processor - Added by gstrauss over 1 year ago

Well, it seems like you haven't read the documentation or searched this site for an answer to those questions.
(Hint: you're not the first embedded system to use lighttpd with a broken clock. You're probably not using full paths to run lighttpd and its config paths.)

Also, since you do not seem to know the etiquette to staying on topic, I won't be responding further.

RE: cross-compilation for blackfin processor - Added by msink over 1 year ago

Well, sorry for not knowing etiquette, it was not intentional.

Where to use full paths, I do not understand.
Our current setup, if needed:

/etc/inittab

kd-oper:unknown:/sbin/nice -n 0 /usr/bin/lighttpd -D -f /etc/lighttpd.conf

/etc/lighttpd.conf

server.document-root="/usr/share/kd-oper-web" 
server.port = 80
server.modules = ( "mod_rewrite", "mod_fastcgi" )

url.rewrite-once = (
    "^/media/([a-zA-Z_0-9\-]+\.[a-z]+)$" 
        => "/?PATH=$1&PAGE=MEDIA",

    "^/$"    => "/?PATH=login&PAGE=login",

    "^/([a-z_0-9]+)/$" 
        => "/?PATH=$1&PAGE=$1",

    "^/([a-z_0-9]+)/\?(.+)$" 
        => "/?PATH=$1&PAGE=$1&$2",

    "^/([a-z_0-9]+)/([0-9]+)/([a-z_0-9]+)/$" 
        => "/?PATH=$1/$2/$3&PAGE=$1_$3&INDEX=$2",

    "^/([a-z_0-9]+)/([0-9]+)/([a-z_0-9]+)/\?(.+)$" 
        => "/?PATH=$1/$2/$3&PAGE=$1_$3&INDEX=$2&$4",
)

fastcgi.server = ( "/" =>
  (( 
    "host" => "127.0.0.1",
    "port" => "666",
    "bin-path" => "/usr/bin/kd-oper-web",
    "min-procs" => 1,
    "max-procs" => 1,
    "max-load-per-proc" => 4,
  ))
)

set_datetime.c

...

static int set_datetime(const char *year,
                        const char *month,
                        const char *day,
                        const char *hour,
                        const char *min,
                        const char *sec)
{
        if (!year || !month || !day || !hour || !min || !sec)
                return -1;

        char command[100];
        snprintf(command, sizeof(command),
                "date -s %s.%s.%s-%s:%s:%s",
                year, month, day, hour, min, sec);
        system(command);

        system("hwclock -wu");

        return 0;
}

...

RE: cross-compilation for blackfin processor - Added by msink 9 months ago

We've tried several methods of respawn - seems that uClinux for Blackfin is buggy in that area.
And finally we solved our problem by just disabling respawn:

server.feature-flags = (
    "server.clock-jump-restart" => 0
}

RE: cross-compilation for blackfin processor - Added by gstrauss 9 months ago

We've tried several methods of respawn - seems that uClinux for Blackfin is buggy in that area.

(note: that is not an issue in lighttpd, but is a huge red flag for ancient software if your system init is buggy!)

If you have a broken clock and disable lighttpd restart when the clock jumps, then you may have a variety of difficulties with TLS libraries, such as openssl used by lighttpd mod_openssl.

Modern lighttpd makes an effort to use the system monotonic clock rather than the realtime clock whenever possible, so lighttpd by itself should generally handle clock jumps gracefully, but when other libraries are involved, such as TLS libraries, those libraries might not handle large clock jumps so well.


With modern lighttpd, an embedded system might consider "min-procs" => 0, in your fastcgi.server config, and lighttpd will start up the backend when required, and kill it when idle for some time. This can save memory by not having the backend using resources if the backend is seldom used.

If lighttpd itself is seldom used, you can have inetd or xinetd start lighttpd as needed.

Additionally, since lighttpd 1.4.70, if you disable HTTP/2 in lighttpd ("server.h2proto" => "disable" in server.feature-flags), then you will save some memory by lighttpd not loading HTTP/2 support.

    (1-18/18)