[Solved] 1 of 3 tests failed on on macOS 10.13.6
Added by ryandesign about 2 years ago
After compiling lighttpd 1.4.64 on macOS 10.13.6 I ran the tests, some of which failed and it asked me to report the results here. So here they are.
$ sudo -u macports make check
Making check in src
/Applications/Xcode.app/Contents/Developer/usr/bin/make check-am
/Applications/Xcode.app/Contents/Developer/usr/bin/make check-TESTS
PASS: t/test_common
PASS: t/test_configfile
PASS: t/test_mod
==================
All 3 tests passed
==================
Making check in doc
Making check in config
Making check in conf.d
make[3]: Nothing to be done for `check'.
Making check in vhosts.d
make[3]: Nothing to be done for `check'.
make[3]: Nothing to be done for `check-am'.
Making check in scripts
make[2]: Nothing to be done for `check'.
Making check in systemd
make[2]: Nothing to be done for `check'.
Making check in outdated
make[2]: Nothing to be done for `check'.
make[2]: Nothing to be done for `check-am'.
Making check in tests
Making check in docroot
make[2]: Nothing to be done for `check'.
/Applications/Xcode.app/Contents/Developer/usr/bin/make fcgi-responder scgi-responder
depbase=`echo fcgi-responder.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
/usr/bin/clang -DHAVE_CONFIG_H -I. -I.. -I/opt/local/include -D_REENTRANT -D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGE_FILES -pipe -Os -arch x86_64 -pipe -Wall -W -Wshadow -pedantic -MT fcgi-responder.o -MD -MP -MF $depbase.Tpo -c -o fcgi-responder.o fcgi-responder.c &&\
mv -f $depbase.Tpo $depbase.Po
/bin/sh ../libtool --tag=CC --mode=link /usr/bin/clang -pipe -Os -arch x86_64 -pipe -Wall -W -Wshadow -pedantic -L/opt/local/lib -Wl,-headerpad_max_install_names -arch x86_64 -o fcgi-responder fcgi-responder.o
libtool: link: /usr/bin/clang -pipe -Os -arch x86_64 -pipe -Wall -W -Wshadow -pedantic -Wl,-headerpad_max_install_names -arch x86_64 -o fcgi-responder fcgi-responder.o -L/opt/local/lib
depbase=`echo scgi-responder.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
/usr/bin/clang -DHAVE_CONFIG_H -I. -I.. -I/opt/local/include -D_REENTRANT -D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGE_FILES -pipe -Os -arch x86_64 -pipe -Wall -W -Wshadow -pedantic -MT scgi-responder.o -MD -MP -MF $depbase.Tpo -c -o scgi-responder.o scgi-responder.c &&\
mv -f $depbase.Tpo $depbase.Po
/bin/sh ../libtool --tag=CC --mode=link /usr/bin/clang -pipe -Os -arch x86_64 -pipe -Wall -W -Wshadow -pedantic -L/opt/local/lib -Wl,-headerpad_max_install_names -arch x86_64 -o scgi-responder scgi-responder.o
libtool: link: /usr/bin/clang -pipe -Os -arch x86_64 -pipe -Wall -W -Wshadow -pedantic -Wl,-headerpad_max_install_names -arch x86_64 -o scgi-responder scgi-responder.o -L/opt/local/lib
/Applications/Xcode.app/Contents/Developer/usr/bin/make check-TESTS
Testing in build directory: '..' and cwd: '/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_www_lighttpd/lighttpd/work/lighttpd-1.4.64/tests'
preparing infrastructure PASS: prepare.sh
./core-condition.t .. 1/25 2022-01-21 13:17:44: (configfile.c.2333) command "echo cmd_ok=456" exited non-zero: 1
2022-01-21 13:17:44: (configfile.c.2153) source: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_www_lighttpd/lighttpd/work/lighttpd-1.4.64/tests/var-include-sub.conf line: 35 pos: 14 parser failed somehow near here: (EOL)
2022-01-21 13:17:44: (configfile.c.2153) source: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_www_lighttpd/lighttpd/work/lighttpd-1.4.64/tests/condition.conf line: 90 pos: 1 parser failed somehow near here: (EOL)
#
# empty response
# Failed test 'config deny'
# at ./core-condition.t line 28.
#
# connect failed: Connection refused
# Failed test '2nd child of chaining'
# at ./core-condition.t line 36.
#
# connect failed: Connection refused
# Failed test '3rd child of chaining'
# at ./core-condition.t line 44.
#
# connect failed: Connection refused
# Failed test 'nesting'
# at ./core-condition.t line 52.
#
# connect failed: Connection refused
# Failed test 'url subdir'
# at ./core-condition.t line 60.
#
# connect failed: Connection refused
# Failed test 'url subdir with path traversal'
# at ./core-condition.t line 68.
#
# connect failed: Connection refused
# Failed test 'condition: handle if before else branches'
# at ./core-condition.t line 75.
#
# connect failed: Connection refused
# Failed test 'condition: handle if before else branches \#2'
# at ./core-condition.t line 82.
#
# connect failed: Connection refused
# Failed test 'basic test'
# at ./core-condition.t line 89.
#
# connect failed: Connection refused
# Failed test 'string_append'
# at ./core-condition.t line 122.
#
# connect failed: Connection refused
# Failed test 'number2'
# at ./core-condition.t line 122.
#
# connect failed: Connection refused
# Failed test 'servername2'
# at ./core-condition.t line 122.
#
# connect failed: Connection refused
# Failed test 'number_append'
# at ./core-condition.t line 122.
#
# connect failed: Connection refused
# Failed test 'env'
# at ./core-condition.t line 122.
#
# connect failed: Connection refused
# Failed test 'concat'
# at ./core-condition.t line 122.
#
# connect failed: Connection refused
# Failed test 'array_append'
# at ./core-condition.t line 122.
#
# connect failed: Connection refused
# Failed test 'number1'
# at ./core-condition.t line 122.
#
# connect failed: Connection refused
# Failed test 'servername3'
# at ./core-condition.t line 122.
#
# connect failed: Connection refused
# Failed test 'myvar'
# at ./core-condition.t line 122.
#
# connect failed: Connection refused
# Failed test 'servername1'
# at ./core-condition.t line 122.
#
# connect failed: Connection refused
# Failed test 'include_shell'
# at ./core-condition.t line 122.
#
# connect failed: Connection refused
# Failed test 'var.myvar'
# at ./core-condition.t line 122.
#
# connect failed: Connection refused
# Failed test 'include'
# at ./core-condition.t line 122.
# Looks like you failed 23 tests of 25.
./core-condition.t .. Dubious, test returned 23 (wstat 5888, 0x1700)
Failed 23/25 subtests
./mod-fastcgi.t ..... ok
./mod-scgi.t ........ ok
./request.t ......... 150/176 #
# status failed: expected '200', got '403'
# Failed test 'secdownload (hmac-sha1)'
# at ./request.t line 1664.
#
# status failed: expected '404', got '403'
# Failed test 'secdownload - timeout (hmac-sha1)'
# at ./request.t line 1699.
#
# status failed: expected '200', got '403'
# Failed test 'secdownload (hmac-sha256)'
# at ./request.t line 1713.
#
# status failed: expected '200', got '403'
# Failed test 'secdownload (hmac-sha256) with hash-querystr'
# at ./request.t line 1727.
#
# status failed: expected '404', got '403'
# Failed test 'secdownload - timeout (hmac-sha256)'
# at ./request.t line 1762.
# Looks like you failed 5 tests of 176.
./request.t ......... Dubious, test returned 5 (wstat 1280, 0x500)
Failed 5/176 subtests
(less 1 skipped subtest: 170 okay)
Test Summary Report
-------------------
./core-condition.t (Wstat: 5888 Tests: 25 Failed: 23)
Failed tests: 2-24
Non-zero exit status: 23
./request.t (Wstat: 1280 Tests: 176 Failed: 5)
Failed tests: 160, 163-165, 168
Non-zero exit status: 5
Files=4, Tests=235, 2 wallclock secs ( 0.06 usr 0.02 sys + 0.59 cusr 0.47 csys = 1.14 CPU)
Result: FAIL
Failed 2/4 test programs. 28/235 subtests failed.
FAIL: run-tests.pl
cleaning up PASS: cleanup.sh
========================================================================
1 of 3 tests failed
Please report to https://redmine.lighttpd.net/projects/lighttpd/boards/2
========================================================================
make[3]: *** [check-TESTS] Error 1
make[2]: *** [check-am] Error 2
make[1]: *** [check-recursive] Error 1
make: *** [check-recursive] Error 1
Replies (12)
RE: 1 of 3 tests failed - Added by gstrauss about 2 years ago
./core-condition.t .. 1/25 2022-01-21 13:17:44: (configfile.c.2333) command "echo cmd_ok=456" exited non-zero: 1
That looks to me like a shell issue. More likely a problem with this test on that platform. Maybe it should be /bin/echo instead of echo
If you want to try changing it, that comes from tests/var-include-sub.conf
cmd = "echo cmd_ok=456"
include_shell cmd
In any case, if the syntax fails, then lighttpd fails to start up with that config, and all the subsequent core-condition.t tests will, of course, fail because they can not even start to run.
As to the secdownload tests failing, that is expected if lighttpd is not built with TLS libraries and not built with Nettle to provide the HMAC and SHA-256 algos.
RE: 1 of 3 tests failed on on macOS 10.13.6 - Added by ryandesign about 2 years ago
No change if I use "/bin/echo" instead of "echo".
Yes, the secdownload tests no longer fail if I enable nettle support. You might consider reporting these failures as "expected failures" or "XFAIL" in such situations, to avoid lots of people reporting non-problems to you. You might have the test output explain that the failures are expected because of missing nettle support so that those wishing to see if the tests pass will know what to enable.
The test output could be made easier to understand. The above output included "All 3 tests passed", "failed 23 tests of 25", "failed 5 tests of 176", "Failed 2/4 test programs", and "1 of 3 tests failed"; one hardly knows what to make of it.
RE: 1 of 3 tests failed on on macOS 10.13.6 - Added by gstrauss about 2 years ago
No change if I use "/bin/echo" instead of "echo".
Sigh. WTH is wrong with MacOS that it can't provide 'echo' as a shell-builtin or as an executable in the standard PATH?
Yes, the secdownload tests no longer fail if I enable nettle support.
The secdownload tests should have been skipped by the test harness if lighttpd was not built with crypto libs or with Nettle. Would you please share lighttpd -V using the path to the lighttpd which was built without crypto libs and without Nettle?
The test output could be made easier to understand.
Regarding the test output, that is Perl Test::Harness and Test::More, unfortunately.
RE: 1 of 3 tests failed on on macOS 10.13.6 - Added by gstrauss about 2 years ago
No change if I use "/bin/echo" instead of "echo".
Sigh. WTH is wrong with MacOS that it can't provide 'echo' as a shell-builtin or as an executable in the standard PATH?
I have a MacBook Air from 2014 running MacOS Big Sur 11.6.2 and lighttpd tests all pass. My shell is /bin/bash and /bin/echo is present.
Before MacOS 11 was released, these same lighttpd tests passed when run under macOS X on my system.
./core-condition.t .. 1/25 2022-01-21 13:17:44: (configfile.c.2333) command "echo cmd_ok=456" exited non-zero: 1
When lighttpd executes commands from lighttpd.conf, lighttpd uses the SHELL environment variable, if present, or else uses /bin/sh, and execs commands using $SHELL -c. I just tested with SHELL=/bin/sh, SHELL=/bin/zsh, and SHELL=/bin/bash on my system, and the lighttpd tests pass with any of them. The lighttpd tests also pass on my system when SHELL is unset.
There is something in your environment that is preventing the lighttpd tests from passing, but I'll need help figuring it out since I do not have access to your system.
RE: 1 of 3 tests failed on on macOS 10.13.6 - Added by ryandesign about 2 years ago
macOS does provide echo both as a shell builtin and as a separate /bin/echo.
I should have mentioned that I was running these tests as the unprivileged "macports" user that MacPorts uses, because my goal is to be able to allow MacPorts to run the lighttpd tests (with "sudo port test lighttpd"). For security reasons the macports user does not have a shell.
$ sudo -u macports sh -c 'echo $SHELL'
/usr/bin/false
If I run the tests as my normal macOS user in a lighttpd source directory owned by my normal macOS user, the tests succeed. If I override the SHELL environment variable when running the tests as the macports user, that works too, so I can do that. ("sudo -u macports SHELL=/bin/sh make check") Thanks for helping me diagnose this.
When the secdownload tests were failing, I was building lighttpd with openssl support but without nettle support.
$ ./src/lighttpd -V
lighttpd/1.4.64 (ssl) - a light and fast webserver
Event Handlers:
- select (generic)
+ poll (Unix)
- epoll (Linux)
- /dev/poll (Solaris)
- eventports (Solaris)
+ kqueue (FreeBSD)
- libev (generic)Network handler:
- linux-sendfile
- freebsd-sendfile
+ darwin-sendfile
- solaris-sendfilev
+ writev
+ write
- mmap supportFeatures:
+ IPv6 support
+ zlib support
+ zstd support
- bzip2 support
+ brotli support
+ crypt support
+ OpenSSL support
- mbedTLS support
- NSS crypto support
- GnuTLS support
- WolfSSL support
- Nettle support
+ PCRE support
- MySQL support
- PgSQL support
- DBI support
- Kerberos support
- LDAP support
- PAM support
- FAM support
+ LUA support
- xml support
- SQLite support
+ Y2038 supportRE: 1 of 3 tests failed on on macOS 10.13.6 - Added by gstrauss about 2 years ago
When the secdownload tests were failing, I was building lighttpd with openssl support but without nettle support.
What version of openssl?
lighttpd detected this: + OpenSSL support and expected to be able to run the secdownload tests, but for some reason that was failing. I'll see if I can reproduce that.
RE: 1 of 3 tests failed on on macOS 10.13.6 - Added by gstrauss about 2 years ago
Some history: #2910 where lighttpd replaced popen() used in lighttpd config_parse_cmd()
I guess this might work well enough for lighttpd config processing:
--- a/src/configfile.c
+++ b/src/configfile.c
@@ -2299,6 +2299,9 @@ int config_parse_cmd(server *srv, config_t *context, const char *cmd) {
char *shell = getenv("SHELL");
char *args[4];
pid_t pid;
+ if (shell && (0 == strcmp(shell, "/usr/bin/false")
+ || 0 == strcmp(shell, "/bin/false")))
+ shell = NULL;
*(const char **)&args[0] = shell ? shell : "/bin/sh";
*(const char **)&args[1] = "-c";
*(const char **)&args[2] = cmd;
RE: 1 of 3 tests failed on on macOS 10.13.6 - Added by gstrauss about 2 years ago
I built lighttpd on Linux with OpenSSL (and without other TLS libs or Nettle) and the secdownload tests pass. My openssl is OpenSSL 1.1.1l
In your case, those test failed, but at least failed "closed" with 403 Forbidden. It is strange that the tests would pass when built with Nettle, but not with OpenSSL, so I wonder if it is the OpenSSL version, or something else in your test environment that OpenSSL requires.
RE: 1 of 3 tests failed on on macOS 10.13.6 - Added by ryandesign about 2 years ago
MacPorts recently switched to using openssl 3.x by default. Currently it's 3.0.1. openssl 1.1.x and 1.0.x remain available should individual ports need to use them.
RE: 1 of 3 tests failed on on macOS 10.13.6 - Added by ryandesign about 2 years ago
/sbin/nologin or /usr/sbin/nologin is another popular value that a user's shell might be set to if the user is not supposed to be able to log in.
RE: 1 of 3 tests failed on on macOS 10.13.6 - Added by gstrauss about 2 years ago
/sbin/nologin or /usr/sbin/nologin is another popular value that a user's shell might be set to if the user is not supposed to be able to log in.
I think at some point the answer will be unset SHELL or define it to something workable if you want lighttpd to be able to use it when running commands in lighttpd.conf
RE: 1 of 3 tests failed on on macOS 10.13.6 - Added by gstrauss about 2 years ago
The following works with OpenSSL 3.0, but I have to use (unsigned int) instead of (size_t) for earlier versions of OpenSSL, so the patch I have committed to my dev branch is slightly different.
--- a/src/algo_hmac.c
+++ b/src/algo_hmac.c
@@ -99,10 +99,11 @@ li_hmac_sha1 (unsigned char digest[SHA_DIGEST_LENGTH],
return 0;
return 1;
#elif defined(USE_OPENSSL_CRYPTO)
+ size_t dlen = SHA_DIGEST_LENGTH;
return (NULL != HMAC(EVP_sha1(),
(const unsigned char *)secret, (int)slen,
(const unsigned char *)msg, mlen,
- digest, NULL));
+ digest, &dlen));
#elif defined(USE_GNUTLS_CRYPTO)
return 0 ==
gnutls_hmac_fast(GNUTLS_MAC_SHA1,
@@ -174,10 +175,11 @@ li_hmac_sha256 (unsigned char digest[SHA256_DIGEST_LENGTH],
return 0;
return 1;
#elif defined(USE_OPENSSL_CRYPTO)
+ size_t dlen = SHA256_DIGEST_LENGTH;
return (NULL != HMAC(EVP_sha256(),
(const unsigned char *)secret, (int)slen,
(const unsigned char *)msg, mlen,
- digest, NULL));
+ digest, &dlen));
#elif defined(USE_GNUTLS_CRYPTO)
return 0 ==
gnutls_hmac_fast(GNUTLS_MAC_SHA256,