[Solved] 500 Internal Server Error with lighttpd and Python WSGI App
Added by temp.sha almost 7 years ago
Hi,
I am running a basic Python WSGI test program in the below setup.
Web browser <---------------> lighttpd <-----------via Fast CGI-----------> Python WSGI AppHowever I get "socket.error: [Errno 98] Address already in use" in lighttpd log.
my sample python wsgi code is:-
#!/usr/bin/python
from fcgi import WSGIServer
def simple_app(environ, start_response):
start_response('200 OK', [('Content-Type', 'text/html')])
return('''Hello---------- python world!''')
WSGIServer(simple_app, bindAddress = '/tmp/fcgi.sock').run()
I am using Allan Saddi's fastcgi implementation downloaded from the below github location
https://github.com/keikubo/Python-FastCGI-Sample/blob/master/lib/fcgi.py
lighttpd server log:-
2017-06-05 17:46:31: (response.c.591) -- file found
2017-06-05 17:46:31: (response.c.592) Path : /var/www/html/cgi-bin/a.py
2017-06-05 17:46:31: (response.c.753) -- handling subrequest
2017-06-05 17:46:31: (response.c.754) Path : /var/www/html/cgi-bin/a.py
2017-06-05 17:46:31: (mod_access.c.148) -- mod_access_uri_handler called
Traceback (most recent call last):
File "/var/www/html/cgi-bin/a.py", line 10, in <module>
WSGIServer(simple_app, bindAddress = '/tmp/fcgi.sock').run()
File "/var/www/html/cgi-bin/fcgi.py", line 1069, in run
sock = self._setupSocket()
File "/var/www/html/cgi-bin/fcgi.py", line 1029, in _setupSocket
sock.bind(self._bindAddress)
File "/usr/lib/python2.7/socket.py", line 228, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 98] Address already in use
2017-06-05 17:46:32: (response.c.122) Response-Header: \nHTTP/1.0 500 Internal Server Error\r\nContent-Type: text/html\r\nContent-Length: 369\r\nConnection: close\r\nDate: Mon, 05 Jun 2017 12:16:32 GMT\r\nServer: Apache 1.3.29\r\n\r\n
my lighttpd.conf:-
server.document-root = "/var/www/html"
server.max-request-size = 65000
server.port = 86
server.bind = "localhost"
server.name = "www.example.org"
server.tag = "Apache 1.3.29"
server.dir-listing = "enable"
server.pid-file = "/var/run/lighttpd.pid"
server.username = "www-data"
server.groupname = "www-data"
server.upload-dirs = ( "/var/cache/lighttpd/uploads" )
compress.cache-dir = "/var/cache/lighttpd/compress/"
"mod_rewrite",
"mod_setenv",
"mod_secdownload",
"mod_access",
"mod_auth",
"mod_authn_file",
"mod_status",
"mod_expire",
"mod_simple_vhost",
"mod_redirect",
"mod_fastcgi",
"mod_cgi",
"mod_compress",
"mod_userdir",
"mod_ssi",
"mod_accesslog",
)
- MODULE CONFIG ############################
setenv.add-environment = (
"TRAC_ENV" => "tracenv",
"SETENV" => "setenv",
)
fastcgi.debug = 1
cgi.assign = (
".py" => "/usr/bin/python",
)
fastcgi.server = ( ".fcgi" =>
( "localhost" =>
(
"min-procs" => 4,
"socket" => "/tmp/fcgi.sock",
)
)
)
lighttpd version 1.4.45
OS ubuntu 16.4
python version 2.7
I verified using netstat utility as well that no other process is using the same port/address.
Is there any problem with the configuration ? can any one help me in this ?
Thanks....
Replies (6)
RE: [Solved] 500 Internal Server Error with lighttpd and Python WSGI App - Added by gstrauss almost 7 years ago
Based on this question and your previous question in https://redmine.lighttpd.net/boards/2/topics/7423, I suggest that you spend some more time reading the documentation on this site.
e.g. HowToPythonWSGI
In the above config, you appear to be running your WSGI program as a CGI, not as FastCGI, but the script your are running appears to expect FastCGI. Your config above runs URLs ending with .py via CGI under /usr/bin/python, and uses FastCGI for URLs ending .fcgi, though in your config you need to start the FastCGI backend independently from lighttpd since you have not configured lighttpd to do so. Please read the doc on this site more carefully.
RE: [Solved] 500 Internal Server Error with lighttpd and Python WSGI App - Added by temp.sha almost 7 years ago
Hi gstrauss,
Thanks for your help and suggestions. I went through the documents suggested by you and did the change in my configuration file accordingly. However I still do not get expected response while using uwsgi with lighttpd. When I try to access python app using web browser, in response I get static folder listing of the document root instead of app's dynamic response. I do not see such issue when I use uwsgi with nginx. It means some where in the lighttpd config or uwsgi config is incorrect.
Could you please help me further?
lighttpd version 1.4.45
OS ubuntu 16.4
python version 2.7
uwsgi command used:
uwsgi -L --fastcgi-socket 127.0.0.1:8080 --plugin python --pythonpath /var/www/pyapp -H /var/www/pyapp/pvenv/ --callable app -w hello
lighttpd configuration
debug.log-request-handling = "enable"
debug.log-request-header = "enable"
debug.log-response-header = "enable"
debug.log-condition-handling = "enable"
server.document-root = "/var/www/pyapp"
server.max-request-size = 65000
server.port = 80
server.bind = "localhost"
server.name = "www.example.org"
server.tag = "Apache 1.3.29"
server.dir-listing = "enable"
server.pid-file = "/var/run/lighttpd.pid"
server.username = "www-data"
server.groupname = "www-data"
server.upload-dirs = ( "/var/cache/lighttpd/uploads" )
compress.cache-dir = "/var/cache/lighttpd/compress/"
server.modules = (
"mod_rewrite",
"mod_setenv",
"mod_secdownload",
"mod_access",
"mod_auth",
"mod_authn_file",
"mod_status",
"mod_expire",
"mod_simple_vhost",
"mod_redirect",
"mod_fastcgi",
"mod_cgi",
"mod_compress",
"mod_userdir",
"mod_ssi",
"mod_accesslog",
)
server.indexfiles = (
"index.php",
"index.html",
"index.htm",
"default.htm",
)
######################## MODULE CONFIG ############################
ssi.extension = (
".shtml",
)
mimetype.assign = (
".png" => "image/png",
".jpg" => "image/jpeg",
".jpeg" => "image/jpeg",
".gif" => "image/gif",
".html" => "text/html",
".htm" => "text/html",
".pdf" => "application/pdf",
".swf" => "application/x-shockwave-flash",
".spl" => "application/futuresplash",
".txt" => "text/plain",
".tar.gz" => "application/x-tgz",
".tgz" => "application/x-tgz",
".gz" => "application/x-gzip",
".c" => "text/plain",
".conf" => "text/plain",
)
compress.filetype = (
"text/plain",
"text/html",
)
setenv.add-environment = (
"TRAC_ENV" => "tracenv",
"SETENV" => "setenv",
)
fastcgi.debug = 1
fastcgi.server = ( ".fcgi" =>
( "localhost" =>
(
"min-procs" => 4,
"host" => "127.0.0.1" ,
"port" => "8080" ,
)
)
)
python application/script:
from flask import Flask
app = Flask(__name__)
@app.route("/")
def hello():
return "Hello World!!!"
if __name__ == "__main__":
app.run(host='0.0.0.0', port=8080)
lighttpd log:
root@HP-Z420:/var/www/pyapp# /usr/local/sbin/lighttpd -D -f /var/www/pyapp/pyapp_lighttpd.conf 2017-06-08 16:00:56: (log.c.217) server started 2017-06-08 16:01:06: (request.c.436) fd: 6 request-len: 277 \nGET / HTTP/1.0\r\nUser-Agent: w3m/0.5.3+git20151119\r\nAccept: text/html, text/*;q=0.5, image/*, application/*, audio/*, x-scheme-handler/*, inode/*, video/*, x-content/*, misc/*\r\nAccept-Encoding: gzip, compress, bzip, bzip2, deflate\r\nAccept-Language: en;q=1.0\r\nHost: localhost\r\n\r\n 2017-06-08 16:01:06: (response.c.261) run condition 2017-06-08 16:01:06: (response.c.350) -- splitting Request-URI 2017-06-08 16:01:06: (response.c.351) Request-URI : / 2017-06-08 16:01:06: (response.c.352) URI-scheme : http 2017-06-08 16:01:06: (response.c.353) URI-authority : localhost 2017-06-08 16:01:06: (response.c.354) URI-path (raw) : / 2017-06-08 16:01:06: (response.c.355) URI-path (clean): / 2017-06-08 16:01:06: (response.c.356) URI-query : 2017-06-08 16:01:06: (mod_access.c.148) -- mod_access_uri_handler called 2017-06-08 16:01:06: (response.c.490) -- before doc_root 2017-06-08 16:01:06: (response.c.491) Doc-Root : /var/www/pyapp 2017-06-08 16:01:06: (response.c.492) Rel-Path : / 2017-06-08 16:01:06: (response.c.493) Path : 2017-06-08 16:01:06: (response.c.542) -- after doc_root 2017-06-08 16:01:06: (response.c.543) Doc-Root : /var/www/pyapp 2017-06-08 16:01:06: (response.c.544) Rel-Path : / 2017-06-08 16:01:06: (response.c.545) Path : /var/www/pyapp/ 2017-06-08 16:01:06: (response.c.562) -- logical -> physical 2017-06-08 16:01:06: (response.c.563) Doc-Root : /var/www/pyapp 2017-06-08 16:01:06: (response.c.564) Basedir : /var/www/pyapp 2017-06-08 16:01:06: (response.c.565) Rel-Path : / 2017-06-08 16:01:06: (response.c.566) Path : /var/www/pyapp/ 2017-06-08 16:01:06: (response.c.583) -- handling physical path 2017-06-08 16:01:06: (response.c.584) Path : /var/www/pyapp/ 2017-06-08 16:01:06: (response.c.591) -- file found 2017-06-08 16:01:06: (response.c.592) Path : /var/www/pyapp/ 2017-06-08 16:01:06: (response.c.753) -- handling subrequest 2017-06-08 16:01:06: (response.c.754) Path : /var/www/pyapp/ 2017-06-08 16:01:06: (mod_indexfile.c.153) -- handling the request as Indexfile 2017-06-08 16:01:06: (mod_indexfile.c.154) URI : / 2017-06-08 16:01:06: (mod_access.c.148) -- mod_access_uri_handler called 2017-06-08 16:01:06: (mod_compress.c.839) -- handling file as static file 2017-06-08 16:01:06: (mod_dirlisting.c.1165) -- handling the request as Dir-Listing 2017-06-08 16:01:06: (mod_dirlisting.c.1166) URI : / 2017-06-08 16:01:06: (response.c.765) -- subrequest finished 2017-06-08 16:01:06: (response.c.122) Response-Header: \nHTTP/1.0 200 OK\r\nContent-Type: text/html\r\nContent-Length: 6632\r\nConnection: close\r\nDate: Thu, 08 Jun 2017 10:31:06 GMT\r\nServer: Apache 1.3.29\r\n\r\n
uwsgi log:
^C(pvenv) root@HP-Z420:/var/www/pyapp# uwsgi -L --fastcgi-socket 127.0.0.1:8080 --plugin python --pythonpath /var/www/pyapp -H /var/www/pyapp/pvenv/ --callable app -w hello *** Starting uWSGI 2.0.12-debian (64bit) on [Thu Jun 8 16:04:59 2017] *** compiled with version: 5.3.1 20160412 on 13 April 2016 08:36:06 os: Linux-4.8.0-52-generic #55~16.04.1-Ubuntu SMP Fri Apr 28 14:36:29 UTC 2017 nodename: HP-Z420 machine: x86_64 clock source: unix pcre jit disabled detected number of CPU cores: 4 current working directory: /var/www/pyapp detected binary path: /usr/bin/uwsgi-core uWSGI running as root, you can use --uid/--gid/--chroot options *** WARNING: you are running uWSGI as root !!! (use the --uid flag) *** *** WARNING: you are running uWSGI without its master process manager *** your processes number limit is 111903 your memory page size is 4096 bytes detected max file descriptor number: 1024 lock engine: pthread robust mutexes thunder lock: disabled (you can enable it with --thunder-lock) uwsgi socket 0 bound to TCP address 127.0.0.1:8080 fd 3 Python version: 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609] Set PythonHome to /var/www/pyapp/pvenv/ *** Python threads support is disabled. You can enable it with --enable-threads *** Python main interpreter initialized at 0x242b400 your server socket listen backlog is limited to 100 connections your mercy for graceful operations on workers is 60 seconds mapped 72768 bytes (71 KB) for 1 cores *** Operational MODE: single process *** added /var/www/pyapp/ to pythonpath. WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x242b400 pid: 18016 (default app) *** uWSGI is running in multiple interpreter mode *** spawned uWSGI worker 1 (and the only) (pid: 18016, cores: 1)
Thanks.....
RE: [Solved] 500 Internal Server Error with lighttpd and Python WSGI App - Added by gstrauss almost 7 years ago
Could you please help me further?
Your FastCGI config is handling .fcgi. Your request is "GET / ...". Does that end in .fcgi?
Please read the documentation Docs_ModFastCGI (linked from HowToPythonWSGI) and get simple configurations working before trying to do anything more complex.
You'll save yourself a lot of hassle.
RE: [Solved] 500 Internal Server Error with lighttpd and Python WSGI App - Added by temp.sha almost 7 years ago
I went through the specified doc also and did the change in my configuration. However issues are still not resolved. Looks like some where either in lighttpd or uwsgi side configuration is incorrect which I am not able to catch. JFYI I am very new to the web programming, its hardly one week when I started working on this front. By considering the fact, I hope it will not be very irritating for you to respond to my silly issues if at all they are trivial in any form. Your help and support will be highly appreciated.
issue now facing while accessing from browser:
web url typed: http://127.0.0.1 web response: HTTP/1.1 404 Not Found
lihttpd.conf :
debug.log-request-handling = "enable"
debug.log-request-header = "enable"
debug.log-response-header = "enable"
debug.log-condition-handling = "enable"
server.document-root = "/var/www/pyapp"
server.max-request-size = 65000
server.port = 80
server.name = "www.example.org"
server.tag = "Apache 1.3.29"
server.dir-listing = "enable"
server.pid-file = "/var/run/lighttpd.pid"
server.username = "www-data"
server.groupname = "www-data"
server.upload-dirs = ( "/var/cache/lighttpd/uploads" )
compress.cache-dir = "/var/cache/lighttpd/compress/"
server.modules = (
"mod_rewrite",
"mod_setenv",
"mod_secdownload",
"mod_access",
"mod_auth",
"mod_authn_file",
"mod_status",
"mod_expire",
"mod_simple_vhost",
"mod_redirect",
"mod_fastcgi",
"mod_cgi",
"mod_compress",
"mod_userdir",
"mod_ssi",
"mod_accesslog",
)
server.indexfiles = (
"index.php",
"index.html",
"index.htm",
"default.htm",
)
######################## MODULE CONFIG ############################
ssi.extension = (
".shtml",
)
mimetype.assign = (
".png" => "image/png",
".jpg" => "image/jpeg",
".jpeg" => "image/jpeg",
".gif" => "image/gif",
".html" => "text/html",
".htm" => "text/html",
".pdf" => "application/pdf",
".swf" => "application/x-shockwave-flash",
".spl" => "application/futuresplash",
".txt" => "text/plain",
".tar.gz" => "application/x-tgz",
".tgz" => "application/x-tgz",
".gz" => "application/x-gzip",
".c" => "text/plain",
".conf" => "text/plain",
)
compress.filetype = (
"text/plain",
"text/html",
)
setenv.add-environment = (
"TRAC_ENV" => "tracenv",
"SETENV" => "setenv",
)
fastcgi.debug = 1
fastcgi.server = (
"/" =>
(
"python-fcgi" =>
(
"host" => "127.0.0.1",
"port" => "8080",
"bin-path" => "hello.py",
"check-local" => "disable",
"max-procs" => 1,
)
))
uwsgi command used:
uwsgi -L --fastcgi-socket 127.0.0.1:8080 --plugin python --pythonpath /var/www/pyapp -H /var/www/pyapp/pvenv/ --callable app -w hello
python script hello.py:
from flask import Flask
app = Flask(__name__)
@app.route("/")
def hello():
return "Hello World!!!"
if __name__ == "__main__":
app.run()
-- Point to note: I am running uwsgi and lighttpd in a virtual environment under folder pvenv.
application dir structure:
(pvenv) /var/www/pyapp ---
hello.py
pyapp_lighttpd.conf
pvenv
lighttpd log:
(pvenv) /var/www/pyapp#/usr/local/sbin/lighttpd -D -f /var/www/pyapp/pyapp_lighttpd.conf 2017-06-09 12:31:57: (log.c.217) server started 2017-06-09 12:31:57: (mod_fastcgi.c.1485) --- fastcgi spawning local \n\tproc: hello.py \n\tport: 8080 \n\tsocket \n\tmax-procs: 1 2017-06-09 12:31:57: (mod_fastcgi.c.1509) --- fastcgi spawning \n\tport: 8080 \n\tsocket \n\tcurrent: 0 / 1 2017-06-09 12:32:04: (request.c.436) fd: 6 request-len: 74 \nHEAD / HTTP/1.1\r\nHost: localhost\r\nUser-Agent: curl/7.47.0\r\nAccept: */*\r\n\r\n 2017-06-09 12:32:04: (response.c.261) run condition 2017-06-09 12:32:04: (response.c.350) -- splitting Request-URI 2017-06-09 12:32:04: (response.c.351) Request-URI : / 2017-06-09 12:32:04: (response.c.352) URI-scheme : http 2017-06-09 12:32:04: (response.c.353) URI-authority : localhost 2017-06-09 12:32:04: (response.c.354) URI-path (raw) : / 2017-06-09 12:32:04: (response.c.355) URI-path (clean): / 2017-06-09 12:32:04: (response.c.356) URI-query : 2017-06-09 12:32:04: (mod_access.c.148) -- mod_access_uri_handler called 2017-06-09 12:32:04: (mod_fastcgi.c.3500) handling it in mod_fastcgi 2017-06-09 12:32:04: (response.c.490) -- before doc_root 2017-06-09 12:32:04: (response.c.491) Doc-Root : /var/www/pyapp 2017-06-09 12:32:04: (response.c.492) Rel-Path : / 2017-06-09 12:32:04: (response.c.493) Path : 2017-06-09 12:32:04: (response.c.542) -- after doc_root 2017-06-09 12:32:04: (response.c.543) Doc-Root : /var/www/pyapp 2017-06-09 12:32:04: (response.c.544) Rel-Path : / 2017-06-09 12:32:04: (response.c.545) Path : /var/www/pyapp/ 2017-06-09 12:32:04: (response.c.562) -- logical -> physical 2017-06-09 12:32:04: (response.c.563) Doc-Root : /var/www/pyapp 2017-06-09 12:32:04: (response.c.564) Basedir : /var/www/pyapp 2017-06-09 12:32:04: (response.c.565) Rel-Path : / 2017-06-09 12:32:04: (response.c.566) Path : /var/www/pyapp/ 2017-06-09 12:32:05: (mod_fastcgi.c.2875) got proc: pid: 0 socket: tcp:127.0.0.1:8080 load: 1 2017-06-09 12:32:05: (mod_fastcgi.c.1642) released proc: pid: 0 socket: tcp:127.0.0.1:8080 load: 0 2017-06-09 12:32:05: (response.c.122) Response-Header: \nHTTP/1.1 404 Not Found\r\nContent-Type: text/html\r\nContent-Length: 233\r\nDate: Fri, 09 Jun 2017 07:02:05 GMT\r\nServer: Apache 1.3.29\r\n\r\n
uwsgi log:
(pvenv) /var/www/pyapp#uwsgi -L --fastcgi-socket 127.0.0.1:8080 --plugin python --pythonpath /var/www/pyapp -H /var/www/pyapp/pvenv/ --callable app -w hello *** Starting uWSGI 2.0.12-debian (64bit) on [Fri Jun 9 12:31:52 2017] *** compiled with version: 5.3.1 20160412 on 13 April 2016 08:36:06 os: Linux-4.8.0-52-generic #55~16.04.1-Ubuntu SMP Fri Apr 28 14:36:29 UTC 2017 nodename: HP-Z420 machine: x86_64 clock source: unix pcre jit disabled detected number of CPU cores: 4 current working directory: /var/www/pyapp detected binary path: /usr/bin/uwsgi-core uWSGI running as root, you can use --uid/--gid/--chroot options *** WARNING: you are running uWSGI as root !!! (use the --uid flag) *** *** WARNING: you are running uWSGI without its master process manager *** your processes number limit is 111903 your memory page size is 4096 bytes detected max file descriptor number: 1024 lock engine: pthread robust mutexes thunder lock: disabled (you can enable it with --thunder-lock) uwsgi socket 0 bound to TCP address 127.0.0.1:8080 fd 3 Python version: 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609] Set PythonHome to /var/www/pyapp/pvenv/ *** Python threads support is disabled. You can enable it with --enable-threads *** Python main interpreter initialized at 0x1032400 your server socket listen backlog is limited to 100 connections your mercy for graceful operations on workers is 60 seconds mapped 72768 bytes (71 KB) for 1 cores *** Operational MODE: single process *** added /var/www/pyapp/ to pythonpath. WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x1032400 pid: 23822 (default app) *** uWSGI is running in multiple interpreter mode *** spawned uWSGI worker 1 (and the only) (pid: 23822, cores: 1)
I have started pkt capture on 127.0.0.1:8080 and looks like listener uwsgi is receiving req from lighttpd and responding with "404 Not Found" error back to lighttpd. Not sure why.
below is the log
/var/www/pyapp#tcpdump -vv -nn -i lo port 8080
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes
13:34:03.158316 IP (tos 0x0, ttl 64, id 17463, offset 0, flags [DF], proto TCP (6), length 123)
127.0.0.1.8080 > 127.0.0.1.48542: Flags [P.], cksum 0xfe6f (incorrect -> 0xed5c), seq 9:80, ack 766, win 354, options [nop,nop,TS val 83104056 ecr 83104056], length 71: HTTP, length: 71
Status: 404 NOT FOUND
Content-Type: text/html
Content-Length: 233
13:34:03.158324 IP (tos 0x0, ttl 64, id 9573, offset 0, flags [DF], proto TCP (6), length 52)
127.0.0.1.48542 > 127.0.0.1.8080: Flags [.], cksum 0xfe28 (incorrect -> 0xf00b), seq 766, ack 80, win 342, options [nop,nop,TS val 83104056 ecr 83104056], length 0
/var/www/pyapp#lsof -i:8080 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME uwsgi 23989 root 3u IPv4 1303951 0t0 TCP localhost:http-alt (LISTEN) /var/www/pyapp#
Please let me know if any more info is needed to help me in figuring out the issue.
Once again thanks for the help.
RE: [Solved] 500 Internal Server Error with lighttpd and Python WSGI App - Added by gstrauss almost 7 years ago
Doesn't matter how new you are. Read the documentation. Start simple. Get simple things working before trying to get more complex things working. Once simple things are working, make small incremental changes and test each one to make sure simple things still work. Then follow the examples perfectly to get slightly more complex things working.
"bin-path" is supposed to be a path, not a filename, and should not be specified if you start the backend service manually.
Read the docs more carefully and follow the examples perfectly. If not specific enough for you, then use your favorite search engine to find other simple How-To examples, and try to learn by following those.
RE: [Solved] 500 Internal Server Error with lighttpd and Python WSGI App - Added by temp.sha almost 7 years ago
the problem I am facing here is the most of the documents and How-TO on Internet deals with (nginx + uwsgi + python) with increasing complexities of examples. that is why I am successfully able to run (nginx + uwsgi + python) however I still face issues with (lighttpd + uwsgi + python) I do not get proper examples of this at least proper understandably by a person having this much of web experience.
I do several several configuration changes and testing before posting the query here.
I will change the config and try again.
thanks....