Project

General

Profile

Actions
Copy link

Bug #1717

closed

Google's urlfetch from appEngine and lighttpd HTTP 400 response

Added by Anonymous over 16 years ago. Updated over 8 years ago.

Status:
Obsolete
Priority:
Normal
Category:
core
Target version:
1.5.0
ASK QUESTIONS IN Forums:

Description

Hi,
I started using urlfetch from google's appEngine (http://code.google.com/appengine/docs/urlfetch/overview.html) and I noticed that it doesn't work for some sites powered by lighttpd. I tried the example given in the link above but replaced the URL with lighttpd.net:


from google.appengine.api import urlfetch

url = "http://www.lighttpd.net/" 
result = urlfetch.fetch(url)
if result.status_code == 200:
  self.response.out.write(result.content)
else:
  self.response.out.write(result.status_code)

I always get back a 400 (Bad Request) response in return. I tried with youtube.com with the same result. I am rather new with Python so it maybe is something I don't do right.

I also tried this on a local lighttpd 1.5 instance with the same result. Here's the debug trace I see:


configfile-glue.c.500: (trace) === start of 1 condition block ===
configfile-glue.c.258: (trace) is condition [3] (global/HTTPhost=~(^|\.)local.com[:0-9]*$) already valid ? nej
configfile-glue.c.521: (trace) [1] result: unknown
2008-07-11 01:55:01: (response.c.137) Response-Header:
HTTP/1.1 400 Bad Request
Content-Type: text/html
Content-Length: 349
Connection: close
Date: Fri, 11 Jul 2008 01:55:01 GMT
Server: lighttpd

Please help!
florin

-- florin

Related issues 2 (0 open — 2 closed)

Has duplicate Bug #1351: lighttpd returns "400 Bad Request" for Nokia web browser requestDuplicateActions
Has duplicate Bug #2208: HEAD method returns 404 if request has Content-Length: 0Duplicate2010-05-21Actions
Actions
Copy link
 #1

Updated by stbuehler over 16 years ago

The request would be the interesting thing.

Actions
Copy link
 #2

Updated by stbuehler over 16 years ago

The framework sends a "Content-Length: 0" header in a GET request:

rfc 2616, 4.3:


   The presence of a message-body in a request is signaled by the
   inclusion of a Content-Length or Transfer-Encoding header field in
   the request's message-headers. A message-body MUST NOT be included in
   a request if the specification of the request method (section 5.1.1)
   does not allow sending an entity-body in requests. A server SHOULD
   read and forward a message-body on any request; if the request method
   does not include defined semantics for an entity-body, then the
   message-body SHOULD be ignored when handling the request.

So i guess google shouldn't send the content-length header in that case.

Actions
Copy link
 #3

Updated by ralf about 16 years ago

stbuehler wrote:

The framework sends a "Content-Length: 0" header in a GET request:

rfc 2616, 4.3:

[...]

So i guess google shouldn't send the content-length header in that case.

hehe, the words of a developer.

i think lighttpd should read 0 bytes (as lighttpd do if there is no content-length header).

Every (ok, the mostly used) webserver - Apache - works with the request.

Actions
Copy link
 #4

Updated by james82@gmail.com almost 16 years ago

  • Status changed from New to Patch Pending

stbuehler, what are your thoughts on this patch? This patch is copied from Bug #1351, and allows lighttpd servers to be accessed using browsers that send "Content-Length: 0" as one of the request headers. (It turns out that many HTTP implementations do this, including the implementations used by some Nokia and Motorola phones. For compatibility with such browsers we should allow browsers to send "Content-Length: 0" with their requests.)

Index: request.c
===================================================================
--- request.c   (revision 2407)
+++ request.c   (working copy)
@@ -542,7 +542,7 @@ int http_request_parse(server *srv, connection *co
        case HTTP_METHOD_GET:
        case HTTP_METHOD_HEAD:
                /* content-length is forbidden for those */
-               if (con->request.content_length != -1) {
+               if (con->request.content_length != -1 && con->request.content_length != 0) {
                        /* content-length is missing */
                        if (srv->srvconf.log_request_header_on_error) {
                                ERROR("GET/HEAD with content-length: %d", 400);
Actions
Copy link
 #6

Updated by stepancheg over 14 years ago

Proposed patch helps.

Actions
Copy link
 #7

Updated by bestis over 12 years ago

What's the situation of this? In 1.4.31 it seems to have something as the if is just != 0, but I'm seeing this sometimes as a "storm" of requests.

For example:

2012-07-17 08:50:53: (request.c.1131) GET/HEAD with content-length -> 400
192.89.0.X - - [17/Jul/2012:08:50:23 +0300] "GET /datafiles/styles/style.css HTTP/1.0" 
400 349 "-" "Mozilla/4.0 (compatible; MSIE 4.01; Windows 95)"

As the log line doesn't contain what is the value of content-length it's hard to say how badly the client behaves. But after grepping logs it seems to be from same company all the time. So probably they are doing something wrong and it works for the most of the people.

Actions
Copy link
 #8

Updated by stbuehler about 12 years ago

  • Assignee deleted (jan)
  • Missing in 1.5.x set to No

I think 1.4 "always" accepted "Content-Length: 0" for HEAD and GET.

Actions
Copy link
 #9

Updated by gstrauss over 8 years ago

  • Missing in 1.5.x changed from No to Yes
Actions
Copy link
 #10

Updated by gstrauss over 8 years ago

  • Status changed from Patch Pending to Obsolete
Actions
Copy link

Also available in: Atom