Project

General

Profile

Bug #1717

Google's urlfetch from appEngine and lighttpd HTTP 400 response

Added by Anonymous over 11 years ago. Updated over 3 years ago.

Status:
Obsolete
Priority:
Normal
Assignee:
-
Category:
core
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Missing in 1.5.x:
Yes

Description

Hi,
I started using urlfetch from google's appEngine (http://code.google.com/appengine/docs/urlfetch/overview.html) and I noticed that it doesn't work for some sites powered by lighttpd. I tried the example given in the link above but replaced the URL with lighttpd.net:


from google.appengine.api import urlfetch

url = "http://www.lighttpd.net/" 
result = urlfetch.fetch(url)
if result.status_code == 200:
  self.response.out.write(result.content)
else:
  self.response.out.write(result.status_code)

I always get back a 400 (Bad Request) response in return. I tried with youtube.com with the same result. I am rather new with Python so it maybe is something I don't do right.

I also tried this on a local lighttpd 1.5 instance with the same result. Here's the debug trace I see:


configfile-glue.c.500: (trace) === start of 1 condition block ===
configfile-glue.c.258: (trace) is condition [3] (global/HTTPhost=~(^|\.)local.com[:0-9]*$) already valid ? nej
configfile-glue.c.521: (trace) [1] result: unknown
2008-07-11 01:55:01: (response.c.137) Response-Header:
HTTP/1.1 400 Bad Request
Content-Type: text/html
Content-Length: 349
Connection: close
Date: Fri, 11 Jul 2008 01:55:01 GMT
Server: lighttpd

Please help!
florin

-- florin


Related issues

Has duplicate Bug #1351: lighttpd returns "400 Bad Request" for Nokia web browser requestDuplicate

Actions
Has duplicate Bug #2208: HEAD method returns 404 if request has Content-Length: 0Duplicate2010-05-21

Actions

History

#1

Updated by stbuehler over 11 years ago

The request would be the interesting thing.

#2

Updated by stbuehler over 11 years ago

The framework sends a "Content-Length: 0" header in a GET request:

rfc 2616, 4.3:


   The presence of a message-body in a request is signaled by the
   inclusion of a Content-Length or Transfer-Encoding header field in
   the request's message-headers. A message-body MUST NOT be included in
   a request if the specification of the request method (section 5.1.1)
   does not allow sending an entity-body in requests. A server SHOULD
   read and forward a message-body on any request; if the request method
   does not include defined semantics for an entity-body, then the
   message-body SHOULD be ignored when handling the request.

So i guess google shouldn't send the content-length header in that case.

#3

Updated by ralf about 11 years ago

stbuehler wrote:

The framework sends a "Content-Length: 0" header in a GET request:

rfc 2616, 4.3:

[...]

So i guess google shouldn't send the content-length header in that case.

hehe, the words of a developer.

i think lighttpd should read 0 bytes (as lighttpd do if there is no content-length header).

Every (ok, the mostly used) webserver - Apache - works with the request.

#4

Updated by james82@gmail.com over 10 years ago

  • Status changed from New to Patch Pending

stbuehler, what are your thoughts on this patch? This patch is copied from Bug #1351, and allows lighttpd servers to be accessed using browsers that send "Content-Length: 0" as one of the request headers. (It turns out that many HTTP implementations do this, including the implementations used by some Nokia and Motorola phones. For compatibility with such browsers we should allow browsers to send "Content-Length: 0" with their requests.)

Index: request.c
===================================================================
--- request.c   (revision 2407)
+++ request.c   (working copy)
@@ -542,7 +542,7 @@ int http_request_parse(server *srv, connection *co
        case HTTP_METHOD_GET:
        case HTTP_METHOD_HEAD:
                /* content-length is forbidden for those */
-               if (con->request.content_length != -1) {
+               if (con->request.content_length != -1 && con->request.content_length != 0) {
                        /* content-length is missing */
                        if (srv->srvconf.log_request_header_on_error) {
                                ERROR("GET/HEAD with content-length: %d", 400);

#6

Updated by stepancheg over 9 years ago

Proposed patch helps.

#7

Updated by bestis over 7 years ago

What's the situation of this? In 1.4.31 it seems to have something as the if is just != 0, but I'm seeing this sometimes as a "storm" of requests.

For example:

2012-07-17 08:50:53: (request.c.1131) GET/HEAD with content-length -> 400
192.89.0.X - - [17/Jul/2012:08:50:23 +0300] "GET /datafiles/styles/style.css HTTP/1.0" 
400 349 "-" "Mozilla/4.0 (compatible; MSIE 4.01; Windows 95)" 

As the log line doesn't contain what is the value of content-length it's hard to say how badly the client behaves. But after grepping logs it seems to be from same company all the time. So probably they are doing something wrong and it works for the most of the people.

#8

Updated by stbuehler about 7 years ago

  • Assignee deleted (jan)
  • Missing in 1.5.x set to No

I think 1.4 "always" accepted "Content-Length: 0" for HEAD and GET.

#9

Updated by gstrauss over 3 years ago

  • Missing in 1.5.x changed from No to Yes
#10

Updated by gstrauss over 3 years ago

  • Status changed from Patch Pending to Obsolete

Also available in: Atom