Project

General

Profile

Feature #208

Limit connections per IP (patch)

Added by Anonymous about 14 years ago. Updated over 2 years ago.

Status:
Missing Feedback
Priority:
Low
Assignee:
-
Category:
3rd party
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Missing in 1.5.x:

Description

Would be awesome to be able to limit a number of connections per ip address for each different domain.

So, for example, I have a file archive and when people use download managers, they open 10-80 connections to download files faster. This kicks out my HDD. I found a way in stopping people from using RANGE requests, but this isn't a perfect solution.

So as mentioned above, I'd want to limit number of connections per each ip.

-- lighttpd.net

mod_throttle.c (14.5 KB) mod_throttle.c -- petopeto Anonymous, 2008-06-15 06:28
throttle-origurl.diff (2.75 KB) throttle-origurl.diff -- petopeto Anonymous, 2008-06-15 06:28
example.conf (1.14 KB) example.conf -- petopeto Anonymous, 2008-06-15 06:29

History

#1

Updated by Anonymous almost 14 years ago

I would agree, a feature/module to limit connections per IP/given time would be of great use.

I hope to see this feature soon, it is especially useful when serving a high traffic server and to prevent hotlinking, etc.

-- aaron

#2

Updated by Anonymous almost 14 years ago

I too am interested in limiting connections per IP, please add this function!

-- GM

#3

Updated by Anonymous almost 14 years ago

Also voting for this one, would make a huge difference for us

-- toasty

#4

Updated by Anonymous about 13 years ago

I'd like to see this too ;)

-- joel

#5

Updated by Anonymous almost 13 years ago

Check this out! :)
http://trac.lighttpd.net/trac/ticket/195

-- DoomStar

#6

Updated by Anonymous over 11 years ago

evasive doesn't allow anything but refusing the request entirely; it's not a good solution for most uses. Instead, mod_throttle.

- each configuration block defines a simple token bucket and maximum concurrency
- token bucket allows limiting the rate of requests; maximum concurrency limits the number of simultaneous requests
- requests beyond limits are simply delayed; when a request finishes, the next one(s) are started in a roughly FIFO manner
- multiple blocks can be configured.

The original purpose of this is for limiting expensive requests; in our case, we wanted to allow only 2-3 concurrent Rails requests per second. Since Rails servers are 50-100 megs a piece, a reliable server can't let a single user monopolize them, and our application allows several long-running requests which can hold one up for several seconds.

At the same time, we can limit the number of large simultaneous file downloads to, say, 2. This doesn't break download managers; if they start a lot of threads on the same file, two will start and the rest will simply block until others finish (though in practice the client will probably time them out and restart them; this is harmless).

Token buckets are the normal paradigm: you start with a specified number of tokens, each request uses one token, and the tokens refill back to the max at a specified rate. This gives some flexibility in limiting the connection rate, by allowing bursts of activity to avoid affecting responsiveness. Each rule has its own separate token buffer, so you can give different limits to different things (eg. pages of thumbnails may need to allow more rapid requests than Rails requests).

Any rule can be defined as a "ban" rule; if a request in the rule is denied, the IP is banned for a specified length of time (all requests are refused).

Lighttpd's configuration syntax makes this a bit of a hack in cases. Each configuration conditional becomes a separate rule, so to set up two independent rules on the same conditional (eg. a throttle rule and a ban rule), you need to use different conditionals or the configurations will be merged:
$HTTPurl =~ "/requests" {
throttle.max-concurrent-connections = 3
}
$HTTPurl =~ "/requests|abcdefg" {
throttle.bucket-size = 300
throttle.tokens-per-second = 1
throttle.ban-when-empty = 1
}

This also adds the $HTTPorigurl conditional, which exposes con->request.orig_uri. This is needed to match against the real request when using error-handler-404 for fcgi requests; otherwise only the redirected URL can be matched.

This could probably use more cleaning up, but with another ticket I've opened with a simple patch not getting a reply in three weeks (#1675), I don't really expect this big one to be applied (is development dead?)--I'm just making this available to people who google their way here. Patch is in the public domain (merge it if you like); this code has been live on a medium-traffic site for several weeks and seems stable. Patch doesn't add documentation.

tokens-per-second should refill smoothly; 2 should start a new connection every half second, not two connections every second. There's no timer interface to do this and I don't need it right now.

One significant limitation: if a client starts an expensive Rails request, and then closes the connection, the connection closure is not communicated to Rails, so the expensive request will continue but the user will get his parallel connection back and can start another. Whether this is a lighttpd fcgi bug or a Rails bug or somewhere in between I'm not sure; fcgi does specify how to communicate dropped connections, and it's important that Rails code be able to catch a signal of some kind (eg. a callback that can return or throw) to decide whether or not to continue with the request after the client has disappeared.

(There's no good reason for this code to be C instead of C++, which would make it substantially simpler, and make it trivial to use better data structures for some of the data, which would make it faster; I can't be bothered to do that in C and it's fast enough for my use.)

-- petopeto

#7

Updated by Anonymous about 11 years ago

I am a newbie in using lighttpd. Is there any instruction to compile this module?

#8

Updated by gstrauss over 3 years ago

  • Description updated (diff)
  • Category changed from core to 3rd party
  • Assignee deleted (jan)
  • Priority changed from Normal to Low

For #2199, "[mod_evasive] redirect if maximum connections exceeded", a new evasive.location directive was added to allow redirection when connection limit is reached, instead of rejecting request. That might be a solution for some use-cases.

#9

Updated by gstrauss over 2 years ago

  • Status changed from New to Missing Feedback

Linked this ticket to Docs_UserWrittenMods

If there is still interest in this module, perhaps some of the desired features could be merged into mod_evasive

Also available in: Atom