Project

General

Profile

[Solved] Some advice on RESTful Application and some info and direction on fixing the "no memory release" feature

Added by garym about 6 years ago

Hi

Great product !!

I'm looking at lighthttpd for a restful project and I wanted to ask advice on an approach. BTW, I was using another package which didn't really work well.

I have to write an high load, low latency, RESTful application that can bidirectionally, concurrently transfer up to 100TB of data on a ipv6 transport. Don't worry about the transport, its provided to me. The URI of the namespace of object where the data is residing "cannot" be exposed in the URI.

1) Question is should I write the application as a custom module or use the fastCGI interface ?

I have a couple of questions on the "memory for later use" issue. From my understanding its a file buffering issue.

2) Will I only bump into the issue on file reads using fastCGI or will writes to a URI manifest the same behavior ?
3) If I write the app as fastCGI, can I hook the raw HTTP request and chunk out my own data ?
4) If I write the app as a module, can I hook the raw HTTP request and chunk out my own data ?
5) Is the problem caused by the buffering of the HTTP message ?
6) Where is the code is this memory/file buffering occurring ? Sorry, I'm not familiar with the code yet. I'll adjust it if its easy only a day or two of work.

b/r
gary


Replies (2)

RE: Some advice on RESTful Application and some info and direction on fixing the "no memory release" feature - Added by garym about 6 years ago

Ok,

I give up... Moving to another platform. Owners don't seem to be interested any more...

Hello, NGINX

RE: [Solved] Some advice on RESTful Application and some info and direction on fixing the "no memory release" feature - Added by gstrauss 7 months ago

While waiting only three days for a response to many detailed questions might be a little impatient, waiting 5 years definitely is not. I hope that you found a solution to meet your needs.

To answer your questions briefly: given the size of data involved, writing a FastCGI server will probably be easier to prototype and to maintain than writing a lighttpd module, though both will be able to do what you have described, except that you should let lighttpd send out HTTP chunked-encoded data rather than doing it yourself.

Since lighttpd 1.4.40, you'll want to use server.stream-request-body = 1 and server.stream-response-body = 1

    (1-2/2)