[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-core-devel
Subject:    kio_http, persistent connections
From:       Rik Hemsley <rik () kde ! org>
Date:       2001-06-04 15:35:01
[Download RAW message or body]

I'm busy implementing persistent connection support in my web server
(kpf).

My server supports simultaneous-connection limiting, intended to keep
resource usage down.

While testing persistent connection support using konqueror, I noticed
that if I set a low value (e.g. 4) for the simultaneous connection
limit, there is a problem with kio_http.

When requesting a web page which includes quite a few (about 10) small
images, it appears that kio_http creates about 4 connections to the
server. This, I would guess, is to allow 'parallel' retrieval of the
various components that make up the page.

kio_http tells the server that it wants to keep the connection open (it
doesn't send "Connection: close" and it explicitly sends "Connection:
Keep-Alive").

This is fine, if kio_http will send "Connection: close" in a future
request on the same connection. Unfortunately, it doesn't. Instead, it
seems try to open another connection to the server instead.

This happens when kio_http has not yet retrieved the full response from
the server.

According to RFC2616, clients should simply pipeline requests without
waiting for a response from the server.

I can see that pipelining all the requests necessary for one page, on
one connection, would lose the benefit of parallelism, so it makes sense
that kio_http would use more than one connection.

If kio_http were to limit itself to 4 connections and, when it has
reached this limit, send further requests (pipelined) on the existing
4 connections, with "Connection: close" as a header in the final
request, there would be no problem.

With the current implementation of kio_http, I need to set quite a
high simultaneous connection limit in the server, to handle the case
when kio_http fires off what appears to be an unlimited number of
connections.

I can see that generating "Connection: close" intelligently must be
difficult, as kio_http would have to know some information about
the web page it is accessing, i.e. the number of images (etc.) that
will be requested. Even with this knowledge, a perfect implementation
would be difficult, as the user may quickly select a link on the
page, so some requests are cancelled and new ones are added.

I think that the best solution would simply be to ensure that no
more than 4 connections are made to one host.

I would still need a large simultaneous limit in my server, because
kio_http does not close the connections when finished - it just leaves
them to timeout.

After a page with a few graphics has been accessed, I will have 4
connections waiting 60 seconds to timeout, so the connection pool will
still be artificially filled, but this is better than > 4 connections,
at least.

Perhaps I will have to decide that persistent connections are a bad
idea, due to the way that kio_http makes connections (and fails to
disconnect when it should.)

In that case, I will simply go back to sending "Connection: close"
with each response, though I would prefer not to do this, as
persistent connections can save a lot of time and resources.

Rik

[prev in list] [next in list] [prev in thread] [next in thread] 

Configure | About | News | Add a list | Sponsored by KoreLogic