From kde-core-devel Mon Jun 04 15:35:01 2001 From: Rik Hemsley Date: Mon, 04 Jun 2001 15:35:01 +0000 To: kde-core-devel Subject: kio_http, persistent connections X-MARC-Message: https://marc.info/?l=kde-core-devel&m=99167562432322 I'm busy implementing persistent connection support in my web server (kpf). My server supports simultaneous-connection limiting, intended to keep resource usage down. While testing persistent connection support using konqueror, I noticed that if I set a low value (e.g. 4) for the simultaneous connection limit, there is a problem with kio_http. When requesting a web page which includes quite a few (about 10) small images, it appears that kio_http creates about 4 connections to the server. This, I would guess, is to allow 'parallel' retrieval of the various components that make up the page. kio_http tells the server that it wants to keep the connection open (it doesn't send "Connection: close" and it explicitly sends "Connection: Keep-Alive"). This is fine, if kio_http will send "Connection: close" in a future request on the same connection. Unfortunately, it doesn't. Instead, it seems try to open another connection to the server instead. This happens when kio_http has not yet retrieved the full response from the server. According to RFC2616, clients should simply pipeline requests without waiting for a response from the server. I can see that pipelining all the requests necessary for one page, on one connection, would lose the benefit of parallelism, so it makes sense that kio_http would use more than one connection. If kio_http were to limit itself to 4 connections and, when it has reached this limit, send further requests (pipelined) on the existing 4 connections, with "Connection: close" as a header in the final request, there would be no problem. With the current implementation of kio_http, I need to set quite a high simultaneous connection limit in the server, to handle the case when kio_http fires off what appears to be an unlimited number of connections. I can see that generating "Connection: close" intelligently must be difficult, as kio_http would have to know some information about the web page it is accessing, i.e. the number of images (etc.) that will be requested. Even with this knowledge, a perfect implementation would be difficult, as the user may quickly select a link on the page, so some requests are cancelled and new ones are added. I think that the best solution would simply be to ensure that no more than 4 connections are made to one host. I would still need a large simultaneous limit in my server, because kio_http does not close the connections when finished - it just leaves them to timeout. After a page with a few graphics has been accessed, I will have 4 connections waiting 60 seconds to timeout, so the connection pool will still be artificially filled, but this is better than > 4 connections, at least. Perhaps I will have to decide that persistent connections are a bad idea, due to the way that kio_http makes connections (and fails to disconnect when it should.) In that case, I will simply go back to sending "Connection: close" with each response, though I would prefer not to do this, as persistent connections can save a lot of time and resources. Rik