[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-core-devel
Subject:    some thoughts on libkio
From:       Lars Knoll <knoll () mpi-hd ! mpg ! de>
Date:       1999-12-06 14:21:42
[Download RAW message or body]

Hi,

I just had a look at the http/1.1 specs, and recognized some features, which
would make loading of from a web server much faster, if we could support them
in kio. Mainle the two features needed are persistent connections and
pipelineing. 

We can sort of do persistent connections already, but we can't do them in a
very good way. To do it the right way, the kio library would have to know,
which slave is connected to which server, so it can schedule http requests
accordingly. Ths means, one opens up a connection to a web server to get the
first file. Then after retrieving the file, more requests to the same server
(and also some to different servers) will follow. If kiolib would know which
slave is connected to a specific server, it could schedule all requests to this
server to the slave connected to it, and start another slave for requests to
other servers. 

If the kiolib doesn't know which server the slave is connected to (the situatin
we have now), then it is just pure luck, if we can use the open connection or
if the slave has to do a reconnect (which takes time).

The second feature would be pipelineing. This is, that one can request a new
file from the server, when the old one still hasent' been completely
transmitted. Basically it allows to request several files at once.

So I would propose an approach which is a bit different to the current one,
which would be very useful for http requests, but also other network related
protocols (like ftp, pop, gopher) could profit from it.

The kiolib can ask a slave to open/close a connection to a specific host. The
slave returns a signal, when it is connected/disconnected.

Then, you have the usual methods for get and put, but the slave returns an 
identifier for this request. When the slave has data for this request, it sends
the identifier along with the data, and a signal, when all data for this
request has arrived.

This would allow a quite easy implementation of persistent connections and
pipelineing in libkio. Then, for example khtml/konqueror could just pass all
files it needs to libkio as soon as they are parsed. kio will do the scheduling
onto different slaves internally. If pipelineing is not available, kio could
just push all requests onto a queue, and process them one after the other,
giving priority to requests, where no reconnect is needed (grouping together
requests to the same host).

This could also help speeding up kio_file, if the slave has the possibility,
to pass over a group of files in one chunk.

What do you think?

Cheers,
Lars

[prev in list] [next in list] [prev in thread] [next in thread]