From kde-core-devel Thu Aug 20 15:04:22 2009 From: David Faure Date: Thu, 20 Aug 2009 15:04:22 +0000 To: kde-core-devel Subject: Re: KIO: Mass Copy of Files from Different Sources to Different Message-Id: <200908201704.22504.faure () kde ! org> X-MARC-Message: https://marc.info/?l=kde-core-devel&m=125078073415748 On Thursday 20 August 2009, Dawit A. wrote: > "Direct scheduling" guarantees that if I make 4 requests one > after the other, all those requests will immediately spawn a kioslave. Which is a dangerous guarantee, if you end up with "FTP server: too many connections". So we need this to be configurable somehow (and I think the somehow is "by the user" rather than "through the kio api", since apps shouldn't have to care whether they're talking to often-limited-FTP or to fish/sftp/smb/file/etc.). > In the scheduled case, no matter what everything is queued and > processed serially one after the other. Are you sure? That would be bad, we don't want that, we want "process things at the pace of N slaves per protocol or host". But this might be what maelcum started working on and that isn't finished, though. In that case, we have to postpone this change indeed. For kio_file I certainly want the behavior where N is quite big, so that things happen in parallel -- but not too much, cf the initial post in this thread. N should be maybe 50, but not infinite. > Well the "not waiting much" is a very important point here. And as an > example I will give you a download manager that provides the user the > option to set the number of connections to make per site. How would it be > able to do that if the default is changed to "Scheduled" and "Direct" is > completely gone ? By having that option being honoured by the scheduler ;-) But this looks like the heart of our disagreement here; I'm thinking of a situation where the kio scheduler would implement that, and therefore -all- apps benefit from "N connections per site" limitations (which would fix a very old kde bug), rather than the user _having_ to use a download manager in order to get that feature. > In addition to the fact that KIO::Scheduler::scheduleJob does not allow > anything but a SimpleJob (dunno why that restriction) to be scheduled, Because only a SimpleJob is talking to an actual kioslave. A CopyJob is a high-level job which takes care of subjobs: one for doing a stat (file or directory?), one for doing a recursive listing of that directory, one job for copying each file, one mkdir job per directory to create, etc. That "high-level" job is totally unknown to the scheduler, since the scheduler is about giving slaves to jobs ;) But the highlevel CopyJob doesn't need any slave for itself, it's just a qobject with timers and slots and subjobs. > the function name itself is the source of confusion. It would have been > better if the function was named queueJobs or something along that line and > allowed you to provide the maximum number of ioslaves that can be spawned > to service any request per site/location. As I said, I don't think that decision belongs to the application. It belongs to the KDE configuration (i.e. the user, with sensible defaults). The fact that ftp.foo.org only accepts 1 connection per IP is not something that all apps should have to care about. Or the fact that the KIO scheduler shouldn't see 150 incoming jobs at the same time. We can take care of such problems inside KIO, without duplicating logic in the apps. An app which -wants- to control this, like a download manager, should of course be able to tune settings in the kio scheduler, I'm not rejecting that possibilty of course, but this is only for special kinds of apps, not something all apps should have to do. Oh and with my patch all kio jobs automatically get scheduled so scheduleJob() becomes completely internal, no need to call it from anywhere, and therefore its name doesn't really matter :-)) -- David Faure, faure@kde.org, sponsored by Qt Software @ Nokia to work on KDE, Konqueror (http://www.konqueror.org), and KOffice (http://www.koffice.org).