[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-core-devel
Subject:    Re: KIO: Mass Copy of Files from Different Sources to Different
From:       David Faure <faure () kde ! org>
Date:       2009-08-19 15:12:12
Message-ID: 200908191712.12871.faure () kde ! org
[Download RAW message or body]

On Thursday 18 June 2009, Alejandro Wainzinger wrote:
> Using KIO::copy( ... ) to generate more than 150 jobs causes the KIO
> scheduler to complain:
>
> "WARNING - KIO::Scheduler got more than 150 jobs! This shows a misuse
> in your app (yes, a job is a QObject)."
>
> What misuse is this, isn't the KIO::Scheduler supposed to schedule
> jobs?  I notice that you can directly tell the scheduler to schedule a
> job with KIO::Scheduler::scheduleJob(...) which will queue jobs when
> no slaves are left over and possibly avoid this error, but that only
> works for SimpleJobs, and a CopyJob is not a SimpleJob.
>
> Up until now I've worked around it by manually keeping track of jobs I
> create and jobs that finish so as not to make the Scheduler angry, but
> shouldn't this be part of the Scheduler's job?
>
> I notice most applications don't run into this issue because they use
> KIO to copy a folder that could contain more than 150 files which is
> fine, or copy more than 150 files to a single destination, but in my
> case each file I copy can come from a different source and go to a
> different destination so I can not aggregate the jobs.

This could be another reason for "all jobs should be scheduled",
as we were discussing in another thread ("Patch KIO::AccessManager").

I just gave it a try, as an experiment. Shows that scheduleJob has to support
being called twice for a job, but apart from that, the unittest "jobtest" 
still passes, which is good enough for me :-)

Can you give the attached patch a try?

-- 
David Faure, faure@kde.org, sponsored by Qt Software @ Nokia to work on KDE,
Konqueror (http://www.konqueror.org), and KOffice (http://www.koffice.org).

["schedule_all.diff" (text/x-patch)]

Index: job.cpp
===================================================================
--- job.cpp	(revision 1012540)
+++ job.cpp	(working copy)
@@ -308,6 +308,7 @@
     }
 
     Scheduler::doJob(q);
+    Scheduler::scheduleJob(q); // experiment
 }
 
 
Index: scheduler.cpp
===================================================================
--- scheduler.cpp	(revision 1012540)
+++ scheduler.cpp	(working copy)
@@ -401,15 +401,17 @@
 }
 
 void SchedulerPrivate::scheduleJob(SimpleJob *job) {
+
     newJobs.removeOne(job);
     const JobData& jobData = extraJobData.value(job);
 
     QString protocol = jobData.protocol;
 //    kDebug(7006) << "protocol=" << protocol;
     ProtocolInfo *protInfo = protInfoDict.get(protocol);
-    protInfo->joblist.append(job);
-
-    slaveTimer.start(0);
+    if (!protInfo->joblist.contains(job)) { // scheduleJob already called for this job?
+        protInfo->joblist.append(job);
+        slaveTimer.start(0);
+    }
 }
 
 void SchedulerPrivate::cancelJob(SimpleJob *job) {


[prev in list] [next in list] [prev in thread] [next in thread]