[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kmail-devel
Subject:    Re: PATCH: Asynchronous filtering
From:       Marc Mutz <mutz () kde ! org>
Date:       2003-10-27 19:16:12
[Download RAW message or body]

[Attachment #2 (multipart/signed)]


On Sunday 26 October 2003 21:27, Till Adam wrote:
> On Sunday 26 October 2003 20:24, Marc Mutz wrote:
<snip>
> The idea is to get a bunch
> of messages and hand them off to the filtering subsystem as quickly
> as possible to be able to give back control to the user sooner. The
> overall time spent filtering is of secondary concern if the user can
> continue working while it happens. Or so the theory.

I do understand the goal of the changes, of course. I just don't want 
threading introduced into KMail except for those very special cases 
like compaction, provided the complexity payed in locking issues is 
well worth it. Consider for a moment the result of a compator thread 
locking a folder and the user clikcing on that folder. With simple 
QMutex-protection, the GUI-thread will _block_, waiting for the 
compactor to release the lock.

> > > - there is only a one time overhead of thread creation at
> > > startup, which is rather small, and no kprocess overhead for each
> > > filtered mail
> >
> > Cough, cough. In the "pipe through" action (and as I understand it,
> > that's what current ThreadWeaver is all about, right?), you still
> > need to fork off a child processes, right? How else would you
> > implement pipe through "/usr/bin/spamc"
> > ?
> >
> > This argument is thus void, right?
>
> Yes, the popen is still there, but no KProcess. If that is
> negligible, oh well, yes, the argument is void.

How can there be no KProcess (or fork()/exec() pair) in _any_ 
implementation of a pip-through action???

> > > - we want to encapsulate other tasks, such as pgp
> > > signature verification, trust db rebuilding etc into weaver jobs
> > > as well
> >
> > Do we? You know, starting child processes (and gpg and the
> > pipe-through commands _are_ and ever will be child processes) on a
> > multiprocessing OS like Unix in _threads_ to avoid blocking makes
> > so unbelievably no sense that it is rediculous. What could
> > _possibly_ be the benefit of using threads if we use them only to
> > fork off child processes? Hello? select()? QEventLoop?
> > QSocketNotifier? KProcIO?
>
> Be my guest. :) No one has implemented the solutions you propose,
> although they are of course perfectly doable. The weaver is a very
> straight forward way to encapsulate tasks that would otherwise block
> the gui and be notified when they are done. It might not be the most
> efficient way to do it, but it's simple and works. It abstracts away
> a lot of the unpleasantness of dealing with asyncronous jobs. Feel
> free to provide us and the rest of kde with a similarly straight
> forward implementation of that using processes.

Come on, you don't want to sell me threads as being easier than 
KProcess, now do you? If that's true for you, you most likely forgot 
the locking issues. "Works for me" doesn't count anymore when doing MT. 
You can't really test MT programs. They just fail when the other CPU 
skips a cycle and that will never be reproducable again :-(

I know you know all this, but I wonder why you ignore it?

> > > Move actions come to mind as well.
> >
> > No, they don't. KIO slaves are already async external processes.
> > That eliminates POP as source and IMAP folders as source and target
> > of threaded move operations. Moves between maildirs are atomic
> > moves, moves from/to maildir folders are file operations the KIO
> > subsystem can handle internally, so no need for threading, either.
> > That leaves mbox folders as targets and sources of mails. Why
> > should those be threaded? They're not compute-intensive at all?
>
> Again, this is not about performance. It's about gui responsiveness.
> Try moving 20000 mails from maildir to maildir folder. Atomic for
> each mail or not, it takes forever and completely blocks the gui for
> _minutes_. Sure, singleShotTimers, event loop, all perfectly doable.
> But why bother, if there is a tool for the job?

Yes, exactly! Why bother if there's KIO::move()? :-o

My words: You use threads to cover up the fact that KMail is too C-ish 
and ignorant of chunking and KIO in that area. ;-)

> > Summary: Using threads for anything that doesn't involve heavy
> > computations is IMNSHO trying to cover up the realities of a bad
> > design. Whether you post an event or you emit a signal is not that
> > different.
>
> Summary: KMails blocking operations (and there are several) are IMHO
> its one major flaw. Apart from that it is an exellent tool for the
> job. Users have very low tolerance for blocking guis and there are
> perfectly avoidable. We need a way to perform those operations
> without blocking the gui and using threads (especially with the
> ThreadWeaver) is a straightforward way to achieve that.

You sell threads as if it's easy as 123.
$ grep -Ri mutex asyncfiltering/ | wc -l
0
Yes, _then_ it's easy ;-)

> > I'm open to new arguments for using threads to do work that is not
> > compute-intensive, but I believe those can be rebutted just as
> > easily. :-)
>
> Marc, believe it or not, in principle I agree.

I do believe you :-)

> Threads are not the be 
> all and end all of software design. But they are a valuable tool. A
> blunt axe, maybe, but it gets the fucking wood cut, if you catch my
> drift ;).
<snip>

Yes, except that it's not about cutting wood, but more about peeling 
oranges. Of course, you _can_ use an ax for that, but who does? I'd 
rather take a sharp knive. :-)

Marc

-- 
[...] the USA needs a German product: Vergangenheitsbewältigung
   -- Craig Morris: US boycott of products from countries against war?
      www.telepolis.de



[Attachment #5 (application/pgp-signature)]

_______________________________________________
KMail Developers mailing list
kmail@mail.kde.org
http://mail.kde.org/mailman/listinfo/kmail

[prev in list] [next in list] [prev in thread] [next in thread]