[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kdepim-users
Subject:    Re: [kdepim-users] Fwd: KMail + IMAP painfully slow
From:       Ingo =?iso-8859-1?q?Kl=F6cker?= <kloecker () kde ! org>
Date:       2007-05-16 21:18:02
Message-ID: 200705162318.05013 () erwin ! ingo-kloecker ! de
[Download RAW message or body]

[Attachment #2 (multipart/signed)]


On Wednesday 16 May 2007 21:30, Ross Boylan wrote:
> > I've spent the day watching KMail be unresponsive (using most of
> > the CPU, not updating the screen, not responding to the mouse)
> > using a local (Cyrus) IMAP server.
> >
> > Moving messages takes many minutes, sometimes (even when it's not
> > many messages).
>
> Ingo asked (sorry about the single > for all quotes)
>
> > Moving messages from one IMAP folder to the other should be quite
> > fast because it happens on the server.
>
> I can confirm this is quite fast when I do it with other programs.
>
> > What happens during those many
> > minutes? Is KMail eating CPU like mad? Or does is just hang and do
> > nothing?
>
> KMail eats CPU like mad.  It does so in all the scenarios I
> described.  I noticed today that when I moved some messages to a
> folder the columns on the left for unread and total started updating
> after I move the messages in.  It appears that after the messages are
> moved in KMail scans the target folder; I'm guessing kmail builds up
> its indices.

If the total column is visible then KMail updates it at least on the 
first start to get an initial value. You might try whether disabling 
the total column helps.

Also, the first time KMail accesses a folder it does indeed create its 
indices.

> While it was doing this I attempted to conduct a new quick search by
> typing into the search bar.  The program did not respond to this
> until it finished whatever it was doing with the target box.  The
> target box was not open in my display (except for showing on the left
> hand pane).

Yes, there are still lots of blocking operations in KMail.

> I've also noticed some strange behavior.  I had about 7000 messages
> on a remote server.  After I moved them to the local server I opened
> the remote folder.  kmail showed the messages that were previously
> there in the upper right pane (this is with the default setup for
> panes--not sure what their official name is).  It then started
> reducing the counts shown on the left, and eliminating message lines
> shown in the upper right.
>
> I got bored waiting for this to finish and I switched to another box.
>  This made the countdown process stop.  When I switched back to the
> now-empty box kmail said "no new mail" and left the display as it
> was, i.e., the left pane showed the  box had quite a few messages,
> and the upper riight pane also showed quite a few messages.  I was
> eventually able to force a reread (I think I switched to another box
> on the remote server and then back to the empty one).

Sounds like an update problem.

> Perhaps consistent with this, I have a lot of boxes which show 
> partial counts in the unread column and nothing at all in the Total
> column.  When I switch to them these values update.  So perhaps what
> I'm seeing is the result of other times I switched into and then out
> of a folder.

KMail caches the last known values.

> Finally, I once switched into a box after having accessed the
> contents of the box from another program (I can't recall if it was a
> read or a read and delete).  This was on the remote server.  When I
> switched into the box the left hand display started counting down,
> and the upper right started deleting headers.  This was very
> alarming, since it went way beyond the number (if any) I had deleted.
>  I switched out of the box and used another program to copy its
> entire contents somewhere else.  The I reopened the folder in kmail,
> which eventually showed the proper counts and contents.  I thought it
> was deleting messages, but I guess it was again updating its indices,
> since the mail did remain in the folder.

Yeah, I guess so.

> Since the information in the counts is available from a single call
> to the IMAP server I conclude that kmail is not getting the
> information that way, but is instead building its own indices.

I'm not sure how we do get the count. It's quite possible that we just 
count the number of messages marked as unread in our caches instead of 
using the number given to us by the IMAP server so that we can use the 
same code also for local folders (where we have no other choice but to 
count ourselves).

> > Getting the text of a message to display can take forever.
> >
> > I disabled automatic checking, and this seemed to help some.  The
> > manual check remains slow, I guess because it insists on visiting
> > every folder.
> >
> > Am I doing something wrong?  Is my mailstore bigger than KMail was
> > designed for?  Or is KMail just not that good at IMAP?
>
> Ingo replies
>
> > Especially with IMAP the size of the mailstore should be more or
> > less irrelevant. A possible bottleneck is the fact that we use the
> > mbox format for caching the message headers.
>
> I suspect that the slowness is the result of the local indices,
> caches, or whatever that kmail is creating.  Perhaps once they are
> fully created things aren't so bad.

Yes, I think so too. The first import of a large IMAP account does use a 
lot of resources but afterwards it should work okay. It does for me, 
but my largest IMAP folder only holds about 12000 messages.

> I experience kmail as semi-multithreaded.  It does seem to be doing
> several things at once sometimes (e.g., updating counts on the left;
> showing a moving progress bar on the lower right; checking mail;
> displaying messages I've selected).  But often it locks up while
> doing them.  Since threads are designed to support a responsive user
> interface while background processing happens (e.g., updating
> index/cache for a folder), and since the interface is often
> unresponsive, I'm not sure what's going on.

KMail is single-threaded, but we make extensive use of Qt's signal-slot 
mechanism together with splitting jobs (like moving loads of messages) 
up into many small tasks (like moving a few messages) to keep the GUI 
responsive. As you have noticed this is not used for every operation so 
KMail does still lock up during some operations.

> Also, as I reported above with the folder I emptied, kmail sometimes
> seems to get confused when I interrupt a task by switching folders or
> mailboxes.

We do have a few bottlenecks in updating the message list (i.e. the 
upper right pane). This list is often redrawn after each change and 
that's painfully slow if lots of changes happen. When you leave the 
folder then redrawing is obviously no longer necessary. I guess this 
explains some of the behavior you saw.

> > I have 2 imap accounts; a small remote one and a large local one. 
> > The local one has about 115 folders (with several levels of
> > nesting), some of which have a lot of messages (several in the 10's
> > of thousands, the largest around 80k).  KMail's area under .kde is
> > about 335Mg.  I have yet to visit most of the folders.
>
> Ingo asked
>
> > Do the problems occur with both accounts?
>
> Primarily the large local one, though as noted above I've seen some
> oddities with the remote one too.
>
> > kmail 3.5.5.dfsg.1-6 (the about panel says it's 1.9.5) on Debian
> > GNU/Linux. 3GHz Pentium4 with a decent SATA disk.
> >
> > It's doing better than evolution, which can't define the accounts,
> > and arguably better than mutt, which hangs every time it exits a
> > folder.
>
> P.S. about the quick search: I keep entering the search term and then
> hitting enter.  The enter brings up the currently selected message, I
> think.  You might consider making the enter key a no-op in the search
> bar.  I'm accustomed to enter meaning "do the search".  I guess that,
> since the search is incremental, that's redundant in this case.

You might want to file a bug/wish at bugs.kde.org for this.

Regards
Ingo

[Attachment #5 (application/pgp-signature)]

_______________________________________________
KDE PIM users mailing list
kdepim-users@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-users


[prev in list] [next in list] [prev in thread] [next in thread]