'Re: [Kde-pim] Review Request 117975: Change libmaildir listNew and listCurrent functions to explicet'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-pim
Subject:    Re: [Kde-pim] Review Request 117975: Change libmaildir listNew and listCurrent functions to explicet
From:       "Martin Steigerwald" <martin () lichtvoll ! de>
Date:       2014-05-15 10:09:18
Message-ID: 20140515100918.26550.9552 () probe ! kde ! org
[Download RAW message or body]

> On May 5, 2014, 3:08 p.m., Sergio Luis Martins wrote:
> > Commit to stable too ?
> 
> Laurent Montel wrote:
> Please wait after 4.13.1 it will tag the 8 may.
> Apply it in 4.13.2 if safe
> thanks
> 
> Martin Steigerwald wrote:
> Okay, will wait. I think its good if someone else than me actually also tests it. \
> For me it works really nicely. 
> Martin Steigerwald wrote:
> Hi! So did anyone test with it? It works very well for me. I see higher I/O \
> throughput on accessing folders, but I think thats just due to the fact that there \
> is way less CPU overhead now. It will stat() mail files in random order instead of \
> alphabetically now, I think. Depending on the filesystem implementation and the \
> order in which those files will be created, this may make a difference â€“ \
> accessing in inode number order may help on some filesystems. In any case, \
> subjectively I think it is way better than before. What do you experience? 
> So did anyone test and okay with committing to 4.13.2? How can I do it? Rebase my \
> nosorting branch to 4.13.2 branch and then merge my nosorting branch with the two \
> commits into 4.13.2 branch? I did so with my commits to master branch, so I think \
> this would work.

Andras wrote:
> I updated master after the commit and didn't see any bad sideeffects. Can't
> tell if it is faster or not.

Okay, don ´t see your comment on reviewboard. How large is your maildir?

find ~/.local/share/local-mail -type f

martin@merkaba:~> du -sh ~/.local/share/local-mail
11G     /home/martin/.local/share/local-mail
martin@merkaba:~> find ~/.local/share/local-mail -type f | wc -l
875128

Anyway, I will commit this to 4.13.2 in a few days if no one objects.

- Martin

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://git.reviewboard.kde.org/r/117975/#review57321
-----------------------------------------------------------

On May 5, 2014, 12:07 p.m., Martin Steigerwald wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://git.reviewboard.kde.org/r/117975/
> -----------------------------------------------------------
> 
> (Updated May 5, 2014, 12:07 p.m.)
> 
> 
> Review request for KDEPIM, KDEPIM-Libraries, Andras Mantia, David Faure, and Sergio \
> Luis Martins. 
> 
> Bugs: 334218
> http://bugs.kde.org/show_bug.cgi?id=334218
> 
> 
> Repository: kdepim-runtime
> 
> 
> Description
> -------
> 
> On trying to find the cause for Akonadi maildir resource hogging up a Sandybridge \
> core for minutes without any notable MySQL or disk activity, Sergio recommended \
> doing a callgrind run. This revealed a bottleneck with QDir.entryList and there \
> specifically in QAlgorihmsPrivate::qSortHelper[1]. 
> Calling setSorting(QDir::NoSort) on the QDir object gets rid of the sorting. This I \
> did twice in maildir.ccp and twice in keycache.cpp for listCurrent and listNew \
> functions there. 
> It is not yet clear whether this change is safe, but on my tests it appears to do. \
> Moving and filtering mails still work okay. I will CC Andras on this as Sergio \
> tried to ping him about this change. 
> I think it is fair to say that this change has a *huge* impact on Akonadi maildir \
> performance with large maildir and I recommend to backport this for stable branch \
> as well once changing to no sorting is considered to be safe. 
> Thanks to Sergio and David for pointing out the sorting issue, telling how to \
> change to no sorting and general help. 
> See also:
> 
> Bug 334218 - synchronizations of large folders with filesystem contents hogs a \
> Sandybridge core for minutes https://bugs.kde.org/334218
> 
> [1] https://bugs.kde.org/show_bug.cgi?id=334218#c4
> 
> [2] Bug 334206 - While maildir resources synchronizes a folder KMail blocks on \
> switching to a different folder: https://bugs.kde.org/334206
> 
> 
> Diffs
> -----
> 
> resources/maildir/libmaildir/keycache.cpp f0af9c47cef63dac45ae68dd94fa17a040dc1593 
> resources/maildir/libmaildir/maildir.cpp 9bd380201a1a250ecfe354dd99946e8dae0ab668 
> 
> Diff: https://git.reviewboard.kde.org/r/117975/diff/
> 
> 
> Testing
> -------
> 
> In my tests with KMail SC 4.12.4 Debian unstable packages and Akonadi and \
> kdepimlibs from Git it works well. Well basically Akonadi maildir resource changed \
> from being a CPU hog for minutes to hardly every appearing in a 10 second \
> measurement average snapshot of atop anymore. KMail hardly blocks on folder changes \
> anymore, which I still believe to be a different issue tough[2]. 
> Mail filtering and moving mails to a different folder still works ago. And KMail is \
> subjectively much faster. It completely changes my KMail experience from being \
> unbearable at times to quite pleasant. 
> 
> Thanks,
> 
> Martin Steigerwald
> 
> 

_______________________________________________
KDE PIM mailing list kde-pim@kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/

[prev in list] [next in list] [prev in thread] [next in thread] 