[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kmail-devel
Subject:    Re: for kmail people: a discussion of mail summary files
From:       Don Sanders <don () sanders ! org>
Date:       2000-05-16 6:34:36
[Download RAW message or body]

Another thing we need to do is reduce the size of the index file. Just loading
this file is going to take a susbtantial amount of time for large folders.

Strip the email address out of "From" and "To" fields, cut down size from 100
chars to 50 say. Use MD5 checksums on message id and parent id (or whatever
it's called) and resize them from 100 to 24 chars (or whoever many chars is
needed to store a 128bit value in ascii). That will halve the size of the index
file (down from 541 to 291 IIRC)

Create new methods FromStripped, ToStripped, MsgIdMD5, ReplyIdMD5 in KMMsgBase.
Use these when creating KMHeaderItems.

If we flyweight KMMsgInfos, KMHeaderItems and do this we should see a marked
improvement in speed (at least for non-threaded folders).

BFN,
Don.

On Tue, 16 May 2000, Don Sanders wrote:
> On Tue, 16 May 2000, Waldo Bastian wrote:
> > On Mon, 15 May 2000, Don Sanders wrote:
> > > Something that takes up a lot of time in KMail when
> > > showing a list of message headers (if not taiking up the most time) is the
> > > cost of operator new. A KMMsgInfo object and a QListViewItem object has to
> > > be constructed for each message header in the list. This means a lot of
> > > calls to operator new which is very slow. I tried to speed this up by
> > > recycling QListViewItems instead of deleting them and creating new ones
> > > this helped a lot but I never really finished this work. 
> > 
> > Do you use KMMsgInfo::init() ?? I just noticed that the QString arguments 
> > misses a "&". E.g. it is "const QString subject" instead of "const QString 
> > &subject". Changing that spares a call to the QString-copy operator.
> > 
> > (They are missing at other places as well I see...)
> 
> I wouldn't think it makes much of a difference since QString is shared
> (reference counted).
> 
> > Something else you might consider is to use a bucket-allocator as replacement 
> > for new: You basically allocate a large array of non-initialized KMMsgInfo 
> > objects and make a linked-list from the ones you don't use. This allows you 
> > to allocate memory for new KMMsgInfo objects quite fast. Of course the actual 
> > allocation is only part of the time spent by new, the other time is spent in 
> > the constructors of the various objects inside it.
> 
> Yes, yes, this is what I was talking about, it's called the "Flyweight" pattern
> in "Design Patterns". It does speed things up a lot, I made a half-hearted
> attempt to do it for KMHeaderItems (derived from QListViewItem) in
> kmheaders.cpp.
> 
> Actually you might as well throw away the array and just use the linked
> list. For a further improvement keep a count of the number of items on
> the linked list and 
> while (number-of-items-on-linked-list + number-of-items shown is greater than
> say 5000) {
>    delete items from the list.
>   stopping when list is empty
> }
> 
> > It makes only sense if you have a large number of the same objects. 
> 
> yes we do, both for KMMsgInfo and KMHeaderItem.
> 
> some comments from kmheaders.cpp
> 
> <quote>
>  // About 60% of the time spent in populating the list view in spent
>   // in operator new and QListViewItem::QListViewItem. Reseting an item
>   // instead of new'ing takes only about 1/3 as long. Hence this attempt
>   // reuse list view items when possibly.
>   //
> 
> <cut>
> 
>    // We can gain some speed by reusing QListViewItems hence
>     // avoiding expensive calls to operator new and the QListViewItem
>     // constructor
>     //
>     // We should really do this by takeItems out of the QListView and
>     // store them in a list, and then resuse them from that list as
>     // needed
>     //
>     // The following is a bit of a cludge to achieve a similar end
>     // (when moving a folder to another folder with around the same
>     // number of items) but it does unspeakable things if nested
>     // messages is turned on.
> </quote>
> 
> If nested messages are turned on you must take all the children of an item
> before taking its parent otherwise KMail will crash.
> 
> BFN,
> Don.

[prev in list] [next in list] [prev in thread] [next in thread] 

Configure | About | News | Add a list | Sponsored by KoreLogic