[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kmail-devel
Subject:    Re: kmail
From:       Don Sanders <don () sanders ! org>
Date:       2000-05-03 1:41:13
[Download RAW message or body]

> Example (real world, folders just renamed for easier reading):
> * I've have folders A, B and C
> * A --> 5 messages
> * B --> ~4500 messages
> * C --> ~9000 messages
> 
> 1. After startup, VMDATA is ~1.9 MB
> 2. After selecting folder B, VMDATA goes up to ~9.5 MB
> 3. After selecting folder A again, memory stays at 9.5
> 4. After selecting folder C, memory goes up to 17.2 MB
> 5. After selecting folder A again, it stays at 17.2
> 6. After selecting folder B, marking all messages, and using "Message/Set 
> Status/Read", memory goes up to 57MB
> 7. After selecting folder A again, memory stays at 57 MB
> 8. After selecting folder C, marking all messages and mark all read, memory 
> goes up to 109MB
> 9. After repeating step 6, memory only slightly increases to 111MB
> 
> As it seems, we have HUGE malloc fragmentation here. I'll try to check this 
> too.

Right so there are two problems here.

The first I have already talked about, a lot of actions like setting message
status are inefficient and load all messages operated on at once. By
relying on a KMMsgPtr KMFolder::getMsgPtr(int idx) method that returns a
reference counted KMMessage you should be able to one by one analyse these
actions (empty folder, move message, copy messages, set status of messages,
compact folder) and reduce their memory use to a sensible level.

Furthermore the horrible
KMMessage* KMHeaders::getMsg (int msgId)

needs to be eliminated entirely and the

KMMessageList* KMHeaders::selectedMsgs(int idx)

method needs to be updated to return a list of int messages indexes rather than
a list of messages.


The second problem is that loading the list of message headers is taking up a
lot of ram. In your other message you talked about using a memory mapped file.
I don't think this will help much. The problem is that for each message header
a QListViewItem is created. This QListViewItem stores various strings used for
sorting and for displaying text.

You will be able to reduce the amount of memory allocated by a significant
amount by using MD5 checksum for the messageId and messageReplyId fields. This
will reduce the size of an index entry from 541 bytes down to say about 390
bytes. You could only store a personas name (and not their to and from
address) in the index and use only 50 bytes instead of 100 for each of these
new fields. This way you should also be able to benefit from the data sharing
of QString.

Some of the relevant methods to revise here are

const QString KMMsgBase::asIndexString(void) const
int KMFolder::createIndexFromContents()
const QString KMMsg*::replyToId(void) const
const QString KMMsg*::msgId(void) const

(The last two should probably have their names changed to replyToIdMD5 and
msgIdMD5 too, you will need new method for fromName, and toName as stored in
the index as well).

By using MD5 checksums I mean instead of storing a messageId/replyId store the
MD5 checksum of the messageId, preferably BASE64 encoded.

BFN,
Don.

[prev in list] [next in list] [prev in thread] [next in thread]