[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kmail-devel
Subject:    Re: KMMsgDict slowness
From:       Ronen Tzur <rtzur () shani ! net>
Date:       2002-01-03 17:47:54
[Download RAW message or body]


Okay, so now we keep at it just for fun.

On Thursday 03 January 2002 09:30, Michael Häckel wrote:
> On Thursday 03 January 2002 21:39, Ronen Tzur wrote:
> > Because it's not right (in my opinion) to separate serial number from
> > message. If not physically by design in the program, at least
> > conceptually they should be considered a part of the message.
>
> It's not about philosophy here, but about programming :-)

What I really meant was that if Stefan had thought in the beginning
that messages should have unique identifier, then they would (probably)
be coupled tightly by design, and such a splitting of the process
would (probably) not be considered.

  >  > > But X messages per second also counts for the physical move of X 
messages
> > between folders, ie disk I/O.  I think as the dictionary becomes faster,
> > the part of disk I/O becomes more and more dominant factor in the X
> > messages per second.
>
> My hard disc seems to be able to copy about 45MB data per second. If I
> assume an average mail size of mailinglist mails with 4KB, that would be
> 1000 mails per second which is still far away from the speed we have now.

From what I know about hard disks, that is a sequential transfer rate.
Moving messages is not at all sequential, because you append to three
different files for each message, which involves the much slower
disk-head seeks.

But then again, the system usually has write-ahead cache memory, certainly
in the operating system and probably also in the disk itself.

> > So it seems to me that the claim that X messages for a folder with N
> > messages is exactly (X/m) messages on a folder with Nm messages,
> > is not exactly right.
>
> For small folders it is not, but for big folders it is. For big enough
> numbers n every a*n^2 term becomes bigger than every b*n term, even if a is
> much smaller than b.

Alright, that seem logical.   But I would have preferred to put this to test.

> > Also, did you really think that first patch was going to make a
> > difference between ten hours and fifteen minutes??  But you gave the idea
> > of speedying up the dictionary a chance, and it proved itself, at least
> > partially.   Why stop before we've exhausted all the options there?
>
> Yet another 4 bytes per messages means for example for me yet another MB
> more RAM KMail requires and that only because there is some O(n^2) code
> that could also be changed to O(n) without a big problem.

Without big problem is not so definate.  And, also in response your following
paragraph, "if it works why break it."

Changing the way it works today, without a good enough reason, is
also a possible error source.   However, I know what we're really arguing
about here, is if the reason is actually good enough.

> > Then you'll be happy that it's fast, and I'll be happy that consistency
> > is still maintained, and the world will rejoyce.   (Oh, and I'll even try
> > to reduce the memory requirements, by not using QDict at all.)
> If you introduce additional pointers to the items in the dictionary, isn't
> that just duplicate information and therefore just another possible error
> source?

Not really.  If the reverse dictionary holds the serial number itself,
or if it keeps a pointer to an object in the main dictionary (which
contains the serial number), what's the difference?

And on that note, QDict is kinda simple, but wastes memory because
it is too generic.  If instead of using it, the message dictionary
implements its own hash table (which would work similiar to QDict),
then it can have all of the functionality described earlier, with even
less memory requirements than today.


But, we already decided on a course of action for now.  :-)
_______________________________________________
kmail Developers mailing list
kmail@mail.kde.org
http://mail.kde.org/mailman/listinfo/kmail
[prev in list] [next in list] [prev in thread] [next in thread]