[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-devel
Subject:    Re: Kmail slow
From:       aj () stack ! nl (Arend-Jan Wijtzes)
Date:       2000-06-09 10:33:50
[Download RAW message or body]

> > Well, this is not exclusively for kmail. I will develop it anyway, since I
> > want to improve it for KRN, but of course the more programs use it, the
> > better :-)
> 
> I don't think idea of using a database was ruled out, just no one volunteered 
> to do it.

It seems that there are (at least) two people that like to put some time it ;-)

> 
> I'm all for the database idea, I said it would be a cool area for someone to 
> research.
> 
> The main reason I'm interested in it is increased speed for changing folders. 
> As I said in my previous mail on this list there are two problems. A database 
> can solve the second problem, which is the need to incrementally index 
> messages by all fields that messages can be sorted by.
> 
> The first problem is that it takes quite some time (a few seconds) to create 
> and insert thousands of items into a QListView (try editing the QT listviews 
> example to see what I mean). To overcome this probelm you need to use a list 
> view that only inserts items as they are made visible (this requires a new 
> list view class to be coded), actually this could be a bit tricky for 
> threaded messages view (its a lot of work anyway). 

Sounds like fun!

> 
> I'm even potentially interested in storing message bodies in the 'database'. 
> As this could speed up other operations (eg moving messages from one folder to 
> another, and eliminate the need for compacting folders perhaps)

If you only store the message bodies in seperate files (no headers) than the 
only difference in performance will be the case when you need to search
(regexp) in the body. It seems to me that when using a database, the text 
bodies should definitaly be stored in the database.
That rules out compatibility with other mail clients.

> 
> But if all that is done is porting the index files over to a database then we 
> still  retain the advantage of storing the messages in normal text format 
> (obviously).

I think just storing the index in a 'real' database will not gain performance
that much. only for more complicated tasks.

> 
> Aah, while I remember I also need the database to keep a tree structure and 
> incrementally update it for threaded message views.

That's easy :) Just keep a parent id with each mail header. And perhaps also
a child id for better performance.

> 
> Anyway I think it's a good idea, and definitely worth researching.

To resume this far:

* for performance it would be better to use a database
* if we choose to drop compatibility this can be realised, perhaps with
  an import/export function to support the old mbox format.
* it would take a new class (based on Qlistview) to process only 
  those messages visible at that time, it could be made even a more
  general class so it can be used for any large list.
* someone has to set up a whislist for the mail client so we can design
  a database format that will be flexible and efficient.
* We need to know if it's worth the trouble (effort). 

About the last item:
Are there many users with _big_ mailfolders? If so, scrolling a window with
the content being 50K of messages will not do, you would need other ways to
find the message you are looking for like a search with regexp, filters 
and perhaps multi-level sorting the list. 
For newsgroups (KRN) this is obvious, those lists _are_ big :)
Are there other features that we can not implement without the database setup?
I'd hate to put my time in something nobody needs :-)


Then a last thing. This mailing list is crowded enough as it is. Should we
continue this discussion somewhere else ?


aj
 
>> Visit http://master.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<

[prev in list] [next in list] [prev in thread] [next in thread]