[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kmail-devel
Subject:    Bug#45694: acknowledged by developer (kmail fails to update its folder index properly)
From:       Don Sanders <sanders () kde ! org>
Date:       2002-07-30 9:30:00
[Download RAW message or body]

On Tuesday 30 July 2002 17:49, V K wrote:
> On Tue 30 Jul 2002 18:26:12 NZST +1200, Don Sanders wrote:
> > On Tuesday 30 July 2002 08:07, Ingo Klöcker wrote:
> > > Today it's clear that it was a bad design decision to assume
> > > that KMail is the only program which accesses ~/Mail.
> >
> > Ingo I strongly disagree.
>
> Don I strongly disagree with you.
>
> > File locking of ~/Mail is the bad idea and should be discouraged.
>
> File locking is always a good idea.

Not on NFS systems where it hangs.

> The alternative of stuffing
> sufficiently randomised filenames into a directory doesn't
> convince. Perhaps there's another alternative. Many programs can
> obviously cope with different locking standards, that's no
> argument. Locking with e.g. procmail and mutt works extremely well.
> True that applications need to agree on locking standard(s), but
> that technique is commonly used in many places. To the contrary of
> your argument, *not* locking is very dangerous.

KMail considers its mail folder files private so that locking is 
unnecessary. Modifying KMail private files is the same as modifying 
the private files of any other application, if you try it without 
knowing what you are doing you are asking for trouble.

> > These flaws in ~/Mail  locking  are why Maildir was developed.
>
> There may be flaws, but that's not a good reason to do nothing.
> Besides, one can argue about "flaws". "Some problems which can be
> solved" is more accurate.
>
> Maildir solves some problems and creates new difficulties. I am not
> the only one who finds it difficult to handle. In contrast, I can
> run one grep on 5 mailboxes easily with mbox, forget about doing
> that with maildir. Maildir's lack of subdirectories is a laugh and
> something I don't find acceptable.

Maildir can be used with subdirectories e.g. in KMail. A graphical 
interface or shell script can be used for searching.

> > But my main problem with locking of ~/Mail is that it is a
> > non-scalable design, it makes it impossible to index the mbox
> > files efficiently. Under the ~/Mail locking design external
> > programs can not only append to mbox files but modify them
> > randomly, some programs do so without changing the file date.
> > Thus the only way to handle mbox files under the ~/Mail locking
> > design is to rescan them completely.
>
> The indexing in kmail is broken. Before accessing any file, it's
> size *and* time needs to be checked, and the file needs to be
> recanned if necessary.

When reading a large file it would be massivley inefficient to check 
file size and date before every fread.

> I don't see any problem at all with
> rescanning, certainly not with rescanning just one file. kmail
> seems to scan the whole big lot when it starts (in some cases
> anyway), which is neither fast nor necessary.

If KMail rescans your mbox files then it indicates something is wrong 
like the files have been modified by an external program.

> Not rescanning is
> dangerous and can lead to data loss. If the choice is between data
> loss and rescanning a file your reasons about not rescanning go out
> the door.

Hitting your computer with a large hammer is also dangerous and can 
lead to data loss. Like corrupting files KMail uses I suggest you do 
not do it.

> There is no problem with scalability. We're only talking mbox here.
> I never found the time it takes mutt to scan a folder when opening
> it to be any problem at all.

I've found mutt to be extremely slow and resource intensive for large 
mbox files.

> If you really need to have 150MB in a
> mail folder, either use maildir or put up with the time it takes to
> scan the file. 

No, I will not, if I like I will use a mail client that can handle 
large mbox files efficiently.

>Scan intelligently, like mutt does, and yes it works
> and I never lost anything there.

It doesn't really work if you grow old and die while you are waiting 
for the operation to complete. I find the performance of mutt on 
large folders to be unacceptable slow.

> In any case, file locking of mbox
> files is mandatory and does not cause additional disadvantages
> other than sacn time. You could put an option "don't lock if you
> know no other app will access the file but lose data if some app
> does" option.

A feature request to add optional support for ~/Mail locking is more 
sensible but I won't be implementing it.

> Saying mbox doesn't scale as well isn't true. Access time is
> proprtional to size. "Bad scaling" is when time is more than
> proportional to size.

I don't say mbox doesn't scale well. I say ~/Mail locking doesn't 
scale.

I say this because ~/Mail locking results in unacceptably slow 
performance for large but realistically sized mail folders.

> Also, any mbox file should be locked, not just those under ~/Mail,
> but I supppose that's arguable (e.g. locking will fail if it's on
> CD). An good mail client should also be able to just open an mbox
> file from anywhere, including /tmp (if I restored some very old
> mail from CD I want to look at).
>
> Sorry Don but your arguments don't hold. "Not as fast" and "has
> more issues to solve but solutions are readily available by
> copy/paste from other GPL software" is no reason to not lock files.

You are misquoting me I did not write that.

> The locking problem has been solved. The reason it's not
> implemented appear to be rather of a political nature to me.

Politics relates to government.

If you want to believe there's some kind of government conspiracy 
preventing KMail from supporting ~/Mail locking then I doubt I will 
be able to persuade you otherwise.

> > Scalable mail clients do no support ~/Mail locking. Evolution,
> > Mozilla and KMail all keep index files and do not support ~/Mail
> > locking.
>
> All these clients are tailored for the computer-illiterate user at
> the other end of a pop3. There's nothing wrong with that, unless
> the literate person is ignored.
>
> The assumption that $MAILCLIENT is the only program writing under
> ~/Mail is seriously flawed. It's not with Unix tradition of having
> a system of parts interoperating with each other.

True it breaks with Unix tradition, this is a valid criticism.

> It's ignoring
> procmail, which is a very fine mail filter.

False procmail is supported, see the FAQ, I don't buy arguments that 
the procedure in that FAQ is too difficult to follow.

> And it's (indirectly)
> forcing a dependence on just one mail client,

True.

> via a dependence on
> filtering.

False, the dependency is enforced due to incompatible index file 
formats. But because KMail supports conventional file formats for 
mail it is not that hard to switch from one mail client to another.

> I don't have a problem with kmail implementing its own
> filtering, but I object to being forced to use it. Implementing the
> same filter rules in more than one place is not desirable. kmail
> does not interoperate with procmail, neither do those not bothering
> about locking.

Procmail support is covered in the FAQ.

> Sure you could just say don't use kmail then, but that's beside the
> point (all other GUI clients are the same wrt interoperability).
> I'd love to move up to a GUI client from text based ones, but find
> that GUI ones are not an extension (in terms of features) to
> traditional ones. GUI clients simply can't do what text based
> clients are able to (ok, pine might be different).

I think Balsa doesn't use index files, you could try that.

> > Really the problem is with users who mistakenly assume KMail
> > supports a broken design like ~/Mail locking. Unfortunately such
> > an assumption
>
> Then kmail (and similar clients) is lacking in features. Of course,
> for newbies the issue doesn't exist (no more than 5 mailboxes,
> never uses grep, fetchmail or procmail).
>
> I don't agree with your opinion Don that locking is a broken
> design. It can be made to work reliably, proof is there. You are
> taking the lazy way out.

There is no proof, conventiional ~/Mail locking cannot be supported 
efficiently.

Show me a mail client that can handle large mbox files efficiently and 
supports ~/Mail locking. I've tried both mutt and pine and found 
neither can. The main place they fail is when a mail is deleted and 
the user opens a new folder.

Don.


(Complete bug history is available at http://bugs.kde.org/db/45/45694.html)
_______________________________________________
KMail Developers mailing list
kmail@mail.kde.org
http://mail.kde.org/mailman/listinfo/kmail
[prev in list] [next in list] [prev in thread] [next in thread]