[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-bugs-dist
Subject:    Bug#45694: acknowledged by developer (kmail fails to update its folder index properly)
From:       Don Sanders <sanders () kde ! org>
Date:       2002-08-01 5:46:09
[Download RAW message or body]

On Thursday 01 August 2002 09:50, V K wrote:
> On Tue 30 Jul 2002 21:30:00 NZST +1200, Don Sanders wrote:
> > > File locking is always a good idea.
> >
> > Not on NFS systems where it hangs.
>
> It doesn't hang on all. A turn-off feature would solve that easily.

Well if you have a reliable solution for determining which systems 
locking hangs on please send in a patch and I'll gladly apply it.

I don't want to add a lock mbox folders in ~/Mail option as I feel 
that will disguise rather than solve the problem. Basically I don't 
want to add more options.

If you want to enable locking for mbox files in ~/Mail simply modify 
KMFolderMbox::KMFolderMbox in kmfoldermbox.cpp and change
"mLockType = None;" to your preferred locking type (FCNTL, 
procmail_lockfile, mutt_dotlock, or muttdotlock_priviledge).

IMO FCNTL is by far the best if your system supports it.

I would like to enable FCNTL by default, but I also don't want to 
upset NFS users.

> > KMail considers its mail folder files private so that locking is
> > unnecessary.
>
> The files are not kmail's, they're mine, but I'm sure you'll
> disagree.

The mbox folders are like configuration files etc, if you modify them 
without knowing what you are doing then you can corrupt them and 
cause data loss.

> > > Maildir solves some problems and creates new difficulties. I am
> > > not the only one who finds it difficult to handle. In contrast,
> > > I can run one grep on 5 mailboxes easily with mbox, forget
> > > about doing that with maildir. Maildir's lack of subdirectories
> > > is a laugh and something I don't find acceptable.
> >
> > Maildir can be used with subdirectories e.g. in KMail. A
> > graphical
>
> Well, I have several subdirectories in ~/Mail/, and kmail doesn't
> show *any* of them in the folder list under ~/Mail. Yes I have read
> the manual, and that symlink-diddling can best be described as
> cumbersome. Kmail doesn't usefully support subdirs, if you're
> saying otherwise you're having me on. (I did say this in a previous
> mail already?)

KMail supports its own format for subfolders. Supporting the formats 
of other mail clients may be a nice feature I would consider a patch.

> Worse, the instructions in the manual don't work. It says: "If you
> want to add a whole remote mail directory use ln -s
> /somewhere/Mail/.remotedir.directory
> ~/Mail/.mymailboxfile.directory." but doing it does absolutely
> nothing. It also requires touch ~/Mail/mymailboxfile

The website faq is correct http://kmail.kde.org/manual/faq.html states 

"If  you want to add a whole remote mail directory use ln -s 
/somewhere/Mail ~/Mail/.remotedir.directory. For that case you also 
need to create a new empty folder named remotedir with KMail."

> The last bit is what makes ln -s existingdirunderMail
> ~/Mail/.existingdirunderMail.directory a pain and impossible.

Possibly It's not supported because this design for subfolders is a 
stupid design. If I call a mail directory foo then I can't create an 
mbox file called foo in the same directory, hence I can't put mail in 
folder foo only in subfolders of foo.

Unless you specially name an mbox somewhere that corresponds to 
directory foo but then you have created something equivalent to the 
current format.

Basically the design you are asking to be supported sucks because if a 
folder has child folders you can only put mail in the child folders 
and not in the parent folder.

Nevertheless I would consider supporting it if a patch was submitted. 
I definitely don't think KMail should create folders in such a 
format.

> > interface or shell script can be used for searching.
>
> Perhaps true. Grep is easier and doesn't require time to get going.

For many people grep is harder to use than a GUI. I also don't buy 
your argument that grep is too difficult to use on Maildir.

> > When reading a large file it would be massivley inefficient to
> > check file size and date before every fread.
>
> Every *write* only, I see that mutt does it, so it's not really a
> problem, I have not noticed any delays so far.

You wrote "access" in the paragraph you have deleted. access and write 
are very different things.

Anyway as I have repeatedly stated just because a files date and size 
has remained constant doesn't imply the file hasn't been modified. 
mutt for some reason forces the date of a file to remain the same 
even after modifying the file. So it's possible both the date and 
size of a file may be constant even after mutt has modified the 
contents of the file.

Checking file data and size and regenerating the index wouldn't 
allowing sharing of mbox files it would just make it much more 
difficult to reproduce and understand mail losing behavior.

> > If KMail rescans your mbox files then it indicates something is
> > wrong like the files have been modified by an external program.
>
> Which is nothing wrong, but we had that before.

It indicates something is wrong because it indicates an external 
program may have corrupted KMail files.

> > Hitting your computer with a large hammer is also dangerous and
> > can lead to data loss. Like corrupting files KMail uses I suggest
> > you do not do it.
>
> This is a bit silly.

What I am saying is if something causes you pain then maybe you 
shouldn't do it. If corrupting files is causing you data loss then I 
suggest you stop corrupting files.

> > I've found mutt to be extremely slow and resource intensive for
> > large mbox files.
>
> Then our experiences differ considerably.

Well here's my experience, take a typical mbox file with about 10000 
messages (40MB) in it. mutt takes about 4 seconds to open this file 
while KMail takes well under a second.

I'm assumming KMail has already generated its index file here, but 
this should always be the case for normal operation of KMail.

Deleting some mails in this mbox folder and changing to a different 
mbox file took 18 seconds under mutt and about 4 seconds under KMail.

Here KMail is definitely too slow and I do plan to optimize this 
further but mutt is unacceptably slow.

Perhaps the slow performance of mutt is acceptable to you but it's not 
acceptable to me, it's not acceptable to many KMail users and I don't 
plan to make KMail slower to add support for a meritless feature.

> > No, I will not, if I like I will use a mail client that can
> > handle large mbox files efficiently.
>
> I am looking for one which handles mbox files at any speed and
> *with* interaction with procmail.
>
> > It doesn't really work if you grow old and die while you are
> > waiting for the operation to complete. I find the performance of
> > mutt on large folders to be unacceptable slow.
>
> Let's disagree.
>
> > A feature request to add optional support for ~/Mail locking is
> > more sensible but I won't be implementing it.
>
> Fair enough to both.
>
> > I don't say mbox doesn't scale well. I say ~/Mail locking doesn't
> > scale.
>
> It's the same in practice.

It's not the same because mbox can be handled efficiently but a system 
that requires completely rescanning mbox files everytime they are 
opened cannot be efficient.

> > I say this because ~/Mail locking results in unacceptably slow
> > performance for large but realistically sized mail folders.
>
> I disagree, sorry.

I have made empirical tests on several occasions and found mutt to be 
much slower than KMail. I give example results above.

> > > Sorry Don but your arguments don't hold. "Not as fast" and "has
> > > more issues to solve but solutions are readily available by
> > > copy/paste from other GPL software" is no reason to not lock
> > > files.
> >
> > You are misquoting me I did not write that.
>
> Wasn't meant to be a quote of anyone, the marks delimit the
> phrases. And what you're saying boils down to those phrases - you
> object to locking because it's too slow for you.

Quote marks indicate quotation. If you are not quoting someone then I 
suggest you do not use quote marks.

KMail is currently too slow for me and I work to make it faster, I 
definitely don't want to make it as slow as clients that rely on 
convnetional ~/Mail locking, that would be a step backwards.

> > > The assumption that $MAILCLIENT is the only program writing
> > > under ~/Mail is seriously flawed. It's not with Unix tradition
> > > of having a system of parts interoperating with each other.
> >
> > True it breaks with Unix tradition, this is a valid criticism.
>
> I don't have a problem at all with kmail not locking mbox folders,
> as long as it's optional...

It is optional in the sense you can change one line of source as I 
have described above.

> > False procmail is supported, see the FAQ, I don't buy arguments
> > that the procedure in that FAQ is too difficult to follow.
>
> I had said in one of my first posts that this feature is useless.
> Kmail is unable to operate on folders procmail delivers to. If I
> try, kmail will immediately empty out the whole folder and stash it
> away in some other place I am forced to specify. Why do you think I
> have procmail deliver to that folder in the first place? Perhaps
> because I want the mail to be there? Besides, doing this for
> several dozen folders is not what I call user-friendly (even if it
> did achieve the purpose, which it doesn't). The bottom line is
> kmail does not usefully work with procmail, which is the main
> reason I can only read about 5% of mail with kmail.

It is true we do not support the inefficient non-scalable way of using 
procmail that you insist on using.

We do support an efficient scalable way of using procmail.

> > > And it's (indirectly)
> > > forcing a dependence on just one mail client,
> >
> > True.
>
> Not acceptable. kmail doesn't work via dialup when I'm not at home.
> Falling back on something like mutt in that case must remain a
> possibility.

This is a legitimate and serious problem. IMAP potentially offers a 
solution but has problems of it's own. A text based interface to 
KMail is another solution, this might not be too hard to do using 
dcop.

> > formats. But because KMail supports conventional file formats for
> > mail it is not that hard to switch from one mail client to
> > another.
>
> As long as they work with procmail.

Procmail is irrelevant here. You can switch to another mail client 
easily as long as it supports mbox or maildir.

> > Procmail support is covered in the FAQ.
>
> See above. And please stop pointing to the FAQ when it comes to
> procmail. The only way to make it work is useless.

I disagree. The method outlined in the FAQ is only useless if you 
insist on handling mail in an inefficient way.

> > Show me a mail client that can handle large mbox files
> > efficiently and supports ~/Mail locking. I've tried both mutt and
> > pine and found neither can. The main place they fail is when a
> > mail is deleted and the user opens a new folder.
>
> I haven't had trouble with mutt yet. Let me rephrase that, its
> configuration is taking far too much time, but it does the
> important jobs. It doesn't do well with supporting different of
> what kmail calls identities. That's one of the real pluses of
> kmail. Others are it's neat integration into the desktop and its
> pleasing look and feel.

I've had trouble with mutt because it cannot efficiently handle large 
folders. Really 10000 mails is not even large it's moderately sized. 
We do handle moderately sized folders ok, better than any other *nix 
client I've examined but there's still a lot of room for improvement. 
Supporting conventionay ~/Mail locking would retard the progress we 
have made and prevent further improvement.

Don.


(Complete bug history is available at http://bugs.kde.org/db/45/45694.html)
[prev in list] [next in list] [prev in thread] [next in thread]