[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kmail-devel
Subject:    Re: [Kde-pim] Re: CIA proposal (was: ClientInterface)
From:       Don Sanders <sanders () kde ! org>
Date:       2003-07-27 23:59:54
[Download RAW message or body]

On Friday 25 July 2003 21:57, Marc Mutz wrote:
> On Friday 25 July 2003 08:30, Don Sanders wrote:
> > On Friday 25 July 2003 00:15, Marc Mutz wrote:
>
> <snip>
>
> > > Of course, MUAs wishing to modify ~/Mail need to conform to
> > > CIA/4, but maildir-compliant MDAs work without modifications.
> >
> > Ok, but what about the case of another (non-KMail) mail client
> > moving mail from new/ to cur/ while KMail is not running.
>
> <snip>
>
> KMail detects new mail has arrived after last exit (e.g. checking
> the last-modified-date of new/ and cur/) and updates it's index
> after obtaining the write lock. For maildir, the link between the
> file and the index is the file's basename (without the status bits
> at the end), which never changes, so incremental index updates are
> relatively cheap.

So extra work beyond the CIA proposal is required? And does the CIA 
approach make this extra work significantly easier?

> However, the server approach has the same problem, hasn't it?

It's true that the client/server approach doesn't solve this problem. 
It's also true that I never claimed it did.

Perhaps I need to explain why I asked this question about support for 
external modification of Maildirs.

I'm trying to access the relative merits of two different approaches 
of solving a problem. I'm not judging solely on the basis of how well 
those approaches help solve that one particular problem. Instead I'm 
also taking into consideration the affect those approaches will have 
on solving other problems. Specifically I'm interested in whether an 
approach will make solving other problems listed on kontact.org/votes 
easier or harder. (kontact.org/votes since I want to implement 
features that are good for the users, and I think normally what the 
users want is what's good for them)

For example the CIA proposal would seem to have the consequence of 
making folder changing after deleting a message quicker. (Due to only 
marking deleted messages as deleted in the index, instead of 
completely rewriting the index as is done currently). So I consider 
that a plus for the CIA proposal (since the server/client model has 
no such benefit).

If the CIA proposal fixed the issue of external apps modifying Maildir 
folders, or made it significantly easier to do so that would be an 
additional plus for the CIA proposal.

> > How will you set the timeout for the lock? Some operations in
> > KMail take a long time so perhaps there's a danger of mistakenly
> > thinking the lock is stale when really a KMail client is busy,
> > right?
>
> <snip>
>
> If you re-read my earlier mails, then of course the complementary
> goal is to make KMail non-blocking for all long operations. In
> additon, a timeout triggers a message to the user, who will know if
> it's other KMail is doing some long operation.

I see this limitation of the CIA approach as being very unfortunate.

> You have the same thing with the client/server approach. Either you
> make the server non-blocking or KMail clients need a way to
> distinguish the case that the server died from the case that the
> server is serving a blocking request.

The difference is: with the server/client approach the user doesn't 
run the risk of losing all the mail in a folder when a client 
mistakenly thinks a lock file is stale.

> > Half a year ago I spent some time trying to delay the loading of
> > the dictionary to make startup of KMail faster.
> >
> > But personally I couldn't manage it. We use serial numbers quite
> > a lot now as they provide a safe way of keeping a reference to a
> > message, even at startup the dictionary is used (in KMReaderWin)
> > when the initial message is displayed.
>
> This is just a workaround for the fact that there's no ref counting
> for either messages or folders.
>
> > It seems to me that any client that wants to display messages
> > will use the dict. I would have thought that this is a pretty
> > common case, would you not agree?
>
> Displaying messages (or parsing them to extract an iCal like KO'd
> do) is prettry common, yes. But there's no need for sernums here.
> If KMail is able to lose references to messages during the lifetime
> of an KMail instance, then there's something wrong. In this case,
> it's missing refcounting.

I'm not sure that lack of ref counting is the problem here. But 
regardless this shows that the CIA approach has an additional burden 
that the client/server approach does not suffer. Namely requiring the 
avoidance of the use of serial numbers to achieve efficiency.

To me this is a mark against the CIA approach, as everything else 
being equal I prefer flexible solutions that do not impose additional 
restrictions on the design of KMail.

> > Unfortunately the case of the full text index seems to be an even
> > more difficult one. As in order to keep the full text index up to
> > date it must be running whenever mail is discovered, altered,
> > copied, or moved. Wouldn't this be a pretty typical kind of thing
> > for clients todo?
>
> You coul do lazy updates. Only update the index when it needs to be
> there. No need for updating it unless the user performs a search or
> opens a search folder.

The full text index needs to be up to date in order to maximize the 
speed of searching.

Besides requiring a rewrite of the full text index I think delaying 
updates of the full text index until the time when the user makes a 
search would result in an unfortunate and irritating delay for users.

Having said that I do think a good time to prompt the user and ask 
whether they would like to turn the full text index on is when they 
perform their first search.

> > I wonder if a client can't efficiently handle either of these
> > cases what useful task could a client do efficiently?
>
> There was life before the full text index. It probably was slower.
> There's a speed/space tradeoff here and each clients need to decide
> what's more important for them.

The important point to me is that the CIA approach requires additional 
work to keep the full text index operative while the client/server 
approach does not.

To me that's a plus for the server/client approach.

> > > > Besides folder files there are also other files than need to
> > > > be handled. The config file is a problem if it is desirable
> > > > to have each client keep their config dialogs and general
> > > > configuration info in sync.
> > >
> > > I'm cool with having that break. If a user is dumb enough to
> > > keep the config dialogs of two KMail's open at the same time,
> > > then he shouldn't complain if the second-to-be-closed one
> > > overwrites the settings of the first-to-be-closed one. I don't
> > > see a need to make that work.
> >
> > Ok. But even if that was acceptable wouldn't there be a problem
> > on say session exit with all the clients updating the config file
> > at exit?
>
> Nope. The last session wins.

I think this means that with the CIA approach if I open two clients, 
add a new account in one, and exit KDE via session management then I 
run the risk of losing that new account. (Depending on the order the 
clients close).

I think that's a very serious problem.

The client/server approach doesn't suffer from this problem (for 
server config data at least), so I regard this as a minus for the CIA 
approach.

> > I mean say a new account was created in one client, couldn't that
> > new account be lost if another client closed after the client
> > where the new account was created?
>
> Nope. Writing the complete config file on exit is nonsense (even if
> KMail currently does that). Of course, you write only the session
> config and then it's "last session wins" again. It's simply not
> necessary to make every conceivable thing possible with concurrent
> access. Keep the perspective: As Ingo said, the typical scenario is
> to have at most two or three full KMail instances running at the
> same time, only one of which is interactive in the sense that the
> user does something with it.

Even if a client doesn't allow network config changes to be made, that 
client will still rewrite the network config state at application 
shutdown. This is because currently all KMail clients are designed to 
write all network config prefs at shutdown.

I think it's unfortunate that you do not consider this issue to be 
important.

> <snip pro-client/server stuff>
>
> Most stuff needs to be solved either way. For one thing, the
> client/ server approach is more convenient, for others it's CIA.
> The net result as I see it is that CIA is both more performant
> (almost no socket traffic), works (for the most part) over NFS and
> is easier to maintain once the necessary changes have been made. It
> is also the way to go if there's anything like a multithreaded
> KMail coming along in the distant future.

I don't regard less socket traffic as being an advantage as the socket 
connections in the client/server approach are local.

I don't see any reason why the CIA approach has an advantage for NFS. 
I don't think the CIA approach will be easier to maintain. I don't 
see any reason why the CIA approach is preferable if KMail is 
multithreaded in the future. I don't see evidence supporting your 
claims to the contrary.

> CIA has techniques that help also a normal KMail to reduce resource
> usage.

Yes.

> It doesn't add much in the way of complexity for 
> single-instance mode.

I don't believe that's true. The CIA approach would add significant 
complexity for serial number generation, recording mails left on the 
pop server and updating the full text index. Keeping mbox support 
would also require more complexity on top of the proposed CIA design.

> IOW, it optimizes (also) the common case, not 
> the rare one, as the c/s approach does.

Yes the CIA approach would make some things faster. That's a plus for 
the CIA approach. But not a big plus as those same optimizations can 
be just as easily be made on top of the client/server approach as on 
top of the current KMail.

> CIA also covers the MDA-delivers-into-~/Mail case, which the server
> has to solve, too, probably with the exact same techniques that CIA
> uses.

I don't believe the CIA approach as given handles delivery of mail 
into ~/Mail. Instead it requires additional work in order to do that, 
and as far as I can tell that additional work could just as easily be 
added to the client/server model.

> So you can also say that the server needs CIA techniques in
> addition to it's own complexity. So what's left is full
> client/server complexity vs. the need to lock a few files.

'need' is too strong a word. The client/server approach provides a 
solution to the problem of embedding KMail gui parts in other apps 
without the need for any of the CIA techniques.

> The "Composer freezes on new mail check" is _not_ an argument
> against CIA or for the client/server approach. It's again the
> reasoning that I ranted about earlier: Instead of fixing the
> problems, let's work around them using a server. Not.

I see running the GUI in a separate process as being a valid solution 
to the problem of the GUI freezing when syncing (newly retrieved) 
mail to disk and when waiting for filters.

Hence as by the bug system the GUI freezing on mail checking is 
considered a significant problem I see that as a big plus for the 
cient/server approach.

However the main reason why I prefer the client/server approach to the 
CIA approach is not the advantages of the client/server approach but 
rather the disadvantages of the CIA approach.

The CIA approach introduces the risk of losing mail if an operation in 
KMail takes a long time, due to the frailties inherent in the file 
locking mechanism that the CIA approach relies on.

The CIA approach does not appear to be compatible with the full text 
index implementation.

The CIA approach will either be inefficient due to duplicating the 
msgdict, or fragile due to requiring the KMail code to be modified to 
avoid use of the msgdict.

The CIA approach requires extra complexity to be introduced in the 
serial number and leave-mail-on-server code, and that extra 
complexity will have the side effect of making KMail more fragile.

The CIA approach is also limited, and does no cater for mbox users 
without additional modifications.

The CIA approach as given in CIA/4 is not well defined as it does not 
take into account index entries that are not covered by the index 
mmap.

The CIA approach also has the disadvantage of introducing 
last-client-wins kernel config file data loss semantics.

For these reasons I'm unconvinced by the arguments given in support of 
the CIA approach and prefer the client/server (or perhaps more aptly 
the componentization) approach.

Don.
_______________________________________________
KMail Developers mailing list
kmail@mail.kde.org
http://mail.kde.org/mailman/listinfo/kmail
[prev in list] [next in list] [prev in thread] [next in thread]