[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-pim
Subject:    Re: [Kde-pim] Making offline/online handling sane
From:       Andras Mantia <amantia () kde ! org>
Date:       2012-12-05 20:02:49
Message-ID: 20121205200252.42C314EC4008 () mail ! kdab ! com
[Download RAW message or body]

Hi,

 first of all, I guess we are mostly on the same page now with your naming 
of Offline to StayDisconnected or DoNotUseNetwork. :)
I kept using offline, as that was the term KMail used all the time for "do 
not access network". The meaning changed (got broader) only when Akonadi was 
introduced. But I don't mind changing the terms for Akonadi resources, 
although you should be aware that KMail still uses the offline term in its 
UI.

Kevin Krammer wrote:
>> I see no contradiction. Offline will not mean anymore the resource is not
>> allowed to perform *any* work, it will mean it won't be allowed to
>> perform network operations.
> 
> Again assuming Offline as StayDisconnected: isn't that independent of the
> backend access mechanism? E.g. if the backend is accessed via Unix socket?

With network access I mean anything that can cause network traffic. Think of 
using internet in roaming or paid per-data-volume transfer, so you want to 
limit your applications in not using the network and not cause extra 
traffic. Or the case Laurent mentioned: you work on two computers and you 
want to keep them in sync somehow, and to avoid conflicts you limit one of 
your computers to access the network, while the other is allowed to do. Very 
useful for POP3 sources. 

>> The reason this cannot be in ResourceBase or AgentBase is that we can't
>> control what the resource does or not. We can either completely shut it
>> down (like we do now) or not.
> 
> The resource handles that if it wants to (it can reimplement doSetOnline).
> I think the problem is that this is not precise enough, since we have
> currently multiple concepts applied to a single boolean.
> 
> Or do you want to make sure every resource handles the state changes, as
> in making something pure virtual?

This is an implementation detail question. :) Indeed, we can add an extra 
pure virtual function (setNetworkUsageMode() or such), but that will be BIC, 
and I always forget if we need to keep BC in pimlibs or not (I guess we 
need). So maybe the only way to do it now is to have a setter and getter for 
this boolean and trust the resources they comply to it. Not optimal, I know.

>> What I propose is to make the offline a flag
>> that the resource can use to turn the network operations off, if it needs
>> network. If you still want the current offline=turned off behavior, we
>> can keep it, but call it the right way.
> 
> Well, offline means both things so we need replacement for both meanings.
> (if we reuse offline for one of them I expect confusion).

That's the whole point, to not reuse it, but separate the one "offline" 
boolean into more than one, just as you point out later.

> Of course not offline as in "cannot create a TCP connection", but it could
> still not be able to reach its backend, e.g. removable volume removed.

That is indeed true. But that is not a user indicated behavior, but a change 
in the backend reachability. That'd be like the IMAP resource turning itself 
off if it cannot log in to the server. That is what happens now (just to 
make the meaning of offline more confusing :) ).

> Just like a user wanting the IMAP resource to cease network operations
> (e.g. expensive connection), the same user might not want a maildir
> resource to block a mount point.

Ok, makes sense indeed there as well.

>> KMail Offline means that resources are manually set to offline mode. That
>> is what KMail does. (File->Work Offline menu)
> 
> Reusing my suggested terms this would then switch them to
> StayDisconnected, right?

Right.

>> We can introduce a different state for that, if needed.
> 
> Right. I think we will need it, user's don't want to get errors if the
> network is not available, they expect software to detect that.

Sure. Actually mishandling of this case was the cause of a bug I fixed 
recently: you set KMail to offline mode (it set the resources to offline). 
You disconnected from the net and reconnected (eg. after suspend/resume). 
The resources realized there is network, and they happily switched 
themselves back to online mode, while KMail (and the user) asked them to 
stay disconnected. And your mail was downloaded when you didn't want to.
This also suggest that the resource internal state regarding "can I use the 
network" should be different from the user's request about "should I use the 
network".

>> >> If it
>> >> is in that state, it starts to report errors back to the Akonadi
>> >> server.
>> > 
>> > If course, it is being asked for things it can't do right now.
>> 
>> Sorry, it is not clear for me what you mean here. Right now (with the
>> current code), of course cannot report errors, as nothing reaches the
>> resource in offline mode.
> 
> Hmm, misunderstanding then. I thought you were getting an error from code
> in resource base.

Well, I got from the resource base indeed, that the resource if offline and 
the item cannot be retrieved. Why the resource was asked at all when in 
offline mode is something I wrote that still don't know, but neverthless the 
error lead me to think about the offline meaning at all.

>> This is something I need to investigate. A quick check at that time shown
>> that "setCacheOnly(true)" was used, so it looked correct, but probably I
>> missed something.
> 
> Right. This is currently your main problem. The IMAP resource is asked for
> something it should be asked for, in a state where it can't do what it is
> asked for. So the only thing it can do is report that as an error.
> It should not have gotten the request at all.

Exactly.

>> > As far as I can tell we have three different reasons why an agent is
>> > offline: 1) user's explicit choice
>> > 2) backend/service not available, e.g. backend is a remote server, no
>> > network 3) agent has reached a "broken" state and doesn't want its
>> > scheduler to dispatch new tasks until the problem is fixed.
>> > 
>> > Any other interpretation/reason come to mind?
>> 
>> I agree with the above, and I find it wrong.
> 
> Agreed, what I wanted to have is a list of things we currently map into
> that one boolean. We need to demulitplex them into individual states so
> each use case can change the appropriate one.

Agreed.


>> >> Anyway, I'd not mind if somebody would try to reproduce what I see to
>> >> confirm it. How to do it?
>> >> - set up a maildir and a disconnected IMAP account
>> >> - sync them, especially the disconnected IMAP so you are sure mail
>> >> content is downloaded
>> > 
>> > What we expect after this stage is that all IMAP emails are fully
>> > cached by Akonadi and all maildir mails have their envelopes cached,
>> > right?
>> 
>> Well, not really.
> 
> Are you sure?
> After a complete sync with full payload caching I would expect mails to be
> served from cache (IMAP) and after a complete sync with envelope I would
> expect envelopes to be served from cache.

ah, sorry. Indeed, you're right. A complete sync with full payload caching 
(disconnected mode), should deliver the full payload regardless of the 
network status or even regardless if the resource works at all.
If only the envelope is cached, we have several cases if the body is 
requested:
- resource turned off: error that the resource is off
- resource set to "StayDisconnected" state: error that the content cannot be 
delivered because network access is not allowed
- resource is on, it is allowed to access the network, but there is a 
network error (e.g it cannot log in, server failure, etc): you said the user 
shouldn't see such errors. I wonder if it is really how it should behave.
Probably the resource should do some automatic retries, but give up and 
inform the user after a while.

>> The main problem is that even *cached* items are not
>> reachable right now. With disconnected IMAP (when the full mail is
>> cached), you cannot read the mails, because the IMAP resource is accessed
>> in offline mode, but it cannot react because it is "turned off".
> 
> We agree on that is the problem, but this is not the expected behavior.
> The expected behavior as described above is that the mails are delivered
> from cache, regardless of resource state. No?

Right.

>> But what I would expect here is that offline means no network usage.
>> Nothing else.
> 
> So: TurnedOff, NoBackendAcces and NoNetworkAccess?

I wonder about the need to separate the last two, but let's see at the end 
of the mail.

>> >> - unplug any network cable, shut down any wireless acount, if possible
>> >> disable the wireless completely. Now try to read mail on the IMAP
>> >> account and switch folders. Alternatively restart Akonadi/KMail after
>> >> you disabled the network.
>> > 
>> > Since the resources are already offline by use intervention, we would
>> > expect the situation to remain unchanged?
>> 
>> As I wrote somewhere above, turning the resources offline or unplugging
>> the cable results in different behavior, even if visually in both cases
>> the IMAP resource is just "offline".
> 
> Interesting. So it behaves correctly when switched off by the user (one of
> the previous steps) but starts acting differently when network is no
> longer available?

That is what I saw, but as I said, I'd like some confirmation. That is also 
the reason why this bug was hard to notice: just switching the resource ot 
offline mode doesn't trigger the problem.

> I would have expected that there is no difference to the previous step
> (when KMail has turned off the resources).

I'd expect that as well. :)

>> Sure, that is what "offline" would mean (and what I fixed recently, when
>> Akonadi silently turned the resources online, even if they were requested
>> to stay offline). My suggestion would be a step forward: separate the "do
>> not download my mails" case from "stop working for now completely".
> 
> Lets assume we have a distinct state "user does not want network
> operations". If that is active, it will depend on what the resource's
> operations require.
> 
> The IMAP resource would be in the same state as before if its server is
> not a local server, i.e. it cannot add/modify/remove items/collections nor
> retrieve items/collections.
> 
> A resource that only uses network for additional information, e.g.
> downloading avatar images, would continue to work normally.
> 
> Not a bad idea.

Right. The idea was something like that.

> Hmm. I guess if we have a state where we can tell resources not to access
> the backends, then most "offline" use cases are covered.
> Still might make sense for agents, e.g. temporarily turning of the Nepomuk
> feeder agent.

Indeed, that is also good use case to the "turned off" state.

>> If you still want that, we can keep, but I will/want to rename the
>> feature to "Turned off" instead of "Offline" and make "Offline" do what
>> it should do: work, but don't touch the network.
> 
> As written above we'll need several more user switchable states.
> 
> Right now we have
> - TurnedOff (if we want that)
> - DoNotAccessBackend
> - DoNotUseNetwork

What would be the difference between DoNotAccessBackend and DoNotUseNetwork?
Or you mean like DoNotAccessBackend would mean the IMAP would not be able to 
access also local servers (on the same machine), while DoNotUseNetwork would 
disable access only to remote servers? Although not that bad idea, it might 
make things more complicated inside the resources. Of course, they are free 
to treat both the same way as a start.
 
> Due to user expectations of software not attempting network access if
> there is not network available we might need a state that reflects the
> deteced unavailability of network, e.g. HasNoNetwork, and one that
> reflects detected unavailability of backend, e.g. HasNoBackend 
> detected as opposed to encountered during operation attempt. The latter
> would result in an error and "broken" resource state.

I agree with the need of such state, but the question about the distinction 
of the two is still there. If there is no network, on a network based 
resource that implies HasNoBackend. Network state in itself can easily be 
queried from Solid, so we might just keep the HasNoBackend state, no?

Andras


_______________________________________________
KDE PIM mailing list kde-pim@kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/
[prev in list] [next in list] [prev in thread] [next in thread]