[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-pim
Subject:    Re: [Kde-pim] RFC: pimd
From:       Reinhold Kainhofer <reinhold () kainhofer ! com>
Date:       2004-12-31 13:24:24
Message-ID: 200412311424.27036.reinhold () kainhofer ! com
[Download RAW message or body]

[Attachment #2 (multipart/signed)]


On Friday 31 December 2004 14:05, Mark Bucciarelli wrote:
> On Monday 06 December 2004 09:48, Reinhold Kainhofer wrote:
> > On Sunday 05 December 2004 17:53, Mark Bucciarelli wrote:
> > > There is another way to address this problem that seems much closer to
> > > what we have now--client pull.  If the library API is modified so
> > >
> > > (1) a client can only select a subset of incidents, and
> >
> > But for syncing we need the whole list of items.
>
> If the PC ID stored on the Palm matches the PC ID of the machine you are
> currently synching with, you can do a fast sync and you don't need the
> entire calendar.  (At least this is how the old Palm Pilot stuff worked).

But that relied on korganizer properly setting a status flag to indicate that 
an incidence  was changed. If you sync a calendar file of another calendar 
application, that's totally broken.

> Even if you are doing a slow sync, technically you could do it one by one.
> - get the palm pilot record
> - retrieve the matching ical record using the Palm record id
> - check for differences
> - sync accordingly.

You need to detect which incidences are new, and which have been changed since 
the time of the last sync. For both you need the whole list of incidences... 
In particular, since not all changes are done through KDE-PIM applications 
(e.g. on groupware servers), and some applications / groupware servers don't 
store any proper timestamps / last modified times.

Also, the "slow sync" only means that we try to fetch as few records from the 
handheld as possible. On the PC side we don't have to care about slow serial 
lines... In the conduit we really go through all entries on the PC. 

> > > then the client can periodically pull that "recent changes" list, see
> > > if any of the changes were to incidents it is viewing, and refresh
> > > data if so.
> > >
> > > "Select UID from events where last_change > now - 10 minutes"

Quite often you don't know the time of the last change (in particular you 
might not even know if an incidence has changed).


> > But for a local file it needs to reload the whole file...
>
> For a local file, the method would be different.  Each time an incidence is
> changed, an entry would be written to a log file.  You then have to query
> the log file, which will be fast.

And that will only work if the data you access can only be changed through 
kde-pim applications...

> I am suggesting that instead of adding yet another layer to the software
> stack (a daemon), we fix libical to do the following:
>
> - be indexed by incidence date (and related-to?)

Yes, that would be nice.

> - load subsets of data into memory

Yes, that's certainly needed anyway.

> - provide changedSince(QDate&) message

As explained above, often you do not know the time of any change, or even if 
the incidence has been changed at all.

> - incident-level and calendar-level locking

Yes. Actually, both should already be implemented...

However, this does not solve the performance problem. Currenly, if you change 
something in korganizer, and then want to sync the file with KPilot, you need 
the absolutely latest version of the calendar on disk. So, KOrganizer has to 
save the file after each and every small change. And then all applications 
that use the calendar (which are quite a few already!) need to reload that 
data (at least those changed incidences).

> We could write it in pure QT to be cross-platform, and remove three layers:
> FAM (or gamin), KDirWatcher, and some libkcal chunks.  A side benefit of
> giving these responsibilities to libical is that resource code for local
> and groupware backends would become more similar.

Are you seriously suggesting to work on libical? Have you ever looked at that 
code? 
Another problem is that the groupware backends don't use libical, since 
typically the format of groupware servers is not iCalendar, but some custom 
format. libkcal is KDE's calendar library, and libical is just a parsing 
library for one particular calendar format.

> My assumption is that on the desktop, people will (most of the time) only
> be looking at a small subset of the calendar (especially after a couple
> years).

Absolutely correct.

> A daemon would also have to load the entire calendar into memory.

Why? It might also load on demand.

> > > If you can select subsets of data to view, do you still need caching
> > > resources?
> >
> > Yes, because the caching is used to optimize writing to disk. You just
> > can't save the whole huge calendar file to disk after each small file.
>
> Why not?  Reading and writing files is fast.  

Yes, but all other applications need to parse that file again... And that is 
dead slow, in particular six or seven applications are accessing your 1MB 
calendar file!

> performance), reading and writing data was ten times as slow as using a
> file-based database!  From this I concluded file-based storage is really
> really fast.

Yes, the pure filesystem-storage is fast. But first you need to format the 
incidences into the iCalendar text format, and on loading you have to parse 
the text...


Reinhold

[Attachment #5 (application/pgp-signature)]

_______________________________________________
kde-pim mailing list
kde-pim@kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
kde-pim home page at http://pim.kde.org/

[prev in list] [next in list] [prev in thread] [next in thread]