[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-devel
Subject:    Re: Storage implementation in KDE
From:       Stefano Borini <munehiro () ferrara ! linux ! it>
Date:       2003-09-07 20:48:09
[Download RAW message or body]

On Sun, Sep 07, 2003 at 02:01:50PM +0100, Luke Chatburn wrote:
> I don't weigh in too often on these threads, but I'd like to make a few
> points (feel free to flame :) ).

naah, i'm here to learn

> - Metadata does not scale well. It's like having a desk in an office and
> filling it with pieces of paper. It works fine if you have a limited
> quantity of pieces of paper, but for business and for regular home use over
> a period of time, you do need organisations. In the real world, this is in
> the form Room->Filing Cabinet/Box->Folder->Document. It's a normal
> heirarchical file system. That's not to say that a relational look-up can't
> help with a few things, but is can't be used for everything.

right, but don't forget that kde should be used also by home users, that
usually doesn't handle a lot of files, and prefer to store them
accordingly to a different scheme. I mean... people store their
real life photos in a shelf, so the primary classification is "by date".
But if the user have to look up all the photos which have a friend in
it, only a systematical lookup can extract the data. 

> - Metadata searching is very, very expensive in processing terms and worse
> for larger document sets. Moreover, you limit yourself by I/O abilities.

databases are optimized for performance, i'm not sure this is a real
problem, now in 3 GHz world.

> - Using a database for metadata means that you have to have the service
> running constantly in the background.

yes, and no. Relying on postgres mean this, but selecting as a backend
berkelydb does not involve running a bg process, IIRC. I have no
experience with berkeleydb, so i may be wrong.

> Security considerations should be
> inserted here. Moreover, it is taking up cycles and a very significant chunk
> of memory. The more files, the worse the footprint. SQL dbs are huge,
> really, especially when you add the interpretation stage for SQL on top of
> the myriad of different functions they have to perform. You could write a
> completely fresh, light-weight DB for this, if you liked, but it is hardly
> ideal, and we'd spend two or three years bug and security-fixing the thing.

i agree.

> - Namespace collisions. Every run across a problem where you had to work
> with documents with the same name, one for one purpose and one with a few
> changes for another, in another folder (clearly marked as the other purpose,
> hopefully!)? The system will have to display many results and offer a clear
> indication to the user where the file was contained. Moreover, often users
> will identify files based upon other files in the same location.Metadata
> can compensate for this, but it isn't easy.

agree... but don't forget that this is not "the definitive replacement
to a traditional filesystem". This is a storing facility that the user
may or may not use depending how he feels comfortable with it.

> - Users are used to thinking in a heirarchical manner.

here i can't agree. Managing hierarchical is a pain for the average joe
user, in special way if the hierarchy is deep as a traditional unix
filesystem. I have under my eyes people that aren't able to find stuff
in the kde menu since they don't directly relate the "office" submenu to
a word processor (the target application)

> With all of that having been said (and there are ther arguments), I do agree
> that metadata can be valuable for augmenting existing search facilities. In
> order to do this effectively and within a reasonable time, however, it needs
> to be done at the filesystem level, not by a high-level system embedded in
> KDE. ReiserFS 4 should allow us to do these things with relatively little
> system slowdown, which is why I look forward to it so avidly. Smarter
> filesystems are the key, really, but not SQL database-based FSs.

maybe... but isn't a filesystem a database with the kernel as dbms ?
we can rely on sql db as an option. Don't forget that networking a
filesystem is really easy both in security and in configuration if this
fs is hosted on a sql db. This is not the case with traditional
approaches like nfs or ftp, or even samba (however slightly better)

> The point most worth making, however, is that Windows is stuck at a point
> where they seemingly can't develop further in application or DE
> usability/functionality and they're going for frills. To be frank, though,
> most users don't have a difficult time finding their files (assuming they
> understand the concepts of directories, however). They spend 99% of their
> time sitting staring at an application trying to do work of some kind. I
> believe that there is significant innovation taking place in other aspects
> of the KDE project and applications, and that efforts are better served
> focusing there. When Reiser 4 turns up, this stuff will be trivial to
> implement, but until then, it's duplication of that effort.

maybe you are right. But don't forget that, if gnome and windows goes on
this line, even it should appear as unuseful or poor, kde could find
itself without this facility that, at last, could have some minor but
critical uses.

> To be honest, there are a number of areas where genuine progress can be made
> to make a real difference, almost immediately. Merging of instant messaging
> with PIM components is a start. Some sort of development to the KDE Notes
> system to assign notes to any document in any application and when the same
> program opens the same document again, they get reopened would be a great
> improvement to workflow.

yep. I suggested this facility to the mozilla team a lot ago, but they
redirected me at mozillanotes, which is not the same stuff.

> If you really want to try something radical, implement this idea I floated
> with the Slicker folks (they seem to be dead atm):
> 
> http://www.linuxcomment.com/slicker.htm

i love slicker and i'd like to see them working on kde.

> KDE is already very strong in network transparency

right... but also storage transparency could be a goal.

 
>> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<
[prev in list] [next in list] [prev in thread] [next in thread]