[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-devel
Subject:    Re: Status on indexing system
From:       Manuel Amador <rudd-o () amautacorp ! com>
Date:       2005-02-25 15:59:19
Message-ID: 1109347507.10715.19.camel () master ! amauta
[Download RAW message or body]

El jue, 24-02-2005 a las 11:33 +0000, Matteo Merli escribió:
> On Wed, 23 Feb 2005 16:33:12 -0500, Manuel Amador <rudd-o@amautacorp.com> wrote:
> > Hi,
> > 
> > I'm advancing, slowly, with the project I proposed to the list a few
> > days ago.  I'm struggling with scalability/database issues right now,
> > but as soon as I get over them, I'll start working on a simple frontend
> > a la Beagle.  For what it's worth, searching songs by The Beatles on
> > 70.000 files now takes under 0.4 seconds.
> 
> Hi, I'm am working from some time on (not-yet-released) information
> retreival system very similar to your. I used basically the same
> approach: python + ZODB + BerkeleyDB reached after many tries with
> other systems.

Utterly amazing! =)  I knew that was the route!

> I faced your same problems with the index performances and space
> occupation and came out with a few solutions..

> Maybe we can share ideas and code..

I will be more than willing, I'll mail you an attachment with what's
done right now.  More important than the code you'll see is the
documentation.

I'll dissect what you've done quickly, in comparison to Metadata
services.  This is only with the desire to see how can we cross-
pollinate.
> 
> The main features already implemented in my system are:
> - Directory Scanner
Yes.  Per-volume.
> - Pugins to handle different mime-types
Yes. Plugins in Metadata service have access to platform-independent
abstractions for each common platform (KDE, GNOME)
> - Very fast text tokenizer and indexer
Yes. Using ZCatalog.
> - Full Unicode support ( all the text is converted to unicode )
Over here, I've chosen to store text in the system locale (mainly
because I did not want to mess around with the catalog, and the catalog
seems to operate in the locale's context), and convert to UTF-8 when
communicating with the user.
> - Query Parser (does handle logical queries with AND OR "("  )
Done too.
> - Basic web interface with Twisted/Nevow
I'm exposing two XML-RPC interfaces, one via sockets and the other via
TCP.
> - SOAP interface
> 
> I haven't released nothing yet because there is no documentation and I
> am working on rationalising the package structure.

let's keep in touch, Matteo!

> 
> Best Regards
> Matteo Merli
> 
-- 
Manuel Amador <rudd-o@amautacorp.com>
Amauta
 
>> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<

[prev in list] [next in list] [prev in thread] [next in thread]