[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-devel
Subject:    RE: A search service for KDE
From:       Manuel Amador <rudd-o () amautacorp ! com>
Date:       2005-02-23 23:15:17
Message-ID: 1109200517.30814.50.camel () master ! amauta
[Download RAW message or body]

El mar, 22-02-2005 a las 08:58 -0500, Alan Bryan escribió:
> My only question is why Phython and not straight C++? It seems would be
> fasterr and definately more memory conservative.

Python is very memory-conscious.  The KDE libs count for at least 50% of
the memory footprint of the indexer, currently.  Plus, Python has
exceptions, is very fast, there's no manual memory management involved
(I know, I know, Qt and memory management rock, but Python is, frankly,
better) and one of the options that I'm studying (in practice) is to
store everything into ZODB, which transparently persists objects instead
of having to store them in relational form.  I can do this:

>>> a = database.file_list["/home"]
>>> print a
<Directory /home, with 88 elements>
>>> get_transaction()
>>> a.label = "Directory for home users"
>>> get_transaction.commit()

and it's all stored.  An equivalent abort() call rolls all changes back.
I can define classes really really fast.  Python has "batteries
included" - utilities for just about everything under the sun, safe
transparent threading and good locking and queueing primitives,
excellent IPC.  I can tag any object with any property I want, at
runtime.  And I don't have to compile.

just a few reasons.

> 
> Alan
> 
> -----Original Message-----
>     From: "Manuel Amador" <rudd-o@amautacorp.com>
>     Sent: 02/18/2005 6:32:02 PM
>     To: "kde-devel@kde.org" <kde-devel@kde.org>
>     Subject: A search service for KDE
>     
>     Hello,
>     
>     Can I interest you guys in a KDE-technology based desktop search
> engine?
>     I've built one from scratch, using Python, Zope and the KFile
> bindings
>     for Python, which I'm currently working on, and I'm very close to a
>     preview release (we're talking days here, and this is only because I
>     don't have an Internet connection at home).
>     
>     Basically what it does is spin in an idle loop.  Well, it does more
> than
>     that =).  It crawls a directory and indexes plugin-rendered
>     representations of files into the ZODB (Zope Object Database), while
>     accepting search queries via an XML-RPC interface.  Searches are
> very
>     fast: searching for The Beatles in my collection of 13000 music
> files
>     takes 0.4 seconds, while searching for any number of words through
> my
>     information collection (HTML, PDF and text files) takes 0.6 seconds.
>     The indexing process automatically throttles processing if the
> system
>     load spikes, and continually tunes the throttling values.
>     
>     And it's only 6000 lines of code.
>     
>     Core functionality is pretty much done right now, after 8 days work.
> I'm
>     currently writing a simple client application a la Beagle, which
> will
>     support DCOP for other client applications to use and integrate.
>     
>     Basically what I wanted to ask is:
>     - any pointers or ideas to keep in mind?
>     - would people here support integrating it into KDE?  How would that
> be
>     accomplished best?  I'm thinking KFind should support it by default
>     (there's nothing mysterious about the interface/API to the search
>     server, it's just an XML-RPC server)
>     - why the hell does the indexing process use 160 MB of RAM? (a
> tongue-
>     in-cheek tuning question, which really means "do the python bindings
>     leak memory?")
>     
>     Well, let's check those followups.  I hope this stirs a good
> controversy
>     and leads into constructive discussion.
>     -- 
>     Manuel Amador <rudd-o@amautacorp.com>
>     Amauta
>      
>     >> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to
> unsubscribe <<
> 
-- 
Manuel Amador <rudd-o@amautacorp.com>
Amauta
 
>> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<

[prev in list] [next in list] [prev in thread] [next in thread]