[prev in list] [next in list] [prev in thread] [next in thread] 

List:       gentoo-dev
Subject:    Re: [gentoo-dev] Portage metadata
From:       Nick Jones <carpaski () twobit ! net>
Date:       2003-02-28 7:33:09
[Download RAW message or body]

>> 2. For later speedups, is anybody looking into using actual database
>> formats for the data? (GDBM/BDB/NDBM/etc.)
> 
> Python internal DB is cache in memory and is quite fast, for certain task a DB 
> would be faster but most of the time, the python DB seems to kick ass unless 
> you do description search.

Used a 2.0.47-series?  ~20-60 seconds depending on your box. That
includes printing all the descriptions, checking versions, etc.

This of course assumes you're using an rsync via 'emerge sync' and
you aren't anhiliating/damaging /var/cache/edb/dep in some fashion.

> 
> I have written in the past a bash script that generated an XML files out of 
> the /var/db/pkg.  If you can import that 4 Megs files inside a DB then you 
> could run some query and time them and see how things get faster.

Databases are not going to be that great of a speedup as it incurs the
overhead of the DB. Portage's DB is an on-demand one-file-per-ebuild
cache that is trivial to parse as it's one line per field. Ensuring
that the db is proper would take just as long, if not longer, than using
the server side cache (metadata/cache) as 'emerge sync' does already.

--NJ

--
gentoo-dev@gentoo.org mailing list

[prev in list] [next in list] [prev in thread] [next in thread]