[Rpm-maint] Rpm Database musings

Michael Schroeder mls at suse.de
Mon Mar 4 10:21:20 UTC 2013

On Sun, Mar 03, 2013 at 05:46:10PM +0200, Panu Matilainen wrote:
> Right, in this context compression does indeed seem quite attractive. When 
> we talked about this in the devconf, I was thinking about the way rpm 
> itself currently keeps (re)loading the headers from Packages and adding 
> repeated decompression to the other costs of header loading didn't seem 
> like a way to make it faster. But for roughly halving the amount of io 
> needed for scanning through it exactly once (which is of course the way 
> libsolv operates) its quite a different thing.
> 0.2s is not a whole lot, for many operations absolutely nothing really, but 
> I'd think some kind of cache would be in order to avoid having to read 
> through all of packages just for those simple 'rpm -qf /foo' kind of 
> queries.

Yes, I think it's too much. A rpm query call should be pretty
instantaneous. Plus, reading all headers will kill 10 MBytes precious
block buffer cache.

> Such as, store the in-memory index structures into a memory mapped 
> cache file. The cache could perhaps be write-once and read-only for other 
> uses so there's no need for locking within the cache: eg recreate it from 
> scratch at the end of transactions and atomically replace the old one so 
> the cache itself is always coherent.

Yes, but that's pretty much the way rpm currently works. (The "but"
is not meant negative, we don't need to change everything just
for change's sake.)

> Or something... this isn't that far 
> from libsolv's .solv files.
> Speaking of which... a funny little idea I got at the end of the devconf: 
> regardless of future rpmdb format changes, it should be now possible to 
> write an rpm plugin that creates + updates a .solv file for the rpmdb, so 
> you should never have to actually read through the entire rpmdb in libsolv 
> and its users like libzypp, dnf etc.

Actually libsolv can do a "incremental" update if it has an old
solv file available, i.e. it takes the unchanged content from the
old solv file and only queries new headers from the rpm database.
Ales doesn't yet use this in method in hawkey.


Michael Schroeder                                   mls at suse.de
SUSE LINUX Products GmbH,  GF Jeff Hawn, HRB 16746 AG Nuernberg

More information about the Rpm-maint mailing list