[Rpm-maint] Feature request: Improved speed for 'rpm -qa'

[Axel Liljencrantz]

On 12/19/06, James Olin Oden <james.oden at gmail.com> wrote:
> On 12/19/06, Axel Liljencrantz <liljencrantz at gmail.com> wrote:
> > On 12/19/06, James Olin Oden <james.oden at gmail.com> wrote:
> > > <snip>
> > >
> > > > Even on my 300 MHz system, it only takes about 0.02 seconds. Is there
> > > > any chance that regular rpm could be made this fast, or is this too
> > > > much of a hack to include there? I don't think that bundling a special
> > > > command for querying the rpm database together with a general purpose
> > > > OS agnostic commandline shell is the proper way to do this...
> > > >
> > > > By the way, why is this a hack? Is the index not updated when an entry
> > > > is removed or something? If not, indexes are there to be used!
> > > >
> > >
> > > I think the main thing you have to ask is why did some customers want
> > > every header whether it be in a package or database to always have its
> > > signature verified?  Did they see something we do not?
> > >
> > > I always thought that it was enough to check the header at install
> > > time when it was being inserted into the DB, but now I can see why one
> > > might want to do this check even on queries, as a query is often used
> > > as input into some larger process.  The person running the "process"
> > > may have all authority to do what they are going to do based on the
> > > inputs from the rpm DB, and have no malicious intent, but rogue data
> > > may make them do something they would not have otherwise done.
> > >
> > > Security is always a trade off with something else, though.  I'm just
> > > trying to make sure everyone is considering that the way rpm works now
> > > was driven by someones customers somewhere.
> >
> > Your arguments make sense, security by default is the right thing. I
> > would note that hiding unsigned packages could be used to hide the
> > existance of an 'evil' package, but that is a topic for a different
> > discussion.
> >
> > What a shell needs for tab completions is a list of _all_ installed
> > packages, even unsigned/b0rked/evil ones. Actually, the user is _more_
> > likely to want to deal with one of those rouge packages, so they are
> > extra important to include in the completion list. And for an
> > interactive shell, it is very important for the list to be generated
> > quickly, verifying each signature seems to take far too much time.
> >
> > So the added security is actually bad in two separate ways when it
> > comes to tab completions.  I would argue that since command specific
> > tab completions are becoming more and more common, it makes lots of
> > sense for rpm to be 'completion friendly', though as a shell developer
> > I may be biased.
> >
> > It would be fine with me if the current interface was kept, but an
> > additional switch was added which makes things go really fast.
> >
> Agreed.   As it is, and I think its been mentioned, you can turn off
> the digest checking both within the API and on the CLI (--nodigests),
> and this should speed things up immensely.
>
> Is there something else your looking for?

The most vital thing is performance. Even with no signature checking
and no digest checking, the listing still takes up to 13 seconds on a
slow system (Down from 40 seconds with digest+signature and no
readahead). The program snippet that Michael Shroeder sent to the list
does the same thing in 0.02 seconds on the same system. That extra
performance increase would be nice! :-)

Going beyond that, it would also be nice if one could get the summary
field of each package as well. Fish supports the showing of a
description for every completion. If you are completing a command name
or a manual page for the man command, fish will show you the whatis
data for the command/manual page as the description, if you are
completing a username, the full name is shown, etc.. But I can see how
adding support for retrieving summary data would make things slow, so
I guess that is 'and a pony' territory. But in a perfect world, I
would like something like:

# rpm --query --prefix=als -- fields=name,summary
alsa-lib,The Advanced Linux Sound Architecture (ALSA) library
alsa-utils,Advanced Linux Sound Architecture (ALSA) utilities

>
> Cheers...james
>

-- 
Axel