[Rpm-maint] [PATCH 4/5] Extend header size to 64MB due to file signatures

[Stefan Berger]

Lubos Kardos <lkardos at redhat.com> wrote on 04/29/2016 06:40:16 AM:

> 
> It is just a thought. Rpm transaction can be divided in two phases.
> In the first phase in the beginning of transaction rpm loads all file 
infos to
> perform transactions checks and then releases them. In the second phase 
rpm
> reloads single file infos to install single packages in row. The memory 
peak
> happens in the first phase when all file infos are loaded. These file 
infos
> contain also file signatures but in the first phase they needn't to 
contain
> them because the signature checking is performed only in the second 
phase.
> 
> So if the file signatures blow up the file infos so much so we need 
> to increase
> maximum header size then maybe it would be nice not to load file 
signatures
> into file infos during the first phase of transaction when the rpm 
memory peak
> happens.

I agreed and it would be a separate patch.

I didn't look very deeply, but how does one detect the phases? I suppose 
the part
to skip would be in lib/trpmfi.c::rpmfilePopulate where the flag 
RPMFI_NOFILESIGNATURES is
checked. If that's the case, maybe the 1st phase could call this function 
with this
flag always set?

    Stefan

> 
> Lubos
> 
> ----- Original Message -----
> > From: "Florian Festi" <ffesti at redhat.com>
> > To: "Stefan Berger" <stefanb at us.ibm.com>
> > Cc: rpm-maint at lists.rpm.org
> > Sent: Friday, April 29, 2016 10:27:39 AM
> > Subject: Re: [Rpm-maint] [PATCH 4/5] Extend header size to 64MB 
> due to file signatures
> > 
> > On 04/27/2016 09:47 PM, Stefan Berger wrote:
> > > "Rpm-maint" <rpm-maint-bounces at lists.rpm.org> wrote on 04/27/2016
> > > 05:50:54 AM:
> > > 
> > > 
> > >>
> > >> Well changing header size limit needs a bit more thought. The main
> > >> problem is that packages with bigger header will look broken on 
older
> > >> rpm versions and the usual way of dealing with this (adding 
rpmlib()
> > >> Requires) won't work it needs reading the header.
> > > 
> > > These huge headers are only occurring in a few very large packages 
and
> > > only if one applies the per-file signatures. So most users probably
> > > won't notice.
> > > 
> > >>
> > >> Also I wonder if we should increase the header size even more, to 
get
> > >> rid of this topic for a longer time. I thought about 256MB which 
gives a
> > >> 4 times increase over the 16MB. I am kinda tempted to go even 
further.
> > >> Otoh the limit is there for a reason. And having rpm chew through 
one GB
> > >> of broken data doesn't sound like a pleasant experience.
> > > 
> > > Anything >=16 MB works with signed files for all packages in Fedora 
23.
> > > Let me know if you want me to resubmit the patch with a higher 
limit.
> > 
> > Yes, please. 256MB is probably the way to go. Let's hope we don't 
reach
> > that any time soon.
> > 
> > Florian
> > 
> > --
> > 
> > Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
> > Commercial register: Amtsgericht Muenchen, HRB 153243,
> > Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham,
> > Michael O'Neill
> > _______________________________________________
> > Rpm-maint mailing list
> > Rpm-maint at lists.rpm.org
> > http://lists.rpm.org/mailman/listinfo/rpm-maint
> > 
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rpm.org/pipermail/rpm-maint/attachments/20160429/d660caa2/attachment-0001.html>