[Rpm-ecosystem] Some points about zchunk

[Neal Gompa]

On Thu, Sep 27, 2018 at 1:49 PM Jonathan Dieter <jdieter at gmail.com> wrote:
>
> On Thu, 2018-08-09 at 10:49 +0200, Jonathan Dieter wrote:
> > On Sun, 2018-07-15 at 11:25 -0400, Neal Gompa wrote:
> > > On Thu, Jul 12, 2018 at 3:27 PM Jonathan Dieter <jdieter at gmail.com>
> > > wrote:
> > > > I'd go with <foo>_zck since <foo> is, by default, xml, but, other
> > > > than
> > > > that, I think what you (and Michael) are suggesting makes sense.
> > > >
> > > > Michael, I agree that there should be some way to manually enable
> > > > zchunk (with a method that will work with future algorithms) in
> > > > librepo.
> > > >
> > > > I don't mind writing the code, but I'd like to hear some
> > > > consensus on
> > > > the method and element name.
> > >
> > > I'm okay with this, but does this mean we could also have (cheaply)
> > > zck of arbitrary xml files appended using tools like modifyrepo_c,
> > > then?
> >
> > Ok, I've gone with creating a new record <foo>_zck, that uses all the
> > same attributes available for <foo> plus <header-checksum> and
> > <header-
> > size>.  My updated pull request for createrepo_c has these changes
> > now.
>
> Apologies that it's taken so long for me to follow up on this.  So,
> I've been working on getting librepo and libdnf up-to-date with this
> change and it's far more difficult than you would expect.  Currently
> the process is something like this:
>
> ✔ libdnf requests primary
> ✔ librepo cleverly changes primary to primary_zck if primary_zck exists
> ✔ librepo downloads primary_zck
> ✔ libdnf gets path of primary from librepo
> ✔ libdnf passes primary path to libsolv
> ✘ libsolv can't open primary path, because we downloaded primary_zck
>
> I think the way forward is to make libdnf more aware of zchunk with the
> following (simpler) flow:
>
> ✔ libdnf requests primary_zck.  If that fails, it requests primary
> ✔ librepo downloads primary_zck
> ✔ libdnf gets path of primary_zck from librepo
> ✔ libdnf passes primary_zck path to libsolv
> ✔ libsolv opens primary_zck path
>
> There are three things that I want to verify before I move forward with
> this:
>    1. dnf is now using libdnf, so I'm not going to have to repeat the code
>       twice, right?
>    2. Are the librepo consumers happy with me moving some logic to libdnf?
>       Is there anybody who's losing zchunk support with this move?
>    3. Is there a better way to do this?
>

The problem if librepo can't do it is that pure-librepo consumers are
probably going to have issues. The main upcoming pure-librepo consumer
is Koji to replace its yum.repoMDObject stuff and other repomd centric
actions using the YUM API.

DNF is now using libdnf, so you shouldn't need to repeat it twice.

But that said, the path you describe makes sense.


-- 
真実はいつも一つ！/ Always, there's only one truth!