[Rpm-ecosystem] RFE: Stackable namespaced repositories

[Orion Poplawski]

    I've been working on trying to package python virtual environments with
rpm (outline here:
http://orionlibre.blogspot.com/2015/05/deploying-scientific-python-environment.html).
 While this particular project is python based, the same basic concept can be
applied to anything else that allows manipulating the load path - which is
pretty much everything (PATH, LD_LIBRARY_PATH, PERLINC, etc).  I'm most
familiar with the HPC world which does this a lot - providing different
versions of various packages in a hierarchy and using environment modules to
allow users to load the desired stack at runtime.  This is also essentially
the same thing that SCLs do, though SCLs are more isolated from the base system.
    The big problem that this exposes in rpm packaging is "namespacing" the
rpms: rpms in the "stacked" repo can depend on packages in the system repos,
but not vice-versa.  So we need to put them into a different namespace
manually (I'm using a %{?pyvenv_name_prefix} macro, SCLs use %{scl_prefix}).
Complicating this is that one needs to add the prefix to the names of other
packages in the repository when specifying requirements - which may not be
known ahead of time.  It would be very nice if this could be handled
automatically.
    What I think would work is if a repository could be marked as a "stacked"
repository and given a namespace - essentially a prefix to the names in the
repository.  For example, my pyvenv-nwra repo could be given that prefix - and
marked as such - say with pyvenv-nwra:.  Then when depsolving dnf would allow
dependencies in the pyvenv-nwra repo to be satisfied by packages in base
repositories but not allow the reverse.  Installed packages/depnames would be
given the pyvenv-nwra: prefix to keep them separate from the system resources.
    This strikes me as a complicated technical challenge, so I really would
like to hear back from the packaging ecosystem folks their thoughts about it.
 But I really think this could move us into the model of providing software
stacks targeting different applications with different package/version
requirements that build upon a stable OS core while still allowing for the
tracking and deployment benefits of using rpm packages.  And by having
pre-populated buildroots that make any needed macro changes (say %_prefix),
many packages could be built unmodified from the Fedora git repos as opposed
to having to hack in the namespace prefixes into the spec files.

- Orion

-- 
Orion Poplawski
Technical Manager                     303-415-9701 x222
NWRA/CoRA Division                    FAX: 303-415-9702
3380 Mitchell Lane                  orion at cora.nwra.com
Boulder, CO 80301              http://www.cora.nwra.com