[Rpm-maint] [rpm-software-management/rpm] RFE: Automatic (sub)package generators (#329)

[nim-nim]

Anyway, I needed to solve quite a lot of the problems involved in automated packages to prepare the switch of Fedora Go packages to Go modules.

I will push soonish the result to redhat-rpm-config (not because the Go automation is finished, I’d say it’s 90% done but in need of lots of testing, but because the infra is shared with font packages and Fedora i18n is getting nervous now that Fedora 33 change deadlines loom and they need the font automation finished for their own changes, which are getting critical now that apps have started rejecting some legacy font formats).

Of course, it won’t be able to depend on `%prep` and post-`%prep` processing for a lot of things, since that is locked down in current rpm, but this is just a little part of what automated generation needs, and I now know where that limited my implementation.
(for the curious,
https://src.fedoraproject.org/fork/nim/rpms/redhat-rpm-config/commits/forge-with-patches
though that is still changing a lot, not because the core logic needs fixing, but because refactorings and naming cleanups tend to touch a lot of lines)

Things I learnt doing full package automation:

1. it completely inverts functional tag/variable inheritance logic, you want processing to set variables, which are then pushed to `%package` sections (in tags), which are then turned into SRPM tags (the SRPM tags are computed from the `%package` tag values and not the reverse). That’s the normal case when your automated generation picks up a single thing to auto-package – you want the SRPM tags to reflect the single package you are auto-producing. That’s also the normal case when you are generating multiple auto-packages – you want to compose the SRPM license from all the package licenses, and not the reverse

2. that means SRPM tags need to be yanked from the preamble, or the preamble turned into a proper `%package` section, that may occur very late in the spec file (the preamble needs to be just another `%package` section, with a marker that says it’s the source package section, and nothing else, and not forced to occur at the start of the spec).

3. that also means you need global state variables computation and re-computation at each stage of the pipeline, not just a lua dump at preamble time. You could do it implicitly in each section (like it is done implicitly in the preamble now) but that is likely to lead to grief when people disagree on ordering and what using and setting actually means.

    So, I’d rather have an explicit `%x_init` section added to existing sections, so you know that (for example) lua code evaluation for `%build` is done in `%build_init`, and can parse files produced in previous sections (like `%prep`), and no further lua evaluation will occur before the next `x_init` section, and shell code in `%build` can rely on all lua or rpm variables it uses to be set and finalized before the `%build` section starts.

4. you will compute and re-compute a lot of rpm variables, and current infra to read/set/inherit variables is crap. I managed to streamline pretty much every set of the auto-packaging process, and turn it in nice easily maintained macros (including for horribly complex stuff like go modules) except for the variable initialisation part (that was, already the case for `%forgemeta` 3 years ago, the logic in `%prep` is dead simple, computing the variables it needs is definitely not dead simple).

    Most of my additions to redhat-rpm-config are just helpers and routines to read and set rpm variables and make it less painful.

    I know that variables are just macros, but they are special macros without any argument. And that is critical when you are composing auto-generators. In an auto-generator world, you pass state from one stage to the next as variables, you do not serialise it to CLI arguments that need de-serialization at the next step (you could serialize/deserialize at all steps, that’s completely needless complexity, the rpm argument parser will drive you crazy before you finish implementing).

5. right now, I’ve not found a case where I needed to interleave generators (ie apply `%build` generation for a Go component, then `%build` generation for a font component, then return to `%build` generation for a Go component). I *did* find cases where the generation order mattered, so right now my automation implementation permits declaring that `%go_prep` MUST occur after `%forge_prep`, for example. So far, a `%foo_after` variable that contains the list of things `%foo` must be executed after, is sufficient for my needs.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/329#issuecomment-643595899
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rpm.org/pipermail/rpm-maint/attachments/20200613/3e0b7643/attachment.html>