[Rpm-maint] [rpm-software-management/rpm] Handle Python extras trough reverse requirements encoded in provides (#1061)

[Miro Hrončok]

In Python, we have a concept of [extras](https://packaging.python.org/specifications/core-metadata/#provides-extra-multiple-use) a way to specify an optional dependency with the ability to install the software with that dependency enabled trough the "extra" part of a name in square brackets.

A real world example:
 - `cachecontrol` has an extra called `[filecache]` and it extras_requires `lockfile`
 - `poetry` requires `cachecontrol[filecache]`

This is what happens:
 - `pip install cachecontrol` installs `cachecontrol` without the `lockfile` dependency
 - `pip install cachecontrol[filecache]` installs `cachecontrol` with the `lockfile` dependency
 - `pip install poetry` installs `poetry` with the `cachecontrol[filecache]` dependency (with the `lockfile` dependency)

(I've omitted version specifiers from the examples to make them simpler. `pip install cachecontrol[filecache] < 5` means `cachecontrol < 5` with the `[filecache]` extras.)

## Naïve RPM idea

At first, this is the idea we had:

`python3-lockfile`:
 - Provides `python3.Xdist(lockfile)`

`python3-cachecontrol`:
 - Provides `python3.Xdist(cachecontrol)`
 - Provides `(python3.Xdist(cachecontrol[filecache]) if python3.Xdist(lockfile))`

`python3-poetry`:
 - Requires `python3.Xdist(cachecontrol[filecache])`

We were told that this is not possible:
- Rich provides are not available in RPM yet.
- Rich provides are a can of worms and most likely won't be available ever.

Notes:
- I cannot find where exactly this was said, feel free to provide more accurate information.
- If `[]` are not valid in provides/requires, we can encode this differently, such as `python3.Xdist(cachecontrol:filecache)` or similar -- that part s not so important in this context.

## Subpackages idea

The second idea we had (and packagers already do this somewhat similarly, but manually, in some cases) is to create meta(sub)packages with the extras:

`python3-lockfile`:
 - Provides `python3.Xdist(lockfile)`

`python3-cachecontrol`:
 - Provides `python3.Xdist(cachecontrol)`
 - Suggests `python3-cachecontrol+filecache = %{?epoch:%{epoch}:}%{version}-%{release}` (possibly)

`python3-cachecontrol+filecache`:
 - Provides `python3.Xdist(cachecontrol[filecache])`
 - Requires `python3.Xdist(lockfile)`
 - Requires `python3-cachecontrol = %{?epoch:%{epoch}:}%{version}-%{release}`
 - Has no files.

`python3-poetry`:
 - Requires `python3.Xdist(cachecontrol[filecache])`

The problem with this approach is:
 - Generating subpackages from provides/requires generators is hard (currently not possible?).
 - Possible bloat in total package count (however Python packages don't tend to have dozens of extras).


## Encoding reverse requirements in provides

Here is the new idea I've just got. It is a bit hackish, but I think it should work:

`python3-lockfile`:
 - Provides `python3.Xdist(lockfile)`

`python3-cachecontrol`:
 - Provides `python3.Xdist(cachecontrol)`
 - **Requires `(python3.Xdist(lockfile) if python3.Xextrasreq(cachecontrol[filecache]))`**

`python3-poetry`:
 - Requires `python3.Xdist(cachecontrol)`
 - **Provides `python3.Xextrasreq(cachecontrol[filecache])`**

Here is how that works:
 1. install `python3-poetry`
 2. `python3-cachecontrol` is dragged in, trough classic `python3.Xdist(cachecontrol)` dependency
 3. since `python3-poetry` provides `python3.Xextrasreq(cachecontrol[filecache])`, `python3-cachecontrol` now also requires `python3.Xdist(lockfile)`
 4. profit?

The `python3.Xextrasreq(...)` name is not perfect and might need to change.

We are able to generate those reverse requires/provides using the current tools.

I don't really like the idea that we say what a package depends on trough a field that is supposed to say what other packages may depend on to get this package. However we use Provides for all sorts of things already: `bundled(...)`, `deprecated()` etc. We will obviously need to document that manually requiring `python3.Xextrasreq(...)` is not useful and possibly ban it via policies.


@encukou @ignatenkobrain @Conan-Kudo @gordonmessmer @decathorpe Please provide feedback. I am especially interested in "this doesn't work, because..." kind of feedback and less in "this is so ugly we'd rather not have anything" (however "if we tweak it like this, it's nicer" is also appreciated).



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/1061
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rpm.org/pipermail/rpm-maint/attachments/20200209/1bc271d7/attachment.html>