[Rpm-maint] [rpm-software-management/rpm] WIP: Use Python Stable ABI for the bindings (PR #2674)

[Petr Viktorin]

These are the changes needed to use Python's stable ABI. This allows the extension to be used without recompilation with multiple versions of Python (from 3.7 until an ABI break).

Hack to make this easier:
- type objects are allocated when the module loads, stored in global pointers, and never released
- the module may only be loaded once per process, so each type is leaked at most once, and the global pointers aren't clobbered.

There is no support for installing for, nor testing with, multiple versions of Python at once. That's left up to distributors for now. (AFAIK it's a bit of uncharted territorry for projects that aren't Python-first.)

The individual commits carry explanations for each kind of change, but the in-between commits won't necessarily build. (i.e. you'd need a Git *merge commit* to  preserve both messages and bisectability.)

I am asking for help with the build system. As it is (in the last commit), it's a bit chaotic:
- With CMake 3.26+, stable ABI will be used by default.
  - This requires Python 3.7+
  - This can be turned off using `cmake -DENABLE_PYTHON_SABI=OFF` (e.g. to allow an older Python version)
- With older CMake, or `ENABLE_PYTHON_SABI=OFF`, stable ABI is not used

If that's fine, I can put it in the build documentation.
If it would be better to build stable ABI even with older CMake, it shouldn't be hard to set the flag manually instead of using CMake's integration.

I've built and ran the tests with Python 3.6-3.12. Sadly I don't have easy access to 3.5 and below.

Fixes: #2345

You can view, comment on, or merge this pull request online at:

  https://github.com/rpm-software-management/rpm/pull/2674

-- Commit Summary --

  * Python stable ABI: Remove PyType_Ready calls
  * Python stable ABI: Add helper for adding a heap type
  * Python stable ABI: Call helper for adding a heap type
  * Python stable ABI: Make calls to `tp_free` go through PyType_GetSlot
  * Python stable ABI: Make calls to `tp_alloc` go through PyType_GetSlot
  * Python stable ABI: Change type definitions to heap type specs
  * Python stable ABI: Adjust type usage & declarations
  * Python stable ABI: Convert uses of Py${container}_SET_ITEM
  * Python stable ABI: Add error handling to Py${container}_SetItem
  * Python stable ABI: Avoid initializing the module more than once
  * Python stable ABI: Disable instantiation "manually" for some types
  * Python stable ABI: Define Py_TPFLAGS_IMMUTABLETYPE as no-op if missing
  * Python Stable ABI: Use Stable ABI, by default on CMake 3.26 (needs Python 3.7+)

-- File Changes --

    M CMakeLists.txt (11)
    M python/CMakeLists.txt (8)
    M python/header-py.c (90)
    M python/header-py.h (5)
    M python/rpmarchive-py.c (73)
    M python/rpmarchive-py.h (5)
    M python/rpmds-py.c (84)
    M python/rpmds-py.h (5)
    M python/rpmfd-py.c (65)
    M python/rpmfd-py.h (5)
    M python/rpmfiles-py.c (180)
    M python/rpmfiles-py.h (10)
    M python/rpmii-py.c (118)
    M python/rpmii-py.h (5)
    M python/rpmkeyring-py.c (130)
    M python/rpmkeyring-py.h (6)
    M python/rpmmi-py.c (93)
    M python/rpmmi-py.h (5)
    M python/rpmmodule.c (156)
    M python/rpmps-py.c (75)
    M python/rpmps-py.h (5)
    M python/rpmstrpool-py.c (70)
    M python/rpmstrpool-py.h (3)
    M python/rpmsystem-py.h (6)
    M python/rpmtd-py.c (7)
    M python/rpmte-py.c (73)
    M python/rpmte-py.h (5)
    M python/rpmts-py.c (79)
    M python/rpmts-py.h (5)
    M python/rpmver-py.c (66)
    M python/rpmver-py.h (5)
    M python/spec-py.c (134)
    M python/spec-py.h (10)

-- Patch Links --

https://github.com/rpm-software-management/rpm/pull/2674.patch
https://github.com/rpm-software-management/rpm/pull/2674.diff

-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/2674
You are receiving this because you are subscribed to this thread.

Message ID: <rpm-software-management/rpm/pull/2674 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rpm.org/pipermail/rpm-maint/attachments/20230921/d82dbe6a/attachment-0001.html>