[Rpm-maint] [PATCH 00/15] first pass at Python 3 bindings for RPM

[David Malcolm]

I've been investigating adding Python 3 support to librpm's python bindings, and in a "release early" spirit, the following sequence of patches is what I've got so far.  There are plenty of FIXMEs and hairy issues, so I wanted to get some feedback before diving in further.

According to configure.ac, "rpm-python is based on python-2.5, with legacy hacks to allow building against python >= 2.3".  I'm seeking to generalize the code to compile against both 2.* and 3.*, so that it's "based on python-2.6 and python-3.1, with legacy hacks to allow building against python >= 2.3"

Ultimately, I want to be able to build both python 2 and python 3 support during the same build, using the same source.  But I think it's easier to start by making sure that the source can be built against either major-release of Python.

So these patches merely generalize the existing code so that it can be built against both python 2.6 and python 3.1

So far, I'm only testing this by hand, against 2.6.3 and 3.1.1 (on a Fedora 11 box).  I suspect that these patches break the build on Python earlier than 2.6, due to the uses of the "Bytes" macro.

I'm uneasy with purely manual testing of this code.  Is there an existing unittest suite for the rpm-python code?

With these patches I can do this:
[david at brick rpm]$ PYTHONPATH=/home/david/coding/python3/rpm-python-bindings/install-prefix/lib/python3.1/site-packages python3
Python 3.1.1 (r311:74480, Oct  1 2009, 12:20:21) 
[GCC 4.4.1 20090725 (Red Hat 4.4.1-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import rpm
/home/david/coding/python3/rpm-python-bindings/install-prefix/lib/python3.1/site-packages/rpm/__init__.py:9: DeprecationWarning: Type rpm.hdr defines tp_reserved (formerly tp_compare) but not tp_richcompare. Comparisons may not behave as intended.
  from rpm._rpm import *
# (clearly I need to fix this)
>>> rpm.addMacro("_dbpath", "/var/lib/rpm")
>>> ts = rpm.TransactionSet()
>>> mi = ts.dbMatch()
>>> for h in mi:
...     print("%s-%s-%s" % (h['name'], h['version'], h['release']))  # note that print has become a function
b'im-chooser'-b'1.2.6'-b'3.fc11'
(etc)

This highlights one of the big unknowns to me at the moment: what guarantees (if any) are there as to the encoding of the bytes in a value of RPM_STRING_CLASS?  Currently in python/rpmtd-py.c:rpmtd_ItemAsPyobj I punt the issue by returning them as Bytes objects, and hence you see the bytes objects in the output. So as written, users of the API would have to deal with decoding these back to py3k's unicode strings.

Thoughts?
Dave

David Malcolm (15):
  Generalize python configuration code to support both Python 2.* and
    Python 3.*
  Generalize access to ob_type so that they work with both Python 2.*
    and Python 3.*
  Generalize type object initialization to work with both Python 2.*
    and Python 3.*
  Convert relative imports within rpm package to absolute imports
  Convert all usage of two-expression exception raising to modern
    syntax
  Update module initialization to work with both Python 2.* and Python
    3.*
  Use macros to hide "int" implementation differences between Python 2
    and 3
  Use PyBytes_ rather than PyString_ for RPM header data
  Use PyBytes_ API when reading from file descriptors and for public
    keys
  Port tagNumFromPyObject on Python 3 to use unicode objects
  Fixup another key refernce to use PyBytes rather than PyString
  Use PyUnicode_ on Python 3 when setting up the various RPMTAG_
    constants
  str() returns a PyUnicode for Python 3
  Generalize tag data -> python conversion to work with both Python 2
    and Python 3
  Hack up various dark corners so that they compile with Python 3

 configure.ac              |    2 +-
 python/header-py.c        |   18 +++++++---
 python/header-py.h        |    7 ++++
 python/rpm/__init__.py    |    9 ++---
 python/rpm/transaction.py |   22 ++++++------
 python/rpmbmodule.c       |   37 +++++++++++++++++++--
 python/rpmds-py.c         |   10 +++--
 python/rpmfd-py.c         |   30 +++++++++++++---
 python/rpmfi-py.c         |    5 +--
 python/rpmkeyring-py.c    |   12 +++----
 python/rpmmi-py.c         |    5 +--
 python/rpmmodule.c        |   79 ++++++++++++++++++++++++++++++++++++--------
 python/rpmps-py.c         |    8 ++---
 python/rpmsystem-py.h     |   11 ++++++
 python/rpmtd-py.c         |   14 +++++---
 python/rpmte-py.c         |    3 +-
 python/rpmts-py.c         |   30 ++++++++++-------
 python/spec-py.c          |    5 +--
 18 files changed, 216 insertions(+), 91 deletions(-)