[Rpm-maint] [PATCH 00/15] first pass at Python 3 bindings for RPM

[Panu Matilainen]

On Thu, 15 Oct 2009, David Malcolm wrote:

> I've been investigating adding Python 3 support to librpm's python 
> bindings, and in a "release early" spirit, the following sequence of 
> patches is what I've got so far.

Nice, and nice timing too as the python bindings are in the a middle of a 
serious spring-clean/facelift.

> There are plenty of FIXMEs and hairy issues, so I wanted to get some 
> feedback before diving in further.
>
> According to configure.ac, "rpm-python is based on python-2.5, with 
> legacy hacks to allow building against python >= 2.3".  I'm seeking to 
> generalize the code to compile against both 2.* and 3.*, so that it's 
> "based on python-2.6 and python-3.1, with legacy hacks to allow building 
> against python >= 2.3"
>
> Ultimately, I want to be able to build both python 2 and python 3 
> support during the same build, using the same source.  But I think it's 
> easier to start by making sure that the source can be built against 
> either major-release of Python.
>
> So these patches merely generalize the existing code so that it can be 
> built against both python 2.6 and python 3.1
>
> So far, I'm only testing this by hand, against 2.6.3 and 3.1.1 (on a 
> Fedora 11 box).  I suspect that these patches break the build on Python 
> earlier than 2.6, due to the uses of the "Bytes" macro.

Supporting Python 2.3 (which is pretty ancient by now) is by no means a 
hard requirement, it's just that there hasn't been any strong reason to 
require anything newer. I've for a while suspected that in order to 
semi-sanely support both Python 2.x and 3.x, something has got to give. 
All the major distros are already at Python 2.6 so I dont find it 
unreasonable to bump up the requirements to Python 2.6 if that's what it 
takes to support Python 3.x too.

> I'm uneasy with purely manual testing of this code.  Is there an 
> existing unittest suite for the rpm-python code?

Unfortunately no. And yes a test-suite for it is desperately needed.

>
> With these patches I can do this:
> [david at brick rpm]$ PYTHONPATH=/home/david/coding/python3/rpm-python-bindings/install-prefix/lib/python3.1/site-packages python3
> Python 3.1.1 (r311:74480, Oct  1 2009, 12:20:21)
> [GCC 4.4.1 20090725 (Red Hat 4.4.1-2)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import rpm
> /home/david/coding/python3/rpm-python-bindings/install-prefix/lib/python3.1/site-packages/rpm/__init__.py:9: DeprecationWarning: Type rpm.hdr defines tp_reserved (formerly tp_compare) but not tp_richcompare. Comparisons may not behave as intended.
>  from rpm._rpm import *
> # (clearly I need to fix this)
>>>> rpm.addMacro("_dbpath", "/var/lib/rpm")
>>>> ts = rpm.TransactionSet()
>>>> mi = ts.dbMatch()
>>>> for h in mi:
> ...     print("%s-%s-%s" % (h['name'], h['version'], h['release']))  # note that print has become a function
> b'im-chooser'-b'1.2.6'-b'3.fc11'
> (etc)
>
> This highlights one of the big unknowns to me at the moment: what 
> guarantees (if any) are there as to the encoding of the bytes in a value 
> of RPM_STRING_CLASS?  Currently in python/rpmtd-py.c:rpmtd_ItemAsPyobj I 
> punt the issue by returning them as Bytes objects, and hence you see the 
> bytes objects in the output. So as written, users of the API would have 
> to deal with decoding these back to py3k's unicode strings.

There are no guarantees whatsoever :-/ The spec file has no concept of 
encoding so the headers end up containing whatever was used in the spec, 
even mixtures of encodings within a package. See http://rpm.org/ticket/30, 
this is a long-standing issue but so far nothing concrete has been done 
about it. And even if/when rpm starts requiring UTF-8 encoding, anything 
dealing with rpms will have to deal with legacy packages without such 
a guarantee for a long, long time.

Further comments to the individual patches to follow...

 	- Panu -