[Rpm-ecosystem] DNF team would like to take over libdnf ownership

[Neal Gompa]

On Mon, Oct 30, 2017 at 6:48 AM, Daniel Mach <dmach at redhat.com> wrote:
> Hi everyone,
> My name is Daniel Mach and I'm leading the DNF team.
>
>
> TL;DR:
>
> The DNF team wants to take libdnf over and re-start development, which would
> possibly include using C++ (only to limited and reasonable extent). The goal
> is to get existing DNF code into libdnf. The current APIs would remain
> intact.
> We're looking for your use cases.
>

Fantastic!

>
> More details follow:
>
> We are getting requests to implement more features in microdnf,
> such as modularity[1][2], unify repo cache and overall business logic.
> Porting big portions of existing DNF code (Python) into a C library
> seems to be the only feasible option to avoid code duplication.
>
> In the first place, we'd like to clarify libdnf ownership and
> responsibilities.
> Libdnf is currently co-maintained by several teams (DNF, PackageKit,
> rpm-ostree), which makes it an unmaintained orphan.
> There are gaps in roadmap, development and response time to the customer
> issues.
> The code suffers with a technical debt, especially the libhif/hawkey merge
> hasn't been finished.
>
> From organizational perspective, we are proposing following:
> * DNF team to own libdnf and be fully responsible for it.
> * DNF team to guarantee existing C and Python APIs.

This would probably be a mistake, given how the existing APIs are. I
would suggest picking which base of APIs you want to guarantee and
work from there.

> * DNF team to discuss any API changes with the stakeholders in timely
> manner.

This means you should include soname bumps every time API/ABI changes occur.

> * PackageKit to become a libdnf stakeholder.
> * rpm-ostree to become a libdnf stakeholder.
>
> We are looking for a way how to boost the development speed by using a more
> high-level programming language than C. We are considering C++ at the
> moment,
> because it offers some features we (mainly Python programmers) would
> appreciate,
> including strongly typed lists and dictionaries, less work with pointers and
> last, but not least - native objects. It also allows us to wrap and re-use
> existing C code without major changes.
>

I don't have a problem with C++. In fact, I'm more versed in C++ than
I am in C, so this is pretty nice for me. That said, I do have a
couple of concerns about it:

* Binding from C++ to languages is trickier than C, so do you plan to
provide a C API for this? Or maybe do you plan to wire in SWIG[1] to
dynamically generate them as libsolv does?
* Using C++ makes glib2 usage mostly pointless, as glib2 is by and
large replaced by the C++ STL. The only advantage of GLib2 is GObject
Introspection, but if you're using C++, you might as well do something
else for bindings generation (like SWIG).
* Please avoid Boost. The Zypper stack uses it and it's a major pain
(it's why Yocto dropped Zypper years ago).

[1]: http://www.swig.org/

> Short-term plan: Fix libdnf technical debt
> * Finish libhif and hawkey merge.
> * Remove unused code.
> * Fix code duplicates.
> * Hide private functions which were published by mistake.
>

This conflicts with guaranteeing all C/Python APIs. Again, I suggest
picking what you want to stabilize and go from there.

> Mid-term plan:
> * Merge DNF Base class with libdnf's context.
> * Re-shuffle existing C code into new structures, but still keep existing
> APIs.
> * Implement modularity[1]
>
> Long-term plan:
> * Full rewrite/cleanup
> * Move major part of DNF's code into libdnf
> * New, well-defined, fully supported API
>
> The Holy Grail:
> * libdnf becomes "The Software Management Library"
> * libdnf has fully documented and tested API
> * libdnf has bindings to infinite amount of languages :)

Though if we move from C to C++ (and consequently drop glib2
dependency), then GObject introspection goes away, which is what could
give us this (sorta). SWIG (as mentioned above) can offer this for us,
but you need to build this in from the beginning...

>
> I'd like to ask you for your feedback on the plan.
> If you have any use cases you'd like implemented, please open an issue[3] or
> a bug[4].
> I understand that the C++ part sounds controversial, but as long as we keep
> the
> C API available and intact, nothing should change for the existing users.
> Please keep the discussion technical.
>

The big question for me is if you decide to use C++, do you plan to
keep the glib2 dependency? From my point of view, glib2 is a wasteful
and unnecessary extra dependency for C++ libraries and applications.

Also, where do librepo and libcomps fit into this plan? Both libraries
are also maintained similarly to libdnf, and they've caused their own
share of issues.

In addition, it would be good if libdnf is capable of being used as a
static link library, mainly for building a statically linked version
of microdnf that would work in the initramfs for system recovery and
such.

-- 
真実はいつも一つ！/ Always, there's only one truth!