[Rpm-ecosystem] Testing the Dependency Chain

Radek Holy rholy at redhat.com
Fri Sep 11 10:48:05 UTC 2015



----- Original Message -----
> From: "Pavel Odvody" <podvody at redhat.com>
> To: rpm-ecosystem at lists.rpm.org
> Sent: Monday, August 24, 2015 4:09:49 PM
> Subject: Re: [Rpm-ecosystem] Testing the Dependency Chain
> 
> On Mon, 2015-08-24 at 09:21 -0400, Honza Šilhan wrote:
> > > From: "Pavel Odvody" <podvody at redhat.com>
> > > 
> > > On Mon, 2015-08-24 at 11:29 +0200, Michael Schroeder wrote:
> > > > As you're basically testing libsolv, you might as well use libsolv's
> > > > testcase support.
> > >
> > > That looks interesting, Michael.
> > > Can this framework do without actually writing all the specfiles?
> > > 
> > > Is there a documentation for the *.t file syntax or do I have to
> > > reverse-engineer that from tests/* ? :)
> > 
> > The syntax is pretty straightforward. There is some docs of flags [1] or
> > look
> > into code for SOLVER_(FLAG) constants.
> Thanks for the link.
> 
> > (The generated testcases from `dnf ... --debugsolver` can be seen in
> > "debugdata/testcase.t")
> > 
> > Pavel, what is the aim of the tests?
> > 1) Have a guide for package maintainers, the reference
> > 2) make sure all DNF versions always honor the guide
> > 3) test that RPM works too
> > 
> > For 1) we can do that in Behave (easily readable) and convert it to
> > libsolv's testcases.
> > (btw the newest Fedora libsolv build contains the `testsolv` executable.)
> > 
> 
> I'm not sure about the factual merits, people unfamiliar with Behave are
> going to struggle with parsing it's sentences anyway, it's like writing
> software in COBOL for the sake of it being "self-documenting" :)
> OTOH, if we had some syntax highlight for Behave test cases it could be
> nice.

Look rather for a Cucumber or Gherkin syntax highlight. At least Github, Bitbucket and PyCharm are already able to highlight it. Although there is not much to highlight :-)
BTW, PyCharm also already has some helpers. AFAIK, it may e.g. inform you that a particular sentence does not match any step definition.

> Anyway, I'm going to try to rewrite a few tests into Behave to see how
> it turns out.
> 
> > If we want to support 2) too then the test should run on DNF
> > level. Hawkey adds some flags to libsolv by default and some of them
> > are added by different DNF command. So you would test DNF by
> > `dnf ... --assumeno --debugsolver` then inspect "debug data/solver.result".
> > Injecting the different env of installed packages would be harder, maybe
> > by setting different --installroot.
> > 
> > Having tested all rpm stack 3) you should continue with what you are doing.
> 
> All 3, the aim is to be:
> 
>    1) Technology neutral (can test dnf, libsolv, pkcon, rpm ...)
>    2) Closest to the natural usage as possible (command line,
> lightweight bindings)
> 
> > 
> > It would be a good idea to have these tests written the same way as planned
> > DNF functional tests. If we don't use Beaker then docker will be a good
> > choice
> > for creating a separate environment without the need of setup and teardown
> > (just kill and run again the container). It should not be a problem to run
> > it under the Jenkins - I've seen some posts mentioning running docker
> > there.
> > 
> 
> Definitely, do you have some documentation/roadmap regarding the
> functional tests?
> 
> > [1]
> > https://github.com/openSUSE/libsolv/blob/master/doc/libsolv-bindings.txt
> > 
> > Honza
> > _______________________________________________
> > Rpm-ecosystem mailing list
> > Rpm-ecosystem at lists.rpm.org
> > http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
> 
> 
> --
> Pavel Odvody <podvody at redhat.com>
> Software Engineer - EMEA ENG Developer Experience
> 5EC1 95C1 8E08 5BD9 9BBF 9241 3AFA 3A66 024F F68D
> Red Hat Czech s.r.o., Purkyňova 99/71, 612 45, Brno
> 
> 
> _______________________________________________
> Rpm-ecosystem mailing list
> Rpm-ecosystem at lists.rpm.org
> http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
> 

-- 
Radek Holý
Associate Software Engineer
Software Management Team
Red Hat Czech


More information about the Rpm-ecosystem mailing list