[Rpm-ecosystem] [v2] Testing the Dependency Chain (w/ Behave)

Pavel Odvody podvody at redhat.com
Fri Sep 11 12:48:30 UTC 2015


On Fri, 2015-09-11 at 08:11 -0400, Radek Holy wrote:
> 
> ----- Original Message -----
> > From: "Pavel Odvody" <podvody at redhat.com>
> > To: rpm-ecosystem at lists.rpm.org
> > Sent: Friday, September 11, 2015 1:48:03 PM
> > Subject: Re: [Rpm-ecosystem] [v2] Testing the Dependency Chain (w/
> > Behave)
> > 
> > On Fri, 2015-09-11 at 07:12 -0400, Radek Holy wrote:
> > > 
> > > ----- Original Message -----
> > > > From: "Pavel Odvody" <podvody at redhat.com>
> > > > To: rpm-ecosystem at lists.rpm.org
> > > > Sent: Monday, September 7, 2015 3:20:50 PM
> > > > Subject: Re: [Rpm-ecosystem] [v2] Testing the Dependency Chain
> > > > (w/
> > > > Behave)
> > > > 
> > > > Update notice!
> > > > 
> > > > Rich dependencies now share common syntax across RPM and
> > > > libsolv:
> > > > - the Dockerfile was updated to rebuild both from git master
> > > > - all testing repositories were ported to the new syntax
> > > > 
> > > > The operators are: or, and, if, else
> > > > 
> > > > New test cases are coming, stay tuned!
> > > > 
> > > > On Fri, 2015-08-28 at 19:05 +0200, Pavel Odvody wrote:
> > > > > Hello,
> > > > > 
> > > > > I've pushed an updated branch to the git repo [1], which now
> > > > > implements
> > > > > the tests on top of the Behave framework. One test is
> > > > > currently
> > > > > converted to the new specification (test-1).
> > > > > 
> > > > > Example run:
> > > > > 
> > > > > $ ./test-launcher.py test-1
> > > > > Running test:
> > > > >  test-1
> > > > > Starting container:
> > > > >  docker run -i -v /richdeps-docker/repo:/build:Z -v /richdeps
> > > > > -docker/features:/behave:Z pavelo/richdeps:1.0.0 test-1
> > > > > 
> > > > > Feature: Richdeps/Behave test # behave/test-1.feature:1
> > > > >   TestA requires (TestB OR TestC), TestA recommends TestC
> > > > >   Scenario: Install TestA from repository "test-1"    #
> > > > > behave/test
> > > > > -1.feature:4
> > > > >     Given I use the repository "test-1"               #
> > > > > behave/steps/test_behave.py:51
> > > > >     When I "install" a package "TestA" with "dnf"     #
> > > 
> > > 
> > > I think you can use "context.config.userdata" in order to make it
> > > technology agnostic (i.e. get rid of 'with "dnf"'). Content of
> > > this
> > > dictionary can be defined in the behave config as well as from
> > > command line.
> > 
> > The goal is to be able to mix&match package managers within the
> > same
> > test case specification, for example to test package erasure
> > through
> > RPM after installing with DNF etc.
> > Both RPM and libsolv have their own implementation of the
> > dependency
> > resolution so I guess it makes sense to test both.
> > I suppose it could be used to "memoize" the last package manager
> > used,
> > allowing for constructs like:
> > 
> > When I "install" a package "PkgA" with "dnf"
> >  And I "install" a package "PkgB"
> > Then package "PkgA, PkgB" should be "installed"
> > 
> > But this dims the expressiveness of Behave and essentially is a
> > shift
> > from explicit to implicit definition (which is especially frowned
> > upon
> > in the land of Python).
> 
> What problems do you expect to find/prevent with these tests? In such
> case, it doesn't make much sense to run these tests during the
> continuous integration of a single package manager. Then I need to
> ask who is going to run/maintain these tests?
> 
> If you insist on this style of tests, as an alternative, I can also
> imagine something like:
> 
>     When I "install" a package "PkgA" with the user-defined package
> manager
>      And I "remove" a package "PkgA" with another package manager
>     Then I am not really surprised that it works
> 

These tests are not meant for unit nor integration testing phase, and
are better fit to V&V phase.
I wonder how is that different from the specification of test-2 below:

Scenario: 
  Given I use the repository "test-1"
  When I "install" a package "TestB" with "rpm"
  Then package "TestB" should be "installed"
  When I "install" a package "TestA" with "dnf"
  Then package "TestA, TestC" should be "installed"

RPM has no notion of repository and conversely libsolv has no notion of
"single package" and since both technologies use their own dependency
resolution mechanism to act upon a *single* database, it makes sense to
offer that level of granularity to test writers.
The split of "WHEN install X with Y" and "THEN check if X is installed"
is to catch situations where given tool returned a wrong return code
for the action that took effect.


> > 
> > 
> > > 
> > > 
> > > > > behave/steps/test_behave.py:61
> > > > >     Then package "TestA, TestC" should be "installed" #
> > > > > behave/steps/test_behave.py:71
> > > > >     And package "TestB" should be "absent"            #
> > > > > behave/steps/test_behave.py:71
> > > > > 
> > > > > 1 feature passed, 0 failed, 0 skipped
> > > > > 1 scenario passed, 0 failed, 0 skipped
> > > > > 4 steps passed, 0 failed, 0 skipped, 0 undefined
> > > > > Took 0m5.615s
> > > > > OK
> > > > > 
> > > > > 
> > > > > [1]:
> > > > > https://github.com/shaded-enmity/richdeps-docker/tree/behave-
> > > > > int
> > > > > egration
> > > > > _______________________________________________
> > > > > Rpm-ecosystem mailing list
> > > > > Rpm-ecosystem at lists.rpm.org
> > > > > http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
> > > > 
> > > > --
> > > > Pavel Odvody <podvody at redhat.com>
> > > > Software Engineer - EMEA ENG Developer Experience
> > > > 5EC1 95C1 8E08 5BD9 9BBF 9241 3AFA 3A66 024F F68D
> > > > Red Hat Czech s.r.o., Purkyňova 99/71, 612 45, Brno
> > > > 
> > > > 
> > > > 
> > > > _______________________________________________
> > > > Rpm-ecosystem mailing list
> > > > Rpm-ecosystem at lists.rpm.org
> > > > http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
> > > > 
> > > 
> > 
> > --
> > Pavel Odvody <podvody at redhat.com>
> > Software Engineer - EMEA ENG Developer Experience
> > 5EC1 95C1 8E08 5BD9 9BBF 9241 3AFA 3A66 024F F68D
> > Red Hat Czech s.r.o., Purkyňova 99/71, 612 45, Brno
> > 
> > 
> > 
> > _______________________________________________
> > Rpm-ecosystem mailing list
> > Rpm-ecosystem at lists.rpm.org
> > http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
> > 
> 

-- 
Pavel Odvody <podvody at redhat.com>
Software Engineer - EMEA ENG Developer Experience
5EC1 95C1 8E08 5BD9 9BBF 9241 3AFA 3A66 024F F68D
Red Hat Czech s.r.o., Purkyňova 99/71, 612 45, Brno


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.rpm.org/pipermail/rpm-ecosystem/attachments/20150911/e44ba224/attachment.asc>


More information about the Rpm-ecosystem mailing list