[Rpm-ecosystem] Testing suggestions for "dnf --quiet repolist" & dnf --quiet list"?

[Honza Šilhan]

> > From: "Nick Coghlan" <ncoghlan at gmail.com>
> >
> > As a way to get started on dnf development, I'm taking a look at
> > https://bugzilla.redhat.com/show_bug.cgi?id=1236310
> > 
> > This bug points out that "dnf --quiet list" produces output, while
> > "dnf --quiet repolist" does not.
> > 
> > The actual bug is fairly straightforward: "dnf repolist" is currently
> > using "logger.info" for command output, when it should be using
> > "print" (as per
> > https://docs.python.org/3/howto/logging.html#when-to-use-logging)

You are on the right path. You can create pull request and we can discuss
the code on github ;).

> > >> Specifically, I'm thinking that behave scenarios could be a good way
> > >> of testing some end-to-end user facing CLI behaviour for "given this
> > >> command line, we expect this command output":
> > >> http://pythonhosted.org/behave/tutorial.html#scenario-outlines
> > >>
> > >> We wouldn't want to go overboard with these, but I think they'd be a
> > >> good way to check the basic interconnections between components are in
> > >> place.
> > >
> > > I started to write some Behave tests (I'd call them "system tests" rather
> > > than "integration tests" but it doesn't matter I think) already because I
> > > wanted to make sure that my documentation of the "--installroot" switch
> > > matches the reality. Take a look at it
> > > https://github.com/rholy/dnf-extra-tests . Maybe you find it worth
> > > extending. If you do, I'm ready to move them under
> > > https://github.com/rpm-software-management/ if the rest of the team
> > > agrees.
> > 
> > +1, that sounds like a good idea to me. I actually also like the idea
> > of keeping these kinds of system/acceptance tests separate from the
> > main dnf repo, as that helps ensure consistency across releases and
> > makes it easier for QE folks to contribute specifically to the
> > acceptance tests.

We are missing the functional tests. The current unit tests are testing only parts
of the code and especially transactions and command output is hardly testable currently.
I also think it would be beneficial for us and QE to collaborate on these tests.
But I don't know they would agree about using Behave framework instead of their
beaker libs. I wanted to talk to Karel Srot (QE responsible for yum who will
probably at some time have to take care of DNF) so he can set up the beaker environment
with some tests from yum. He is unfortunately PTO this week. The question is if dnf-QE
alliance would work in the long term. If we could get write access into beaker to write
the new tests on our own and QE would not be afraid of us erasing some of the tests. We
want to run these tests in our CI checking the regression in the upstream and in pull request.
What am I saying now is to not make any radical changes or start to intensively work
on the tests before we talk to QE.

Honza