[prev in list] [next in list] [prev in thread] [next in thread] 

List:       rpmorg-ecosystem
Subject:    [Rpm-ecosystem] Testing the Dependency Chain
From:       podvody () redhat ! com (Pavel Odvody)
Date:       2015-09-11 11:06:35
Message-ID: 1441969595.3960.7.camel () redhat ! com
[Download RAW message or body]

On Fri, 2015-09-11 at 06:48 -0400, Radek Holy wrote:
> 
> ----- Original Message -----
> > From: "Pavel Odvody" <podvody at redhat.com>
> > To: rpm-ecosystem at lists.rpm.org
> > Sent: Monday, August 24, 2015 4:09:49 PM
> > Subject: Re: [Rpm-ecosystem] Testing the Dependency Chain
> > 
> > On Mon, 2015-08-24 at 09:21 -0400, Honza ?ilhan wrote:
> > > > From: "Pavel Odvody" <podvody at redhat.com>
> > > > 
> > > > On Mon, 2015-08-24 at 11:29 +0200, Michael Schroeder wrote:
> > > > > As you're basically testing libsolv, you might as well use
> > > > > libsolv's
> > > > > testcase support.
> > > > 
> > > > That looks interesting, Michael.
> > > > Can this framework do without actually writing all the
> > > > specfiles?
> > > > 
> > > > Is there a documentation for the *.t file syntax or do I have
> > > > to
> > > > reverse-engineer that from tests/* ? :)
> > > 
> > > The syntax is pretty straightforward. There is some docs of flags
> > > [1] or
> > > look
> > > into code for SOLVER_(FLAG) constants.
> > Thanks for the link.
> > 
> > > (The generated testcases from `dnf ... --debugsolver` can be seen
> > > in
> > > "debugdata/testcase.t")
> > > 
> > > Pavel, what is the aim of the tests?
> > > 1) Have a guide for package maintainers, the reference
> > > 2) make sure all DNF versions always honor the guide
> > > 3) test that RPM works too
> > > 
> > > For 1) we can do that in Behave (easily readable) and convert it
> > > to
> > > libsolv's testcases.
> > > (btw the newest Fedora libsolv build contains the `testsolv`
> > > executable.)
> > > 
> > 
> > I'm not sure about the factual merits, people unfamiliar with
> > Behave are
> > going to struggle with parsing it's sentences anyway, it's like
> > writing
> > software in COBOL for the sake of it being "self-documenting" :)
> > OTOH, if we had some syntax highlight for Behave test cases it
> > could be
> > nice.
> 
> Look rather for a Cucumber or Gherkin syntax highlight. At least
> Github, Bitbucket and PyCharm are already able to highlight it.
> Although there is not much to highlight :-)
> BTW, PyCharm also already has some helpers. AFAIK, it may e.g. inform
> you that a particular sentence does not match any step definition.
> 

Vim has support for Cucumber/Gherkin highlight too, very bare bones
though. But for me, the parts that are not in quotes usually "blur",
when I look at the line:

When I "install" a package "TestA" with "dnf"

I see:

When..install..TestA..dnf :)

In general I like the expressiveness and maturity of Behave. I'll be
adding a few more tests/repos later today.


> > Anyway, I'm going to try to rewrite a few tests into Behave to see
> > how
> > it turns out.
> > 
> > > If we want to support 2) too then the test should run on DNF
> > > level. Hawkey adds some flags to libsolv by default and some of
> > > them
> > > are added by different DNF command. So you would test DNF by
> > > `dnf ... --assumeno --debugsolver` then inspect "debug
> > > data/solver.result".
> > > Injecting the different env of installed packages would be
> > > harder, maybe
> > > by setting different --installroot.
> > > 
> > > Having tested all rpm stack 3) you should continue with what you
> > > are doing.
> > 
> > All 3, the aim is to be:
> > 
> >    1) Technology neutral (can test dnf, libsolv, pkcon, rpm ...)
> >    2) Closest to the natural usage as possible (command line,
> > lightweight bindings)
> > 
> > > 
> > > It would be a good idea to have these tests written the same way
> > > as planned
> > > DNF functional tests. If we don't use Beaker then docker will be
> > > a good
> > > choice
> > > for creating a separate environment without the need of setup and
> > > teardown
> > > (just kill and run again the container). It should not be a
> > > problem to run
> > > it under the Jenkins - I've seen some posts mentioning running
> > > docker
> > > there.
> > > 
> > 
> > Definitely, do you have some documentation/roadmap regarding the
> > functional tests?
> > 
> > > [1]
> > > https://github.com/openSUSE/libsolv/blob/master/doc/libsolv-bindi
> > > ngs.txt
> > > 
> > > Honza
> > > _______________________________________________
> > > Rpm-ecosystem mailing list
> > > Rpm-ecosystem at lists.rpm.org
> > > http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
> > 
> > 
> > --
> > Pavel Odvody <podvody at redhat.com>
> > Software Engineer - EMEA ENG Developer Experience
> > 5EC1 95C1 8E08 5BD9 9BBF 9241 3AFA 3A66 024F F68D
> > Red Hat Czech s.r.o., Purky?ova 99/71, 612 45, Brno
> > 
> > 
> > _______________________________________________
> > Rpm-ecosystem mailing list
> > Rpm-ecosystem at lists.rpm.org
> > http://lists.rpm.org/mailman/listinfo/rpm-ecosystem
> > 
> 

-- 
Pavel Odvody <podvody at redhat.com>
Software Engineer - EMEA ENG Developer Experience
5EC1 95C1 8E08 5BD9 9BBF 9241 3AFA 3A66 024F F68D
Red Hat Czech s.r.o., Purky?ova 99/71, 612 45, Brno


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.rpm.org/pipermail/rpm-ecosystem/attachments/20150911/37f2fa61/attachment.asc>

[prev in list] [next in list] [prev in thread] [next in thread]