[prev in list] [next in list] [prev in thread] [next in thread] 

List:       freebsd-hackers
Subject:    Re: Testing with lua/atf-lua reviews
From:       Kyle Evans <kevans () freebsd ! org>
Date:       2020-10-29 1:03:00
Message-ID: CACNAnaGQ-ek5yV6WCSN2J=jwh3qARfAJ6u3MRn65BH1W3hftCg () mail ! gmail ! com
[Download RAW message or body]

On Wed, Oct 28, 2020 at 7:43 PM Kyle Evans <kevans@freebsd.org> wrote:
> 
> On Wed, Oct 28, 2020 at 7:23 PM Enji Cooper <yaneurabeya@gmail.com> wrote:
> > 
> > 
> > On Oct 24, 2020, at 9:09 AM, Kyle Evans <kevans@FreeBSD.org> wrote:
> > 
> > Hello!
> > 
> > I've just put up for review some work I've done to allow us to write
> > tests in lua, primarily intended to test the lua libs we're writing.
> > Please feel free to add yourself or drop in for some commentary:
> > 
> > - https://reviews.freebsd.org/D26928 - atf-lua(1)/atf-lua(3) itself
> > - https://reviews.freebsd.org/D26929 - Build glue for atf-lua
> > - https://reviews.freebsd.org/D26930 - atf.tests.mk infrastructure for
> > adding tests
> > - https://reviews.freebsd.org/D26931 - Build glue for atf-lua tests
> > - https://reviews.freebsd.org/D26932 - jail(3lua) tests, as a sample
> > 
> > Note that D26932 has an additional hard dependency on the libjail
> > bindings and some additions I've made to them, notably: D26080,
> > D26756, and D26927.
> > 
> > 
> > Hi Kyle,
> > 
> > I realize that I haven't been fully in the loop lately due to time and focusing \
> > on other things, but I'm not fully onboard with this approach. 
> > In particular, one of the things that jmmv was more onboard with was limiting \
> > atf, not extending it, and I agree with his desire to not do that. 
> 
> Sure-
> 
> > Furthermore, why isn't this using the luaunit framework instead and the support \
> > being added to kyua to support luaunit:  http://lua-users.org/wiki/UnitTesting ? \
> > There are a ton of caveats with ATF that I would rather not support longer than \
> > necessary and having to teach folks how to use a homegrown test infrastructure \
> > instead of leveraging an open source test infrastructure which is supported by an \
> > external group. Doing the latter makes maintenance easy for us and improves the \
> > utility of the support better. 
> 
> I had actually considered this, but ruled it out due to a couple
> factors. The main one was that there's really no benefit with adopting
> yet another test framework for base -- I think we're much less likely
> to get people that are already familiar with luaunit contributing to
> our base tests than we are to get people in one of two other camps:
> 
> 1. They already work with our vast array of other ATF !lua tests and
> would find themselves staring at a really familiar interface, or
> 2. They don't, they want to work on Lua stuff and Lua tests in base,
> then an ATF lua interface will be at least somewhat applicable to
> other areas of our test infrastructure
> 
> I haven't dumped all that much time into this, though, so I have no
> problem taking another aerial glance at it.

Oh, oh dear. luaunit already supports TAP output:
https://luaunit.readthedocs.io/en/latest/#output-formats

I kinda prefer the minimal verbosity text format that it defaults to,
but as far as effort to get off the ground goes... that's likely
impossible to beat.
_______________________________________________
freebsd-hackers@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"


[prev in list] [next in list] [prev in thread] [next in thread]