[prev in list] [next in list] [prev in thread] [next in thread] 

List:       ant-dev
Subject:    Re: Tests in Surefire
From:       Gintautas Grigelionis <g.grigelionis () gmail ! com>
Date:       2018-10-28 17:13:02
Message-ID: CALVNWHUz-wq+nVQrQJKG9Qq9o7VrXwKFF4naSMsXb_3yVwakNA () mail ! gmail ! com
[Download RAW message or body]


On Thu, 25 Oct 2018 at 09:05, Gintautas Grigelionis <g.grigelionis@gmail.com>
wrote:

> On Wed, 24 Oct 2018 at 00:29, Gintautas Grigelionis <
> g.grigelionis@gmail.com> wrote:
>
>> On Tue, 23 Oct 2018 at 12:25, Stefan Bodewig <bodewig@apache.org> wrote:
>>
>>> On 2018-10-20, Gintautas Grigelionis wrote:
>>>
>>> > I believe that in order to execute Ant test suite in Surefire one must
>>> > configure basedir which unfortunately affects Ant test projects.
>>>
>>> I'm not sure I share the goal of trying to run the tests via Surefire at
>>> all.
>>>
>>> Antoine has been the one who set up the maven poms and I don't remember
>>> us ever discussing what we wanted to achieve there. To me they've been a
>>> vehicle to get decent POMs for the jars we publish to Maven Central, not
>>> more. Here "decent" means "specifies the correct dependencies" and not
>>> even "can be used to compile the jar". I think it has outgrown my
>>> personal minimal requirement by far, so others must have had bigger
>>> plans :-)
>>>
>>> I do not believe we've been aware of Surefire bugs that prevented
>>> running. We've most likely never cared (and I still don't, obviously :-)
>>>
>>
>> It was in the README :-) the inability to set basedir/workingDir.
>>
>>
>>> > Perhaps it would make sense to have a magic property that (in
>>> > combination with detection of Surefire environment) would allow
>>> > BuildFileRule to call System.clearProperty("basedir")?
>>>
>>> If that's all it would take to make them work, that's fine with me. If
>>> we need further modifications to the tests them I'm not sure.
>>>
>>
>> Fair enough. I've hit some snags and I commented on them in the POMs.
>>
>
> I am at the point now where I must say "this is the farthest I could get
> without changing the tests".
> Please see the comments in src/etc/poms/ant/pom.xml, especially
> testExcludes section.
> Some tests, in particular in IncludeTest, are not robust (failure modes
> depend on XML parser,
> which may throw an exception because encoding is not specified, which was
> not the original intention).
> AntTest hardcodes directory structure (build/testcases) which apparently
> breaks modular builds in Jenkins
> (works locally). Also, current test failures in Jenkins apparently are an
> oddity of Maven,
> which uses ANT_HOME/jre/java on Java 8 or older SDKs precluding use of
> javac.
>
> Thanks, Gintas
>

It has been eerily quiet here... anyway, here's the state of the test suite
vis-a-vis Surefire and modular Maven builds:

   - most cases that were dependent on assumed structure of build file tree
   are refactored to use build.tests.value property (corresponding to
   testOutputDirectory in Maven);
   - build.tests.value property is redundant where it is used in
   combination with java.class.path property and therefore is removed so that
   the corresponding test classes do not need to assert it;
   - root property is removed from all test build files except one,
   javadoc, where it is used to access Java sources; POMs do not set it, and
   perhaps build.xml needs not set it either;
   - I added build.classes.value property (corresponding to outputDirectory
   in Maven), it's only used in signjar tests currently; I chose build.*.value
   names for the properties in order to avoid accidental name collisions.

Currently, certain Ant core and junit/junitlauncher tests still fail in
Surefire. I would like to get them all working; however, I am afraid that
their logic should be changed quite drastically. Also. please see comments
in pom.xml about some dependencies that must be eliminated (launcher on
core, or core on junit).

I also activated test analysis in Ant_Nightly (build from POMs presents
test analysis automagically). Currently, it seems that Maven executes more
tests than Nightly, and I will look into that discrepancy, too.

Thanks, Gintas


[prev in list] [next in list] [prev in thread] [next in thread]