[prev in list] [next in list] [prev in thread] [next in thread] 

List:       gcc
Subject:    Re: Which Binutils should I use for performing daily regression test on trunk?
From:       Hans-Peter Nilsson <hans-peter.nilsson () axis ! com>
Date:       2011-12-23 7:43:14
Message-ID: 201112230743.pBN7hERg018813 () ignucius ! se ! axis ! com
[Download RAW message or body]

> From: Terry Guo <flameroc@gmail.com>
> Date: Wed, 21 Dec 2011 04:25:46 +0100

> I plan to set up daily regression test on trunk for target
> ARM-NONE-EABI and post results to gcc-testresults mailing
> list.

Nice.  I see others do it for that target, but apparently not
for a pristine tree (the results having many failures I don't
see for cris-elf and the added "Summary adjusted for local
xfails" I don't understand), others restricting to
--enable-languages=c.

> Which
> Binutils should I use, the Binutils trunk or the latest released
> Binutils? And which way is recommended, building from a combined tree
> or building separately? If there is something I should pay attention
> to, please let me know. Thanks very much.

You need to worry about newlib and sim too (the latter assuming
you use the simulator cohabiting with the gdb project and not
e.g. qemu).

Maybe it's better to recap how my autotesters work instead of
pointing out caveats and making decisions for you.  I have one
gcc autotester instance (independent updates etc.) for trunk and
each open release branch for cris-elf, but I'll use singular
below as it's just one script.  It imports tarballs, the latest
one FAIL-free for cris-elf and cris-axis-linux-gnu exported from
my autotester for binutils trunk and similarly one from the sim
autotester.  The tarball for newlib somewhat similarly, but
untested, see below.  The gcc autotester imports those tarballs
when it's regression-free and not busy testing.  It does an
extra test-round to check that results from the update do not
regress, and bails on the update if there is a regression.  All
binutils, newlib, and sim tarballs are imported together; if
there's a regression in that phase I have to investigate anyway,
no use separating the updates.

The sim and binutils are each built separately, as there's
absolutely no warranty that they're combinable (that they'll
build together) at every time, more so as the gcc tree can carry
regressions for quite some time (well, at least several months)
and even more so for the release-branches.  The newlib tarball
is combined into the gcc tree (or rather the other way round as
gcc files override) but it itself is not tested separately.  The
newlib testsuite is dead or dysfunctional; all tests fail, and
no, I have not reported this or investigated.  Mea culpa, but at
least that's not a regression.  Newlib problems visibile as a
regression with a gcc update are dealt with properly.  Thus, I
have no separate newlib autotester, just a newlib auto-checkout-
and-tarball-creator.

Autotesters are trigged by emails from the *-cvs@ lists (a
procmail recipe feeding "batch") but sleeps for 15 minutes to
pick up quick corrections and as a last precaution to avoid
hammering the anon cvs/svn servers (a sure-fire way to get
blacklisted).  I rarely post results ...ok, one sent now.
Results are only sent manually.  Each gcc autotester bugs me
(only me) by email, when there are regressions that I haven't,
in a separate file, marked as known after entering a PR or an
error e.g. updating (anon update can fail temporarily) and
schedules a later restart (with "at") for errors that are
expected to be temporary.  After a tree update I also check
(using "find -newer" on a file created before the update) and
exit early if there are only changes in subdirectories and files
irrelevant to this target, for example Ada, go, libgomp and
target-specific files and directories for other targets.  Only
one instance of each autotester runs at a time, guarded using
"lockfile" and early exits.

And now you may wonder why I don't just post the damn script.
Let's just consider that a gift; less code to look at. 8-}
Actually, the "main" test-bits are in
contrib/regression/btest-gcc.sh.  The calling script with the
"updating" infrastructure haven't seemed generic enough to
warrant interest to take me over the cleanup-and-post threshold.
And it's all so obvious, at least in retrospect. :)  Maybe later.

I believe this is about the same as how Geoff K's autotester for
powerpc-eabisim (IIRC) used to work.  N.B., no update scripts
for that setup were posted.

Happy Holidays, H-P
PS. currently (r182649) regression-free since T0=2007-01-05-16:47:21 (UTC)!
[prev in list] [next in list] [prev in thread] [next in thread]