[prev in list] [next in list] [prev in thread] [next in thread] 

List:       openembedded-core
Subject:    [OE-core] [PATCH 4/6] scripts/resulttool: fix ptests results containing a non reproducible path
From:       Alexis Lothoré via lists.openembedded.org <alexis.lot
Date:       2023-02-28 18:10:50
Message-ID: 20230228181052.4191521-7-alexis.lothore () bootlin ! com
[Download RAW message or body]

From: Alexis Lothoré <alexis.lothore@bootlin.com>

Some ptests results may be wrongly sampled, resulting in some part of the error
being propagated in test name. For example:

"ptestresult.binutils-ld.in testcase \
/home/pokybuild/yocto-worker/qemumips/build/build-st-1666126/tmp/work/mips32r2-poky-linux/binutils-cross-testsuite/2.40-r0/git/ld/testsuite/ld-ctf/ctf.exp"
 "ptestresult.gcc.Couldn't create remote directory /tmp/runtest.455781 on ssh"
"ptestresult.gcc-libstdc++-v3.Couldn't create remote directory /tmp/runtest.3814266 \
on target"

While the root errors must be fixed in corresponding ptests packages for those
tests, the test results history must still be usable even with this issue

Add new filters to detect if temporary test directories (build-st-*,
/tmp/runtime.*) are present in name. If so, truncate test name. As a side
effect, it will aggregate multiple failing errors into one, but the regression
will still be raised

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
---
 scripts/lib/resulttool/regression.py | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/scripts/lib/resulttool/regression.py \
b/scripts/lib/resulttool/regression.py index ad377c596b6..d7f0b16ca98 100644
--- a/scripts/lib/resulttool/regression.py
+++ b/scripts/lib/resulttool/regression.py
@@ -219,6 +219,12 @@ def fixup_ptest_names(results, logger):
                     new = test.split("_-_")[0]
                 elif test.startswith(("ptestresult.curl.")) and "__" in test:
                     new = test.split("__")[0]
+                elif test.startswith(("ptestresult.dbus.")) and "__" in test:
+                    new = test.split("__")[0]
+                elif test.startswith("ptestresult.binutils") and "build-st-" in \
test: +                    new = test.split(" ")[0]
+                elif test.startswith("ptestresult.gcc") and "/tmp/runtest." in test:
+                    new = ".".join(test.split(".")[:2])
                 if new:
                     results[r][i]['result'][new] = results[r][i]['result'][test]
                     del results[r][i]['result'][test]
-- 
2.39.1



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#177852): https://lists.openembedded.org/g/openembedded-core/message/177852
Mute This Topic: https://lists.openembedded.org/mt/97296210/4454766
Group Owner: openembedded-core+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [openembedded-core@marc.info]
-=-=-=-=-=-=-=-=-=-=-=-



[prev in list] [next in list] [prev in thread] [next in thread]