[prev in list] [next in list] [prev in thread] [next in thread]
List: webkit-unassigned
Subject: [Webkit-unassigned] [Bug 248561] New: [EWS] Include flavor (wk1/wk2) in results database request whe
From: bugzilla-daemon () webkit ! org
Date: 2022-11-30 22:50:19
Message-ID: bug-248561-2851 () https ! bugs ! webkit ! org/
[Download RAW message or body]
--1669848619.b156f0.32721
Date: Wed, 30 Nov 2022 14:50:19 -0800
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: https://bugs.webkit.org/
Auto-Submitted: auto-generated
https://bugs.webkit.org/show_bug.cgi?id=248561
Bug ID: 248561
Summary: [EWS] Include flavor (wk1/wk2) in results database
request when evaluating pre-existing failures
Product: WebKit
Version: Other
Hardware: Unspecified
OS: Unspecified
Status: NEW
Severity: Normal
Priority: P2
Component: Tools / Tests
Assignee: webkit-unassigned@lists.webkit.org
Reporter: ryanhaddad@apple.com
The test failures seen in the following WK1 run were consistently failing on ToT WK1 \
until they were rebaselined last night, but the failures were still triggering \
retries in spite of our new results database checks.
https://ews-build.webkit.org/#/builders/73/builds/20801
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&s \
uite=layout-tests&suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-plat \
form-tests%2Fcss%2Fcss-cascade%2Fall-prop-initial-xml.html&test=imported%2Fw3c%2Fweb-p \
latform-tests%2Fgeolocation-API%2FPositionOptions.https.html&test=imported%2Fw3c%2Fweb \
-platform-tests%2Fgeolocation-API%2Fpermission.https.html&test=imported%2Fw3c%2Fweb-pl \
atform-tests%2Fweb-animations%2Fanimation-model%2Fanimation-types%2Faccumulation-per-p \
roperty-002.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fweb-animations%2Fanimation \
-model%2Fanimation-types%2Faddition-per-property-002.html&test=imported%2Fw3c%2Fweb-pl \
atform-tests%2Fweb-animations%2Fanimation-model%2Fanimation-types%2Finterpolation-per-property-002.html
The issue might be that when we query the results database to analyze failures, we \
are only checking for platform (macOS/iOS) and configuration (release/debug), which \
will return results from both WK1 and WK2 and throw off the pass/fail percentage.
Checking Results database for failing tests. Identifier: 257097@main, configuration: \
{'platform': 'mac', 'style': 'release'}
imported/w3c/web-platform-tests/css/css-cascade/all-prop-initial-xml.html: pass_rate: \
72, pre-existing-failure=False Response from results-db: {'audio': 0, 'crash': 0, \
'error': 0, 'fail': 0, 'image': 0, 'pass': 72, 'text': 27, 'timeout': 0, 'warning': \
0}
imported/w3c/web-platform-tests/geolocation-API/PositionOptions.https.html: \
pass_rate: 44, pre-existing-failure=True Response from results-db: {'audio': 0, \
'crash': 0, 'error': 0, 'fail': 0, 'image': 0, 'pass': 44, 'text': 55, 'timeout': 0, \
'warning': 0}
imported/w3c/web-platform-tests/geolocation-API/permission.https.html: pass_rate: 44, \
pre-existing-failure=True Response from results-db: {'audio': 0, 'crash': 0, 'error': \
0, 'fail': 0, 'image': 0, 'pass': 44, 'text': 55, 'timeout': 0, \
'warning': 0}
imported/w3c/web-platform-tests/web-animations/animation-model/animation-types/accumulation-per-property-002.html: \
pass_rate: 44, pre-existing-failure=True Response from results-db: {'audio': 0, \
'crash': 0, 'error': 0, 'fail': 0, 'image': 0, 'pass': 44, 'text': 55, 'timeout': 0, \
'warning': 0}
imported/w3c/web-platform-tests/web-animations/animation-model/animation-types/addition-per-property-002.html: \
pass_rate: 44, pre-existing-failure=True Response from results-db: {'audio': 0, \
'crash': 0, 'error': 0, 'fail': 0, 'image': 0, 'pass': 44, 'text': 55, 'timeout': 0, \
'warning': 0}
imported/w3c/web-platform-tests/web-animations/animation-model/animation-types/interpolation-per-property-002.html: \
pass_rate: 44, pre-existing-failure=True Response from results-db: {'audio': 0, \
'crash': 0, 'error': 0, 'fail': 0, 'image': 0, 'pass': 44, 'text': 55, 'timeout': 0, \
'warning': 0}
If we had been checking for WK1 results specifically, it seems likely that we would \
have ignored the failures and skipped the retry step.
--
You are receiving this mail because:
You are the assignee for the bug.
--1669848619.b156f0.32721
Date: Wed, 30 Nov 2022 14:50:19 -0800
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: https://bugs.webkit.org/
Auto-Submitted: auto-generated
<html>
<head>
<base href="https://bugs.webkit.org/">
</head>
<body><table border="1" cellspacing="0" cellpadding="8">
<tr>
<th>Bug ID</th>
<td><a class="bz_bug_link
bz_status_NEW "
title="NEW - [EWS] Include flavor (wk1/wk2) in results database request when \
evaluating pre-existing failures" \
href="https://bugs.webkit.org/show_bug.cgi?id=248561">248561</a> </td>
</tr>
<tr>
<th>Summary</th>
<td>[EWS] Include flavor (wk1/wk2) in results database request when \
evaluating pre-existing failures </td>
</tr>
<tr>
<th>Product</th>
<td>WebKit
</td>
</tr>
<tr>
<th>Version</th>
<td>Other
</td>
</tr>
<tr>
<th>Hardware</th>
<td>Unspecified
</td>
</tr>
<tr>
<th>OS</th>
<td>Unspecified
</td>
</tr>
<tr>
<th>Status</th>
<td>NEW
</td>
</tr>
<tr>
<th>Severity</th>
<td>Normal
</td>
</tr>
<tr>
<th>Priority</th>
<td>P2
</td>
</tr>
<tr>
<th>Component</th>
<td>Tools / Tests
</td>
</tr>
<tr>
<th>Assignee</th>
<td>webkit-unassigned@lists.webkit.org
</td>
</tr>
<tr>
<th>Reporter</th>
<td>ryanhaddad@apple.com
</td>
</tr></table>
<p>
<div>
<pre>The test failures seen in the following WK1 run were consistently \
failing on ToT WK1 until they were rebaselined last night, but the failures were \
still triggering retries in spite of our new results database checks.
<a href="https://ews-build.webkit.org/#/builders/73/builds/20801">https://ews-build.webkit.org/#/builders/73/builds/20801</a>
<a href="https://results.webkit.org/?suite=layout-tests&suite=layout-tests&sui \
te=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&a \
mp;test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-cascade%2Fall-prop-initial-xml \
.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fgeolocation-API%2FPositionOptions \
.https.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fgeolocation-API%2Fpermissio \
n.https.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fweb-animations%2Fanimation \
-model%2Fanimation-types%2Faccumulation-per-property-002.html&test=imported%2Fw3c% \
2Fweb-platform-tests%2Fweb-animations%2Fanimation-model%2Fanimation-types%2Faddition-p \
er-property-002.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fweb-animations%2Fa \
nimation-model%2Fanimation-types%2Finterpolation-per-property-002.html">https://result \
s.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&sui \
te=layout-tests&suite=layout-tests&suite=layout-tests&test=imported%2Fw3c% \
2Fweb-platform-tests%2Fcss%2Fcss-cascade%2Fall-prop-initial-xml.html&test=imported \
%2Fw3c%2Fweb-platform-tests%2Fgeolocation-API%2FPositionOptions.https.html&test=im \
ported%2Fw3c%2Fweb-platform-tests%2Fgeolocation-API%2Fpermission.https.html&test=i \
mported%2Fw3c%2Fweb-platform-tests%2Fweb-animations%2Fanimation-model%2Fanimation-type \
s%2Faccumulation-per-property-002.html&test=imported%2Fw3c%2Fweb-platform-tests%2F \
web-animations%2Fanimation-model%2Fanimation-types%2Faddition-per-property-002.html&am \
p;test=imported%2Fw3c%2Fweb-platform-tests%2Fweb-animations%2Fanimation-model%2Fanimation-types%2Finterpolation-per-property-002.html</a>
The issue might be that when we query the results database to analyze failures, we \
are only checking for platform (macOS/iOS) and configuration (release/debug), which \
will return results from both WK1 and WK2 and throw off the pass/fail percentage.
Checking Results database for failing tests. Identifier: <a \
href="https://commits.webkit.org/257097@main">257097@main</a>, configuration: \
{'platform': 'mac', 'style': 'release'}
imported/w3c/web-platform-tests/css/css-cascade/all-prop-initial-xml.html: pass_rate: \
72, pre-existing-failure=False Response from results-db: {'audio': 0, 'crash': 0, \
'error': 0, 'fail': 0, 'image': 0, 'pass': 72, 'text': 27, 'timeout': 0, 'warning': \
0}
imported/w3c/web-platform-tests/geolocation-API/PositionOptions.https.html: \
pass_rate: 44, pre-existing-failure=True Response from results-db: {'audio': 0, \
'crash': 0, 'error': 0, 'fail': 0, 'image': 0, 'pass': 44, 'text': 55, 'timeout': 0, \
'warning': 0}
imported/w3c/web-platform-tests/geolocation-API/permission.https.html: pass_rate: 44, \
pre-existing-failure=True Response from results-db: {'audio': 0, 'crash': 0, 'error': \
0, 'fail': 0, 'image': 0, 'pass': 44, 'text': 55, 'timeout': 0, \
'warning': 0}
imported/w3c/web-platform-tests/web-animations/animation-model/animation-types/accumulation-per-property-002.html: \
pass_rate: 44, pre-existing-failure=True Response from results-db: {'audio': 0, \
'crash': 0, 'error': 0, 'fail': 0, 'image': 0, 'pass': 44, 'text': 55, 'timeout': 0, \
'warning': 0}
imported/w3c/web-platform-tests/web-animations/animation-model/animation-types/addition-per-property-002.html: \
pass_rate: 44, pre-existing-failure=True Response from results-db: {'audio': 0, \
'crash': 0, 'error': 0, 'fail': 0, 'image': 0, 'pass': 44, 'text': 55, 'timeout': 0, \
'warning': 0}
imported/w3c/web-platform-tests/web-animations/animation-model/animation-types/interpolation-per-property-002.html: \
pass_rate: 44, pre-existing-failure=True Response from results-db: {'audio': 0, \
'crash': 0, 'error': 0, 'fail': 0, 'image': 0, 'pass': 44, 'text': 55, 'timeout': 0, \
'warning': 0}
If we had been checking for WK1 results specifically, it seems likely that we would \
have ignored the failures and skipped the retry step.</pre> </div>
</p>
<hr>
<span>You are receiving this mail because:</span>
<ul>
<li>You are the assignee for the bug.</li>
</ul>
</body>
</html>
--1669848619.b156f0.32721--
_______________________________________________
webkit-unassigned mailing list
webkit-unassigned@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-unassigned
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic