[prev in list] [next in list] [prev in thread] [next in thread]
List: wine-devel
Subject: TestBot News - MR false positives analysis
From: Francois Gouget <fgouget () codeweavers ! com>
Date: 2022-12-14 3:53:37
Message-ID: 3fcffd9b-c3c0-4428-56b5-5d7eaf3ed619 () free ! fr
[Download RAW message or body]
I have been reviewing the TestBot and GitLab CI test results for the
merged MRs. While doing that I updated the TestBot's known failures list
(https://testbot.winehq.org/FailuresList.pl) in order to drive down the
false positive rate.
Incidentally I also collected the list of test units causing false
positives, so I'll start with that. Specifically, here are the bugs to
fix to help the GitLab CI:
* Bug 53433 - mmdevapi:capture - impacted 18 MRs
* Bug 54064 - ntdll:threadpool - impacted 15 MRs
* Bug 54078 - ntdll:pipe - impacted 11 MRs
* Bug 54140 - mmdevapi:render - impacted 5 MRs
* Bug 54005 - ole32:clipboard - impacted 5 MRs
* Bug 54037 - user32:msg - impacted 5 MRs
* Bug 54074 - ws2_32:sock - impacted 5 MRs
I classified the TestBot / GitLab CI results as follows:
* False positive
Cases where the CI system incorrectly claimed the MR introduces new
failures. This is typically the case when the failures that are
already present in nightly WineTest results.
* Bad merge
MRs that break a test and got merged anyway.
* Collateral Damage from a bad merge
The false positives (aka collateral damage) caused by one of the bad
merges above.
* Outside interference
This identifies false positives that are not random and intrinsic to
the test but that result from change outside the Wine infrastructure,
for instance certificates that expire, or configuration changes to
servers that break the tests that depend on it.
Of those the only ones that a CI can really avoid are the first type,
aka "False positive". So I calculated the corresponding weekly rate:
Adjusted False Positive rate
Week | TestBot | GitLab CI
2022-11-14 | 21.9% | 8.3%
2022-11-21 | 8.0% | 21.6%
2022-11-28 | 14.7% | 28.4%
2022-12-05 | 8.5% | 24.5%
2022-12-12 | 0.0% | 20.0%
Note that the TestBot's 8% rate for the 11-21 week is not representative
because Wine was broken that week (collateral damage) which prevented
the tests from running in Wine, and thus from contributing real "false
positives". Also the 12-12 week is still incomplete obviously.
Even so I think his shows the TestBot is improving.
Here's a list of the incidents for the weeks above:
* 11-14 An external certificate revocation issue caused crypt32:cert to
fail systematically. This impacted 14 merge requests and was fixed in
MR1360.
* 11-17 MR!1399 got merged despite the TestBot detecting that it
prevented 32-bit Wine tests from running to completion. This impacted
39 merge requests. I could have reduced that number if I had been
faster to reconfigure the TestBot to stop running the full 32-bit Wine
test suite. This was fixed in MR!1524.
* 11-17 MR!1398 got merged despite the TestBot detecting that it broke
ntoskrnl.exe:ntoskrnl on Windows 7. This was fixed in MR!1803.
* 11-22 MR!1495 got merged despite the TestBot detecting that it broke
vbscript:run on Windows *. I don't have a record of the impacted MRs
or of when it was fixed.
* 11-23 The b00a831d direct commit broke kernel32:process in Wine. This
got fixed since.
* 12-07 MR!1732 got merged despite the TestBot detecting that it broke
taskschd:scheduler on Windows *. I immediately added a known failure
entry and no MR got impacted. This was fixed in MR!1736.
If not filtering out the failures caused by these incidents, the false
positive rate is:
Raw False Positive rate
Week | TestBot | GitLab CI
2022-11-14 | 52.1% | 27.1%
2022-11-21 | 50.0% | 29.5%
2022-11-28 | 20.0% | 33.7%
2022-12-05 | 19.1% | 57.4%
2022-12-12 | 0.0% | 20.0%
I think that also shows that the TestBot is improving.
I have attached the raw data I collected and shell snippets to
extract various statistics (failures-mr.txt) as well as a spreadsheet
import (failures.xls).
--
Francois Gouget <fgouget@codeweavers.com>
["failures-mr.txt" (text/plain)]
Troublesome test units:
grep -h -v -P '^(#| +[0-9])' failures-mr.txt |
sed -n '/week 12-02$/,/week 11-28$/p' |
sed -e 's/[^a-z0-9:._]/\n/g' | egrep '[a-z0-9]:[a-z0-9]' | sort | uniq -c | sort -n \
-r # or to filter on days instead of weeks
sed -n '/^12-05/,/^12-02/p' |
# or to exclude already diagnosed / filtered out issues
sed -e 's/[a-z][a-z0-9:._]*:[a-z][a-z0-9:._]* ->//g' |
# or to exclude the GitLab CI failures
sed -e 's/([^)]*)//g' |
# or to keep on the GitLab CI failures
sed -e 's/^.*[(]\([^)]*\)[)].*$/\1/' -e t -e d |
TestBot false positive rate and more:
for t in "[.]" "N" "F" "B" "D" "O" "E" "S" "[?]" "."; do count=`sed -n '/week \
12-02$/,/week 11-28$/p' failures-mr.txt | egrep -i "^$t. [0-9][0-9][0-9]" | wc -l`; [ \
$count -ne 0 ] && echo " $t -> $count"; done
GitLab CI false positive rate and more:
for t in "[.]" "N" "F" "B" "D" "O" "E" "S" "[?]" "."; do count=`sed -n '/week \
12-02$/,/week 11-28$/p' failures-mr.txt | egrep -i "^.$t [0-9][0-9][0-9]" | wc -l`; [ \
$count -ne 0 ] && echo " $t -> $count"; done
For importing in spreadsheets:
(p=13;for d in 12 09 08 07 06 05 02 01;do echo -n "2022/12/$p,"; for t in "[.]" "N" \
"F" "B" "D" "O" "E" "S" "[?]" "."; do count=`sed -n "/^12-$p/,/^12-$d/p" \
failures-mr.txt | egrep -i "^$t. [0-9][0-9][0-9]" | wc -l`; echo -n "$count,"; done; \
p=$d; echo; done; p=12-01;for d in 30 29 28 26 24 23 22 21 18 17 16 15 14 11;do echo \
-n "2022/$p,"; for t in "[.]" "N" "F" "B" "D" "O" "E" "S" "[?]" "."; do count=`sed -n \
"/^$p/,/^11-$d/p" failures-mr.txt | egrep -i "^$t. [0-9][0-9][0-9]" | wc -l`; echo -n \
"$count,"; done; p=11-$d; echo; done)
# One character classification codes:
# . Success
# The CI found no failure attributable to the MR.
# N False negative
# An MR that got merged and broke a test with no advance warning of the CI.
# F False positive
# Identifies cases where the CI system incorrectly claimed the MR introduces
# new failures. This is typically the case when the failures that are already
# present in nightly WineTest results.
# B Bad merge
# Identifies MRs that break a test and got merged anyway. Whether an MR is
# F or B is mostly determined after the fact.
# D Collateral Damage from a bad merge
# This identifies the false positives (aka collateral damage) caused by one
# of the bad merges above.
# O Massive outside interference
# This is not an intermittent false positive intrinsic to the test, but a
# failure resulting from a change outside the Wine infrastructure, such as
# when the configuration of a server used for the tests changes and breaks the
# tests that depend on it.
# E BotError
# And internal CI bug. Mostly relevant for TestBot results.
# S Patch series mixup
# Cases where it looks like the TestBot mixed up two patch series, resulting
# in a patch that cannot be applied and thus no actual test results.
# ? Status to be determined
#
# A merge request may have more than one type of failure. In that case the
# priority order is: B N F D O E S ?
12-16 --- week 12-12
12-13
.f 1750 (winhttp:notification)
.f 1802 (ntdll:pipe -> bug 54078)
.. 1803
.. 1808
.. 1776
.f 1804 (wmvcore:wmvcore)
12-12
.. 1793
.. 1796
.. 1766
.. 1792
.. 1768
.. 1772
.. 1763
.. 1791
.. 1789
12-09 --- week 12-02
.o 1751 (winhttp:notification -> bug 54098)
.f 1681 (mmdevapi:capture user32:msg)
(winhttp:notification -> bug 54098)
.o 1740 (winhttp:notification -> bug 54098)
.f 1757 (build:windows-on-unix)
.f 1761 (ole32:clipboard -> bug 54005)
.f 1758 (ntdll:pipe -> bug 54078)
(ws2_32:sock -> bug 54074 + bug 52474)
.. 1759
.f 1631 (ntdll:pipe -> bug 54078)
(wmvcore:wmvcore)
.. 1638
oo 1721 winhttp:notification -> bug 54098
(winhttp:notification -> bug 54098)
.f 1754 (build:windows-on-unix)
.. 1760
.. 1582
12-08
.o 1727 (winhttp:notification -> bug 54098)
.o 1739 (winhttp:notification -> bug 54098)
.o 1738 (winhttp:notification -> bug 54098)
.o 1737 (winhttp:notification -> bug 54098)
fo 1635 ws2_32:afd -> new bug 54113
(winhttp:notification -> bug 54098)
.o 1748 (winhttp:notification -> bug 54098)
.f 1747 (ntdll:threadpool -> bug 54064)
(winhttp:notification -> bug 54098)
.o 1745 (winhttp:notification -> bug 54098)
.o 906 (winhttp:notification -> bug 54098)
.o 1744 (winhttp:notification -> bug 54098)
.f 1742 (winhttp:notification -> bug 54098)
(ws2_32:afd -> new bug 54113)
.f 1736 taskschd:scheduler -> new bug 54109
(winhttp:notification -> bug 54098)
.f 1701 (ntdll:threadpool -> bug 54064)
(winhttp:notification -> bug 54098)
.o 1743 (winhttp:notification -> bug 54098)
.o 1734 (winhttp:notification -> bug 54098)
.o 1735 (winhttp:notification -> bug 54098)
.. 1741
12-07
Bo 1732 taskschd:scheduler -> new bug 54105 + new bug 54106
(winhttp:notification -> bug 54098)
.o 1666 (ntdll:threadpool -> bug 54064)
(winhttp:notification -> bug 54098)
ff 1705 ws2_32:sock -> bug 54073 variant
(ddraw:ddraw7)
(ntdll:pipe -> bug 54078)
(winhttp:notification -> bug 54098)
.o 1726 (winhttp:notification -> bug 54098)
.o 1730 (rpcrt4:server)
(winhttp:notification -> bug 54098)
.o 1729 (ntdll:threadpool -> bug 54064)
(winhttp:notification -> bug 54098)
.o 1728 (ntdll:threadpool -> bug 54064)
(winhttp:notification -> bug 54098)
Fo 1731 msctf:inputprocessor
(winhttp:notification -> bug 54098)
oo 1713 winhttp:notification -> bug 54098
(winhttp:notification -> bug 54098)
So 1717 ???
(winhttp:notification -> bug 54098)
.o 1719 (winhttp:notification -> bug 54098)
oo 1718 winhttp:notification -> bug 54098
(winhttp:notification -> bug 54098)
oo 1720 winhttp:notification -> bug 54098
(winhttp:notification -> bug 54098)
of 1722 winhttp:notification -> bug 54098
(ntdll:threadpool -> bug 54064)
(winhttp:notification -> bug 54098)
oo 1723 winhttp:notification -> bug 54098
(winhttp:notification -> bug 54098)
oo 1716 winhttp:notification -> bug 54098
(winhttp:notification -> bug 54098)
.o 1698 (winhttp:notification -> bug 54098)
.f 1715 (winhttp:notification -> bug 54098)
(ws2_32:sock -> bug 54074)
oo 1724 winhttp:notification -> bug 54098
(winhttp:notification -> bug 54098)
12-06
.. 1712
.. 1704
.. 1702
.. 1692
.. 1693
.. 1618
f. 1708 wmvcore:wmvcore => unique, ignore
ff 1703 quartz:filtermapper -> new bug 54095
(mshtml:misc user32:menu)
.. 1709
.. 1711
.f 1700 (ntdll:threadpool -> bug 54064)
.f 1688 (ntdll:pipe -> bug 54078)
(user32:clipboard)
.. 1697
.f 1699 (urlmon:protocol)
(ws2_32:sock -> bug 54074)
.. 1696
.. 1694
.. 1695
f. 1691 kernel32:process -> new bug 54094
.. 1690
.. 1689
.. 1181
.. 1687
12-05
.f 1630 (ntdll:threadpool -> bug 54064)
.. 1664
.. 1602
.. 1677
.. 1529
.. 1660
.f 1685 (ntdll:threadpool -> bug 54064)
.f 1668 (mmdevapi:capture -> bug 53433)
(qasf:asfreader)
.. 1676
f. 1675 shell32:autocomplete -> new bug 54090
.. 1678
.. 1679
.. 1646
.. 1643
.. 1680
.. 1673
.f 1667 (ws2_32:afd -> new bug 54113)
(ws2_32:sock -> bug 52474)
.f 1669 (wininet:http -> bug 54084)
(ws2_32:sock -> bug 54073)
f. 1547 mmdevapi:capture -> bug 53265 variant
.. 1663
.f 1672 (dxgi:dxgi explorer.exe:explorer ntdll:file)
.. 1671
.. 1670
12-02 --- week 11-28
.f 1622 (user32:menu)
(user32:msg -> bug 54037)
.. 1542
.. 1654
.f 1659 (wmvcore:wmvcore -> bug 54042)
.. 1623
.. 1653
.. 1640
.f 1647 (mmdevapi:capture -> bug 53433)
.. 1644
.. 1645
.. 1649
.. 1651
.. 1629
.. 1655
.f 1656 (ole32:clipboard -> bug 54005)
.. 1657
.f 1658 (mmdevapi:capture -> bug 53433)
(ole32:clipboard -> bug 54005)
.. 1648
.. 1650
.. 1652
.f 1617 (ntdll:pipe -> new bug 54078)
(user32:msg -> bug 54037)
12-01
.. 1642
.. 1641
.. 1639
.. 1632
.f 1523 (timeout)
.. 1634
.. 1606
.. 1636
f. 1619 user32:input -> new bug 54053 + bug 51477
.f 1502 (mmdevapi:capture -> bug 53433)
Ff 1627 mshtml:misc
(ntdll:pipe -> new bug 54078)
.. 1624
.. 1510
.f 1608 (mmdevapi:render -> new bug 54140)
(user32:msg -> bug 54037)
.f 1615 (ntdll:threadpool -> new bug 54064)
.. 1616
.. 1621
.. 1620
.. 1569
.o 1579 (wldap32:parse -> new bug 54075)
.. 1571
11-30
f. 1560 winhttp:notification -> new bug 54052
ff 1598 shell32:shelllink -> new bug 54061
(mmdevapi:capture -> bug 53433)
(mmdevapi:render -> new bug 54140)
(ntdll:pipe -> new bug 54078)
.. 1503
.. 1614
.. 1611
.. 1613
.. 1593
.. 1599
.. 1601
df 1454 MR!1399 fallout
(ntdll:threadpool -> new bug 54064)
f. 1553 winhttp:notification -> new bug 54052
e. 1604 The previous 1 run(s) ...
.. 1556
.f 1607 (mmdevapi:capture -> bug 53433)
.. 1609
ff 1199 wmvcore:wmvcore -> new bug 54039
(mmdevapi:render -> new bug 54140)
.. 1520
.f 1605 (mmdevapi:capture -> bug 53433)
.f 1603 (mmdevapi:capture -> bug 53433)
.. 1567
.f 1564 (mmdevapi:render -> new bug 54140)
11-29
.. 1555
ff 1575 shell32:shelllink -> new bug 54061
advapi32:security -> bug 54014
(mmdevapi:capture -> bug 53433)
oo 1581 wldap32:parse -> new bug 54075
(wldap32:parse -> new bug 54075)
oo 1574 wldap32:parse -> new bug 54075
shell32:autocomplete -> new bug 54090
(wldap32:parse -> new bug 54075)
.. 1522
.. 1539
.. 1551
.. 1597
.f 1326 (ntdll:threadpool -> new bug 54064)
(user32:msg -> new bug 54036)
(user32:win -> gitlab-specific)
.f 1596 (user32:msg -> new bug 54037)
ff 1595 advapi32:security -> bug 54014
(ntdll:pipe -> new bug 54078)
fo 1585 msctf:inputprocessor -> new bug 54062
wldap32:parse -> new bug 54075
(wldap32:parse -> new bug 54075)
fo 1583 advapi32:security -> bug 54014
wldap32:parse -> new bug 54075
wmvcore:wmvcore -> new bug 54039
ws2_32:sock -> new bug 54073
(wldap32:parse -> new bug 54075)
.. 1566
f. 1594 advapi32:security -> bug 54014
.. 1592
.. 1577
11-28
.. 1562
ff 1561 vbscript:run -> new bug 54003
(ddraw:ddraw2)
(ole32:clipboard -> new bug 54005)
(ws2_32:sock -> new bug 54074)
.. 1572
.. 1548
.f 1576 (mmdevapi:capture -> bug 53433)
.f 1573 (mmdevapi:capture -> bug 53433)
f. 1549 d3d9:d3d9ex -> new bug 53979
ntoskrnl.exe:driver_pnp -> new bug 54004
.. 1558
.. 1563
ef 1526 The previous 1 run(s) ...
(ole32:clipboard -> new bug 54005)
.. 1568
.. 1565
.. 1570
f. 1557 winhttp:notification -> new bug 54052
.f 1536 (ntdll:file)
11-26 --- week 11-21
TestBot
[E] -> 2
[S] -> 5
[?] -> 3
[F] -> 33 including 29 MR!1399 fallout
[B] -> 1
[N] -> 0
[.] -> 44
. -> 88
GitLab CI
[c] -> 26
[.] -> 62
. -> 88
12 kernel32:process
4 ws2_32:sock
4 mmdevapi:capture
3 ntdll:threadpool
2 vbscript:run
2 ntdll:pipe
2 kernel32:heap
1 user32:msg
1 user32:menu
1 urlmon:protocol
1 uiautomationcore:uiautomation
1 ucrtbase:cpp
1 shell32:shelllink
1 ntdll:file
1 mshtml:misc
1 mmdevapi:render
1 d3drm:d3drm
1 d3d9:d3d9ex
1 advapi32:security
.f 1546 (ws2_32:sock -> new bug 54073)
f. 1550 uiautomationcore:uiautomation => unique, ignore
.f 1519 (user32:msg -> new bug 54037)
.. 1544
ff 1537 advapi32:security -> bug 54014
(ntdll:threadpool -> new bug 54064)
.. 1535
.. 1540
.. 1541
11-24
d. 1489 MR!1399 fallout
.f 1528 (mmdevapi:render -> new bug 54140)
.. 1422
f. 1534 vbscript:run -> new bug 54003
f. 1538 d3d9:d3d9ex -> new bug 53979
.f 1527 (kernel32:process)
.. 1530
.f 1531 (mmdevapi:capture -> bug 53433)
.. 1532
d. 1427 MR!1399 fallout
11-23
kernel32:process -> c#b00a831d fixed since
dd 1297 MR!1399 fallout (kernel32:process)
.d 1524 (kernel32:process -> fixed)
(ws2_32:sock -> new bug 54073)
.f 1525 (kernel32:process -> fixed)
(ntdll:pipe -> new bug 54078)
(ws2_32:sock -> new bug 54073)
.f 1504 (kernel32:process -> fixed)
(ntdll:pipe -> new bug 54078)
.d 1514 (kernel32:process -> fixed)
.d 1513 (kernel32:process -> fixed)
.d 1511 (kernel32:process -> fixed)
.d 1521 (kernel32:process -> fixed)
.d 1515 (kernel32:process -> fixed)
ef 1506 The previous 1 run(s) ... (build-linux timeout)
.. 1500
d. 1456 MR!1399 fallout
Sf 1512 ??? (kernel32:process -> fixed)
(mshtml:misc user32:menu)
d. 1423 MR!1399 fallout
.. 1384
d. 1491 MR!1399 fallout
.f 1509 (mmdevapi:capture -> bug 53433)
(urlmon:protocol -> new bug 54080)
.. 1507
.f 1508 (ntdll:file)
(ntdll:threadpool -> new bug 54064)
.. 1505
.f 1485 (ws2_32:sock -> new bug 54074)
ef 1447 The previous 1 run(s) ... (build-linux timeout)
11-22
d. 1461 MR!1399 fallout
.. 1453
f. 1470 kernel32:heap => unique, ignore
.f 1499 (kernel32:heap => unique, ignore)
.. 1497
.. 1501
d. 1442 MR!1399 fallout
Bf 1495 vbscript:run -> fixed since
(shell32:shelllink -> new bug 54061)
.. 1496
d. 1490 MR!1399 fallout
.. 1493
d. 1474 MR!1399 fallout
d. 1476 MR!1399 fallout
.. 1482
d. 1464 MR!1399 fallout
.. 1486
.. 1487
d. 1460 MR!1399 fallout
d. 1466 MR!1399 fallout
.. 1479
.. 1481
df 1483 MR!1399 fallout
(mmdevapi:capture -> bug 53433)
.. 1484
.. 1480
d. 1488 MR!1399 fallout
S. 1478 ???
S. 1448 ???
S. 1477 ???
11-21
f. 1473 ucrtbase:cpp => unique, ignore
.. 1475
S. 1459 ???
d. 1435 MR!1399 fallout
df 1462 MR!1399 fallout
(mmdevapi:capture -> bug 53433)
d. 1467 MR!1399 fallout
d. 1451 MR!1399 fallout
df 1450 MR!1399 fallout
(ntdll:threadpool -> new bug 54064)
d. 1452 MR!1399 fallout
.. 1458
.. 1463
f. 1457 d3drm:d3drm -> new bug 53974
d. 1472 MR!1399 fallout
d. 1449 MR!1399 fallout
d. 1343 MR!1399 fallout
.. 1416
d. 1471 MR!1399 fallout
d. 1465 MR!1399 fallout
d. 1468 MR!1399 fallout
d. 1469 MR!1399 fallout
11-18 --- week 11-14
TestBot
[E] -> 3
[S] -> 0
[?] -> 1
[F] -> 43 including 10 MR!1399 fallout
[B] -> 2
[N] -> 1
[.] -> 46
. -> 96
GitLab CI
[c] -> 26
[.] -> 70
. -> 96
38 crypt32:cert
6 d3drm:d3drm
4 advapi32:security
3 winediag:d3d_device_create
3 mmdevapi:capture
3 d3d8:device
2 wmvcore:wmvcore
2 user32:win
2 user32:msg
2 ole32:clipboard
2 httpapi:httpapi
2 d3d9:device
1 shell32:shelllink
1 quartz:systemclock
1 mshtml:htmldoc
1 ntoskrnl.exe:ntoskrnl
1 dinput:hid
1 d3d11:d3d11
d. 1334 MR!1399 fallout
d. 1434 MR!1399 fallout
d. 1426 MR!1399 fallout
?. 1436 MR!1399 fallout + bad patch???
d. 1373 mshtml:htmldoc => only in this MR, ignore
d. 1440 MR!1399 fallout
df 1441 MR!1399 fallout
(shell32:shelllink -> new bug 54061)
d. 1438 MR!1399 fallout
d. 1417 MR!1399 fallout
d. 1433 MR!1399 fallout
.. 1431
.. 1430
.. 1429
.. 1428
eo 1231 The previous 1 run(s) ...
(crypt32:cert -> run on 2022-11-18 but fixed by MR!1360 on 2022-11-14!)
d. 1439 MR!1399 fallout
ff 1413 winediag:d3d_device_create -> lumped with bug 53975
(mmdevapi:capture -> bug 53433)
.. 914
.. 1392
oo 966 crypt32:cert -> run on 2022-11-14, fixed by MR!1360
(crypt32:cert -> run on 2022-11-14, fixed by MR!1360)
.. 1424
N. 1432 good but -> bug 53977
11-17
f. 1419 d3drm:d3drm -> new bug 53975
.. 1420
.. 1340
B. 1398 ntoskrnl.exe:ntoskrnl -> bug 53976
.. 1412
f. 1400 dinput:hid -> bug 53525 fixed context regular expression
.. 1409
e. 1394 The previous 1 run(s) ...
.. 1415
.. 1335
.. 1396
f. 1397 httpapi:httpapi -> new bug 53980
.. 1403
B. 1399 broke winetest -> bug 53963
.f 1405 (build-linux cc error)
.. 1401
.. 1406
f. 1404 d3drm:d3drm -> new bug 53974
.f 1402 (mmdevapi:capture -> bug 53433)
.. 1410
.. 1407
f. 1411 d3drm:d3drm -> new bug 53974
.. 1408
11-16
.. 1389
.. 1374
f. 1391 d3d8:device -> bug 52983 variant
.. 1390
.. 1386
.o 1295 (crypt32:cert -> run on 2022-11-16 but fixed by MR!1360 on 2022-11-14!)
.. 1365
f. 1385 d3drm:d3drm -> new bug 53974
d3d9:device -> bug 52983
.. 1388
.. 1387
f. 1382 ole32:clipboard -> new bug 54005
.o 1351 (crypt32:cert -> run on 2022-11-16 but fixed by MR!1360 on 2022-11-14!)
.. 1381
.. 1380
.. 1376
f. 1379 advapi32:security -> bug 54014
.. 1378
.. 1375
.. 1383
11-15
f. 1344 d3d11:d3d11 -> bug 53252
ef 1370 The previous 1 run(s) ... (quartz:systemclock)
.. 1369
f. 1372 d3drm:d3drm -> bug 53975
fo 1350 crypt32:cert -> fixed by MR!1360
wmvcore:wmvcore -> new bug 54041
(crypt32:cert -> fixed by MR!1360)
oo 1366 crypt32:cert -> fixed by MR!1360
(crypt32:cert -> fixed by MR!1360)
.. 1368
oo 1361 crypt32:cert -> fixed by MR!1360
(crypt32:cert -> fixed by MR!1360)
f. 1298 crypt32:cert -> fixed by MR!1360
d3d8:device -> bug 52983 variant
d3d9:device -> bug 52983
oo 1367 crypt32:cert -> fixed by MR!1360
(crypt32:cert -> fixed by MR!1360)
11-14
fo 1363 crypt32:cert -> fixed by MR!1360
advapi32:security -> bug 54014
(crypt32:cert -> fixed by MR!1360)
oo 1317 crypt32:cert -> fixed by MR!1360
(crypt32:cert -> fixed by MR!1360)
.f 1362 (crypt32:cert -> fixed by MR!1360)
(mmdevapi:capture -> bug 53433)
oo 1353 crypt32:cert -> fixed by MR!1360
(crypt32:cert -> fixed by MR!1360)
oo 1356 crypt32:cert -> fixed by MR!1360
(crypt32:cert -> fixed by MR!1360)
oo 1357 crypt32:cert -> fixed by MR!1360
(crypt32:cert -> fixed by MR!1360)
oo 1359 crypt32:cert -> fixed by MR!1360
(crypt32:cert -> fixed by MR!1360)
oo 1358 crypt32:cert -> fixed by MR!1360
(crypt32:cert -> fixed by MR!1360)
oo 1347 crypt32:cert -> fixed by MR!1360
(crypt32:cert -> fixed by MR!1360)
.. 1360 -> fixes crypt32:cert -> yay
fo 1354 crypt32:cert -> fixed by MR!1360
wmvcore:wmvcore -> new bug 54039
(crypt32:cert -> fixed by MR!1360)
f. 1315 advapi32:security -> bug 54014
of 1342 crypt32:cert -> fixed by MR!1360
(crypt32:cert -> fixed by MR!1360)
(user32:msg -> new bug 54036)
(user32:win)
oo 1352 crypt32:cert -> fixed by MR!1360
(crypt32:cert -> fixed by MR!1360)
.o 1349 (crypt32:cert -> fixed by MR!1360)
.. 1348
.. 1260
f. 1346 winediag:d3d_device_create -> lumped with bug 53975
.. 1345
ff 1341 ole32:clipboard -> new bug 54005
(user32:msg -> new bug 54036)
(user32:win -> gitlab-specific)
f. 1307 d3drm:d3drm -> bug 53975
httpapi:httpapi -> bug 53980
winediag:d3d_device_create -> lumped with bug 53975
.. 1249
.. 1331
f. 1332 d3d8:device -> bug 52983 variant
11-11 -- week 11-07
["failures.xls" (application/vnd.ms-excel)]
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic