[prev in list] [next in list] [prev in thread] [next in thread] 

List:       wine-devel
Subject:    TestBot News - WineTest progress
From:       Francois Gouget <fgouget () codeweavers ! com>
Date:       2023-03-15 18:31:08
Message-ID: fe6f3b5-90c4-942-15c4-7de9fca6d419 () free ! fr
[Download RAW message or body]


There have been quite a lot of new failures in the past few builds which 
makes it easy to forget the Wine test results are getting better.

The progress is not actually visible on the regular patterns page 
because it is too slow to show up across the past two months. It's 
when looking at 8 months of results that one can see it [1]:

  http://fgouget.free.fr/wtb/patterns.html


There one can see that back in July we had 9 test configurations which 
sometimes got under 10 failures while now we have over 40 that achieve 
it pretty consistently! And where the record low was 8 failing test 
units, now it is 3.

In more details:
* The Debian 11 VMs went from about 12 failures to a mere 4 failures 
  now, not just in the English locale but in a couple others too.

* The Debian Testing VM went from about 20 failures to about 8 now.

* The GitLab CI also went from about 15 failures to about 4 now.

* Windows 11 went from 50+ failures to about 25. Big progress there!
  (But the AMD GPU still has an extra 10.)

* Windows 10 22H2 went from over 10 failures in November to about 5 now.

* Windows 10 21H1 went from about 20 failures to just under 10, 
  most of the time.

* The Windows UTF-8 configurations went from about 35 failures to around 
  20 now.

* The full Wine test suite ran successfully not once but 8 times on 
  Windows 10 1507 (that's still only ~7% of successful runs since the 
  first success).

* Windows 10 1809 has been consistently below the 10 failures threshold 
  since mid-October, and 1709 and 1909 joined it mid-January.


But as I said progress is slow and without everyone's efforts the 
results could very well regress.


[1] Note that a lot of links on that page are broken because they point 
    to the main test.winehq site which does not have 8 months of 
    history. Also I could not duplicate the 97 GB of pages to my 
    website.

-- 
Francois Gouget <fgouget@codeweavers.com>


[prev in list] [next in list] [prev in thread] [next in thread]