[prev in list] [next in list] [prev in thread] [next in thread] 

List:       wine-devel
Subject:    Re: TestBot News (Windows 11, PCI passthrough)
From:       Francois Gouget <fgouget () codeweavers ! com>
Date:       2022-11-07 3:24:55
Message-ID: f523df9-78f8-981d-8bae-ab6a3d6f6577 () free ! fr
[Download RAW message or body]

w11pro64_nv went offline a couple of times. That's because QEmu was 
unable to use the graphics card, issuing an "Unknown PCI header type 
¡127¢" error. This typically happens when the GPU is left in a bad 
(~sleep) state after the VM using it is forcefully powered off. The only 
fix is to reboot the host. Yuck!

I think this happened because Windows Update was not really disabled 
(see below) and took its sweet time updating the VM on shutdown, causing 
a timeout and thus a forceful power off.

So I updated the Windows 11 VM and hopefully this will not happen 
anymore.


So what's new with the Windows 11 VM?
-------------------------------------

* It now has all the 21H2 updates up to 2022-11-04.
  (see also 'What about Windows 11 22H2?' below)

* On Windows 10 setting a data cap would automatically also mark the 
  network connection as metered. However in Windows 11 this is no longer 
  the case and it is the metered part that prevents Windows from 
  updating. So when I went through my standard procedure to set up this 
  VM, Windows update was not disabled at all. So in this update I fixed 
  this and now neither Edge nor Windows should be updating while we are 
  running the tests.

* I also changed the Edge and Teams default configuration so they don't 
  start at boot.

* And I cleared a couple of tasks in the security center and set the 
  password to not expire monthly! (Is that local account default another 
  way for Microsoft to push users to create an online account?)

* I changed the w11pro64_amd and w11pro64_nv test configurations to only 
  install the graphics driver and not the corresponding fancy GUIs. This 
  should prevent the QT component of AMD's GUI from interferring with 
  the clipboard tests.

  w11pro64_amd has whql-amd-software-adrenalin-edition-22.5.1-win10-win11-may10.exe
  w11pro64_nv has nvidia-517.48-desktop-win10-win11-64bit-international-dch-whql.exe

* After the update I could not get the AMD RX 6600 graphics card to work 
  with PCI passthrough anymore: the Windows driver would just show a 
  (generic) 'code 43' error.

  Fortunately at some point I found vfio-pci errors on the host in 
  kernel.log:
  vfio-pci 0000:03:00.0: BAR 0: can't reserve [mem 0x4020000000-0x402fffffff 64bit pref]

  Then /proc/iomem showed that part of that memory range was used by the 
  efifb module which some resources online recommend disabling for PCI 
  passthrough. And indeed that fixed things!

  It's strange that this module did not cause trouble before. I suspect 
  it is quite recent and only got loaded when I rebooted the host to fix 
  the NVIDIA graphics card!

  
  As usual, see the WineTestBot VMs wiki page for details of how the 
  TestBot VMs are set up:

  https://wiki.winehq.org/Wine_TestBot_VMs#PCI-Passthrough


Audio vs. PCI passthrough
-------------------------

The regular VMs have an ich9 emulated audio card. That sometimes causes 
trouble when tests do short duration timing tests because of the way 
Spice's reports its buffering to keep the audio and video in sync over 
the network.

So the PCI passthrough VMs do away with ich9 to the tests to use the 
graphics cards sound support instead, that is audio through HDMI / 
DisplayPort. That's real hardware so it should be perfect!

But there's a hitch: the 'screen' that these graphics cards are 
connected to don't support audio. The graphics card detect that and 
report that no 'speakers' are connected. One of the visible impact is 
that this causes Windows to set the sound level to zero and prevent the 
user from changing the volume. And it causes some Wine test units (see 
mmdevapi:*) to skip the tests [1]. They are probably right to but it's 
just galling to have real hardware and not be able to fully use it :-(

I put 'screen' in quotes above because in fact the TestBot VM hosts are 
rack mounted and use 'dummy HDMI / DisplayPort plugs'. The descriptions 
on online stores never specify whether this type of plug supports audio 
which I take to mean that probably none of them does (if you know of 
any that do let me know).

I guess the next step will be to test this with the KVM instead but it's 
not clear if it supports audio through HDMI (online resources are so 
useless for these products!). At least the HDMI KVM dongles advertise 
audio support but don't have a jack connector and lsusb does not show 
any audio device. So there it looks like audio would have to go through 
HDMI.

[1] mmdevapi:mmdevenum has failures probably because it does not 
    detect this case.


What about Windows 11 22H2?
---------------------------

First Windows 11 will not automatically install it through Windows 
Update because that involves a recheck that the PC is compatible which 
obviously fails since the VM has neither UEFI nor TPM 2.0 (see 
'QEmu vs. UEFI' below).

But booting on the 22H2 ISO (or Rufus' USB 'key') results in an 
immediate freeze. It turns out that can be avoided by changing the 
emulated VM 'chipset' from Q35 to i440FX.

That's a bummer because for PCI passthrough Q35 is much recommended 
because it emulates PCI-Express which matches the bus the actual 
graphics cards and their drivers expect. So essentially the choice is 
between PCI passthrough and Windows 11 22H2.

However online there are few reports of trouble with 22H2 and Q35. So I 
suspect this is a QEmu 5.2 bug and that upgrading would likely fix that. 
So I tabled the 22H2 upgrade and will revisit it when I can update QEmu. 
The VM hosts run Debian Stable which only has QEmu 5.2. Debian Testing 
has version 7.1 so I'll see if that can easily be brought there (maybe 
via backports).


QEmu vs. UEFI
-------------

As explained on the WineTestBot VMs page, the normal way to set up a 
Windows 11 VM in QEmu would be to configure the VM to use UEFI to get 
support for secure boot and to add an emulated TPM.

Similarly a number of online resources recommend configuring the VM with 
UEFI for PCI passthrough, though those seem to be from the early PCI 
passthrough days and no longer seem to be relevant.

However there is a blocker that prevents using this approach for the 
TestBot: Libvirt does not support taking snapshots of UEFI VMs, whether 
live or not. More precisely internal qcow2 snapshots, the only type 
supported by Libvirt, are incompatible with UEFI. The QEmu developers 
won't fix that and instead recommend using external snapshots. But that 
requires a lot of work to be supported in Libvirt (i.e. it could still 
take years). Currently it would be possible to create an external 
disk-only snapshot by changing the TestBot's Libvirt code but it would 
not be possible to restore it without running QEmu manually (i.e. give 
up on the snapshot VM XML description file and Libvirt's remote access 
support). So that rules out UEFI entirely.

So this Windows 11 VM uses a regular BIOS and relies on Rufus to strip 
Windows 11's UEFI, secure boot and TPM requirements.

https://wiki.winehq.org/Wine_TestBot_VMs#Windows_11

-- 
Francois Gouget <fgouget@codeweavers.com>



[prev in list] [next in list] [prev in thread] [next in thread]