From kde-devel Sat Feb 24 19:40:29 2024 From: Ben Cooksley Date: Sat, 24 Feb 2024 19:40:29 +0000 To: kde-devel Subject: Re: Post-MegaRelease projects Message-Id: X-MARC-Message: https://marc.info/?l=kde-devel&m=170880357309970 MIME-Version: 1 Content-Type: multipart/mixed; boundary="--000000000000138d88061225da36" --000000000000138d88061225da36 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Feb 24, 2024 at 11:35=E2=80=AFPM Konstantin Kharlamov wrote: > On Sat, 2024-02-24 at 23:07 +1300, Ben Cooksley wrote: > > On Sat, Feb 24, 2024 at 10:27=E2=80=AFPM Konstantin Kharlamov > wrote: > > On Sat, 2024-02-24 at 22:16 +1300, Ben Cooksley wrote: > > On Sat, Feb 24, 2024 at 8:37=E2=80=AFPM Konstantin Kharlamov > wrote: > > On Sat, 2024-02-24 at 16:31 +1300, Ben Cooksley wrote: > > On Fri, Feb 23, 2024 at 11:12=E2=80=AFPM Sune Vuorela = wrote: > > On 2024-02-22, Nate Graham wrote: > > I've started pondering post-megarelease projects. We've spent so long o= n > > porting and bugfixing that I think it might be useful to shift gears to > > feature work, and I'd like to brainstorm potential large-scale projects > > and gauge the level of interest in putting resources into them soon. > > A bit more from the devops end that I'd love to see people tackle: > > - Ensure frameworks and app unit tests interacting with windows can run > on Windows. > More details: The following fails on our windows CI > https://invent.kde.org/sune/windows-test-thingie/-/blob/master/main.cp= p > I find it weird that we are spending resources on putting things in > the windows store and making apps available on windows, but we can't > actually have passing tests in our CI. > > > This unfortunately will not be easy to solve. > > One of the key things that we've learned out of doing CI, as has been > showcased by FreeBSD in particular, is that the builders need to be > ephemeral - that is only around for the build in question that is being r= un. > We're currently accomplishing this by using containers - via Podman (for > Linux/Android/FreeBSD) and Docker (for Windows). > > Containers also offer us the advantage of allowing people to easily > reproduce the CI environment on their local system without too much troub= le. > > For Windows however, Microsoft has limited the container stack to not > allow anything GUI related to work. The underlying libraries may be there= , > but the equivalent display server components are not operational. > > To complicate things further, on Windows certain permissions are > restricted to the interactive console and are not possible to do as eithe= r > a scheduled task or as a system service. > Usage of existing Windows automation frameworks such as Powershell > Remoting or SSH will therefore not work if we want things to perfectly > replicate a end user environment - because those will run the command(s) = as > part of a non-interactive session (even if the user we connect as is the > same one logged in on the desktop console). > > > Idk if it's a silly question, but=E2=80=A6 If Windows native containers h= ave so > many restrictions, why not just use Linux containers with WINE inside? > > > Because Wine is not Windows either, and there could be subtle differences > in how things run / interact with the system. > > > Fair point, no sw is bugless. Although, from my shallow experience WINE > seems to have an excellent test suite. I remember reporting a regression > once, which turned out to be due to some obscure surfaces manipulation by > an old Heroes =E2=85=A2 game. WINE devs not only quickly fixed that, but = they also > added tests for that case, so I'd presume such regression is just not gon= na > happen anymore. > > Plus some of our software would like to test certain system level > infrastructure (like say KDE Connect). > > > Out of curiosity, what does this infrastructure include? I thought KDE > connect only uses network sockets and system tray. > > > No idea, I saw their commentary on debugging issues they were having in > their unit tests in #kde-devel. > Those issues were due to a lack of permissions, specifically around the > interactive console - that is how I know some of our tests need those > additional permissions and why running as a scheduled task / system servi= ce > will not be sufficient for "fully working" CI tests on Windows. > > > Sorry, it seems there's some misunderstanding=E2=80=A6 Judging by what yo= u say you > seem to be referring to the restrictions that Windows containers have on > Windows. But that was the point that started this thread, to which I > replied that running Linux containers with WINE might be a solution =F0= =9F=98=8A > What i'm referring to here is that KDE Connect interacts with various components of the system in order to do what it does. See for instance the ability to share Notifications between devices, or ability to act as a presenter device. That requires accessing various system level APIs which WINE very well may not support - and we wouldn't support a scenario of using it under WINE because there is a native Linux version already. > > Plus, we have to have native Windows to compile things anyway as we need > to use MSVC (otherwise you have no Qt Web Engine support, as that cannot = be > built with MingW) > > > But I presume it can be built with Clang? In particular, Google Chrome on > Windows is being built with Clang =E2=80=94 and Web Engine is basically a= fork of > Chromium. > > > Qt 6 as a whole does not list Clang as a supported compiler - see > https://doc.qt.io/qt-6/supported-platforms.html > > Given Windows is a bit strange in the first place, i'd be quite reluctant > to step outside of what they list as supported. > > > Oh, that's interesting. So Qt claims to support MinGW, but then WebEngine > does not actually build with it? So it's a bug on their side, was it > reported? > > While at it, just to make sure I understand: do you compile with MSVC > everything or just the WebEngine (which Idk if it's possible without the > deps, but I guess asking doesn't hurt). > As you will see on the supported platforms page, certain modules don't support certain platform/compiler combinations - and MingW / WebEngine is one of those unsupported combinations. MSVC and MingW have incompatible C++ compiler ABIs. That means it is an all or nothing approach, so yes everything is built using MSVC and cannot be built with another compiler. Cheers, Ben --000000000000138d88061225da36 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Sat, Feb 24, 2024 at 11:35=E2=80=AFPM = Konstantin Kharlamov <Hi-Angel@yan= dex.ru> wrote:
On Sat, 2024-02-24 at 23:07 +1300, Ben Cooksley wrote:
On Sat, = Feb 24, 2024 at 10:27=E2=80=AFPM Konstantin Kharlamov <Hi-Angel@yandex.ru> wrote:
On Sat, 2024-02-24 at 22:16 +1300, Ben Cooksley wrote:
On Sat, Feb 24, 2024 at 8:37=E2=80=AFPM Konstantin Kharlamov <Hi-Angel@yandex.ru>= wrote:
On Sat, 2024-02-24 at 16:31 +1300, Ben Cooksley w= rote:
On Fri, Feb 23, 2024 at 11:12=E2=80=AFPM Sune Vuorela <nospam@vuorela.dk>= wrote:
On 2024-02-22, Nate Graham <nate@kde.org> wrote:
> I've started = pondering post-megarelease projects. We've spent so long on
> po= rting and bugfixing that I think it might be useful to shift gears to
&= gt; feature work, and I'd like to brainstorm potential large-scale proj= ects
> and gauge the level of interest in putting resources into the= m soon.

A bit more from the devops end that I'd love = to see people tackle:

=C2=A0- Ensure frameworks and app u= nit tests interacting with windows can run
=C2=A0 =C2=A0on Windows.
= =C2=A0 =C2=A0More details: The following fails on our windows CI
=C2=A0 = =C2=A0https://invent.kde.org/= sune/windows-test-thingie/-/blob/master/main.cpp
=C2=A0 =C2=A0I find= it weird that we are spending resources on putting things in
=C2=A0 =C2= =A0the windows store and making apps available on windows, but we can't=
=C2=A0 =C2=A0actually have passing tests in our CI.

<= /div>

This unfortunately will not be easy t= o solve.=C2=A0

One of the key things that we'v= e learned out of doing CI, as has been showcased=C2=A0by FreeBSD in particu= lar, is that the builders need to be ephemeral - that is only around for th= e build in question that is being run.
We're currently accomp= lishing this by using containers - via Podman (for Linux/Android/FreeBSD) a= nd Docker (for Windows).

Containers also offer= us the advantage of allowing people to easily reproduce the CI environment= on their local system without too much trouble.

F= or Windows however, Microsoft has limited the container stack to not allow = anything GUI related to work. The underlying libraries may be there, but th= e equivalent display server components are not operational.
=
To complicate things further, on Windows certain permissions= are restricted to the interactive console and are not possible to do as ei= ther a scheduled task or as a system service.
Usage of exist= ing Windows automation frameworks such as Powershell Remoting or SSH will t= herefore not work if we want things to perfectly replicate a end user envir= onment - because those will run the command(s) as part of a non-interactive= session (even if the user we connect as is the same one logged in on the d= esktop console).

<= div>Idk if it's a silly question, but=E2=80=A6 If Windows native contai= ners have so many restrictions, why not just use Linux containers with WINE= inside?


B= ecause Wine is not Windows either, and there could be subtle differences in= how things run / interact with the system.
<= /div>

Fair p= oint, no sw is bugless. Although, from my shallow experience WINE seems to = have an excellent test suite. I remember reporting a regression once, which= turned out to be due to some obscure surfaces manipulation by an old Heroe= s =E2=85=A2 game. WINE devs not only quickly fixed that, but they also adde= d tests for that case, so I'd presume such regression is just not gonna= happen anymore.

Plus some of our software would li= ke to test certain system level infrastructure (like say KDE Connect).

Out of curiosity, what does t= his infrastructure include? I thought KDE connect only uses network sockets= and system tray.


No = idea, I saw their commentary on debugging issues they were having in their = unit tests in #kde-devel.
Those issues were due to a lack of perm= issions, specifically around the interactive console - that is how I know s= ome of our tests need those additional permissions and why running as a sch= eduled task / system service will not be sufficient for "fully working= " CI tests on Windows.

Sorry, it seems there's some misunderstanding=E2=80=A6 Judging by wh= at you say you seem to be referring to the restrictions that Windows contai= ners have on Windows. But that was the point that started this thread, to w= hich I replied that running Linux containers with WINE might be a solution = =F0=9F=98=8A

What i'm= referring to here is that KDE Connect interacts with various components of= the system in order to do what it does.
See for instance the abi= lity to share Notifications between devices, or ability to act as a present= er device.

That requires accessing various system = level APIs which WINE very well may not support - and we wouldn't suppo= rt a scenario of using it under WINE because there is a native Linux versio= n already.
=C2=A0

Plus, we have to have native Windows to compile things anyway as we n= eed to use MSVC (otherwise you have no Qt Web Engine support, as that canno= t be built with MingW)

Bu= t I presume it can be built with Clang? In particular, Google Chrome on Win= dows is being built with Clang =E2=80=94 and Web Engine is basically a fork= of Chromium.


Qt 6 as= a whole does not list Clang as a supported compiler - see=C2=A0https://= doc.qt.io/qt-6/supported-platforms.html

Given = Windows is a bit strange in the first place, i'd be quite reluctant to = step outside of what they list as supported.
=

Oh, that's interesting. So Qt claims to support Min= GW, but then WebEngine does not actually build with it? So it's a bug o= n their side, was it reported?

While at it, just t= o make sure I understand: do you compile with MSVC everything or just the W= ebEngine (which Idk if it's possible without the deps, but I guess aski= ng doesn't hurt).

As you will see on the supported platforms page, certain modules don'= t support certain platform/compiler combinations - and MingW / WebEngine is= one of those unsupported combinations.

MSVC= and MingW have incompatible C++ compiler ABIs. That means it is an all or = nothing approach, so yes everything is built using MSVC and cannot be built= with another compiler.

Cheers,
Ben=C2= =A0
--000000000000138d88061225da36--