From kde-community Sat Feb 24 10:07:12 2024 From: Ben Cooksley Date: Sat, 24 Feb 2024 10:07:12 +0000 To: kde-community Subject: Re: Post-MegaRelease projects Message-Id: X-MARC-Message: https://marc.info/?l=kde-community&m=170876914828998 MIME-Version: 1 Content-Type: multipart/mixed; boundary="--000000000000e4f8d006121dd743" --000000000000e4f8d006121dd743 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Feb 24, 2024 at 10:27=E2=80=AFPM Konstantin Kharlamov wrote: > On Sat, 2024-02-24 at 22:16 +1300, Ben Cooksley wrote: > > On Sat, Feb 24, 2024 at 8:37=E2=80=AFPM Konstantin Kharlamov > wrote: > > On Sat, 2024-02-24 at 16:31 +1300, Ben Cooksley wrote: > > On Fri, Feb 23, 2024 at 11:12=E2=80=AFPM Sune Vuorela = wrote: > > On 2024-02-22, Nate Graham wrote: > > I've started pondering post-megarelease projects. We've spent so long o= n > > porting and bugfixing that I think it might be useful to shift gears to > > feature work, and I'd like to brainstorm potential large-scale projects > > and gauge the level of interest in putting resources into them soon. > > A bit more from the devops end that I'd love to see people tackle: > > - Ensure frameworks and app unit tests interacting with windows can run > on Windows. > More details: The following fails on our windows CI > https://invent.kde.org/sune/windows-test-thingie/-/blob/master/main.cp= p > I find it weird that we are spending resources on putting things in > the windows store and making apps available on windows, but we can't > actually have passing tests in our CI. > > > This unfortunately will not be easy to solve. > > One of the key things that we've learned out of doing CI, as has been > showcased by FreeBSD in particular, is that the builders need to be > ephemeral - that is only around for the build in question that is being r= un. > We're currently accomplishing this by using containers - via Podman (for > Linux/Android/FreeBSD) and Docker (for Windows). > > Containers also offer us the advantage of allowing people to easily > reproduce the CI environment on their local system without too much troub= le. > > For Windows however, Microsoft has limited the container stack to not > allow anything GUI related to work. The underlying libraries may be there= , > but the equivalent display server components are not operational. > > To complicate things further, on Windows certain permissions are > restricted to the interactive console and are not possible to do as eithe= r > a scheduled task or as a system service. > Usage of existing Windows automation frameworks such as Powershell > Remoting or SSH will therefore not work if we want things to perfectly > replicate a end user environment - because those will run the command(s) = as > part of a non-interactive session (even if the user we connect as is the > same one logged in on the desktop console). > > > Idk if it's a silly question, but=E2=80=A6 If Windows native containers h= ave so > many restrictions, why not just use Linux containers with WINE inside? > > > Because Wine is not Windows either, and there could be subtle differences > in how things run / interact with the system. > Plus some of our software would like to test certain system level > infrastructure (like say KDE Connect). > > > Out of curiosity, what does this infrastructure include? I thought KDE > connect only uses network sockets and system tray. > No idea, I saw their commentary on debugging issues they were having in their unit tests in #kde-devel. Those issues were due to a lack of permissions, specifically around the interactive console - that is how I know some of our tests need those additional permissions and why running as a scheduled task / system service will not be sufficient for "fully working" CI tests on Windows. > > Plus, we have to have native Windows to compile things anyway as we need > to use MSVC (otherwise you have no Qt Web Engine support, as that cannot = be > built with MingW) > > > But I presume it can be built with Clang? In particular, Google Chrome on > Windows is being built with Clang =E2=80=94 and Web Engine is basically a= fork of > Chromium. > Qt 6 as a whole does not list Clang as a supported compiler - see https://doc.qt.io/qt-6/supported-platforms.html Given Windows is a bit strange in the first place, i'd be quite reluctant to step outside of what they list as supported. > > (and you cannot really mix MingW / MSVC binaries due to incompatible > compiler ABIs for C++ code) > > > Well, if for testing purposes Qt was pre-built with Clang, I guess there > won't be any ABI issues. > Would require that everything else we have is rebuilt, and that all downstream users of our Craft cache also switch to Clang. Note that like most open source projects, we don't know the full extent to which Craft is used outside of our own project. Historically we have seen through Microsoft provided data that dependencies which we built and signed have shown up in surprising places (such as Snoretoast - which was used by something Node.js based at one point or another I believe) Cheers, Ben --000000000000e4f8d006121dd743 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Sat, Feb 24, 2024 at 10:27=E2=80=AFPM = Konstantin Kharlamov <Hi-Angel@yan= dex.ru> wrote:
On Sat, 2024-02-24 at 22:16 +1300, Ben Cooksley wrote:
On Sat, = Feb 24, 2024 at 8:37=E2=80=AFPM Konstantin Kharlamov <Hi-Angel@yandex.ru> wrote:
=
=
On Sat, 2024-02-24 at 16:31 +1300, Ben Cooksley wrote:
=
On Fri, Feb 23, 2024 at 11:12=E2=80=AFPM Sune Vuorela <nospam@vuorela.dk> wrote:
=
=
On 2024-02-22, Nate Graham <nate@kde.org> wrote:
> I've started pondering = post-megarelease projects. We've spent so long on
> porting and = bugfixing that I think it might be useful to shift gears to
> featur= e work, and I'd like to brainstorm potential large-scale projects
&= gt; and gauge the level of interest in putting resources into them soon.

A bit more from the devops end that I'd love to see peo= ple tackle:

=C2=A0- Ensure frameworks and app unit tests = interacting with windows can run
=C2=A0 =C2=A0on Windows.
=C2=A0 =C2= =A0More details: The following fails on our windows CI
=C2=A0 =C2=A0https://invent.kde.org/sune/windo= ws-test-thingie/-/blob/master/main.cpp
=C2=A0 =C2=A0I find it weird = that we are spending resources on putting things in
=C2=A0 =C2=A0the win= dows store and making apps available on windows, but we can't
=C2=A0= =C2=A0actually have passing tests in our CI.


This unfortunately will not be easy to solve.= =C2=A0

One of the key things that we've learne= d out of doing CI, as has been showcased=C2=A0by FreeBSD in particular, is = that the builders need to be ephemeral - that is only around for the build = in question that is being run.
We're currently accomplishing = this by using containers - via Podman (for Linux/Android/FreeBSD) and Docke= r (for Windows).

Containers also offer us the = advantage of allowing people to easily reproduce the CI environment on thei= r local system without too much trouble.

For Windo= ws however, Microsoft has limited the container stack to not allow anything= GUI related to work. The underlying libraries may be there, but the equiva= lent display server components are not operational.

To complicate things further, on Windows certain permissions are res= tricted to the interactive console and are not possible to do as either a s= cheduled task or as a system service.
Usage of existing Wind= ows automation frameworks such as Powershell Remoting or SSH will therefore= not work if we want things to perfectly replicate a end user environment -= because those will run the command(s) as part of a non-interactive session= (even if the user we connect as is the same one logged in on the desktop c= onsole).

Idk = if it's a silly question, but=E2=80=A6 If Windows native containers hav= e so many restrictions, why not just use Linux containers with WINE inside?=


Because Wine is not = Windows either, and there could be subtle differences in how things run / i= nteract with the system.
Plus some of our software would like to = test certain system level infrastructure (like say KDE Connect).

Out of curiosity, what does this in= frastructure include? I thought KDE connect only uses network sockets and s= ystem tray.

No idea, I sa= w their commentary on debugging issues they were having in their unit tests= in #kde-devel.
Those issues were due to a lack of permissions, s= pecifically around the interactive console - that is how I know some of our= tests need those additional permissions and why running as a scheduled tas= k / system service will not be sufficient for "fully working" CI = tests on Windows.
=C2=A0

Plus, we have to have native Windows to compile things anyway= as we need to use MSVC (otherwise you have no Qt Web Engine support, as th= at cannot be built with MingW)

But I presume it can be built with Clang? In particular, Google Chrom= e on Windows is being built with Clang =E2=80=94 and Web Engine is basicall= y a fork of Chromium.

Qt = 6 as a whole does not list Clang as a supported compiler - see=C2=A0https://doc.qt.io/qt-6= /supported-platforms.html

Given Windows is a b= it strange in the first place, i'd be quite reluctant to step outside o= f what they list as supported.
=C2=A0

(and you cannot really mix MingW / MSVC binaries= due to incompatible compiler ABIs for C++ code)

Well, if for testing purposes Qt was pre-built with= Clang, I guess there won't be any ABI issues.

Would require that everything else we have is reb= uilt, and that all downstream users of our Craft cache also switch to Clang= .
Note that like most open source projects, we don't know the= full extent to which Craft is used outside of our own project.
<= br>
Historically we have seen through Microsoft provided data tha= t dependencies which we built and signed have shown up in surprising places= (such as Snoretoast - which was used by something Node.js based at one poi= nt or another I believe)

Cheers,
Ben
=C2=A0
--000000000000e4f8d006121dd743--