From kde-devel Sat Sep 16 20:37:32 2023 From: Ben Cooksley Date: Sat, 16 Sep 2023 20:37:32 +0000 To: kde-devel Subject: Re: How to get dependencies into freebsd ci? Message-Id: X-MARC-Message: https://marc.info/?l=kde-devel&m=169489651723510 MIME-Version: 1 Content-Type: multipart/mixed; boundary="--000000000000a652f506057fe104" --000000000000a652f506057fe104 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sun, Sep 17, 2023 at 8:08=E2=80=AFAM Milian Wolff wrot= e: > On Samstag, 16. September 2023 22:00:37 CEST Ben Cooksley wrote: > > On Sun, Sep 17, 2023 at 3:16=E2=80=AFAM Milian Wolff = wrote: > > > Hey all, > > > > Hi Milian, > > > > > While looking at the kate CI setup, I saw that it gets build on > freebsd. I > > > would like to get that coverage too for heaptrack, but when I try to > add > > > > > > > https://invent.kde.org/sysadmin/ci-utilities/raw/master/gitlab-templates/ > > > freebsd.yml > > > > > > Then heaptrack fails to find the elfutils dependency. This works on > linux, > > > but > > > I have zero clue how that is done. > > > > > > Can I add dependencies covered by `craft` to `.kde-ci.yml`? > > > > Dependencies are managed in different ways depending on the platform in > > question. > > > > For Linux, Android and Windows builds, this starts with a Docker image > that > > can be found at https://invent.kde.org/sysadmin/ci-images > > You can have dependencies added to those by filing a merge request > against > > that repository. > > > > As you'll see, in the case of Linux we source just about everything fro= m > > the distro package manager (in our case, it's SUSE Tumbleweed). > > For Android and Windows, the majority of the dependencies come from > Craft. > > > > FreeBSD is the exception to all of this, as those builders are fixed > > permanent machines rather than ephemeral containers that are only aroun= d > > for a single build. > > For these, please file a Sysadmin ticket. > > > > > Where can I find documentation on what to put in there? Furthermore, = is > > > there > > > some best practices when it comes to CI configuration for KDE > projects? By > > > chance I found https://community.kde.org/Infrastructure/GitLab/CI/ > > > Static_Code_Analysis which is interesting - is there more like it > > > somewhere? > > > > I'm afraid we've not done a terribly good job at documenting things, > > however if you are using the templates available at > > > https://invent.kde.org/sysadmin/ci-utilities/-/tree/master/gitlab-templat= es > > then i'd refer you to > > > https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/config-templat= e.y > > ml?ref_type=3Dheads for the options that the system supports. Note that > these > > aren't used by the craft-* or flatpak templates as those are CD jobs > rather > > than CI jobs. > > Thanks. Where can I find out what these options actually do? `force-injec= t- > asan` sounds interesting. > You'll need to read the code of the actual CI tooling in that repository. > Generally, is it possible to compile and run tests with asan enabled - > ideally > configured through a CMakePreset? > ASAN is enabled by default for Linux builds - see https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/run-ci-build.py?= ref_type=3Dheads#L202 Due to the contagion aspect of ASAN (where binaries have to be built with it if any library that it uses has been built with ASAN), it is not recommended to try to enable or disable it within individual projects. (This is why force-inject-asan exists - to allow for distribution provided binaries that load KDE project plugins/libraries to be used) > > > The Static Code Analysis job referred to there is based on legacy Jenki= ns > > infrastructure which has been shutdown and it therefore will no longer > > function, however the new system provides analysis jobs that work with > > cppcheck to provide similar functionality if enabled. > > So the wiki page is obsolete and should be removed? > Correct. > > And cppcheck is nice, but it's a long shot from clang-tidy and clazy. I > understand that's the way it is for now and it's not possible to get > additional checks enabled for projects? > Nobody has expressed any interest in working on those within the CI system, and I don't have the bandwidth i'm afraid. Note that if clang-tidy and clazy need to recompile everything we would need to carefully consider how that is implemented within the CI system, both from a resource utilisation as well as compiler coverage perspective. Currently gcc compiler coverage is the responsibility of the Linux CI jobs (while Clang compiler coverage is the responsibility of FreeBSD, and MSVC of Windows). If it is just needing to be compiled alongside then we can probably make that work without too much pain, although it would be FreeBSD only option. > > -- > Milian Wolff > mail@milianw.de > http://milianw.de Cheers, Ben --000000000000a652f506057fe104 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Sun, Sep 17, 2023 at 8:08=E2=80=AFAM M= ilian Wolff <mail@milianw.de> = wrote:
On Samstag, 16. September 2023 22:00:37 CEST Ben Cooksley= wrote:
> On Sun, Sep 17, 2023 at 3:16=E2=80=AFAM Milian Wolff <mail@milianw.de> wrote:
> > Hey all,
>
> Hi Milian,
>
> > While looking at the kate CI setup, I saw that it gets build on f= reebsd. I
> > would like to get that coverage too for heaptrack, but when I try= to add
> >
> > https://invent.kd= e.org/sysadmin/ci-utilities/raw/master/gitlab-templates/
> > freebsd.yml
> >
> > Then heaptrack fails to find the elfutils dependency. This works = on linux,
> > but
> > I have zero clue how that is done.
> >
> > Can I add dependencies covered by `craft` to `.kde-ci.yml`?
>
> Dependencies are managed in different ways depending on the platform i= n
> question.
>
> For Linux, Android and Windows builds, this starts with a Docker image= that
> can be found at https://invent.kde.org/sysadmin/ci-ima= ges
> You can have dependencies added to those by filing a merge request aga= inst
> that repository.
>
> As you'll see, in the case of Linux we source just about everythin= g from
> the distro package manager (in our case, it's SUSE Tumbleweed). > For Android and Windows, the majority of the dependencies come from Cr= aft.
>
> FreeBSD is the exception to all of this, as those builders are fixed > permanent machines rather than ephemeral containers that are only arou= nd
> for a single build.
> For these, please file a Sysadmin ticket.
>
> > Where can I find documentation on what to put in there? Furthermo= re, is
> > there
> > some best practices when it comes to CI configuration for KDE pro= jects? By
> > chance I found https://community.kde.or= g/Infrastructure/GitLab/CI/
> > Static_Code_Analysis which is interesting - is there more like it=
> > somewhere?
>
> I'm afraid we've not done a terribly good job at documenting t= hings,
> however if you are using the templates available at
> https://invent.kde.o= rg/sysadmin/ci-utilities/-/tree/master/gitlab-templates
> then i'd refer you to
> https://invent.kde.= org/sysadmin/ci-utilities/-/blob/master/config-template.y
> ml?ref_type=3Dheads for the options that the system supports. Note tha= t these
> aren't used by the craft-* or flatpak templates as those are CD jo= bs rather
> than CI jobs.

Thanks. Where can I find out what these options actually do? `force-inject-=
asan` sounds interesting.

You'll ne= ed to read the code of the actual CI tooling in that repository.
=

Generally, is it possible to compile and run tests with asan enabled - idea= lly
configured through a CMakePreset?


Due to the cont= agion aspect of ASAN (where binaries have to be built with it if any librar= y that it uses has been built with ASAN), it is not recommended to try to e= nable or disable it within individual projects.
(This is why forc= e-inject-asan exists - to allow for distribution provided binaries that loa= d KDE project plugins/libraries to be used)
=C2=A0

> The Static Code Analysis job referred to there is based on legacy Jenk= ins
> infrastructure which has been shutdown and it therefore will no longer=
> function, however the new system provides analysis jobs that work with=
> cppcheck to provide similar functionality if enabled.

So the wiki page is obsolete and should be removed?
Correct.
=C2=A0

And cppcheck is nice, but it's a long shot from clang-tidy and clazy. I=
understand that's the way it is for now and it's not possible to ge= t
additional checks enabled for projects?

Nobody has expressed any interest in working on those within the CI system= , and I don't have the bandwidth i'm afraid.

Note that if clang-tidy and clazy need to recompile everything we would = need to carefully consider how that is implemented within the CI system, bo= th from a resource utilisation as well as compiler coverage perspective.
Currently gcc compiler coverage is the responsibility of the Linux = CI jobs (while Clang compiler coverage is the responsibility of FreeBSD, an= d MSVC of Windows).

If it is just needing to be co= mpiled alongside then we can probably make that work without too much pain,= although it would be FreeBSD only option.
=C2=A0

--
Milian Wolff
mail@milianw.de http://m= ilianw.de

Cheers,
Ben
--000000000000a652f506057fe104--