[prev in list] [next in list] [prev in thread] [next in thread]
List: kde-devel
Subject: Re: How to get dependencies into freebsd ci?
From: Ben Cooksley <bcooksley () kde ! org>
Date: 2023-09-16 20:37:32
Message-ID: CA+XidOH4ja7ajWH4f-7JOcHkoTGqH-d95TRb7dY9wTNaoHfNgw () mail ! gmail ! com
[Download RAW message or body]
On Sun, Sep 17, 2023 at 8:08 AM Milian Wolff <mail@milianw.de> wrote:
> On Samstag, 16. September 2023 22:00:37 CEST Ben Cooksley wrote:
> > On Sun, Sep 17, 2023 at 3:16 AM Milian Wolff <mail@milianw.de> wrote:
> > > Hey all,
> >
> > Hi Milian,
> >
> > > While looking at the kate CI setup, I saw that it gets build on
> freebsd. I
> > > would like to get that coverage too for heaptrack, but when I try to
> add
> > >
> > >
> https://invent.kde.org/sysadmin/ci-utilities/raw/master/gitlab-templates/
> > > freebsd.yml
> > >
> > > Then heaptrack fails to find the elfutils dependency. This works on
> linux,
> > > but
> > > I have zero clue how that is done.
> > >
> > > Can I add dependencies covered by `craft` to `.kde-ci.yml`?
> >
> > Dependencies are managed in different ways depending on the platform in
> > question.
> >
> > For Linux, Android and Windows builds, this starts with a Docker image
> that
> > can be found at https://invent.kde.org/sysadmin/ci-images
> > You can have dependencies added to those by filing a merge request
> against
> > that repository.
> >
> > As you'll see, in the case of Linux we source just about everything from
> > the distro package manager (in our case, it's SUSE Tumbleweed).
> > For Android and Windows, the majority of the dependencies come from
> Craft.
> >
> > FreeBSD is the exception to all of this, as those builders are fixed
> > permanent machines rather than ephemeral containers that are only around
> > for a single build.
> > For these, please file a Sysadmin ticket.
> >
> > > Where can I find documentation on what to put in there? Furthermore, is
> > > there
> > > some best practices when it comes to CI configuration for KDE
> projects? By
> > > chance I found https://community.kde.org/Infrastructure/GitLab/CI/
> > > Static_Code_Analysis which is interesting - is there more like it
> > > somewhere?
> >
> > I'm afraid we've not done a terribly good job at documenting things,
> > however if you are using the templates available at
> >
> https://invent.kde.org/sysadmin/ci-utilities/-/tree/master/gitlab-templates
> > then i'd refer you to
> >
> https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/config-template.y
> > ml?ref_type=heads for the options that the system supports. Note that
> these
> > aren't used by the craft-* or flatpak templates as those are CD jobs
> rather
> > than CI jobs.
>
> Thanks. Where can I find out what these options actually do? `force-inject-
> asan` sounds interesting.
>
You'll need to read the code of the actual CI tooling in that repository.
> Generally, is it possible to compile and run tests with asan enabled -
> ideally
> configured through a CMakePreset?
>
ASAN is enabled by default for Linux builds - see
https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/run-ci-build.py?ref_type=heads#L202
Due to the contagion aspect of ASAN (where binaries have to be built with
it if any library that it uses has been built with ASAN), it is not
recommended to try to enable or disable it within individual projects.
(This is why force-inject-asan exists - to allow for distribution provided
binaries that load KDE project plugins/libraries to be used)
>
> > The Static Code Analysis job referred to there is based on legacy Jenkins
> > infrastructure which has been shutdown and it therefore will no longer
> > function, however the new system provides analysis jobs that work with
> > cppcheck to provide similar functionality if enabled.
>
> So the wiki page is obsolete and should be removed?
>
Correct.
>
> And cppcheck is nice, but it's a long shot from clang-tidy and clazy. I
> understand that's the way it is for now and it's not possible to get
> additional checks enabled for projects?
>
Nobody has expressed any interest in working on those within the CI system,
and I don't have the bandwidth i'm afraid.
Note that if clang-tidy and clazy need to recompile everything we would
need to carefully consider how that is implemented within the CI system,
both from a resource utilisation as well as compiler coverage perspective.
Currently gcc compiler coverage is the responsibility of the Linux CI jobs
(while Clang compiler coverage is the responsibility of FreeBSD, and MSVC
of Windows).
If it is just needing to be compiled alongside then we can probably make
that work without too much pain, although it would be FreeBSD only option.
>
> --
> Milian Wolff
> mail@milianw.de
> http://milianw.de
Cheers,
Ben
[Attachment #3 (text/html)]
<div dir="ltr"><div dir="ltr">On Sun, Sep 17, 2023 at 8:08 AM Milian Wolff <<a \
href="mailto:mail@milianw.de">mail@milianw.de</a>> wrote:<br></div><div \
class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Samstag, 16. \
September 2023 22:00:37 CEST Ben Cooksley wrote:<br> > On Sun, Sep 17, 2023 at \
3:16 AM Milian Wolff <<a href="mailto:mail@milianw.de" \
target="_blank">mail@milianw.de</a>> wrote:<br> > > Hey all,<br>
> <br>
> Hi Milian,<br>
> <br>
> > While looking at the kate CI setup, I saw that it gets build on freebsd. \
I<br> > > would like to get that coverage too for heaptrack, but when I try to \
add<br> > > <br>
> > <a href="https://invent.kde.org/sysadmin/ci-utilities/raw/master/gitlab-templates/" \
rel="noreferrer" target="_blank">https://invent.kde.org/sysadmin/ci-utilities/raw/master/gitlab-templates/</a><br>
> > freebsd.yml<br>
> > <br>
> > Then heaptrack fails to find the elfutils dependency. This works on \
linux,<br> > > but<br>
> > I have zero clue how that is done.<br>
> > <br>
> > Can I add dependencies covered by `craft` to `.kde-ci.yml`?<br>
> <br>
> Dependencies are managed in different ways depending on the platform in<br>
> question.<br>
> <br>
> For Linux, Android and Windows builds, this starts with a Docker image that<br>
> can be found at <a href="https://invent.kde.org/sysadmin/ci-images" \
rel="noreferrer" target="_blank">https://invent.kde.org/sysadmin/ci-images</a><br> \
> You can have dependencies added to those by filing a merge request against<br> \
> that repository.<br> > <br>
> As you'll see, in the case of Linux we source just about everything from<br>
> the distro package manager (in our case, it's SUSE Tumbleweed).<br>
> For Android and Windows, the majority of the dependencies come from Craft.<br>
> <br>
> FreeBSD is the exception to all of this, as those builders are fixed<br>
> permanent machines rather than ephemeral containers that are only around<br>
> for a single build.<br>
> For these, please file a Sysadmin ticket.<br>
> <br>
> > Where can I find documentation on what to put in there? Furthermore, is<br>
> > there<br>
> > some best practices when it comes to CI configuration for KDE projects? \
By<br> > > chance I found <a \
href="https://community.kde.org/Infrastructure/GitLab/CI/" rel="noreferrer" \
target="_blank">https://community.kde.org/Infrastructure/GitLab/CI/</a><br> > > \
Static_Code_Analysis which is interesting - is there more like it<br> > > \
somewhere?<br> > <br>
> I'm afraid we've not done a terribly good job at documenting things,<br>
> however if you are using the templates available at<br>
> <a href="https://invent.kde.org/sysadmin/ci-utilities/-/tree/master/gitlab-templates" \
rel="noreferrer" target="_blank">https://invent.kde.org/sysadmin/ci-utilities/-/tree/master/gitlab-templates</a><br>
> then i'd refer you to<br>
> <a href="https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/config-template.y" \
rel="noreferrer" target="_blank">https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/config-template.y</a><br>
> ml?ref_type=heads for the options that the system supports. Note that these<br>
> aren't used by the craft-* or flatpak templates as those are CD jobs \
rather<br> > than CI jobs.<br>
<br>
Thanks. Where can I find out what these options actually do? `force-inject-<br>
asan` sounds interesting.<br></blockquote><div><br></div><div>You'll need to read \
the code of the actual CI tooling in that repository.</div><div><br></div><blockquote \
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"> <br>
Generally, is it possible to compile and run tests with asan enabled - ideally <br>
configured through a CMakePreset?<br></blockquote><div><br></div><div>ASAN is enabled \
by default for Linux builds - see <a \
href="https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/run-ci-build.py?ref_t \
ype=heads#L202">https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/run-ci-build.py?ref_type=heads#L202</a></div><div><br></div><div>Due \
to the contagion aspect of ASAN (where binaries have to be built with it if any \
library that it uses has been built with ASAN), it is not recommended to try to \
enable or disable it within individual projects.</div><div>(This is why \
force-inject-asan exists - to allow for distribution provided binaries that load KDE \
project plugins/libraries to be used)</div><div> </div><blockquote \
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"> <br>
> The Static Code Analysis job referred to there is based on legacy Jenkins<br>
> infrastructure which has been shutdown and it therefore will no longer<br>
> function, however the new system provides analysis jobs that work with<br>
> cppcheck to provide similar functionality if enabled.<br>
<br>
So the wiki page is obsolete and should be \
removed?<br></blockquote><div><br></div><div>Correct.</div><div> </div><blockquote \
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"> <br>
And cppcheck is nice, but it's a long shot from clang-tidy and clazy. I <br>
understand that's the way it is for now and it's not possible to get <br>
additional checks enabled for projects?<br></blockquote><div><br></div><div>Nobody \
has expressed any interest in working on those within the CI system, and I don't \
have the bandwidth i'm afraid.</div><div><br></div><div>Note that if clang-tidy \
and clazy need to recompile everything we would need to carefully consider how that \
is implemented within the CI system, both from a resource utilisation as well as \
compiler coverage perspective.</div><div>Currently gcc compiler coverage is the \
responsibility of the Linux CI jobs (while Clang compiler coverage is the \
responsibility of FreeBSD, and MSVC of Windows).</div><div><br></div><div>If it is \
just needing to be compiled alongside then we can probably make that work without too \
much pain, although it would be FreeBSD only option.</div><div> </div><blockquote \
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"> <br>
-- <br>
Milian Wolff<br>
<a href="mailto:mail@milianw.de" target="_blank">mail@milianw.de</a><br>
<a href="http://milianw.de" rel="noreferrer" \
target="_blank">http://milianw.de</a></blockquote><div><br></div><div>Cheers,</div><div>Ben</div></div></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic