[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-devel
Subject:    Re: flatpak CI and stable builds - Re: KDE Gear projects with failing CI (release/23.08) (12 Septemb
From:       Albert Astals Cid <aacid () kde ! org>
Date:       2023-09-24 20:43:37
Message-ID: 7639365.3KObdgcpn5 () xps15
[Download RAW message or body]

El dijous, 21 de setembre de 2023, a les 9:54:59 (CEST), Ben Cooksley va 
escriure:
> On Thu, Sep 21, 2023 at 8:35 AM Albert Astals Cid <aacid@kde.org> wrote:
> > El dimecres, 20 de setembre de 2023, a les 13:17:22 (CEST), Ben Cooksley
> > va
> > 
> > escriure:
> > > On Wed, Sep 20, 2023 at 9:42 AM Albert Astals Cid <aacid@kde.org> wrote:
> > > > El dimarts, 19 de setembre de 2023, a les 22:18:40 (CEST),
> > > > 
> > > > christoph@cullmann.io va escriure:
> > > > > On 2023-09-19 21:35, Albert Astals Cid wrote:
> > > > > > Please work on fixing them, otherwise i will remove the failing CI
> > > > > > jobs on their 4th failing week, it is very important that CI is
> > > > > > passing
> > > > > > for
> > > > > > multiple reasons.
> > > > > > 
> > > > > > Bad news: We have 2 new repositories failing :/
> > > > > > 
> > > > > > = FLATPAK FAILING =
> > > > > > 
> > > > > > kate:
> > > > > >  * https://invent.kde.org/utilities/kate/-/pipelines/484147
> > > > > >  
> > > > > >   * This highlights a design problem, it's building markdown part
> > 
> > from
> > 
> > > > > > master
> > > > > > instead of from stable branch. We can manually change the branch,
> > 
> > but
> > 
> > > > > > i
> > > > > > would
> > > > > > prefer a solution that doesn't mean changing lots and lots of
> > 
> > flatpak
> > 
> > > > > > manifests when we branch.
> > > > > 
> > > > > Hmm, yes that sounds not nice.
> > > > > 
> > > > > But not sure how that would work without that, seems
> > 
> > https://invent.kde.org/utilities/kate/-/blob/master/.flatpak-manifest.json
> > 
> > > > ?r>
> > > > 
> > > > > ef_type=heads
> > > > > 
> > > > > hard codes what to fetch.
> > > > > 
> > > > > Given one hard codes there the
> > > > > 
> > > > >   "runtime-version": "5.15-22.08",
> > > > 
> > > > That one is "fine", the 22.08 here it's related to the "flatpak kde/
> > > > freedesktop sdk" not to Gear stuff.
> > > > 
> > > > Yes, we will massively have to update them on master when we decide to
> > > > depend
> > > > on a new one, but it won't cause problems on the stable branches like
> > 
> > the
> > 
> > > > oner
> > > > we're experiencing here.
> > > > 
> > > > The problem here is
> > > > 
> > > > {
> > > > 
> > > >   "name": "markdownpart",
> > > >   "buildsystem": "cmake-ninja",
> > > >   "sources": [
> > > >   
> > > >     {
> > > >     
> > > >       "type": "git",
> > > >       "url": "https://invent.kde.org/utilities/markdownpart.git"
> > > >     
> > > >     }
> > > >   
> > > >   ]
> > > > 
> > > > }
> > > > 
> > > > This unconditionally compiles the master branch of markdownpart repo
> > > > 
> > > > As far as i can see there's three solutions:
> > > > 
> > > > A) If this is just "to make sure it builds CI", we don't need
> > 
> > markdownpart
> > 
> > > > nor
> > > > konsole on the flatpak since they are just runtime dependencies. This
> > 
> > is a
> > 
> > > > sub-optimal solution i'd say since it makes it so that we can't offer
> > 
> > the
> > 
> > > > package for testing in the future and makes the diff with the flathub
> > > > manifest
> > > > bigger than it needs to be
> > > 
> > > The overall intention is for the bundles produced by the Flatpak jobs to
> > 
> > be
> > 
> > > useful for people to locally test a project build.
> > > In the not too distant future we'll have them available from a Flatpak
> > > repository for actual mainline/release branches as well.
> > > 
> > > So the answer certainly isn't (a).
> > > 
> > > > B) Depend on released versions, i.e. a tarball in "sources" instead of
> > 
> > a
> > 
> > > > git
> > > > repo. This is probably suboptimal too in the sense that will require
> > > > constant
> > > > updating on master and if we offer the resulting flatpak as "nightly"
> > 
> > in
> > 
> > > > the
> > > > future for testing it's not "nightly" as it could be.
> > > 
> > > Guess it depends on the nature of the dependency, but in the case of
> > > software released together you probably want to build against what you
> > 
> > will
> > 
> > > be shipping against yes.
> > > 
> > > > C) Add a marker in the .json like branch: "kde-same-branch" and then
> > 
> > have
> > 
> > > > the
> > > > code in
> > 
> > https://invent.kde.org/sysadmin/ci-utilities/raw/master/gitlab-templates/f
> > 
> > > > latpak.yml replace that "kde-same-branch" either by "master" or by
> > > > the appropriate stable branch before actually compiling the flatpak. I
> > > > think
> > > > this would be the optimal solution but needs work.
> > > 
> > > This is my preferred solution as well. it wouldn't be too difficult
> > > given
> > > we have a Python script acting as a middle-man already.
> > > In the past we did some rewriting of the .flatpak-manifest.yml already.
> > > 
> > > Depending on our requirements it may be worth tying into the same
> > > branch-rules.yml logic that the rest of the CI system uses but this is
> > > probably best answered by someone who knows the various Flatpak
> > > manifests
> > > we have better.
> > > 
> > > In #flatpak:kde.org it was mentioned that this would mean that people
> > > wouldn't be able to build it as easily themselves, but if we make it
> > > well
> > > documented (comments in the .flatpak-manifest.yml, etc) then I think
> > > this
> > > shouldn't be much of an issue.
> > 
> > I had not thought of that and it's indeed not great.
> > 
> > We could rename all the flatpak manifest that need this feature to .
> > json.in to
> > make it clear "it's not their final form" and that they need to be pre-
> > processed.
> > 
> > 
> > This does not help a lot to the "non CI user" if they want to actually use
> > the
> > manifest since they still need to pre-process it somehow :/
> 
> At least it would make it explicit that something is required before the
> files can be utilised elsewhere.
> 
> Ultimately in the long term we are potentially looking at having release
> builds we make be submitted into Flathub so this does need a solution in
> some form or another.

I don't see how that would work, for flathub we use released tarballs as 
sources for everything, not git branches, i guess we could switch to git tags? 

> 
> Realistically I think we only have two choices here:
> a) Have our release tooling include logic that bumps/updates the
> .flatpak-manifest.yml files to have the right branch in them; or
> b) Have a intermediary script like flatpak-build.py do that for us
> 
> For now the path of least resistance is probably (b) I think?

Probably

Cheers,
  Albert

> 
> > Solution E)
> > 
> > Since usually/mostly we have our projects be backwards API compatible it's
> > usually/mostly not a problem to that kate stable uses markdown master
> > (understanding the flatpaks generated by that CI are nightlies), we're
> > only
> > having this problem right now because of the KF6 transition.
> > 
> > One potentially valid solution is to just disable flatpak CI in stable
> > until
> > this gets fixed, it's not great but it's just a few months.
> > 
> > Cheers,
> > 
> >   Albert
> 
> Cheers,
> Ben
> 
> > > > D) Something smarter I have not thought about.
> > > > 
> > > > Cheers,
> > > > 
> > > >   Albert
> > > 
> > > Cheers,
> > > Ben
> > > 
> > > > > I assume one will need to hard code that, too, if one creates no own
> > > > > scripting.
> > > > > 
> > > > > But I might be wrong.
> > > > > 
> > > > > Greetings
> > > > > Christoph
> > > > > 
> > > > > > = FAILING UNIT TESTS =
> > > > > > 
> > > > > > konsole:
> > > > > >  * https://invent.kde.org/utilities/konsole/-/pipelines/484148
> > > > > >  
> > > > > >   * freebsd_qt515 tests are failing




[prev in list] [next in list] [prev in thread] [next in thread]