From kde-frameworks-devel Tue Jul 06 13:09:46 2021 From: Helio Chissini de Castro Date: Tue, 06 Jul 2021 13:09:46 +0000 To: kde-frameworks-devel Subject: Re: Package License Metadata for Tooling Message-Id: X-MARC-Message: https://marc.info/?l=kde-frameworks-devel&m=162557701203642 MIME-Version: 1 Content-Type: multipart/mixed; boundary="--00000000000079573805c67423e7" --00000000000079573805c67423e7 Content-Type: text/plain; charset="UTF-8" Hello all I give some time to make a good overview over our current state. Let's tackle the issues first. The SPDX definition file over our metadata is possible, but not in the way we expect. There's no formal way to declare "subpackages" per se, like a source that produces several targets. The normalization of SPDX analysis is the main source (tarball/git) that produces a build, and specific version as well. So in raw terms, we can assign as example that the repo/project karchive depends on the repo/project kconfig, but the projects are treated as their own. References itself is the second issue, because we need to assign proper dependencies, not simply a range of "from version xx minimal". In this case, the yocto reference is more aligned. This complicates if we intend to have the formal spdx declaration direct on the repository. So, based on last sentence, one thing i'm working is on the fly generate the official spdx metadata from the Yocto bb packages, as a bitbake class, and the generated files can be a template for distributions as well. The key goal is deployment, so offering a way to generate the current correct output from be Yoco, deb, rpm would be the logical; way, because the final release package has always a 1 <-> 1 mapping fro dependencies. For the repository itself, what is possible is get the metadata generated on current tooling, and update the files based on current state of the repo metadata. This is feasible and automatized, but will do need us have some personal policy that everytime we properly require a new or different version deps to update the metadata correctly. Said that, I think we should see the results on how the yocto output will go for an initial case. Another suggestion that can improve our solution, is to adopt conan files for real. Not necessarily as mandatory usage, but as reference. Scanning tools can create the files using Conan inclusing the proper dependency check. This will theoretically reduce the work on the metadata, and focus in a single place, conan, where we can rely on results for all states. Anyway, this is my initial findings and i hope on the weekend I can have more concrete results. []'s On Tue, Jun 22, 2021 at 9:26 PM Andreas Cord-Landwehr wrote: > Hi, here is a short wrap-up of our today's BoF > (activate participants*: Helio, Volker, Johan, me) > > The whole topic about license checking can be grouped into three areas: > > 1. Correct statements of copyright & license information in source files > - we think that we are on track here with our REUSE/SPDX statements > - it is agreed that it is not a big problem that we do not have full REUSE > compliance in most of the older repositories (specifically, missing > license > information in build system config files) > > 2. Statement of outbound licenses == what is the license of a compiled lib > or > application? > - we discussed the option to extend our yaml files with license > information > about the specific artifacts > - if the artifacts are too complicated, a fallback could be to look folder- > wise (though that would have several draw-backs) > - outbound license information files in the yaml files could be helpful > input > for packagers, definitely they will help for Yocto packaging and for > Helio's > tooling > > 3. Full distribution check of dependencies > - question here: if our lib links to another lib, is this legally right? > - we got the insight that this very much depends on the precise versions > at > compile time (because licenses of libs may change; e.g., see openssl) > - for now we excluded the correctness checks of interdependencies of such > licenses of different libs > - what is thinkable right now: CMake could generate a list of dependencies > with the precise versions as they are used for configuration/compilation > > Next Steps: > 1. come up with a proposal how yaml files could be extended to state > artifact > licenses > 2. Helio will check how helpful the existing yaml metadata are already for > his > tooling > > Stretch: > 3. check if existing ECM outbound license check tooling can be adapted to > use > the metadata from the yaml files > 4. integrate updated metadata files into Yocto tooling to get rid of > manual > maintenance :) > > Cheers, > Andreas > > * == unmuted themselves at least once :) > > On Dienstag, 22. Juni 2021 08:29:11 CEST Andreas Cord-Landwehr wrote: > > Hi, yesterday morning Helio and me had an interesting discussion about > > metadata for license check tooling. There are a few different use cases: > > > > - Helio needs those for his FOSS license tooling > > - I would really like to improve the generation of Yocto license > metadata, > > which we currently have to keep in sync manually > > - maybe this is something that distros would also like to use in their QA > > processes... > > > > Bottom line, we ended in scheduling a BoF for this evening (Tuesday), 6 > PM > > (UTC) i.e. 8 PM in Berlin time. > > > > Key question is: Do we see a convenient way to generate package metadata > for > > the binary artifacts and/or dependencies to other libraries without > > creating a horrible maintenance burden? > > > > Cheers, > > Andreas > > > > > --00000000000079573805c67423e7 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello all

I give some time to make a good overview over our current state.

Let's tackle the issues first. The SPDX def= inition file over our metadata is possible, but not in the way we expect.
There's no formal way to declare "subpackages"= per se, like a source that produces several targets.
The n= ormalization of SPDX analysis=C2=A0is the main source (tarball/git) that=C2= =A0produces a build, and specific version as well.
So i= n raw terms, we can assign as example that the repo/project karchive depend= s on the repo/project kconfig, but the projects are treated=C2=A0as their o= wn.

References itself is the secon= d issue, because we need to assign proper dependencies, not simply a range = of "from version xx minimal". In this case, the yocto reference i= s more aligned.
This complicates if we intend to have the f= ormal spdx declaration direct on the repository.

=
So, based on last sentence, one thing i'm working is on the = fly generate the official spdx metadata=C2=A0from the Yocto bb packages, as= a bitbake class, and the generated files can be a template for distributio= ns as well.
The key goal is deployment, so offering a way t= o generate the current correct output from be Yoco, deb, rpm would be the l= ogical; way, because the final release package has always a 1 <-> 1 m= apping fro dependencies.

For the repos= itory itself, what is possible is get the metadata generated on current too= ling, and update the files based on current state of the repo metadata. Thi= s is feasible and automatized, but will do need us have some personal polic= y that everytime we properly require a new or different version deps to upd= ate the metadata correctly.

Said that,= I think we should see the results on how the yocto output will go for an i= nitial case.

Another suggestion that= can improve our solution, is to adopt conan files for real.
Not necessarily as mandatory usage, but as reference. Scanning tools can = create the files using Conan inclusing the proper dependency check. This wi= ll theoretically reduce the work on the metadata, and focus in a single pla= ce, conan, where we can rely on results for all states.
Anyway, this is my initial=C2=A0findings and i hope on th= e weekend I can have more concrete results.

[]'s=C2=A0





On Tue, Jun 22, 2021 at 9:26 PM Andreas = Cord-Landwehr <cordlandwehr@kde.= org> wrote:
Hi, here is a short wrap-up of our today's BoF
(activate participants*: Helio, Volker, Johan, me)

The whole topic about license checking can be grouped into three areas:

1. Correct statements of copyright & license information in source file= s
- we think that we are on track here with our REUSE/SPDX statements
- it is agreed that it is not a big problem that we do not have full REUSE =
compliance in most of the older repositories (specifically, missing license=
information in build system config files)

2. Statement of outbound licenses =3D=3D what is the license of a compiled = lib or
application?
- we discussed the option to extend our yaml files with license information=
about the specific artifacts
- if the artifacts are too complicated, a fallback could be to look folder-=
wise (though that would have several draw-backs)
- outbound license information files in the yaml files could be helpful inp= ut
for packagers, definitely they will help for Yocto packaging and for Helio&= #39;s
tooling

3. Full distribution check of dependencies
- question here: if our lib links to another lib, is this legally right? - we got the insight that this very much depends on the precise versions at=
compile time (because licenses of libs may change; e.g., see openssl)
- for now we excluded the correctness checks of interdependencies of such <= br> licenses of different libs
- what is thinkable right now: CMake could generate a list of dependencies =
with the precise versions as they are used for configuration/compilation
Next Steps:
1. come up with a proposal how yaml files could be extended to state artifa= ct
licenses
2. Helio will check how helpful the existing yaml metadata are already for = his
tooling

Stretch:
3. check if existing ECM outbound license check tooling can be adapted to u= se
the metadata from the yaml files
4. integrate updated metadata files into Yocto tooling to get rid of manual=
maintenance :)

Cheers,
Andreas

* =3D=3D unmuted themselves at least once :)

On Dienstag, 22. Juni 2021 08:29:11 CEST Andreas Cord-Landwehr wrote:
> Hi, yesterday morning Helio and me had an interesting discussion about=
> metadata for license check tooling. There are a few different use case= s:
>
> - Helio needs those for his FOSS license tooling
> - I would really like to improve the generation of Yocto license metad= ata,
> which we currently have to keep in sync manually
> - maybe this is something that distros would also like to use in their= QA
> processes...
>
> Bottom line, we ended in scheduling a BoF for this evening (Tuesday), = 6 PM
> (UTC) i.e. 8 PM in Berlin time.
>
> Key question is: Do we see a convenient way to generate package metada= ta for
> the binary artifacts and/or dependencies to other libraries without > creating a horrible maintenance burden?
>
> Cheers,
> Andreas




--00000000000079573805c67423e7--