[prev in list] [next in list] [prev in thread] [next in thread]
List: gentoo-soc
Subject: [gentoo-soc] Week 7 Report for Big Data Infrastructure and H2O ebuilds Project
From: "Yuan Liao (Leo)" <liaoyuan () gmail ! com>
Date: 2021-07-26 7:17:08
Message-ID: CACk4dks1cFHEt-MidXBMxB3=_sYa8M0Mj685-DVZeUixdfrL+A () mail ! gmail ! com
[Download RAW message or body]
Hi folks,
Another week has just passed. I have been gradually moving on to the next
part of my project where I would fix the ebuilds in the Spark overlay that
are no longer installable, most of which are caused by changes to ebuilds
in the Gentoo repository since last year's GSoC concluded. At the same
time, a few additional utilities supporting Kotlin on Gentoo have been
created, including an eselect module for Kotlin [1] and some new Kotlin
eclasses.
Many ebuilds in the Spark overlay depend on packages in ::gentoo, but some
dependencies have been broken due to removal of old slots or even deletion
of packages that were really too old. In the former case, the dependency
could be quickly fixed by updating the slot, although the dependee package
would require some integration tests with the dependency's new slot. In
the latter case, I would restore the removed ebuild from the Git history of
::gentoo and add it to the Spark overlay. This is probably not the best
resolution, but as the number of packages in the Spark overlay may suggest,
Apache Spark is a complicated program whose dependency graph spans hundreds
of libraries, so recreating each ebuild removed from ::gentoo from scratch
might take too much time. However, the ebuilds were not restored
willy-nilly. ebuilds that used EAPI 5 were ported to EAPI 7, the latest
EAPI supported by the Java eclasses in ::gentoo. There was an ebuild whose
upstream had been gone and whose SRC_URI was no longer accessible, and I
managed to find a source JAR containing the package's source code on Maven
Central and updated the ebuild to use it instead.
At this point, the next package that needs to be fixed is dev-java/okio,
which is a package containing Kotlin sources and thus should be built with
a Kotlin compiler. Although my knowledge on Kotlin version compatibility
is limited, I believe there can be situations where a package must be built
by certain versions of the Kotlin compiler. This, plus fordfrog's interest
in allowing users to keep multiple Kotlin versions in parallel, motivated
me to think about how to implement support for multiple Kotlin compilers.
On Gentoo, the obvious solution to this problem is to create an eselect
module, but that would only allow coarse-grained system-wide control;
chances are some packages require Kotlin 1.4 whereas others need Kotlin
1.5, thus the Kotlin version must be set independently for each individual
package. For many other programming languages supported by Gentoo,
including Lua, PHP, Python and Ruby, there are USE_EXPAND variables that
can be used for this purpose, but such an approach would not work for
Kotlin yet because USE_EXPAND is defined in Portage profiles, which I have
no power to influence. As a workaround, I created a new
kotlin-utils.eclass, added a variable that would allow ebuild developers to
define either a single Kotlin version or an open interval of versions
required by the package, and declared another variable for the order of
preference on the accepted versions. The eclass can search for all Kotlin
versions installed on the system and pick the best one according to the
preference which satisfies any hard requirement. This would still
compromise the end users' ability to select the Kotlin version used to
build a package, but at least it could allow the ebuild developers to make
a sensible choice for the users.
Support for selecting a Kotlin version from ebuilds is provided by
kotlin-utils.eclass, and the functionality to choose a Kotlin compiler for
the user-space is shipped with the Kotlin eselect module. The module has a
superset of features offered by the java-vm eselect module. It implements
not only the notion of user and system compiler but also compiler
preference settings for each Kotlin feature release version. This feature
is provisioned for any alternative Kotlin compiler packages in the future,
like dev-lang/kotlin with the "-bin" suffix dropped. For instance, if
users have installed both dev-lang/kotlin:1.5 and dev-lang/kotlin-bin:1.5,
then by using the Kotlin eselect module, they can control whether they want
to use dev-lang/kotlin or dev-lang/kotlin-bin for Kotlin 1.5. It will be
possible to use different Kotlin compiler packages for different feature
releases too, so users can select dev-lang/kotlin:1.4 and
dev-lang/kotlin-bin:1.5 at the same time.
The version-specific Kotlin compiler preferences are recognized by both
kotlin-utils.eclass and a set of versioned executables for Kotlin tools.
Similar to commands like lua5.4, python3.9 and ruby26, versioned Kotlin
commands like kotlin1.4 and kotlinc1.5 are installed with the Kotlin
compiler packages and supported by the Kotlin eselect module. With these
versioned commands, users can quickly switch to another Kotlin version
without even invoking eselect. The versioned commands are not standard in
the Kotlin compiler Zip archive distributed by the upstream (which is what
other distributions' Kotlin packages usually depend on): they are exclusive
on Gentoo, so I hope they will convince any GNU/Linux users who would like
to install multiple Kotlin versions via the system package manager to
consider using Gentoo.
The new kotlin-utils.eclass, the Kotlin eselect module and some additional
Kotlin eclasses are still being tested. They are working delightfully as
of now, and I am trying to capture more issues and bugs before announcing
and releasing them to more users. I sincerely hope that these utilities
can form a robust framework for Kotlin packages and programs on Gentoo.
Best regards,
Leo
[1]: https://github.com/Leo3418/eselect-kotlin
[Attachment #3 (text/html)]
<div dir="ltr"><div dir="auto">Hi folks,<br>
<br>
Another week has just passed. I have been gradually moving on to the next part of \
my project where I would fix the ebuilds in the Spark overlay that are no longer \
installable, most of which are caused by changes to ebuilds in the Gentoo repository \
since last year's GSoC concluded. At the same time, a few additional utilities \
supporting Kotlin on Gentoo have been created, including an eselect module for Kotlin \
[1] and some new Kotlin eclasses.<br> <br>
Many ebuilds in the Spark overlay depend on packages in ::gentoo, but some \
dependencies have been broken due to removal of old slots or even deletion of \
packages that were really too old. In the former case, the dependency could be \
quickly fixed by updating the slot, although the dependee package would require some \
integration tests with the dependency's new slot. In the latter case, I would \
restore the removed ebuild from the Git history of ::gentoo and add it to the Spark \
overlay. This is probably not the best resolution, but as the number of packages in \
the Spark overlay may suggest, Apache Spark is a complicated program whose dependency \
graph spans hundreds of libraries, so recreating each ebuild removed from ::gentoo \
from scratch might take too much time. However, the ebuilds were not restored \
willy-nilly. ebuilds that used EAPI 5 were ported to EAPI 7, the latest EAPI \
supported by the Java eclasses in ::gentoo. There was an ebuild whose upstream had \
been gone and whose SRC_URI was no longer accessible, and I managed to find a source \
JAR containing the package's source code on Maven Central and updated the ebuild \
to use it instead.<br> <br>
At this point, the next package that needs to be fixed is dev-java/okio, which is a \
package containing Kotlin sources and thus should be built with a Kotlin compiler. \
Although my knowledge on Kotlin version compatibility is limited, I believe there can \
be situations where a package must be built by certain versions of the Kotlin \
compiler. This, plus fordfrog's interest in allowing users to keep multiple \
Kotlin versions in parallel, motivated me to think about how to implement support for \
multiple Kotlin compilers. On Gentoo, the obvious solution to this problem is to \
create an eselect module, but that would only allow coarse-grained system-wide \
control; chances are some packages require Kotlin 1.4 whereas others need Kotlin 1.5, \
thus the Kotlin version must be set independently for each individual package. For \
many other programming languages supported by Gentoo, including Lua, PHP, Python and \
Ruby, there are USE_EXPAND variables that can be used for this purpose, but such an \
approach would not work for Kotlin yet because USE_EXPAND is defined in Portage \
profiles, which I have no power to influence. As a workaround, I created a new \
kotlin-utils.eclass, added a variable that would allow ebuild developers to define \
either a single Kotlin version or an open interval of versions required by the \
package, and declared another variable for the order of preference on the accepted \
versions. The eclass can search for all Kotlin versions installed on the system and \
pick the best one<span style="color:rgb(0,0,0)"> </span><span \
style="color:rgb(0,0,0)">according to the preference </span>which satisfies any hard \
requirement. This would still compromise the end users' ability to select the \
Kotlin version used to build a package, but at least it could allow the ebuild \
developers to make a sensible choice for the users.</div><div dir="auto"> <br>
Support for selecting a Kotlin version from ebuilds is provided by \
kotlin-utils.eclass, and the functionality to choose a Kotlin compiler for the \
user-space is shipped with the Kotlin eselect module. The module has a superset of \
features offered by the java-vm eselect module. It implements not only the notion \
of user and system compiler but also compiler preference settings for each Kotlin \
feature release version. This feature is provisioned for any alternative Kotlin \
compiler packages in the future, like dev-lang/kotlin with the "-bin" \
suffix dropped. For instance, if users have installed both dev-lang/kotlin:1.5 and \
dev-lang/kotlin-bin:1.5, then by using the Kotlin eselect module, they can control \
whether they want to use dev-lang/kotlin or dev-lang/kotlin-bin for Kotlin 1.5. It \
will be possible to use different Kotlin compiler packages for different feature \
releases too, so users can select dev-lang/kotlin:1.4 and dev-lang/kotlin-bin:1.5 at \
the same time.<br> <br>
The version-specific Kotlin compiler preferences are recognized by both \
kotlin-utils.eclass and a set of versioned executables for Kotlin tools. Similar to \
commands like lua5.4, python3.9 and ruby26, versioned Kotlin commands like kotlin1.4 \
and kotlinc1.5 are installed with the Kotlin compiler packages and supported by the \
Kotlin eselect module. With these versioned commands, users can quickly switch to \
another Kotlin version without even invoking eselect. The versioned commands are \
not standard in the Kotlin compiler Zip archive distributed by the upstream (which is \
what other distributions' Kotlin packages usually depend on): they are exclusive \
on Gentoo, so I hope they will convince any GNU/Linux users who would like to install \
multiple Kotlin versions via the system package manager to consider using Gentoo.<br> \
<br> The new kotlin-utils.eclass, the Kotlin eselect module and some additional \
Kotlin eclasses are still being tested. They are working delightfully as of now, \
and I am trying to capture more issues and bugs before announcing and releasing them \
to more users. I sincerely hope that these utilities can form a robust framework \
for Kotlin packages and programs on Gentoo.<br> <br>
Best regards,<br>
Leo<br>
<br>
[1]: <a href="https://github.com/Leo3418/eselect-kotlin" rel="noreferrer" \
target="_blank">https://github.com/Leo3418/eselect-kotlin</a><br> </div>
</div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic