[prev in list] [next in list] [prev in thread] [next in thread]
List: openembedded-core
Subject: Re: [OE-core] python3 cryptic build failures
From: "Alexander Kanavin" <alex.kanavin () gmail ! com>
Date: 2023-05-31 15:13:54
Message-ID: CANNYZj96XQ7Sx4S+mqvMCQ0qobtfzja1Y5+_XTAwbsXrEeZS1g () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
I'm regularly seeing this when updating various python recipes. It's caused
by empty versioned directories in sysroots that are leftovers from previous
versions. Doesn't happen in freshly made tmpdirs.
Alex
On Wed 31. May 2023 at 16.43, Bruce Ashfield <bruce.ashfield@gmail.com>
wrote:
> On Wed, May 31, 2023 at 10:41 AM Tim Orling <ticotimo@gmail.com> wrote:
> >
> >
> >
> > On Wed, May 31, 2023 at 7:25 AM Bruce Ashfield <bruce.ashfield@gmail.com>
> wrote:
> > >
> > > On Wed, May 31, 2023 at 10:20 AM Trevor Gamblin <tgamblin@baylibre.com>
> wrote:
> > > >
> > > >
> > > > On 2023-05-31 10:04, Bruce Ashfield wrote:
> > > > > On Wed, May 31, 2023 at 9:23 AM Richard Purdie
> > > > > <richard.purdie@linuxfoundation.org> wrote:
> > > > > > Hi Trevor,
> > > > > >
> > > > > > I thought this issue might be something you might be able to spot
> the
> > > > > > solution with quickly.
> > > > > >
> > > > > > I had a failing python3-iniconfig build in a TMPDIR I've reused
> > > > > > heavily:
> > > > > FWIW. I see this in about 6 or 7 python recipes in my extremely
> > > > > heavily reused tmpdir.
> > > > >
> > > > > After I came back from vacation, I iterated through a -c clean on
> the
> > > > > python packages, and they always built the second time through.
> > > >
> > > > Do you recall which those are, and if it's the same ones each time it
> > > > happens? I'm wondering if it may be an issue with some recipes.
> > > >
> > >
> > > roughly this set (as grepped from my history):
> > >
> > > 2783 bitbake -c clean python3-attrs; bitbake core-image-minimal
> > > 2828 bitbake -c clean python3-colorama; bitbake core-image-minimal
> > > 2829 bitbake -c clean python3-jsonschema; bitbake core-image-minimal
> > > 2943 bitbake -c clean python3-hatch-vcs python3-hatch-vcs-native;
> > > bitbake core-image-minimal
> > > 2944 bitbake -c clean python3-attrs; bitbake core-image-minimal
> > > 2945 bitbake -c clean python3-urllib3; bitbake core-image-minimal
> > > 2946 bitbake -c clean python3-colorama; bitbake core-image-minimal
> > > 2947 bitbake -c clean python3-jsonschema; bitbake core-image-minimal
> > > 2948 bitbake -c clean python3-iniconfig; bitbake core-image-minimal
> > >
> > > Apparently I had particular issues with colorama :)
> > >
> > > Bruce
> > >
> >
> > Do you recall when you started seeing this behavior? I'm wondering if it
> is related to some of the wheel build changes rburton made. But that's just
> my spider sense…
>
> As we can all attest, timelines get blurry very quickly ... but I
> recall it popping up with the same / similar error message starting in
> the last 6 months.
>
> Bruce
>
> > >
> > >
> > > > I haven't been able to recreate the issue yet, but my tmpdir is not
> very
> > > > old. When I attempt the suggested reproducer (creating the empty
> > > > directories and rerunning do_compile), it is successful. I'll keep
> > > > poking at it.
> > > >
> > > > Trevor
> > > >
> > > > >
> > > > > Bruce
> > > > >
> > > > > > ERROR: python3-iniconfig-2.0.0-r0 do_compile:
> ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637',
> 1, None, None)
> > > > > > ERROR: Logfile of failure stored in:
> /media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637
>
> > > > > > Log data follows:
> > > > > > > DEBUG: Executing shell function do_compile
> > > > > > > * Getting build dependencies for wheel...
> > > > > > >
> > > > > > > Traceback (most recent call last):
> > > > > > > File
> "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py",
> line 375, in main
> > > > > > > built = build_call(
> > > > > > > ^^^^^^^^^^^
> > > > > > > File
> "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py",
> line 208, in build_package
> > > > > > > out = _build(isolation, builder, outdir, distribution,
> config_settings, skip_dependency_check)
> > > > > > >
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> > > > > > > File
> "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py",
> line 147, in _build
> > > > > > > return _build_in_current_env(builder, outdir, distribution,
> config_settings, skip_dependency_check)
> > > > > > >
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> > > > > > > File
> "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py",
> line 127, in _build_in_current_env
> > > > > > > missing = builder.check_dependencies(distribution)
> > > > > > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > > > > > > File
> "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py",
> line 369, in check_dependencies
> > > > > > > return {u for d in dependencies for u in check_dependency(d)}
> > > > > > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > > > > > > File
> "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py",
> line 369, in <setcomp>
> > > > > > > return {u for d in dependencies for u in check_dependency(d)}
> > > > > > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > > > > > > File
> "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py",
> line 185, in check_dependency
> > > > > > > yield from check_dependency(other_req_string,
> ancestral_req_strings + (normalised_req_string,), req.extras)
> > > > > > > File
> "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py",
> line 179, in check_dependency
> > > > > > > if req.specifier and not
> req.specifier.contains(dist.version, prereleases=True):
> > > > > > >
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > > > > > > File
> "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py",
> line 905, in contains
> > > > > > > item = Version(item)
> > > > > > > ^^^^^^^^^^^^^
> > > > > > > File
> "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py",
> line 196, in __init__
> > > > > > > match = self._regex.search(version)
> > > > > > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > > > > > > TypeError: expected string or bytes-like object, got 'NoneType'
> > > > > > >
> > > > > > > ERROR expected string or bytes-like object, got 'NoneType'
> > > > > > > WARNING: exit code 1 from a shell command.
> > > > > > ERROR: Task
> (/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile)
> failed with exit code '1'
> > > > > >
> > > > > > I wondered why and I isolated it down to this:
> > > > > >
> > > > > > richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$
> > > > > >
> find -type d -empty
> > > > > > ./hatchling-1.13.0.dist-info/licenses
> > > > > > ./pathspec-0.11.0.dist-info
> > > > > > ./importlib_metadata-6.0.0.dist-info
> > > > > > ./setuptools-67.4.0.dist-info
> > > > > > ./wheel-0.38.4.dist-info
> > > > > > ./installer-0.6.0.dist-info
> > > > > > ./setuptools-67.6.0.dist-info
> > > > > > ./packaging-23.0.dist-info
> > > > > > richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$
> > > > > >
> find -type d -empty -delete
> > > > > >
> > > > > > After which the compile succeeds. These empty dist-info directories
> > > > > > really upset the build. They're left behind by the sstate cleanup
> code
> > > > > > from when I have older versions of these python modules in play
> since
> > > > > > the sstate code can race over directory removal. Is there a way to
> make
> > > > > > python ignore them? Otherwise should we remove them somewhere?
> > > > > >
> > > > > > I suspect you can recreate by adding these empty directories and
> > > > > > running a -c compile -f. I've seen it with plenty of other python
> > > > > > modules too.
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > Richard
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > >
> > >
> > >
> > > --
> > > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > > thee at its end
> > > - "Use the force Harry" - Gandalf, Star Trek II
> > >
> > >
> > >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
>
>
>
[Attachment #5 (text/html)]
<div dir="auto">I'm regularly seeing this when updating various python recipes. It's \
caused by empty versioned directories in sysroots that are leftovers from previous \
versions. Doesn't happen in freshly made tmpdirs.</div><div dir="auto"><br></div><div \
dir="auto">Alex</div><div><br><div class="gmail_quote"><div dir="ltr" \
class="gmail_attr">On Wed 31. May 2023 at 16.43, Bruce Ashfield <<a \
href="mailto:bruce.ashfield@gmail.com">bruce.ashfield@gmail.com</a>> \
wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;border-left-color:rgb(204,204,204)">On \
Wed, May 31, 2023 at 10:41 AM Tim Orling <<a href="mailto:ticotimo@gmail.com" \
target="_blank">ticotimo@gmail.com</a>> wrote:<br> ><br>
><br>
><br>
> On Wed, May 31, 2023 at 7:25 AM Bruce Ashfield <<a \
href="mailto:bruce.ashfield@gmail.com" \
target="_blank">bruce.ashfield@gmail.com</a>> wrote:<br> >><br>
>> On Wed, May 31, 2023 at 10:20 AM Trevor Gamblin <<a \
href="mailto:tgamblin@baylibre.com" target="_blank">tgamblin@baylibre.com</a>> \
wrote:<br> >> ><br>
>> ><br>
>> > On 2023-05-31 10:04, Bruce Ashfield wrote:<br>
>> > > On Wed, May 31, 2023 at 9:23 AM Richard Purdie<br>
>> > > <<a href="mailto:richard.purdie@linuxfoundation.org" \
target="_blank">richard.purdie@linuxfoundation.org</a>> wrote:<br> >> > \
>> Hi Trevor,<br> >> > >><br>
>> > >> I thought this issue might be something you might be able to \
spot the<br> >> > >> solution with quickly.<br>
>> > >><br>
>> > >> I had a failing python3-iniconfig build in a TMPDIR I've \
reused<br> >> > >> heavily:<br>
>> > > FWIW. I see this in about 6 or 7 python recipes in my \
extremely<br> >> > > heavily reused tmpdir.<br>
>> > ><br>
>> > > After I came back from vacation, I iterated through a -c clean on \
the<br> >> > > python packages, and they always built the second time \
through.<br> >> ><br>
>> > Do you recall which those are, and if it's the same ones each time \
it<br> >> > happens? I'm wondering if it may be an issue with some \
recipes.<br> >> ><br>
>><br>
>> roughly this set (as grepped from my history):<br>
>><br>
>> 2783 bitbake -c clean python3-attrs; bitbake core-image-minimal<br>
>> 2828 bitbake -c clean python3-colorama; bitbake core-image-minimal<br>
>> 2829 bitbake -c clean python3-jsonschema; bitbake core-image-minimal<br>
>> 2943 bitbake -c clean python3-hatch-vcs python3-hatch-vcs-native;<br>
>> bitbake core-image-minimal<br>
>> 2944 bitbake -c clean python3-attrs; bitbake core-image-minimal<br>
>> 2945 bitbake -c clean python3-urllib3; bitbake core-image-minimal<br>
>> 2946 bitbake -c clean python3-colorama; bitbake core-image-minimal<br>
>> 2947 bitbake -c clean python3-jsonschema; bitbake core-image-minimal<br>
>> 2948 bitbake -c clean python3-iniconfig; bitbake core-image-minimal<br>
>><br>
>> Apparently I had particular issues with colorama :)<br>
>><br>
>> Bruce<br>
>><br>
><br>
> Do you recall when you started seeing this behavior? I'm wondering if it is \
related to some of the wheel build changes rburton made. But that's just my spider \
sense…<br> <br>
As we can all attest, timelines get blurry very quickly ... but I<br>
recall it popping up with the same / similar error message starting in<br>
the last 6 months.<br>
<br>
Bruce<br>
<br>
>><br>
>><br>
>> > I haven't been able to recreate the issue yet, but my tmpdir is not \
very<br> >> > old. When I attempt the suggested reproducer (creating the \
empty<br> >> > directories and rerunning do_compile), it is successful. \
I'll keep<br> >> > poking at it.<br>
>> ><br>
>> > Trevor<br>
>> ><br>
>> > ><br>
>> > > Bruce<br>
>> > ><br>
>> > >> ERROR: python3-iniconfig-2.0.0-r0 do_compile: \
ExecutionError('/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/run.do_compile.2391637', \
1, None, None)<br> >> > >> ERROR: Logfile of failure stored in: \
/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/temp/log.do_compile.2391637<br>
>> > >> Log data follows:<br>
>> > >> | DEBUG: Executing shell function do_compile<br>
>> > >> | * Getting build dependencies for wheel...<br>
>> > >> |<br>
>> > >> | Traceback (most recent call last):<br>
>> > >> | File \
"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", \
line 375, in main<br> >> > >> | built = build_call(<br>
>> > >> | ^^^^^^^^^^^<br>
>> > >> | File \
"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", \
line 208, in build_package<br> >> > >> | out = \
_build(isolation, builder, outdir, distribution, config_settings, \
skip_dependency_check)<br> >> > >> | \
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^<br>
>> > >> | File \
"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", \
line 147, in _build<br> >> > >> | return \
_build_in_current_env(builder, outdir, distribution, config_settings, \
skip_dependency_check)<br> >> > >> | \
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^<br>
>> > >> | File \
"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__main__.py", \
line 127, in _build_in_current_env<br> >> > >> | missing = \
builder.check_dependencies(distribution)<br> >> > >> | \
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^<br> >> > >> | File \
"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", \
line 369, in check_dependencies<br> >> > >> | return {u for d \
in dependencies for u in check_dependency(d)}<br> >> > >> | \
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^<br> >> > >> | \
File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0. \
0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", \
line 369, in <setcomp><br> >> > >> | return {u for d in \
dependencies for u in check_dependency(d)}<br> >> > >> | \
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^<br> >> > >> | \
File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0. \
0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", \
line 185, in check_dependency<br> >> > >> | yield from \
check_dependency(other_req_string, ancestral_req_strings + (normalised_req_string,), \
req.extras)<br> >> > >> | File \
"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/build/__init__.py", \
line 179, in check_dependency<br> >> > >> | if req.specifier \
and not req.specifier.contains(dist.version, prereleases=True):<br> >> > \
>> | \
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^<br> >> > >> | \
File "/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0. \
0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/specifiers.py", \
line 905, in contains<br> >> > >> | item = Version(item)<br>
>> > >> | ^^^^^^^^^^^^^<br>
>> > >> | File \
"/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/ \
recipe-sysroot-native/usr/lib/python3.11/site-packages/packaging/version.py", \
line 196, in __init__<br> >> > >> | match = \
self._regex.search(version)<br> >> > >> | \
^^^^^^^^^^^^^^^^^^^^^^^^^^^<br> >> > >> | TypeError: expected string \
or bytes-like object, got 'NoneType'<br> >> > >> |<br>
>> > >> | ERROR expected string or bytes-like object, got \
'NoneType'<br> >> > >> | WARNING: exit code 1 from a shell \
command.<br> >> > >> ERROR: Task \
(/media/build/poky/meta/recipes-devtools/python/python3-iniconfig_2.0.0.bb:do_compile) \
failed with exit code '1'<br> >> > >><br>
>> > >> I wondered why and I isolated it down to this:<br>
>> > >><br>
>> > >> \
richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ \
find -type d -empty<br> >> > >> \
./hatchling-1.13.0.dist-info/licenses<br> >> > >> \
./pathspec-0.11.0.dist-info<br> >> > >> \
./importlib_metadata-6.0.0.dist-info<br> >> > >> \
./setuptools-67.4.0.dist-info<br> >> > >> ./wheel-0.38.4.dist-info<br>
>> > >> ./installer-0.6.0.dist-info<br>
>> > >> ./setuptools-67.6.0.dist-info<br>
>> > >> ./packaging-23.0.dist-info<br>
>> > >> \
richard@jet:/media/build/poky/build/tmp/work/core2-64-poky-linux/python3-iniconfig/2.0.0-r0/recipe-sysroot-native/usr/lib/python3.11/site-packages$ \
find -type d -empty -delete<br> >> > >><br>
>> > >> After which the compile succeeds. These empty dist-info \
directories<br> >> > >> really upset the build. They're left \
behind by the sstate cleanup code<br> >> > >> from when I have older \
versions of these python modules in play since<br> >> > >> the sstate \
code can race over directory removal. Is there a way to make<br> >> > \
>> python ignore them? Otherwise should we remove them somewhere?<br> >> \
> >><br> >> > >> I suspect you can recreate by adding these \
empty directories and<br> >> > >> running a -c compile -f. I've \
seen it with plenty of other python<br> >> > >> modules too.<br>
>> > >><br>
>> > >> Cheers,<br>
>> > >><br>
>> > >> Richard<br>
>> > >><br>
>> > >><br>
>> > >><br>
>> > >><br>
>> > >><br>
>> > ><br>
>><br>
>><br>
>><br>
>> --<br>
>> - Thou shalt not follow the NULL pointer, for chaos and madness await<br>
>> thee at its end<br>
>> - "Use the force Harry" - Gandalf, Star Trek II<br>
>><br>
>> <br>
>><br>
<br>
<br>
-- <br>
- Thou shalt not follow the NULL pointer, for chaos and madness await<br>
thee at its end<br>
- "Use the force Harry" - Gandalf, Star Trek II<br>
<br>
<br>
<br>
</blockquote></div></div>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#182017): https://lists.openembedded.org/g/openembedded-core/message/182017
Mute This Topic: https://lists.openembedded.org/mt/99241972/4454766
Group Owner: openembedded-core+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [openembedded-core@marc.info]
-=-=-=-=-=-=-=-=-=-=-=-
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic