[prev in list] [next in list] [prev in thread] [next in thread]
List: freebsd-ppc
Subject: Re: main (14-CURRENT) may be unstable on powerpc64
From: Mark Millard via freebsd-ppc <freebsd-ppc () freebsd ! org>
Date: 2021-02-11 16:53:05
Message-ID: AD35A6BF-198B-4846-9040-07D3C56C4B5D () yahoo ! com
[Download RAW message or body]
On 2021-Feb-6, at 17:23, Mark Millard <marklmi at yahoo.com> wrote:
> On 2021-Feb-6, at 12:55, Mark Millard <marklmi at yahoo.com> wrote:
>
> > On 2021-Feb-5, at 23:49, Mark Millard <marklmi at yahoo.com> wrote:
> >
> > > On 2021-Feb-5, at 21:34, Mark Millard <marklmi at yahoo.com> wrote:
> > >
> > > > On 2021-Feb-5, at 10:34, Mark Millard <marklmi at yahoo.com> wrote:
> > > >
> > > > On 2021-Feb-5, at 09:35, Justin Hibbits <chmeeedalf at gmail.com> wrote:
> > > > >
> > > > > > On Fri, 5 Feb 2021 04:05:55 -0800
> > > > > > Mark Millard via freebsd-ppc <freebsd-ppc@freebsd.org> wrote:
> > > > > >
> > > > > > > . . .
> > > > > >
> > > > > > This is probably fallout from 710e45c4b, which has since been reverted.
> > > > > > 710e45c4b broke other things like swig as well, which caused a lot of
> > > > > > poudriere fallout for me (devel/llvm* failed because swig crashed).
> > > > > >
> > > > > > Try updating past 33f0540b1 and testing again.
> > > > > >
> > > > >
> > > > > The above is based on 847dfd2803f6, which is after 33f0540b1 . . .
> > > > >
> > > > > https://cgit.freebsd.org/src/log/?qt=range&q=33f0540b1~1..847dfd2803f6
> > > > > shows:
> > > > >
> > > > > Commit message (Expand) Author Age Files Lines
> > > > > * readelf: do not trucate section name with -W Ed Maste 45 hours 1 -4/+9
> > > > > * readelf: decode LA48 and ASG_DISABLE feature flags Ed Maste 45 \
> > > > > hours 1 -0/+2
> > > > > * Add a VM flag to prevent reclaim on a failed contig allocation Ryan \
> > > > > Stone 45 hours 3 -2/+11
> > > > > * dwmmc: Multiple busdma fixes. Michal Meloun 46 hours 1 -15/+32
> > > > > * linux: remove locks around callout_drain in timerfd_close() shu 46 \
> > > > > hours 1 -2/+0
> > > > > * Revert "Reimplement strlen" Mateusz Guzik 47 hours 2 -53/+108
> > > > >
> > > > > (I Probably should have shown that in the original
> > > > > message, given the difficulty in determining the
> > > > > relative order of referenced commits.)
> > > > >
> > > > > Before updating to be 847dfd2803f6 based, I had also
> > > > > previously hit the swig issue with the llvm10 build.
> > > > > That problem failed reliably until after I'd updated
> > > > > past the revert. (Not trusting the other things built
> > > > > is why I did a -c poudriere bulk after updating to an
> > > > > environment based on after the revert.)
> > > > >
> > > > > The variability in the pkg-static behavior this time
> > > > > suggests race conditions are involved, though not
> > > > > frequent failures.
> > > > >
> > > > >
> > > >
> > > > It still failed to build llvm10 but the error reporoted
> > > > is rather different. At least there is a .tbz for me to
> > > > expand. I may end up with more to report after that.
> > > >
> > > > Here is the log file's report:
> > > >
> > > > [2996/4558] /usr/bin/c++ -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS \
> > > > -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS \
> > > > -Itools/clang/tools/extra/clang-tidy/plugin \
> > > > -I/wrkdirs/usr/ports/devel/llvm10/work/llvm-10. \
> > > > 0.1.src/tools/clang/tools/extra/clang-tidy/plugin \
> > > > -I/wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/include \
> > > > -Itools/clang/include -Iinclude \
> > > > -I/wrkdirs/usr/ports/devel/llvm10/work/llvm
> > > > -10.0.1.src/include -O2 -pipe -DNDEBUG -fstack-protector-strong -isystem \
> > > > /usr/local/include -fno-strict-aliasing -DNDEBUG -isystem /usr/local/include \
> > > > -fPIC -fvisibility-inlines-hidden -Werror=date-ti me \
> > > > -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter \
> > > > -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic \
> > > > -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-defa ult \
> > > > -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor \
> > > > -Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections \
> > > > -fno-common -Woverloaded-virtual -Wno-nested-anon-types
> > > > -O2 -pipe -DNDEBUG -fstack-protector-strong -isystem /usr/local/include \
> > > > -fno-strict-aliasing -DNDEBUG -isystem /usr/local/include -fno-exceptions \
> > > > -std=c++14 -MD -MT tools/clang/tools/extra/clang-tid \
> > > > y/plugin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o -MF \
> > > > tools/clang/tools/extra/clang-tidy/plugin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o.d \
> > > > -o tools/clang/tools/extra/cl \
> > > > ang-tidy/plugin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o -c \
> > > > /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/tools/extra/clang-tidy/plugin/ClangTidyPlugin.cpp
> > > >
> > > > FAILED: tools/clang/tools/extra/clang-tidy/plugin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o \
> > > >
> > > > /usr/bin/c++ -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS \
> > > > -D__STDC_LIMIT_MACROS -Itools/clang/tools/extra/clang-tidy/plugin \
> > > > -I/wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tool \
> > > > s/clang/tools/extra/clang-tidy/plugin \
> > > > -I/wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/include \
> > > > -Itools/clang/include -Iinclude \
> > > > -I/wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/ include -O2 -pipe \
> > > > -DNDEBUG -fstack-protector-strong -isystem /usr/local/include \
> > > > -fno-strict-aliasing -DNDEBUG -isystem /usr/local/include -fPIC \
> > > > -fvisibility-inlines-hidden -Werror=date-time -Werror=u \
> > > > nguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings \
> > > > -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long \
> > > > -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noe xcept-type \
> > > > -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion \
> > > > -fdiagnostics-color -ffunction-sections -fdata-sections -fno-common \
> > > > -Woverloaded-virtual -Wno-nested-anon-types -O2 -pipe -D NDEBUG \
> > > > -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing \
> > > > -DNDEBUG -isystem /usr/local/include -fno-exceptions -std=c++14 -MD -MT \
> > > > tools/clang/tools/extra/clang-tidy/plugin/CMa \
> > > > keFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o -MF \
> > > > tools/clang/tools/extra/clang-tidy/plugin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o.d \
> > > > -o tools/clang/tools/extra/clang-tidy/plu \
> > > > gin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o -c \
> > > > /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/tools/extra/clang-tidy/plugin/ClangTidyPlugin.cpp
> > > > PLEASE submit a bug report to https://bugs.freebsd.org/submit/ and include \
> > > > the crash backtrace, preprocessed source, and associated run script. Stack \
> > > > dump: 0. Program arguments: /usr/bin/c++ -D_GNU_SOURCE \
> > > > -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS \
> > > > -Itools/clang/tools/extra/clang-tidy/plugin -I/wrkdirs/usr/ports/devel/llvm1 \
> > > > 0/work/llvm-10.0.1.src/tools/clang/tools/extra/clang-tidy/plugin \
> > > > -I/wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/include \
> > > > -Itools/clang/include -Iinclude \
> > > > -I/wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/include -O2 -pipe \
> > > > -DNDEBUG -fstack-protector-strong -isystem /usr/local/include \
> > > > -fno-strict-aliasing -DNDEBUG -isystem /usr/local/include -fPIC \
> > > > -fvisibility-inlines-hidden -Werror=date-time \
> > > > -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter \
> > > > -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic \
> > > > -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default \
> > > > -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor \
> > > > -Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections \
> > > > -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O2 -pipe -DNDEBUG \
> > > > -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing \
> > > > -DNDEBUG -isystem /usr/local/include -fno-exceptions -std=c++14 -MD -MT \
> > > > tools/clang/to
ols/extra/clang-tidy/plugin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o \
-MF tools/clang/tools/extra/clang-tidy/plugin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o.d \
-o tools/clang/tools/extra/clang-tidy/plugin/CMakeFiles/obj.clangTidyPlugin.dir/ClangTidyPlugin.cpp.o \
-c /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/tools/extra/clang-tidy/plugin/ClangTidyPlugin.cpp \
> > > > 1. /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/include/clang/Frontend/Utils.h:165:76: \
> > > > current parser token ')' 2. \
> > > > /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/include/clang/Frontend/Utils.h:40:1: \
> > > > parsing namespace 'clang' 3. \
> > > > /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/include/clang/Frontend/Utils.h:149:1: \
> > > > parsing struct/union/class body 'clang::ModuleDependencyCollector' 4. \
> > > > /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/include/clang/Frontend/Utils.h:165:47: \
> > > > parsing function body 'clang::ModuleDependencyCollector::insertSeen' 5. \
> > > > /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/tools/clang/include/clang/Frontend/Utils.h:165:47: \
> > > > in compound statement ('{}') 6. /usr/include/c++/v1/utility:297:29: \
> > > > instantiating class definition \
> > > > 'std::__1::pair<llvm::StringMapIterator<llvm::NoneType>, bool>' 7. \
> > > > /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/include/llvm/ADT/StringMap.h:34:34: \
> > > > instantiating class definition 'llvm::StringMapIterator<llvm::NoneType>' 8. \
> > > > /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/include/llvm/ADT/StringMap.h:488:7: \
> > > > instantiating class definition \
> > > > 'llvm::StringMapIterBase<llvm::StringMapIterator<llvm::NoneType>, \
> > > > llvm::StringMapEntry<llvm::NoneType>>' 9. \
> > > > /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/include/llvm/ADT/iterator.h:67:7: \
> > > > instantiating class definition \
> > > > 'llvm::iterator_facade_base<llvm::StringMapIterator<llvm::NoneType>, \
> > > > std::__1::forward_iterator_tag, llvm::StringMapEntry<llvm::NoneType>, long, \
> > > > llvm::StringMapEntry<llvm::NoneType> *, llvm::StringMapEntry<llvm::NoneType> \
> > > > &>' #0 0x0000000012f52ed0 llvm::sys::PrintStackTrace(llvm::raw_ostream&) \
> > > > (/usr/bin/c+++0x12f52ed0) #1 0x0000000012f50770 \
> > > > llvm::sys::RunSignalHandlers() (/usr/bin/c+++0x12f50770) #2 \
> > > > 0x0000000012ee4778 CrashRecoverySignalHandler(int) (/usr/bin/c+++0x12ee4778) \
> > > > #3 0x0000000813f2b2b4 (/lib/libthr.so.3+0x2f2b4) c++: error: clang frontend \
> > > > command failed due to signal (use -v to see invocation) FreeBSD clang version \
> > > > 11.0.1 (git@github.com:llvm/llvm-project.git \
> > > > llvmorg-11.0.1-0-g43ff75f2c3fe)
> > > > Target: powerpc64-unknown-freebsd14.0
> > > > Thread model: posix
> > > > InstalledDir: /usr/bin
> > > > c++: note: diagnostic msg:
> > > > ********************
> > > >
> > > > PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
> > > > Preprocessed source(s) and associated run script(s) are located at:
> > > > c++: note: diagnostic msg: /tmp/ClangTidyPlugin-21aaa2.cpp
> > > > c++: note: diagnostic msg: /tmp/ClangTidyPlugin-21aaa2.sh
> > > > c++: note: diagnostic msg:
> > > >
> > > >
> > > > Note: Unfortunately, poudriere does not capture those files
> > > > from /tmp/ in the .tbz for the build failure.
> > > >
> > >
> > > No *.core was left behind in the expanded .tbz .
> > > /var/log/messages did not have a line reporting
> > > any pid signal lines in the time frame.
> > >
> > > Repeating the failing command in the expansion of
> > > the .tbz did not repeat the problem.
> > >
> > > It looks to be another example of there being one
> > > or more not-readily-repeatable problems occurring.
> > >
> > > It will be many hours before the new poudriere bulk
> > > can get back to that same point, but with just the
> > > one builder and the one job for the builder. (Still
> > > allowing use of an active process per cpu.) The
> > > normal/prior bulk runs were allowing 2 jobs.
> > >
> > >
> > > Note: /lib/libthr.so.3+0x2f2b4 seems to be in code
> > > from the "static void handle_signal . . ." routine.
> > >
> >
> > The devel/llvm10 rebuild attempt is well past the
> > [2996/4558] point now: working on [3056/4558] .
> > But it looks like it may have 6+ hrs to go to
> > complete (if successful). So far:
> >
> > [FBSDpowerpc64-default] [2021-02-05_23h31m51s] [parallel_build:] Queued: 1 \
> > Built: 0 Failed: 0 Skipped: 0 Ignored: 0 Tobuild: 1 Time: 13:01:28 [01]: \
> > devel/llvm10 | llvm10-10.0.1_3 build (12:50:05 / \
> > 12:59:42) [13:02:12] Logs: \
> > /usr/local/poudriere/data/logs/bulk/FBSDpowerpc64-default/2021-02-05_23h31m51s
> > I've no reasonable way to figure out if the parallel
> > activity from letting 2 jobs run (so: higher load
> > average) would have made a difference or not. Getting
> > a very detailed repetition for such is difficult
> > so any variation in what context it failed in makes
> > useful comparison/contrast problematical.
> >
> > But, if this run completes successfully, there will
> > be 101 more ports to let poudriere bulk try to build,
> > one of which would be devel/llvm11 .
> >
>
> It finished, apparently successfully:
>
> [17:36:24] Built ports: devel/llvm10
> [FBSDpowerpc64-default] [2021-02-05_23h31m51s] [committing:] Queued: 1 Built: 1 \
> Failed: 0 Skipped: 0 Ignored: 0 Tobuild: 0 Time: 17:35:40 [17:36:24] Logs: \
> /usr/local/poudriere/data/logs/bulk/FBSDpowerpc64-default/2021-02-05_23h31m51s
> I've started the attempt to build what is still pending:
>
> . . .
> [FBSDpowerpc64-default] [2021-02-06_17h13m39s] [balancing_pool:] Queued: 101 Built: \
> 0 Failed: 0 Skipped: 0 Ignored: 0 Tobuild: 101 Time: 00:02:44 [00:03:27] \
> Recording filesystem state for prepkg... done [00:03:27] Building 101 packages \
> using 2 builders [00:03:27] Starting/Cloning builders
> [00:04:15] Hit CTRL+t at any time to see build progress and stats
> [00:04:15] [01] [00:00:00] Building graphics/mesa-libs | mesa-libs-20.2.3
> [00:04:15] [02] [00:00:00] Building devel/llvm11 | llvm11-11.0.0
> . . .
The 101 ports built fine.
But after updating to be based on 3acea07c1873:
# ~/fbsd-based-on-what-freebsd-main.sh
merge-base: 3acea07c1873b1e4042f4a4fa8668745ee59f15b
merge-base: CommitDate: 2021-02-08 19:15:21 +0000
c1845d00f818 (HEAD -> mm-src) mm-src snapshot for mm's patched build in git context.
3acea07c1873 (pure-src) Restore the augmented strlen commentary
FreeBSD FBSDG5L2 14.0-CURRENT FreeBSD 14.0-CURRENT mm-src-n244686-c1845d00f818 \
GENERIC64vtsc-NODBG-dcons powerpc powerpc64 1400004 1400004
I updated /usr/ports and started a rebuild, it showing
44 to be rebuilt.
The system crashed while doing (from before the crash):
[05:12:43] [02] [00:18:39] Finished devel/git@lite | git-lite-2.30.1: Success
load: 1.22 cmd: sh 46874 [runnable] 0.00r 0.00u 0.00s 0% 0k
[FBSDpowerpc64-default] [2021-02-10_22h25m46s] [parallel_build:] Queued: 44 Built: 41 \
Failed: 0 Skipped: 0 Ignored: 0 Tobuild: 3 Time: 08:02:20
[01]: devel/boost-libs | boost-libs-1.72.0_3 stage \
(04:05:07 / 05:46:06) [08:03:03] Logs: \
/usr/local/poudriere/data/logs/bulk/FBSDpowerpc64-default/2021-02-10_22h25m46s \
[08:27:48] [01] [06:10:51] Finished devel/boost-libs | boost-libs-1.72.0_3: Success
The crash had no messages, fans going by the time I discovered
the status. No ddb access was available.
So I again have no evidence beyond the crash itself, this time
for the whole system.
It is again trying to build the 3 ports that had not finished:
[FBSDpowerpc64-default] [2021-02-11_08h31m57s] [balancing_pool:] Queued: 3 Built: 0 \
Failed: 0 Skipped: 0 Ignored: 0 Tobuild: 3 Time: 00:01:15 [00:02:29] Recording \
filesystem state for prepkg... done [00:02:29] Building 3 packages using 2 builders
[00:02:29] Starting/Cloning builders
[00:17:48] Hit CTRL+t at any time to see build progress and stats
[00:17:48] [01] [00:00:00] Building devel/boost-libs | boost-libs-1.72.0_3
===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
_______________________________________________
freebsd-ppc@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ppc
To unsubscribe, send any mail to "freebsd-ppc-unsubscribe@freebsd.org"
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic