[prev in list] [next in list] [prev in thread] [next in thread] 

List:       openembedded-core
Subject:    [OE-core] Concerns about hash equivalence reliability
From:       "Richard Purdie" <richard.purdie () linuxfoundation ! org>
Date:       2021-09-29 16:13:23
Message-ID: d56b277b2dc37a0674698c5c4ca069696723afce.camel () linuxfoundation ! org
[Download RAW message or body]

I'm getting a bit worried about where we're at with reproducibility and hash
equivalence. As a test, I ran a build on the autobuilder:


https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/4053

which is a master-next build of MACHINE=qemux86-64, nothing special.

I then tried to use the sstate and hash equivalence public servers to reuse it
(with the same revision of master-next). I didn't get much reuse which was odd.

I therefore reran the build on the autobuilder:

https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/4055

which also didn't reuse much of the sstate.

I then repeated the external test and whilst there was more reuse this time, it
still wasn't right, e.g. the kernel was rebuilding.

So we have a problem, but what is the problem? I had a feeling the output wasn't
reproducing correctly, so how to debug and identify?

I ran the build on an autobuilder worker and also on my local build machine with
sstate and hashequiv disabled so it would build from scratch. I then collected
the depsig files from both builds:

tar -cvzf depsig.tgz tmp/work/*/*/*/temp/depsig.do_*

and then compared them.

I'm seeing -native populate_sysroot differences which is expected as there are
different host distros being used which will result in differing output. I'm
also seeing:

* do_package_write_rpm differences from parallel compression options
* hardcoded paths in pkgdata *.pclist files which break do_packagedata and  
  do_package
* kernel do_deploy filenames change (probably needs to use SDE rather than
DATETIME)
* hardcoded paths in libtool-cross
* hardcoded data in postinst-useradd- scripts
* hardcoded paths in curl-config
* some issue with gnupg binariess differing

I have fixes for some of these, attempts at others and some I haven't even
touched upon yet.

I kind of feel like I'm drowning in problems again. We do need to improve the
tests to avoid these issues.

Cheers,

Richard







-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#156471): https://lists.openembedded.org/g/openembedded-core/message/156471
Mute This Topic: https://lists.openembedded.org/mt/85952348/4454766
Group Owner: openembedded-core+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [openembedded-core@marc.info]
-=-=-=-=-=-=-=-=-=-=-=-



[prev in list] [next in list] [prev in thread] [next in thread]