[prev in list] [next in list] [prev in thread] [next in thread]
List: openembedded-core
Subject: Re: [OE-core] [PATCH] rm_work.bbclass: use HOSTTOOLS 'rm' binary exclusively
From: "Luis Martins" <luis.pinto.martins () gmail ! com>
Date: 2022-12-30 23:27:53
Message-ID: CAHeZdFwNF1yM871oBBuRQ=z6d_wfk25QGhm7c_N83ShRaxby+A () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Hi, I see this was accepted for master, is it okay to port this to dunfell
and kirkstone as well ?
Best regards,
Luís Martins
On Sat, 24 Dec 2022 at 01:49, Luis Martins <luis.pinto.martins@gmail.com>
wrote:
> The do_rm_work() task is using the first available 'rm' binary
> available in PATH to remove files and folders.
> However, depending on the PATH setup and RECIPE_SYSROOT_NATIVE
> contents, the function can be using the 'rm' binary available
> in RECIPE_SYSROOT_NATIVE, a folder that will get removed.
> This causes a sporadic race-condition when trying to access the
> 'rm' binary of a folder already deleted.
> Solve this by exclusively using the HOSTTOOLS 'rm' binary, as
> this folder will not get removed.
>
> Signed-off-by: Luis Martins <luis.pinto.martins@gmail.com>
> ---
> meta/classes/rm_work.bbclass | 15 +++++++++++----
> 1 file changed, 11 insertions(+), 4 deletions(-)
>
> diff --git a/meta/classes/rm_work.bbclass b/meta/classes/rm_work.bbclass
> index 1f28bc7187..8b5fe1b808 100644
> --- a/meta/classes/rm_work.bbclass
> +++ b/meta/classes/rm_work.bbclass
> @@ -33,6 +33,13 @@ BB_SCHEDULER ?= "completion"
> BB_TASK_IONICE_LEVEL:task-rm_work = "3.0"
>
> do_rm_work () {
> + # Force using the HOSTTOOLS 'rm' - otherwise the SYSROOT_NATIVE 'rm'
> can be selected depending on PATH
> + # Avoids race-condition accessing 'rm' when deleting WORKDIR folders
> at the end of this function
> + RM_BIN="$(PATH=${HOSTTOOLS_DIR} command -v rm)"
> + if [ -z "${RM_BIN}" ]; then
> + bbfatal "Binary 'rm' not found in HOSTTOOLS_DIR, cannot remove
> WORKDIR data."
> + fi
> +
> # If the recipe name is in the RM_WORK_EXCLUDE, skip the recipe.
> for p in ${RM_WORK_EXCLUDE}; do
> if [ "$p" = "${PN}" ]; then
> @@ -79,7 +86,7 @@ do_rm_work () {
> # sstate version since otherwise we'd need to leave
> 'plaindirs' around
> # such as 'packages' and 'packages-split' and these can be
> large. No end
> # of chain tasks depend directly on do_package anymore.
> - rm -f -- $i;
> + "${RM_BIN}" -f -- $i;
> ;;
> *_setscene*)
> # Skip stamps which are already setscene versions
> @@ -96,7 +103,7 @@ do_rm_work () {
> ;;
> esac
> done
> - rm -f -- $i
> + "${RM_BIN}" -f -- $i
> esac
> done
>
> @@ -106,9 +113,9 @@ do_rm_work () {
> # Retain only logs and other files in temp, safely ignore
> # failures of removing pseudo folers on NFS2/3 server.
> if [ $dir = 'pseudo' ]; then
> - rm -rf -- $dir 2> /dev/null || true
> + "${RM_BIN}" -rf -- $dir 2> /dev/null || true
> elif ! echo "$excludes" | grep -q -w "$dir"; then
> - rm -rf -- $dir
> + "${RM_BIN}" -rf -- $dir
> fi
> done
> }
> --
> 2.25.1
>
>
[Attachment #5 (text/html)]
<div dir="ltr">Hi, I see this was accepted for master, is it okay to port this to \
dunfell and kirkstone as well ?<br clear="all"><div><div><div dir="ltr" \
class="gmail_signature" data-smartmail="gmail_signature"><br></div><div \
class="gmail_signature" data-smartmail="gmail_signature">Best regards,<br></div><div \
dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">Luís \
Martins<br></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" \
class="gmail_attr">On Sat, 24 Dec 2022 at 01:49, Luis Martins <<a \
href="mailto:luis.pinto.martins@gmail.com">luis.pinto.martins@gmail.com</a>> \
wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">The do_rm_work() task \
is using the first available 'rm' binary<br> available in PATH to remove \
files and folders.<br> However, depending on the PATH setup and \
RECIPE_SYSROOT_NATIVE<br> contents, the function can be using the 'rm' binary \
available<br> in RECIPE_SYSROOT_NATIVE, a folder that will get removed.<br>
This causes a sporadic race-condition when trying to access the<br>
'rm' binary of a folder already deleted.<br>
Solve this by exclusively using the HOSTTOOLS 'rm' binary, as<br>
this folder will not get removed.<br>
<br>
Signed-off-by: Luis Martins <<a href="mailto:luis.pinto.martins@gmail.com" \
target="_blank">luis.pinto.martins@gmail.com</a>><br>
---<br>
meta/classes/rm_work.bbclass | 15 +++++++++++----<br>
1 file changed, 11 insertions(+), 4 deletions(-)<br>
<br>
diff --git a/meta/classes/rm_work.bbclass b/meta/classes/rm_work.bbclass<br>
index 1f28bc7187..8b5fe1b808 100644<br>
--- a/meta/classes/rm_work.bbclass<br>
+++ b/meta/classes/rm_work.bbclass<br>
@@ -33,6 +33,13 @@ BB_SCHEDULER ?= "completion"<br>
BB_TASK_IONICE_LEVEL:task-rm_work = "3.0"<br>
<br>
do_rm_work () {<br>
+ # Force using the HOSTTOOLS 'rm' - otherwise the SYSROOT_NATIVE \
'rm' can be selected depending on PATH<br> + # Avoids race-condition \
accessing 'rm' when deleting WORKDIR folders at the end of this function<br> \
+ RM_BIN="$(PATH=${HOSTTOOLS_DIR} command -v rm)"<br> + if [ -z \
"${RM_BIN}" ]; then<br> + bbfatal "Binary 'rm' not \
found in HOSTTOOLS_DIR, cannot remove WORKDIR data."<br> + fi<br>
+<br>
# If the recipe name is in the RM_WORK_EXCLUDE, skip the recipe.<br>
for p in ${RM_WORK_EXCLUDE}; do<br>
if [ "$p" = "${PN}" ]; then<br>
@@ -79,7 +86,7 @@ do_rm_work () {<br>
# sstate version since otherwise we'd need to leave \
'plaindirs' around<br>
# such as 'packages' and 'packages-split' and \
these can be large. No end<br>
# of chain tasks depend directly on do_package anymore.<br>
- rm -f -- $i;<br>
+ "${RM_BIN}" -f -- $i;<br>
;;<br>
*_setscene*)<br>
# Skip stamps which are already setscene versions<br>
@@ -96,7 +103,7 @@ do_rm_work () {<br>
;;<br>
esac<br>
done<br>
- rm -f -- $i<br>
+ "${RM_BIN}" -f -- $i<br>
esac<br>
done<br>
<br>
@@ -106,9 +113,9 @@ do_rm_work () {<br>
# Retain only logs and other files in temp, safely ignore<br>
# failures of removing pseudo folers on NFS2/3 server.<br>
if [ $dir = 'pseudo' ]; then<br>
- rm -rf -- $dir 2> /dev/null || true<br>
+ "${RM_BIN}" -rf -- $dir 2> /dev/null || true<br>
elif ! echo "$excludes" | grep -q -w "$dir"; \
then<br>
- rm -rf -- $dir<br>
+ "${RM_BIN}" -rf -- $dir<br>
fi<br>
done<br>
}<br>
-- <br>
2.25.1<br>
<br>
</blockquote></div>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#175179): https://lists.openembedded.org/g/openembedded-core/message/175179
Mute This Topic: https://lists.openembedded.org/mt/95856273/4454766
Group Owner: openembedded-core+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [openembedded-core@marc.info]
-=-=-=-=-=-=-=-=-=-=-=-
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic