[prev in list] [next in list] [prev in thread] [next in thread]
List: openembedded-core
Subject: Re: [OE-core] [PATCH] sanity: allow to compile from root in user namespaces
From: Alexander Kanavin <alex.kanavin () gmail ! com>
Date: 2019-12-31 9:29:34
Message-ID: CANNYZj81SBio8vA1eeP5-17kAZ0OHnd0TCB2RC_pNO7uPyA66g () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
I believe it should be possible to create a regular user inside the
container and then run bitbake as that user?
The patch effectively disables the host contamination check, fixes one
recipe that breaks, and likely introduces other issues; the assumption (and
automated testing) throughout the build is that it's run under a regular
user.
Alex
On Tue, 31 Dec 2019 at 11:40, Nikolai Merinov via Openembedded-core <
openembedded-core@lists.openembedded.org> wrote:
> New rootless container subsystems rely on the "user namespaces"
> Linux feature. In order to create a container from a regular user the
> user uid and subuids mapped to the uid space inside of container.
>
> There are different default configurations used for different
> container subsystems:
>
> 1. Rootless Docker[1] maps the user ID to 0, the subuids to range
> starting from 1.
> 2. Rootless RunC[2] from OCI by default maps the user ID to 0 and
> ignores the subuids.
> 3. LXC[3] maps the subuids to range from 0. UID not participates
> in the mapping.
>
> The LXC variant does not allow to work on same files simultaneusly
> inside and outside of a container. Variant suggested by other
> container susbsystems assumes that files owned by user should be
> owned by root in a container environment.
>
> In order to simplify Yocto compilation in such rootless containers I
> want to allow to start the bitbake from root user in user namespace
> and allow software compilation from root user in the described
> configuration. Additionally I want to provide minimal dockerfile
> suitable for the core-image-sato image compilation.
>
> --
> [1] https://docs.docker.com/engine/security/rootless/
> [2] https://github.com/opencontainers/runc#rootless-containers
> [3]
> https://linuxcontainers.org/lxc/getting-started/#creating-unprivileged-containers-as-a-user
> ---
> meta/classes/insane.bbclass | 4 ++--
> meta/classes/sanity.bbclass | 10 +++++++-
> meta/recipes-core/coreutils/coreutils_8.31.bb | 1 +
> scripts/docker/Dockerfile | 19 +++++++++++++++
> scripts/docker/oe-rootless-docker | 24 +++++++++++++++++++
> 5 files changed, 55 insertions(+), 3 deletions(-)
> create mode 100644 scripts/docker/Dockerfile
> create mode 100755 scripts/docker/oe-rootless-docker
>
> diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
> index 0564f9c2a4..3599ce3e6a 100644
> --- a/meta/classes/insane.bbclass
> +++ b/meta/classes/insane.bbclass
> @@ -876,12 +876,12 @@ def package_qa_check_host_user(path, name, d, elf,
> messages):
> raise
> else:
> check_uid = int(d.getVar('HOST_USER_UID'))
> - if stat.st_uid == check_uid:
> + if check_uid != 0 and stat.st_uid == check_uid:
> package_qa_add_message(messages, "host-user-contaminated",
> "%s: %s is owned by uid %d, which is the same as the user running bitbake.
> This may be due to host contamination" % (pn, package_qa_clean_path(path,
> d, name), check_uid))
> return False
>
> check_gid = int(d.getVar('HOST_USER_GID'))
> - if stat.st_gid == check_gid:
> + if check_gid != 0 and stat.st_gid == check_gid:
> package_qa_add_message(messages, "host-user-contaminated",
> "%s: %s is owned by gid %d, which is the same as the user running bitbake.
> This may be due to host contamination" % (pn, package_qa_clean_path(path,
> d, name), check_gid))
> return False
> return True
> diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
> index 63ab6cf3df..a1eba62589 100644
> --- a/meta/classes/sanity.bbclass
> +++ b/meta/classes/sanity.bbclass
> @@ -742,7 +742,15 @@ def check_sanity_everybuild(status, d):
> # it makes sense to always run them.
>
> if 0 == os.getuid():
> - raise_sanity_error("Do not use Bitbake as root.", d)
> + userns = False
> + with open("/proc/self/uid_map") as f:
> + for line in f:
> + fields = line.split()
> + if fields[0] == "0" and fields[1] != "0":
> + userns = True
> + break
> + if not userns:
> + raise_sanity_error("Do not use Bitbake as root.", d)
>
> # Check the Python version, we now have a minimum of Python 3.4
> import sys
> diff --git a/meta/recipes-core/coreutils/coreutils_8.31.bb
> b/meta/recipes-core/coreutils/coreutils_8.31.bb
> index 57b2c1bdba..2f8009331a 100644
> --- a/meta/recipes-core/coreutils/coreutils_8.31.bb
> +++ b/meta/recipes-core/coreutils/coreutils_8.31.bb
> @@ -28,6 +28,7 @@ SRC_URI[sha256sum] =
> "ff7a9c918edce6b4f4b2725e3f9b37b0c4d193531cac49a48b56c4d0d3
> EXTRA_OECONF_class-native = "--without-gmp"
> EXTRA_OECONF_class-target = "--enable-install-program=arch,hostname
> --libexecdir=${libdir}"
> EXTRA_OECONF_class-nativesdk = "--enable-install-program=arch,hostname"
> +EXTRA_OECONF_append = " FORCE_UNSAFE_CONFIGURE=1"
>
> # acl and xattr are not default features
> #
> diff --git a/scripts/docker/Dockerfile b/scripts/docker/Dockerfile
> new file mode 100644
> index 0000000000..4a143d6aa1
> --- /dev/null
> +++ b/scripts/docker/Dockerfile
> @@ -0,0 +1,19 @@
> +FROM ubuntu:18.04
> +
> +RUN DEBIAN_FRONTEND=noninteractive apt-get -y update --fix-missing
> +
> +# Configure locale for Python3
> +RUN DEBIAN_FRONTEND=noninteractive apt-get -y install locales ; \
> + DEBIAN_FRONTEND=noninteractive dpkg-reconfigure locales && \
> + locale-gen en_US.UTF-8 && \
> + update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
> +
> +ENV LANG=en_US.UTF-8
> +ENV LC_ALL=en_US.UTF-8
> +
> +# Install Yocto native dependencies
> +RUN DEBIAN_FRONTEND=noninteractive apt-get -y install \
> + wget git-core diffstat unzip texinfo gcc-multilib \
> + build-essential chrpath socat cpio python python3 python3-pip
> python3-pexpect \
> + xz-utils debianutils iputils-ping python3-git python3-jinja2
> libegl1-mesa libsdl1.2-dev \
> + pylint3 xterm gawk
> diff --git a/scripts/docker/oe-rootless-docker
> b/scripts/docker/oe-rootless-docker
> new file mode 100755
> index 0000000000..1efcc82280
> --- /dev/null
> +++ b/scripts/docker/oe-rootless-docker
> @@ -0,0 +1,24 @@
> +#!/bin/sh
> +
> +usage () {
> +CMD=$(basename $0)
> +cat <<EOF
> +Usage: $CMD [docker-build|docker-run]
> + docker-build Build Dockerfile locally
> + docker-run Run rootless docker image suitable for image compilation
> +EOF
> +}
> +
> +if ! [ -n "${DOCKER_HOST}" -a "$(stat -c '%U' "${DOCKER_HOST#*://}"
> 2>/dev/null)" = "$(whoami)" ] ; then
> + echo "Install rootless docker according to the
> https://docs.docker.com/engine/security/rootless/"
> + exit 1
> +fi
> +
> +if [ "$1" = "docker-build" ]; then
> + docker build -t poky-docker $(dirname $(readlink -f $0))
> +elif [ "$1" = "docker-run" ]; then
> + docker run -ti -v ${HOME}:${HOME} -v ${PWD}:${PWD} --workdir
> ${PWD} -e HOME poky-docker
> +else
> + usage
> + exit 0
> +fi
> --
> 2.17.1
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
[Attachment #5 (text/html)]
<div dir="ltr"><div>I believe it should be possible to create a regular user inside \
the container and then run bitbake as that user? </div><div><br></div><div>The patch \
effectively disables the host contamination check, fixes one recipe that breaks, and \
likely introduces other issues; the assumption (and automated testing) throughout the \
build is that it's run under a regular \
user.</div><div><br></div><div>Alex</div></div><br><div class="gmail_quote"><div \
class="gmail_attr" dir="ltr">On Tue, 31 Dec 2019 at 11:40, Nikolai Merinov via \
Openembedded-core <<a \
href="mailto:openembedded-core@lists.openembedded.org">openembedded-core@lists.openembedded.org</a>> \
wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid">New \
rootless container subsystems rely on the "user namespaces"<br> Linux \
feature. In order to create a container from a regular user the<br> user uid and \
subuids mapped to the uid space inside of container.<br> <br>
There are different default configurations used for different<br>
container subsystems:<br>
<br>
1. Rootless Docker[1] maps the user ID to 0, the subuids to range<br>
starting from 1.<br>
2. Rootless RunC[2] from OCI by default maps the user ID to 0 and<br>
ignores the subuids.<br>
3. LXC[3] maps the subuids to range from 0. UID not participates<br>
in the mapping.<br>
<br>
The LXC variant does not allow to work on same files simultaneusly<br>
inside and outside of a container. Variant suggested by other<br>
container susbsystems assumes that files owned by user should be<br>
owned by root in a container environment.<br>
<br>
In order to simplify Yocto compilation in such rootless containers I<br>
want to allow to start the bitbake from root user in user namespace<br>
and allow software compilation from root user in the described<br>
configuration. Additionally I want to provide minimal dockerfile<br>
suitable for the core-image-sato image compilation.<br>
<br>
--<br>
[1] <a href="https://docs.docker.com/engine/security/rootless/" target="_blank" \
rel="noreferrer">https://docs.docker.com/engine/security/rootless/</a><br> [2] <a \
href="https://github.com/opencontainers/runc#rootless-containers" target="_blank" \
rel="noreferrer">https://github.com/opencontainers/runc#rootless-containers</a><br> \
[3] <a href="https://linuxcontainers.org/lxc/getting-started/#creating-unprivileged-containers-as-a-user" \
target="_blank" rel="noreferrer">https://linuxcontainers.org/lxc/getting-started/#creating-unprivileged-containers-as-a-user</a><br>
---<br>
meta/classes/insane.bbclass | 4 ++--<br>
meta/classes/sanity.bbclass | 10 +++++++-<br>
meta/recipes-core/coreutils/<a href="http://coreutils_8.31.bb" target="_blank" \
rel="noreferrer">coreutils_8.31.bb</a> | 1 +<br> scripts/docker/Dockerfile \
| 19 +++++++++++++++<br> scripts/docker/oe-rootless-docker | 24 \
+++++++++++++++++++<br> 5 files changed, 55 insertions(+), 3 deletions(-)<br>
create mode 100644 scripts/docker/Dockerfile<br>
create mode 100755 scripts/docker/oe-rootless-docker<br>
<br>
diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass<br>
index 0564f9c2a4..3599ce3e6a 100644<br>
--- a/meta/classes/insane.bbclass<br>
+++ b/meta/classes/insane.bbclass<br>
@@ -876,12 +876,12 @@ def package_qa_check_host_user(path, name, d, elf, \
messages):<br> raise<br>
else:<br>
check_uid = int(d.getVar('HOST_USER_UID'))<br>
- if stat.st_uid == check_uid:<br>
+ if check_uid != 0 and stat.st_uid == check_uid:<br>
package_qa_add_message(messages, \
"host-user-contaminated", "%s: %s is owned by uid %d, which is the \
same as the user running bitbake. This may be due to host contamination" % (pn, \
package_qa_clean_path(path, d, name), check_uid))<br> return False<br>
<br>
check_gid = int(d.getVar('HOST_USER_GID'))<br>
- if stat.st_gid == check_gid:<br>
+ if check_gid != 0 and stat.st_gid == check_gid:<br>
package_qa_add_message(messages, \
"host-user-contaminated", "%s: %s is owned by gid %d, which is the \
same as the user running bitbake. This may be due to host contamination" % (pn, \
package_qa_clean_path(path, d, name), check_gid))<br> return False<br>
return True<br>
diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass<br>
index 63ab6cf3df..a1eba62589 100644<br>
--- a/meta/classes/sanity.bbclass<br>
+++ b/meta/classes/sanity.bbclass<br>
@@ -742,7 +742,15 @@ def check_sanity_everybuild(status, d):<br>
# it makes sense to always run them.<br>
<br>
if 0 == os.getuid():<br>
- raise_sanity_error("Do not use Bitbake as root.", d)<br>
+ userns = False<br>
+ with open("/proc/self/uid_map") as f:<br>
+ for line in f:<br>
+ fields = line.split()<br>
+ if fields[0] == "0" and fields[1] != \
"0":<br> + userns = True<br>
+ break<br>
+ if not userns:<br>
+ raise_sanity_error("Do not use Bitbake as root.", d)<br>
<br>
# Check the Python version, we now have a minimum of Python 3.4<br>
import sys<br>
diff --git a/meta/recipes-core/coreutils/<a href="http://coreutils_8.31.bb" \
target="_blank" rel="noreferrer">coreutils_8.31.bb</a> \
b/meta/recipes-core/coreutils/<a href="http://coreutils_8.31.bb" target="_blank" \
rel="noreferrer">coreutils_8.31.bb</a><br> index 57b2c1bdba..2f8009331a 100644<br>
--- a/meta/recipes-core/coreutils/<a href="http://coreutils_8.31.bb" target="_blank" \
rel="noreferrer">coreutils_8.31.bb</a><br>
+++ b/meta/recipes-core/coreutils/<a href="http://coreutils_8.31.bb" target="_blank" \
rel="noreferrer">coreutils_8.31.bb</a><br> @@ -28,6 +28,7 @@ SRC_URI[sha256sum] = \
"ff7a9c918edce6b4f4b2725e3f9b37b0c4d193531cac49a48b56c4d0d3<br> \
EXTRA_OECONF_class-native = "--without-gmp"<br> EXTRA_OECONF_class-target \
= "--enable-install-program=arch,hostname --libexecdir=${libdir}"<br> \
EXTRA_OECONF_class-nativesdk = "--enable-install-program=arch,hostname"<br> \
+EXTRA_OECONF_append = " FORCE_UNSAFE_CONFIGURE=1"<br> <br>
# acl and xattr are not default features<br>
#<br>
diff --git a/scripts/docker/Dockerfile b/scripts/docker/Dockerfile<br>
new file mode 100644<br>
index 0000000000..4a143d6aa1<br>
--- /dev/null<br>
+++ b/scripts/docker/Dockerfile<br>
@@ -0,0 +1,19 @@<br>
+FROM ubuntu:18.04<br>
+<br>
+RUN DEBIAN_FRONTEND=noninteractive apt-get -y update --fix-missing<br>
+<br>
+# Configure locale for Python3<br>
+RUN DEBIAN_FRONTEND=noninteractive apt-get -y install locales ; \<br>
+ DEBIAN_FRONTEND=noninteractive dpkg-reconfigure locales && \<br>
+ locale-gen en_US.UTF-8 && \<br>
+ update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8<br>
+<br>
+ENV LANG=en_US.UTF-8<br>
+ENV LC_ALL=en_US.UTF-8<br>
+<br>
+# Install Yocto native dependencies<br>
+RUN DEBIAN_FRONTEND=noninteractive apt-get -y install \<br>
+ wget git-core diffstat unzip texinfo gcc-multilib \<br>
+ build-essential chrpath socat cpio python python3 python3-pip \
python3-pexpect \<br> + xz-utils debianutils iputils-ping python3-git \
python3-jinja2 libegl1-mesa libsdl1.2-dev \<br> + pylint3 xterm gawk<br>
diff --git a/scripts/docker/oe-rootless-docker \
b/scripts/docker/oe-rootless-docker<br> new file mode 100755<br>
index 0000000000..1efcc82280<br>
--- /dev/null<br>
+++ b/scripts/docker/oe-rootless-docker<br>
@@ -0,0 +1,24 @@<br>
+#!/bin/sh<br>
+<br>
+usage () {<br>
+CMD=$(basename $0)<br>
+cat <<EOF<br>
+Usage: $CMD [docker-build|docker-run]<br>
+ docker-build Build Dockerfile locally<br>
+ docker-run Run rootless docker image suitable for image compilation<br>
+EOF<br>
+}<br>
+<br>
+if ! [ -n "${DOCKER_HOST}" -a "$(stat -c '%U' \
"${DOCKER_HOST#*://}" 2>/dev/null)" = "$(whoami)" ] ; \
then<br> + echo "Install rootless docker according to the <a \
href="https://docs.docker.com/engine/security/rootless/" target="_blank" \
rel="noreferrer">https://docs.docker.com/engine/security/rootless/</a>"<br> + \
exit 1<br> +fi<br>
+<br>
+if [ "$1" = "docker-build" ]; then<br>
+ docker build -t poky-docker $(dirname $(readlink -f $0))<br>
+elif [ "$1" = "docker-run" ]; then<br>
+ docker run -ti -v ${HOME}:${HOME} -v ${PWD}:${PWD} --workdir ${PWD} -e \
HOME poky-docker<br> +else<br>
+ usage<br>
+ exit 0<br>
+fi<br>
-- <br>
2.17.1<br>
<br>
-- <br>
_______________________________________________<br>
Openembedded-core mailing list<br>
<a href="mailto:Openembedded-core@lists.openembedded.org" \
target="_blank">Openembedded-core@lists.openembedded.org</a><br> <a \
href="http://lists.openembedded.org/mailman/listinfo/openembedded-core" \
target="_blank" rel="noreferrer">http://lists.openembedded.org/mailman/listinfo/openembedded-core</a><br>
</blockquote></div>
--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic