[prev in list] [next in list] [prev in thread] [next in thread]
List: openembedded-core
Subject: Re: [OE-core] [PATCH 3/3] kernel-devsrc: restructure for out of tree (and on target) module builds
From: Bruce Ashfield <bruce.ashfield () windriver ! com>
Date: 2018-02-28 17:38:44
Message-ID: f3935874-0721-ba2f-df81-c2e1af75bfae () windriver ! com
[Download RAW message or body]
On 02/28/2018 11:51 AM, Bruce Ashfield wrote:
> Unfortunately, I can't reproduce that here.
>
> This has been built and tested on all arches by myself and others, but
> I can try again .. just no promises when I'll get back to it.
Ahaha. 32 bit ARM with a little bit of a manual hack on my machine
and it triggered.
64 bit works fine. Will send a v2 shortly.
Bruce
>
> Bruce
>
> On 02/28/2018 11:38 AM, Burton, Ross wrote:
> > Sorry, but this breaks on ARM:
> >
> > ERROR: kernel-devsrc-1.0-r0 do_install: Function failed: do_install
> > (log file is located at
> > /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/log.do_install.25822) \
> >
> > ERROR: Logfile of failure stored in:
> > /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/log.do_install.25822 \
> >
> > Log data follows:
> > > DEBUG: Executing shell function do_install
> > > cp: with --parents, the destination must be a directory
> > > Try 'cp --help' for more information.
> > > WARNING:
> > /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/run.do_install.25822:1 \
> > exit 1 from 'rm -rf $kerneldir/build/include'
> > > ERROR: Function failed: do_install (log file is located at
> > /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/kernel-devsrc/1.0-r0/temp/log.do_install.25822) \
> >
> >
> > Ross
> >
> > On 28 February 2018 at 02:06, Bruce Ashfield
> > <bruce.ashfield@windriver.com <mailto:bruce.ashfield@windriver.com>>
> > wrote:
> >
> > The existing kernel-devsrc package starts with a full copy of the
> > kernel
> > source and then starts to strip out elements that are not required.
> >
> > This results in extra time (I/O) and extra space being taken up in
> > the
> > final package. The main purpose of the kernel-devsrc package has
> > been to
> > build modules against the running kernel, not to include a full
> > copy of
> > the source code for re-building the kernel. The end result was a
> > 600M kernel-devsrc package.
> >
> > This restructuring of the package uses an approach similar to other
> > distros, where the kernel-devsrc package is for building against the
> > running kernel and uses a curated set of copied infrastructure,
> > versus
> > a mass copy of the entire kernel.
> >
> > The differences in this approach versus other is largely due to the
> > architecture support and the split build/source directory of the
> > kernel.
> >
> > The result is a kernel-devsrc package of about 10M, which is capable
> > of running "make scripts" and compiling kernel modules against the
> > running kernel.
> >
> > Along with the changes to the copying of the infrascture, we also
> > have the following changes:
> >
> > - a better/more explicit listing of dependencies for on-target
> > builds of "make scripts" or "make modules_prepare"
> >
> > - The kernel source is installed into /lib/modules/<version>/build
> > and a symlink created from /usr/src/kernel to the new location.
> > This aligns with the standard location for module support
> > code
> >
> > - There is also a symlink from /lib/modules/<version>/source ->
> > build
> > to reserve a spot for a new package that is simply the kernel
> > source. That package is not part of this update.
> >
> > [YOCTO #12535]
> >
> > Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com
> > <mailto:bruce.ashfield@windriver.com>>
> > ---
> > meta/recipes-kernel/linux/kernel-devsrc.bb
> > <http://kernel-devsrc.bb> | 178 ++++++++++++++++++++++-------
> > 1 file changed, 138 insertions(+), 40 deletions(-)
> >
> > diff --git a/meta/recipes-kernel/linux/kernel-devsrc.bb
> > <http://kernel-devsrc.bb>
> > b/meta/recipes-kernel/linux/kernel-devsrc.bb
> > <http://kernel-devsrc.bb>
> > index c1b5b7786d7c..f55b115e0936 100644
> > --- a/meta/recipes-kernel/linux/kernel-devsrc.bb
> > <http://kernel-devsrc.bb>
> > +++ b/meta/recipes-kernel/linux/kernel-devsrc.bb
> > <http://kernel-devsrc.bb>
> > @@ -30,50 +30,148 @@ do_populate_sysroot[noexec] = "1"
> > S = "${STAGING_KERNEL_DIR}"
> > B = "${STAGING_KERNEL_BUILDDIR}"
> >
> > -KERNEL_VERSION = "${@get_kernelversion_headers('${S}')}"
> > -
> > PACKAGE_ARCH = "${MACHINE_ARCH}"
> >
> > +KERNEL_BUILD_ROOT="/lib/modules/"
> > +
> > do_install() {
> > - kerneldir=${D}${KERNEL_SRC_PATH}
> > - install -d $kerneldir
> > -
> > - #
> > - # Copy the staging dir source (and module build support)
> > into the devsrc structure.
> > - # We can keep this copy simple and take everything, since a
> > we'll clean up any build
> > - # artifacts afterwards, and the extra i/o is not significant
> > - #
> > - cd ${B}
> > - find . -type d -name '.git*' -prune -o -path '.debug'
> > -prune -o -type f -print0 | cpio --null -pdlu $kerneldir
> > - cd ${S}
> > - find . -type d -name '.git*' -prune -o -type d -name
> > '.kernel-meta' -prune -o -type f -print0 | cpio --null -pdlu
> > $kerneldir
> > -
> > - # Explicitly set KBUILD_OUTPUT to ensure that the image
> > directory is cleaned and not
> > - # The main build artifacts. We clean the directory to avoid
> > QA errors on mismatched
> > - # architecture (since scripts and helpers are native
> > format).
> > - KBUILD_OUTPUT="$kerneldir"
> > - oe_runmake -C $kerneldir CC="${KERNEL_CC}"
> > LD="${KERNEL_LD}" clean _mrproper_scripts
> > - # make clean generates an absolute path symlink called
> > "source"
> > - # in $kerneldir points to $kerneldir, which doesn't make any
> > - # sense, so remove it.
> > - if [ -L $kerneldir/source ]; then
> > - bbnote "Removing $kerneldir/source symlink"
> > - rm -f $kerneldir/source
> > - fi
> > -
> > - # As of Linux kernel version 3.0.1, the clean target removes
> > - # arch/powerpc/lib/crtsavres.o which is present in
> > - # KBUILD_LDFLAGS_MODULE, making it required to build
> > external modules.
> > - if [ ${ARCH} = "powerpc" ]; then
> > - mkdir -p $kerneldir/arch/powerpc/lib/
> > - cp ${B}/arch/powerpc/lib/crtsavres.o
> > $kerneldir/arch/powerpc/lib/crtsavres.o
> > - fi
> > -
> > - chown -R root:root ${D}
> > + kerneldir=${D}${KERNEL_BUILD_ROOT}${KERNEL_VERSION}
> > + install -d $kerneldir
> > +
> > + # create the directory structure
> > + rm -f $kerneldir/build
> > + rm -f $kerneldir/source
> > + mkdir -p $kerneldir/build
> > +
> > + # for compatibility with some older variants of this package, we
> > + # create a /usr/src/kernel symlink to
> > /lib/modules/<version>/source
> > + mkdir -p ${D}/usr/src
> > + (
> > + cd ${D}/usr/src
> > + ln -s ${KERNEL_BUILD_ROOT}${KERNEL_VERSION}/source kernel
> > + )
> > +
> > + # for on target purposes, we unify build and source
> > + (
> > + cd $kerneldir
> > + ln -s build source
> > + )
> > +
> > + # first copy everything
> > + (
> > + cd ${S}
> > + cp --parents $(find -type f -name "Makefile*" -o -name
> > "Kconfig*") $kerneldir/build
> > + )
> > +
> > + # then drop all but the needed Makefiles/Kconfig files
> > + rm -rf $kerneldir/build/Documentation
> > + rm -rf $kerneldir/build/scripts
> > + rm -rf $kerneldir/build/include
> > +
> > + # now copy in parts from the build that we'll need later
> > + (
> > + cd ${B}
> > +
> > + cp Module.symvers $kerneldir/build
> > + cp System.map* $kerneldir/build
> > + if [ -s Module.markers ]; then
> > + cp Module.markers $kerneldir/build
> > + fi
> > +
> > + cp .config $kerneldir/build
> > +
> > + # TODO: this scripts copy will likely blow up QA, so for
> > now, we
> > + # require a more complex 'make scripts' to restore
> > these, versus
> > + # copying them here. Left as a reference to indicate
> > that we
> > + # know the scripts must be dealt with.
> > + # cp -a scripts $kerneldir/build
> > +
> > + if [ -f tools/objtool/objtool ]; then
> > + cp -a tools/objtool/objtool
> > $kerneldir/build/tools/objtool/
> > + fi
> > + if [ -d arch/${ARCH}/scripts ]; then
> > + cp -a arch/${ARCH}/scripts $kerneldir/build/arch/${ARCH}
> > + fi
> > + if [ -f arch/${ARCH}/*lds ]; then
> > + cp -a arch/${ARCH}/*lds $kerneldir/build/arch/${ARCH}
> > + fi
> > +
> > + rm -f $kerneldir/build/scripts/*.o
> > + rm -f $kerneldir/build/scripts/*/*.o
> > +
> > + if [ "${ARCH}" = "powerpc" ]; then
> > + cp -a --parents arch/powerpc/lib/crtsavres.[So]
> > $kerneldir/build/
> > + fi
> > +
> > + cp -a include $kerneldir/build/include
> > + )
> > +
> > + # now grab the chunks from the source tree that we need
> > + (
> > + cd ${S}
> > +
> > + cp -a scripts $kerneldir/build
> > + if [ "${ARCH}" = "arm64" ]; then
> > + # arch/arm64/include/asm/xen references arch/arm
> > + cp -a --parents arch/arm/include/asm/xen
> > $kerneldir/build/
> > + # arch/arm64/include/asm/opcodes.h references arch/arm
> > + cp -a --parents arch/arm/include/asm/opcodes.h
> > $kerneldir/build/
> > + fi
> > +
> > + # include the machine specific headers for ARM variants, if
> > available.
> > + if [ "${ARCH}" = "arm" ]; then
> > + cp -a --parents arch/${ARCH}/mach-*/include
> > $kerneldir/build/
> > +
> > + # include a few files for 'make prepare'
> > + cp -a --parents arch/arm/tools/gen-mach-types
> > $RPM_BUILD_ROOT/lib/modules/$KernelVer/build/
> > + cp -a --parents arch/arm/tools/mach-types
> > $RPM_BUILD_ROOT/lib/modules/$KernelVer/build/
> > + fi
> > +
> > + if [ -d arch/${ARCH}/include ]; then
> > + cp -a --parents arch/${ARCH}/include $kerneldir/build/
> > + fi
> > +
> > + cp -a include $kerneldir/build
> > +
> > + if [ "${ARCH}" = "x86" ]; then
> > + # files for 'make prepare' to succeed with kernel-devel
> > + cp -a --parents arch/x86/entry/syscalls/syscall_32.tbl
> > $kerneldir/build/
> > + cp -a --parents arch/x86/entry/syscalls/syscalltbl.sh
> > $kerneldir/build/
> > + cp -a --parents arch/x86/entry/syscalls/syscallhdr.sh
> > $kerneldir/build/
> > + cp -a --parents arch/x86/entry/syscalls/syscall_64.tbl
> > $kerneldir/build/
> > + cp -a --parents arch/x86/tools/relocs_32.c
> > $kerneldir/build/
> > + cp -a --parents arch/x86/tools/relocs_64.c
> > $kerneldir/build/
> > + cp -a --parents arch/x86/tools/relocs.c $kerneldir/build/
> > + cp -a --parents arch/x86/tools/relocs_common.c
> > $kerneldir/build/
> > + cp -a --parents arch/x86/tools/relocs.h $kerneldir/build/
> > + cp -a --parents tools/include/tools/le_byteshift.h
> > $kerneldir/build/
> > + cp -a --parents tools/include/tools/be_byteshift.h
> > $kerneldir/build/
> > + cp -a --parents arch/x86/purgatory/purgatory.c
> > $kerneldir/build/
> > + cp -a --parents arch/x86/purgatory/sha256.h
> > $kerneldir/build/
> > + cp -a --parents arch/x86/purgatory/sha256.c
> > $kerneldir/build/
> > + cp -a --parents arch/x86/purgatory/stack.S
> > $kerneldir/build/
> > + cp -a --parents arch/x86/purgatory/string.c
> > $kerneldir/build/
> > + cp -a --parents arch/x86/purgatory/setup-x86_64.S
> > $kerneldir/build/
> > + cp -a --parents arch/x86/purgatory/entry64.S
> > $kerneldir/build/
> > + cp -a --parents arch/x86/boot/string.h $kerneldir/build/
> > + cp -a --parents arch/x86/boot/string.c $kerneldir/build/
> > + cp -a --parents arch/x86/boot/ctype.h $kerneldir/build/
> > + fi
> > + )
> > +
> > + # Make sure the Makefile and version.h have a matching
> > timestamp so that
> > + # external modules can be built
> > + touch -r $kerneldir/build/Makefile
> > $kerneldir/build/include/generated/uapi/linux/version.h
> > +
> > + # Copy .config to include/config/auto.conf so "make prepare" is
> > unnecessary.
> > + cp $kerneldir/build/.config
> > $kerneldir/build/include/config/auto.conf
> > +
> > + chown -R root:root ${D}
> > }
> > +
> > # Ensure we don't race against "make scripts" during cpio
> > do_install[lockfiles] = "${TMPDIR}/kernel-scripts.lock"
> >
> > -PACKAGES = "kernel-devsrc"
> > -FILES_${PN} = "${KERNEL_SRC_PATH}"
> > -RDEPENDS_${PN} = "bc"
> > +FILES_${PN} = "${KERNEL_BUILD_ROOT} ${KERNEL_SRC_PATH}"
> > +FILES_${PN}-dbg += "${KERNEL_BUILD_ROOT}*/build/scripts/*/.debug/*"
> > +RDEPENDS_${PN} = "bc glibc-utils"
> > --
> > 2.5.0
> >
> >
>
--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic