'[Openembedded-architecture] OEDEM - Setup program'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       openembedded-architecture
Subject:    [Openembedded-architecture] OEDEM - Setup program
From:       Mark Hatle <mark.hatle () windriver ! com>
Date:       2016-11-16 22:12:24
Message-ID: cf24d19d-9d1b-69ca-0484-1e00fa98f727 () windriver ! com
[Download RAW message or body]

The setup program that was demonstrated at OEDEM is now available.

https://github.com/Wind-River/wr-lx-setup

My expectation is that if OpenEmbedded decided to take ownership of this project
that they will fork a copy of this, and then I will stop development of this
github project.  If OpenEmbedded does fork the project, a suitable name should
be selected at that time.

To begin, the setup program uses the contents from the layer index.  So the
results are only going to be as good as the layer index you are using.  In
addition, the current layer index does not yet have distro indexing.  This is
available in the paule/django18 branch, but has not yet gone into the production
version.  When the setup program uses an index without any distribution
information, it simply adds a single 'nodistro' entry for OpenEmbedded.

Once the setup program loads the layer index, it will use the users settings to
discover which layers are necessary to construct the project.  It creates a repo
.xml file and uses repo to manage and pull down the layers.  The system will
also construct/copy various sample configuration, README and other files.

Below are some basic usage instructions.  See the README.md file, but I've
duplicated some of the contents below to give you an idea of the workflow.

To create a new project.  The minimal workflow becomes:

mkdir my-project
cd my-project
git clone git://github.com/Wind-River/wr-lx-setup setup
./setup/setup.sh --machine qemux86-64

At this point the program will contact the layer index and determine what
provides the BSP qemux86-64 and will construct a system that includes this BSP.

This should look similar to what you are used to by manually cloning oe-core and
bitbake:

bitbake  default.xml  meta               oe-init-build-env-memres  scripts
config   layers       oe-init-build-env  README                    setup

You'll see the addition of a 'default.xml' file, a layers directory, a config
directory, a customized README and of course the 'setup' program.   All layers
and other components downloaded end up in the 'layers' directory.  Default
configurations and log information go in the 'config' directory.  Only specific
files are symlinked to the root (OpenEmbedded and the related bitbake of course.)

Now for the more advanced information.  This is how everything is controlled.

First the shell script wrapper, setup.sh, is responsible for making the current
directory into a git tree, identifying the branch the setup program was run
from, and has the ability to run various extension scripts.

The setup program then exports certain information, such as the base download
directory and branch and calls the bin/setup.py program.  (If you use --help and
do not have python3 available it will use a special python version to display
basic help.)

The setup.py can replace a special '#BASE_URL#' field when processing URLs with
the url that the setup program was loaded from.  (This is VERY useful when
handling internal mirrors.)  The branch requested from the layer index is the
same branch as the setup program is running from by default as well.  This
allows you to have special configuration files or other files per branch.

Below is a series of files or directories that are used by the setup program to
specify various defaults, or files that get put into the constructed project.

bin/settings.py:
  This file controls the defaults for the setup program.  Including the location
of the layer index, branch settings, special 'search replace' operations,
default values, etc.   Take a look at this file.  If you have any questions let
me know.

As mentioned above, the #BASE_URL# is a magic flag..  if you want to use this on
a live layer-index, you can put in a manual search/replace operation.  This only
affects repository URLs BTW.

data/samples:
  This directory contains sample files for the bblayers.conf, local.conf,
conf-notes, README and a special README-MIRROR.

  Each of these files is processed when copying into the constructed project.

  Special key works can be used to replace contents with dynamic content based
on the configured:

    ####LAYERS#### - replace with each layer, format is for bblayers.conf

    ####SETUP_ARGS#### -- replace with the current arguments to setup

    ####MACHINES##### -- replace with a list of each machine available in the
current layers

    ####DEFAULTMACHINE#### -- replace with the -first- machine in the list

    ####DISTROS#### -- replace with a list of each distro available in the
current layers (required layer index support)

    ####DEFAULTDISTRO#### --- replace with the -first- distro in the list

data/xml:

  There are two types of files that can be in this set, .inc and .xml.  All
files are named 'layer name' (per the layer index) with the appropriate suffix.
A special 'bitbake' one is implemented as well.

The .inc files are included inline in the project element for the layer.  This
is the mechanism to add symlinks or other repo tags that must be inline with in
a 'project' tag.

The .xml files are included -after- the close of the project tag for the layer.
This can be used to include additional (non-layer) components into the generated
repo default.xml file when a specific layer is included.

Advanced features:

There is a --mirror option to the setup program.  This will take the
configuration you specified and create a local 'repo mirror' of the contents, as
well as a copy of the layer index (only the pieces that affect the mirror) and
any corresponding XML fragments.

The mirror allows you to start new projects from the mirror without having to
contact the upstream layer index or original sources.  This can also be useful
to share a specific configuration with others, especially if you want to curate
a specifically supported set of layers.

The contents of the mirror can also be shared on a local git server, making it
even easier to share.

In many environments local git servers can not handle sub directories.  (github,
stash, etc..)  So there is a 'flatten_mirror.py' command in bin.. When run from
a mirror project, it will create a flattened tree that may be easier for some
people to share on a git server (or other resources.)

There are other features, but overall the program isn't very large.  I'm happy
to answer any questions related to this.

--Mark
_______________________________________________
Openembedded-architecture mailing list
Openembedded-architecture@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-architecture
[prev in list] [next in list] [prev in thread] [next in thread] 