Revert "poky: subtree update:b23aa6b753..ad30a6d470"
This reverts commit af5e4ef732faedf66c6dc1756432e9de2ac72988.
This commit introduced openbmc/openbmc#3720 and no solution has been
forthcoming. Revert until we can get to the bottom of this.
Change-Id: I2fb0d81eb26cf3dadb2f2abdd1a1bb7a95eaf03c
diff --git a/poky/documentation/overview-manual/history.rst b/poky/documentation/overview-manual/history.rst
index 6fc700a..0273d28 100644
--- a/poky/documentation/overview-manual/history.rst
+++ b/poky/documentation/overview-manual/history.rst
@@ -1,4 +1,4 @@
-.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
+.. SPDX-License-Identifier: CC-BY-2.0-UK
***********************
Manual Revision History
diff --git a/poky/documentation/overview-manual/overview-manual-concepts.rst b/poky/documentation/overview-manual/overview-manual-concepts.rst
index d9f50e5..6ce5f80 100644
--- a/poky/documentation/overview-manual/overview-manual-concepts.rst
+++ b/poky/documentation/overview-manual/overview-manual-concepts.rst
@@ -1,4 +1,4 @@
-.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
+.. SPDX-License-Identifier: CC-BY-2.0-UK
**********************
Yocto Project Concepts
diff --git a/poky/documentation/overview-manual/overview-manual-concepts.xml b/poky/documentation/overview-manual/overview-manual-concepts.xml
new file mode 100644
index 0000000..58b64bd
--- /dev/null
+++ b/poky/documentation/overview-manual/overview-manual-concepts.xml
@@ -0,0 +1,3235 @@
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
+"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
+[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
+<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
+
+<chapter id=' overview-manual-concepts'>
+<title>Yocto Project Concepts</title>
+
+ <para>
+ This chapter provides explanations for Yocto Project concepts that
+ go beyond the surface of "how-to" information and reference (or
+ look-up) material.
+ Concepts such as components, the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>OpenEmbedded build system</ulink>
+ workflow, cross-development toolchains, shared state cache, and so
+ forth are explained.
+ </para>
+
+ <section id='yocto-project-components'>
+ <title>Yocto Project Components</title>
+
+ <para>
+ The
+ <ulink url='&YOCTO_DOCS_REF_URL;#bitbake-term'>BitBake</ulink>
+ task executor together with various types of configuration files
+ form the
+ <ulink url='&YOCTO_DOCS_REF_URL;#oe-core'>OpenEmbedded-Core</ulink>.
+ This section overviews these components by describing their use and
+ how they interact.
+ </para>
+
+ <para>
+ BitBake handles the parsing and execution of the data files.
+ The data itself is of various types:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis>Recipes:</emphasis>
+ Provides details about particular pieces of software.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Class Data:</emphasis>
+ Abstracts common build information (e.g. how to build a
+ Linux kernel).
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Configuration Data:</emphasis>
+ Defines machine-specific settings, policy decisions, and
+ so forth.
+ Configuration data acts as the glue to bind everything
+ together.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ BitBake knows how to combine multiple data sources together and
+ refers to each data source as a layer.
+ For information on layers, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and Creating Layers</ulink>"
+ section of the Yocto Project Development Tasks Manual.
+ </para>
+
+ <para>
+ Following are some brief details on these core components.
+ For additional information on how these components interact during
+ a build, see the
+ "<link linkend='openembedded-build-system-build-concepts'>OpenEmbedded Build System Concepts</link>"
+ section.
+ </para>
+
+ <section id='usingpoky-components-bitbake'>
+ <title>BitBake</title>
+
+ <para>
+ BitBake is the tool at the heart of the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>OpenEmbedded build system</ulink>
+ and is responsible for parsing the
+ <ulink url='&YOCTO_DOCS_REF_URL;#metadata'>Metadata</ulink>,
+ generating a list of tasks from it, and then executing those
+ tasks.
+ </para>
+
+ <para>
+ This section briefly introduces BitBake.
+ If you want more information on BitBake, see the
+ <ulink url='&YOCTO_DOCS_BB_URL;'>BitBake User Manual</ulink>.
+ </para>
+
+ <para>
+ To see a list of the options BitBake supports, use either of
+ the following commands:
+ <literallayout class='monospaced'>
+ $ bitbake -h
+ $ bitbake --help
+ </literallayout>
+ </para>
+
+ <para>
+ The most common usage for BitBake is
+ <filename>bitbake <replaceable>packagename</replaceable></filename>,
+ where <filename>packagename</filename> is the name of the
+ package you want to build (referred to as the "target").
+ The target often equates to the first part of a recipe's
+ filename (e.g. "foo" for a recipe named
+ <filename>foo_1.3.0-r0.bb</filename>).
+ So, to process the
+ <filename>matchbox-desktop_1.2.3.bb</filename> recipe file, you
+ might type the following:
+ <literallayout class='monospaced'>
+ $ bitbake matchbox-desktop
+ </literallayout>
+ Several different versions of
+ <filename>matchbox-desktop</filename> might exist.
+ BitBake chooses the one selected by the distribution
+ configuration.
+ You can get more details about how BitBake chooses between
+ different target versions and providers in the
+ "<ulink url='&YOCTO_DOCS_BB_URL;#bb-bitbake-preferences'>Preferences</ulink>"
+ section of the BitBake User Manual.
+ </para>
+
+ <para>
+ BitBake also tries to execute any dependent tasks first.
+ So for example, before building
+ <filename>matchbox-desktop</filename>, BitBake would build a
+ cross compiler and <filename>glibc</filename> if they had not
+ already been built.
+ </para>
+
+ <para>
+ A useful BitBake option to consider is the
+ <filename>-k</filename> or <filename>--continue</filename>
+ option.
+ This option instructs BitBake to try and continue processing
+ the job as long as possible even after encountering an error.
+ When an error occurs, the target that failed and those that
+ depend on it cannot be remade.
+ However, when you use this option other dependencies can
+ still be processed.
+ </para>
+ </section>
+
+ <section id='overview-components-recipes'>
+ <title>Recipes</title>
+
+ <para>
+ Files that have the <filename>.bb</filename> suffix are
+ "recipes" files.
+ In general, a recipe contains information about a single piece
+ of software.
+ This information includes the location from which to download
+ the unaltered source, any source patches to be applied to that
+ source (if needed), which special configuration options to
+ apply, how to compile the source files, and how to package the
+ compiled output.
+ </para>
+
+ <para>
+ The term "package" is sometimes used to refer to recipes.
+ However, since the word "package" is used for the packaged
+ output from the OpenEmbedded build system (i.e.
+ <filename>.ipk</filename> or <filename>.deb</filename> files),
+ this document avoids using the term "package" when referring
+ to recipes.
+ </para>
+ </section>
+
+ <section id='overview-components-classes'>
+ <title>Classes</title>
+
+ <para>
+ Class files (<filename>.bbclass</filename>) contain information
+ that is useful to share between recipes files.
+ An example is the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-autotools'><filename>autotools</filename></ulink>
+ class, which contains common settings for any application that
+ Autotools uses.
+ The
+ "<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes'>Classes</ulink>"
+ chapter in the Yocto Project Reference Manual provides
+ details about classes and how to use them.
+ </para>
+ </section>
+
+ <section id='overview-components-configurations'>
+ <title>Configurations</title>
+
+ <para>
+ The configuration files (<filename>.conf</filename>) define
+ various configuration variables that govern the OpenEmbedded
+ build process.
+ These files fall into several areas that define machine
+ configuration options, distribution configuration options,
+ compiler tuning options, general common configuration options,
+ and user configuration options in
+ <filename>conf/local.conf</filename>, which is found in the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>.
+ </para>
+ </section>
+ </section>
+
+ <section id='overview-layers'>
+ <title>Layers</title>
+
+ <para>
+ Layers are repositories that contain related metadata (i.e.
+ sets of instructions) that tell the OpenEmbedded build system how
+ to build a target.
+ Yocto Project's
+ <link linkend='the-yocto-project-layer-model'>layer model</link>
+ facilitates collaboration, sharing, customization, and reuse
+ within the Yocto Project development environment.
+ Layers logically separate information for your project.
+ For example, you can use a layer to hold all the configurations
+ for a particular piece of hardware.
+ Isolating hardware-specific configurations allows you to share
+ other metadata by using a different layer where that metadata
+ might be common across several pieces of hardware.
+ </para>
+
+ <para>
+ Many layers exist that work in the Yocto Project development
+ environment.
+ The
+ <ulink url='https://caffelli-staging.yoctoproject.org/software-overview/layers/'>Yocto Project Curated Layer Index</ulink>
+ and
+ <ulink url='http://layers.openembedded.org/layerindex/branch/master/layers/'>OpenEmbedded Layer Index</ulink>
+ both contain layers from which you can use or leverage.
+ </para>
+
+ <para>
+ By convention, layers in the Yocto Project follow a specific form.
+ Conforming to a known structure allows BitBake to make assumptions
+ during builds on where to find types of metadata.
+ You can find procedures and learn about tools (i.e.
+ <filename>bitbake-layers</filename>) for creating layers suitable
+ for the Yocto Project in the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and Creating Layers</ulink>"
+ section of the Yocto Project Development Tasks Manual.
+ </para>
+ </section>
+
+ <section id="openembedded-build-system-build-concepts">
+ <title>OpenEmbedded Build System Concepts</title>
+
+ <para>
+ This section takes a more detailed look inside the build
+ process used by the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>OpenEmbedded build system</ulink>,
+ which is the build system specific to the Yocto Project.
+ At the heart of the build system is BitBake, the task executor.
+ </para>
+
+ <para>
+ The following diagram represents the high-level workflow of a
+ build.
+ The remainder of this section expands on the fundamental input,
+ output, process, and metadata logical blocks that make up the
+ workflow.
+ </para>
+
+ <para id='general-workflow-figure'>
+ <imagedata fileref="figures/YP-flow-diagram.png" format="PNG" align='center' width="8in"/>
+ </para>
+
+ <para>
+ In general, the build's workflow consists of several functional
+ areas:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis>User Configuration:</emphasis>
+ metadata you can use to control the build process.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Metadata Layers:</emphasis>
+ Various layers that provide software, machine, and
+ distro metadata.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Source Files:</emphasis>
+ Upstream releases, local projects, and SCMs.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Build System:</emphasis>
+ Processes under the control of
+ <ulink url='&YOCTO_DOCS_REF_URL;#bitbake-term'>BitBake</ulink>.
+ This block expands on how BitBake fetches source, applies
+ patches, completes compilation, analyzes output for package
+ generation, creates and tests packages, generates images,
+ and generates cross-development tools.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Package Feeds:</emphasis>
+ Directories containing output packages (RPM, DEB or IPK),
+ which are subsequently used in the construction of an
+ image or Software Development Kit (SDK), produced by the
+ build system.
+ These feeds can also be copied and shared using a web
+ server or other means to facilitate extending or updating
+ existing images on devices at runtime if runtime package
+ management is enabled.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Images:</emphasis>
+ Images produced by the workflow.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Application Development SDK:</emphasis>
+ Cross-development tools that are produced along with
+ an image or separately with BitBake.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <section id="user-configuration">
+ <title>User Configuration</title>
+
+ <para>
+ User configuration helps define the build.
+ Through user configuration, you can tell BitBake the
+ target architecture for which you are building the image,
+ where to store downloaded source, and other build properties.
+ </para>
+
+ <para>
+ The following figure shows an expanded representation of the
+ "User Configuration" box of the
+ <link linkend='general-workflow-figure'>general workflow figure</link>:
+ </para>
+
+ <para>
+ <imagedata fileref="figures/user-configuration.png" align="center" width="8in" depth="4.5in" />
+ </para>
+
+ <para>
+ BitBake needs some basic configuration files in order to
+ complete a build.
+ These files are <filename>*.conf</filename> files.
+ The minimally necessary ones reside as example files in the
+ <filename>build/conf</filename> directory of the
+ <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>.
+ For simplicity, this section refers to the Source Directory as
+ the "Poky Directory."
+ </para>
+
+ <para>
+ When you clone the
+ <ulink url='&YOCTO_DOCS_REF_URL;#poky'>Poky</ulink>
+ Git repository or you download and unpack a Yocto Project
+ release, you can set up the Source Directory to be named
+ anything you want.
+ For this discussion, the cloned repository uses the default
+ name <filename>poky</filename>.
+ <note>
+ The Poky repository is primarily an aggregation of existing
+ repositories.
+ It is not a canonical upstream source.
+ </note>
+ </para>
+
+ <para>
+ The <filename>meta-poky</filename> layer inside Poky contains
+ a <filename>conf</filename> directory that has example
+ configuration files.
+ These example files are used as a basis for creating actual
+ configuration files when you source
+ <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>,
+ which is the build environment script.
+ </para>
+
+ <para>
+ Sourcing the build environment script creates a
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>
+ if one does not already exist.
+ BitBake uses the Build Directory for all its work during
+ builds.
+ The Build Directory has a <filename>conf</filename> directory
+ that contains default versions of your
+ <filename>local.conf</filename> and
+ <filename>bblayers.conf</filename> configuration files.
+ These default configuration files are created only if versions
+ do not already exist in the Build Directory at the time you
+ source the build environment setup script.
+ </para>
+
+ <para>
+ Because the Poky repository is fundamentally an aggregation of
+ existing repositories, some users might be familiar with
+ running the <filename>&OE_INIT_FILE;</filename> script
+ in the context of separate
+ <ulink url='&YOCTO_DOCS_REF_URL;#oe-core'>OpenEmbedded-Core</ulink>
+ and BitBake repositories rather than a single Poky repository.
+ This discussion assumes the script is executed from
+ within a cloned or unpacked version of Poky.
+ </para>
+
+ <para>
+ Depending on where the script is sourced, different
+ sub-scripts are called to set up the Build Directory
+ (Yocto or OpenEmbedded).
+ Specifically, the script
+ <filename>scripts/oe-setup-builddir</filename> inside the
+ poky directory sets up the Build Directory and seeds the
+ directory (if necessary) with configuration files appropriate
+ for the Yocto Project development environment.
+ <note>
+ The <filename>scripts/oe-setup-builddir</filename> script
+ uses the <filename>$TEMPLATECONF</filename> variable to
+ determine which sample configuration files to locate.
+ </note>
+ </para>
+
+ <para>
+ The <filename>local.conf</filename> file provides many
+ basic variables that define a build environment.
+ Here is a list of a few.
+ To see the default configurations in a
+ <filename>local.conf</filename> file created by the build
+ environment script, see the
+ <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta-poky/conf/local.conf.sample'><filename>local.conf.sample</filename></ulink>
+ in the <filename>meta-poky</filename> layer:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis>Target Machine Selection:</emphasis>
+ Controlled by the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
+ variable.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Download Directory:</emphasis>
+ Controlled by the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink>
+ variable.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Shared State Directory:</emphasis>
+ Controlled by the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink>
+ variable.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Build Output:</emphasis>
+ Controlled by the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>
+ variable.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Distribution Policy:</emphasis>
+ Controlled by the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink>
+ variable.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Packaging Format:</emphasis>
+ Controlled by the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink>
+ variable.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>SDK Target Architecture:</emphasis>
+ Controlled by the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SDKMACHINE'><filename>SDKMACHINE</filename></ulink>
+ variable.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Extra Image Packages:</emphasis>
+ Controlled by the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink>
+ variable.
+ </para></listitem>
+ </itemizedlist>
+ <note>
+ Configurations set in the
+ <filename>conf/local.conf</filename> file can also be set
+ in the <filename>conf/site.conf</filename> and
+ <filename>conf/auto.conf</filename> configuration files.
+ </note>
+ </para>
+
+ <para>
+ The <filename>bblayers.conf</filename> file tells BitBake what
+ layers you want considered during the build.
+ By default, the layers listed in this file include layers
+ minimally needed by the build system.
+ However, you must manually add any custom layers you have
+ created.
+ You can find more information on working with the
+ <filename>bblayers.conf</filename> file in the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#enabling-your-layer'>Enabling Your Layer</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para>
+
+ <para>
+ The files <filename>site.conf</filename> and
+ <filename>auto.conf</filename> are not created by the
+ environment initialization script.
+ If you want the <filename>site.conf</filename> file, you
+ need to create that yourself.
+ The <filename>auto.conf</filename> file is typically created by
+ an autobuilder:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis><filename>site.conf</filename>:</emphasis>
+ You can use the <filename>conf/site.conf</filename>
+ configuration file to configure multiple
+ build directories.
+ For example, suppose you had several build environments
+ and they shared some common features.
+ You can set these default build properties here.
+ A good example is perhaps the packaging format to use
+ through the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink>
+ variable.</para>
+
+ <para>One useful scenario for using the
+ <filename>conf/site.conf</filename> file is to extend
+ your
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>
+ variable to include the path to a
+ <filename>conf/site.conf</filename>.
+ Then, when BitBake looks for Metadata using
+ <filename>BBPATH</filename>, it finds the
+ <filename>conf/site.conf</filename> file and applies
+ your common configurations found in the file.
+ To override configurations in a particular build
+ directory, alter the similar configurations within
+ that build directory's
+ <filename>conf/local.conf</filename> file.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>auto.conf</filename>:</emphasis>
+ The file is usually created and written to by
+ an autobuilder.
+ The settings put into the file are typically the
+ same as you would find in the
+ <filename>conf/local.conf</filename> or the
+ <filename>conf/site.conf</filename> files.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ You can edit all configuration files to further define
+ any particular build environment.
+ This process is represented by the "User Configuration Edits"
+ box in the figure.
+ </para>
+
+ <para>
+ When you launch your build with the
+ <filename>bitbake <replaceable>target</replaceable></filename>
+ command, BitBake sorts out the configurations to ultimately
+ define your build environment.
+ It is important to understand that the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>OpenEmbedded build system</ulink>
+ reads the configuration files in a specific order:
+ <filename>site.conf</filename>, <filename>auto.conf</filename>,
+ and <filename>local.conf</filename>.
+ And, the build system applies the normal assignment statement
+ rules as described in the
+ "<ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata'>Syntax and Operators</ulink>"
+ chapter of the BitBake User Manual.
+ Because the files are parsed in a specific order, variable
+ assignments for the same variable could be affected.
+ For example, if the <filename>auto.conf</filename> file and
+ the <filename>local.conf</filename> set
+ <replaceable>variable1</replaceable> to different values,
+ because the build system parses <filename>local.conf</filename>
+ after <filename>auto.conf</filename>,
+ <replaceable>variable1</replaceable> is assigned the value from
+ the <filename>local.conf</filename> file.
+ </para>
+ </section>
+
+ <section id="metadata-machine-configuration-and-policy-configuration">
+ <title>Metadata, Machine Configuration, and Policy Configuration</title>
+
+ <para>
+ The previous section described the user configurations that
+ define BitBake's global behavior.
+ This section takes a closer look at the layers the build system
+ uses to further control the build.
+ These layers provide Metadata for the software, machine, and
+ policies.
+ </para>
+
+ <para>
+ In general, three types of layer input exists.
+ You can see them below the "User Configuration" box in the
+ <link linkend='general-workflow-figure'>general workflow figure</link>:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis>Metadata (<filename>.bb</filename> + Patches):</emphasis>
+ Software layers containing user-supplied recipe files,
+ patches, and append files.
+ A good example of a software layer might be the
+ <ulink url='https://github.com/meta-qt5/meta-qt5'><filename>meta-qt5</filename></ulink>
+ layer from the
+ <ulink url='http://layers.openembedded.org/layerindex/branch/master/layers/'>OpenEmbedded Layer Index</ulink>.
+ This layer is for version 5.0 of the popular
+ <ulink url='https://wiki.qt.io/About_Qt'>Qt</ulink>
+ cross-platform application development framework for
+ desktop, embedded and mobile.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Machine BSP Configuration:</emphasis>
+ Board Support Package (BSP) layers (i.e. "BSP Layer"
+ in the following figure) providing machine-specific
+ configurations.
+ This type of information is specific to a particular
+ target architecture.
+ A good example of a BSP layer from the
+ <link linkend='gs-reference-distribution-poky'>Poky Reference Distribution</link>
+ is the
+ <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta-yocto-bsp'><filename>meta-yocto-bsp</filename></ulink>
+ layer.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Policy Configuration:</emphasis>
+ Distribution Layers (i.e. "Distro Layer" in the
+ following figure) providing top-level or general
+ policies for the images or SDKs being built for a
+ particular distribution.
+ For example, in the Poky Reference Distribution the
+ distro layer is the
+ <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta-poky'><filename>meta-poky</filename></ulink>
+ layer.
+ Within the distro layer is a
+ <filename>conf/distro</filename> directory that
+ contains distro configuration files (e.g.
+ <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta-poky/conf/distro/poky.conf'><filename>poky.conf</filename></ulink>
+ that contain many policy configurations for the
+ Poky distribution.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ The following figure shows an expanded representation of
+ these three layers from the
+ <link linkend='general-workflow-figure'>general workflow figure</link>:
+ </para>
+
+ <para>
+ <imagedata fileref="figures/layer-input.png" align="center" width="8in" depth="8in" />
+ </para>
+
+ <para>
+ In general, all layers have a similar structure.
+ They all contain a licensing file
+ (e.g. <filename>COPYING.MIT</filename>) if the layer is to be
+ distributed, a <filename>README</filename> file as good
+ practice and especially if the layer is to be distributed, a
+ configuration directory, and recipe directories.
+ You can learn about the general structure for layers used with
+ the Yocto Project in the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#creating-your-own-layer'>Creating Your Own Layer</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ For a general discussion on layers and the many layers from
+ which you can draw, see the
+ "<link linkend='overview-layers'>Layers</link>" and
+ "<link linkend='the-yocto-project-layer-model'>The Yocto Project Layer Model</link>"
+ sections both earlier in this manual.
+ </para>
+
+ <para>
+ If you explored the previous links, you discovered some
+ areas where many layers that work with the Yocto Project
+ exist.
+ The
+ <ulink url="http://git.yoctoproject.org/">Source Repositories</ulink>
+ also shows layers categorized under "Yocto Metadata Layers."
+ <note>
+ Layers exist in the Yocto Project Source Repositories that
+ cannot be found in the OpenEmbedded Layer Index.
+ These layers are either deprecated or experimental
+ in nature.
+ </note>
+ </para>
+
+ <para>
+ BitBake uses the <filename>conf/bblayers.conf</filename> file,
+ which is part of the user configuration, to find what layers it
+ should be using as part of the build.
+ </para>
+
+ <section id="distro-layer">
+ <title>Distro Layer</title>
+
+ <para>
+ The distribution layer provides policy configurations for
+ your distribution.
+ Best practices dictate that you isolate these types of
+ configurations into their own layer.
+ Settings you provide in
+ <filename>conf/distro/<replaceable>distro</replaceable>.conf</filename> override
+ similar settings that BitBake finds in your
+ <filename>conf/local.conf</filename> file in the Build
+ Directory.
+ </para>
+
+ <para>
+ The following list provides some explanation and references
+ for what you typically find in the distribution layer:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis>classes:</emphasis>
+ Class files (<filename>.bbclass</filename>) hold
+ common functionality that can be shared among
+ recipes in the distribution.
+ When your recipes inherit a class, they take on the
+ settings and functions for that class.
+ You can read more about class files in the
+ "<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes'>Classes</ulink>"
+ chapter of the Yocto Reference Manual.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>conf:</emphasis>
+ This area holds configuration files for the
+ layer (<filename>conf/layer.conf</filename>),
+ the distribution
+ (<filename>conf/distro/<replaceable>distro</replaceable>.conf</filename>),
+ and any distribution-wide include files.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>recipes-*:</emphasis>
+ Recipes and append files that affect common
+ functionality across the distribution.
+ This area could include recipes and append files
+ to add distribution-specific configuration,
+ initialization scripts, custom image recipes,
+ and so forth.
+ Examples of <filename>recipes-*</filename>
+ directories are <filename>recipes-core</filename>
+ and <filename>recipes-extra</filename>.
+ Hierarchy and contents within a
+ <filename>recipes-*</filename> directory can vary.
+ Generally, these directories contain recipe files
+ (<filename>*.bb</filename>), recipe append files
+ (<filename>*.bbappend</filename>), directories
+ that are distro-specific for configuration files,
+ and so forth.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </section>
+
+ <section id="bsp-layer">
+ <title>BSP Layer</title>
+
+ <para>
+ The BSP Layer provides machine configurations that
+ target specific hardware.
+ Everything in this layer is specific to the machine for
+ which you are building the image or the SDK.
+ A common structure or form is defined for BSP layers.
+ You can learn more about this structure in the
+ <ulink url='&YOCTO_DOCS_BSP_URL;'>Yocto Project Board Support Package (BSP) Developer's Guide</ulink>.
+ <note>
+ In order for a BSP layer to be considered compliant
+ with the Yocto Project, it must meet some structural
+ requirements.
+ </note>
+ </para>
+
+ <para>
+ The BSP Layer's configuration directory contains
+ configuration files for the machine
+ (<filename>conf/machine/<replaceable>machine</replaceable>.conf</filename>)
+ and, of course, the layer
+ (<filename>conf/layer.conf</filename>).
+ </para>
+
+ <para>
+ The remainder of the layer is dedicated to specific recipes
+ by function: <filename>recipes-bsp</filename>,
+ <filename>recipes-core</filename>,
+ <filename>recipes-graphics</filename>,
+ <filename>recipes-kernel</filename>, and so forth.
+ Metadata can exist for multiple formfactors, graphics
+ support systems, and so forth.
+ <note>
+ While the figure shows several
+ <filename>recipes-*</filename> directories, not all
+ these directories appear in all BSP layers.
+ </note>
+ </para>
+ </section>
+
+ <section id="software-layer">
+ <title>Software Layer</title>
+
+ <para>
+ The software layer provides the Metadata for additional
+ software packages used during the build.
+ This layer does not include Metadata that is specific to
+ the distribution or the machine, which are found in their
+ respective layers.
+ </para>
+
+ <para>
+ This layer contains any recipes, append files, and
+ patches, that your project needs.
+ </para>
+ </section>
+ </section>
+
+ <section id="sources-dev-environment">
+ <title>Sources</title>
+
+ <para>
+ In order for the OpenEmbedded build system to create an
+ image or any target, it must be able to access source files.
+ The
+ <link linkend='general-workflow-figure'>general workflow figure</link>
+ represents source files using the "Upstream Project Releases",
+ "Local Projects", and "SCMs (optional)" boxes.
+ The figure represents mirrors, which also play a role in
+ locating source files, with the "Source Materials" box.
+ </para>
+
+ <para>
+ The method by which source files are ultimately organized is
+ a function of the project.
+ For example, for released software, projects tend to use
+ tarballs or other archived files that can capture the
+ state of a release guaranteeing that it is statically
+ represented.
+ On the other hand, for a project that is more dynamic or
+ experimental in nature, a project might keep source files in a
+ repository controlled by a Source Control Manager (SCM) such as
+ Git.
+ Pulling source from a repository allows you to control
+ the point in the repository (the revision) from which you
+ want to build software.
+ Finally, a combination of the two might exist, which would
+ give the consumer a choice when deciding where to get
+ source files.
+ </para>
+
+ <para>
+ BitBake uses the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
+ variable to point to source files regardless of their location.
+ Each recipe must have a <filename>SRC_URI</filename> variable
+ that points to the source.
+ </para>
+
+ <para>
+ Another area that plays a significant role in where source
+ files come from is pointed to by the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink>
+ variable.
+ This area is a cache that can hold previously downloaded
+ source.
+ You can also instruct the OpenEmbedded build system to create
+ tarballs from Git repositories, which is not the default
+ behavior, and store them in the <filename>DL_DIR</filename>
+ by using the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></ulink>
+ variable.
+ </para>
+
+ <para>
+ Judicious use of a <filename>DL_DIR</filename> directory can
+ save the build system a trip across the Internet when looking
+ for files.
+ A good method for using a download directory is to have
+ <filename>DL_DIR</filename> point to an area outside of your
+ Build Directory.
+ Doing so allows you to safely delete the Build Directory
+ if needed without fear of removing any downloaded source file.
+ </para>
+
+ <para>
+ The remainder of this section provides a deeper look into the
+ source files and the mirrors.
+ Here is a more detailed look at the source file area of the
+ <link linkend='general-workflow-figure'>general workflow figure</link>:
+ </para>
+
+ <para>
+ <imagedata fileref="figures/source-input.png" width="6in" depth="6in" align="center" />
+ </para>
+
+ <section id='upstream-project-releases'>
+ <title>Upstream Project Releases</title>
+
+ <para>
+ Upstream project releases exist anywhere in the form of an
+ archived file (e.g. tarball or zip file).
+ These files correspond to individual recipes.
+ For example, the figure uses specific releases each for
+ BusyBox, Qt, and Dbus.
+ An archive file can be for any released product that can be
+ built using a recipe.
+ </para>
+ </section>
+
+ <section id='local-projects'>
+ <title>Local Projects</title>
+
+ <para>
+ Local projects are custom bits of software the user
+ provides.
+ These bits reside somewhere local to a project - perhaps
+ a directory into which the user checks in items (e.g.
+ a local directory containing a development source tree
+ used by the group).
+ </para>
+
+ <para>
+ The canonical method through which to include a local
+ project is to use the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-externalsrc'><filename>externalsrc</filename></ulink>
+ class to include that local project.
+ You use either the <filename>local.conf</filename> or a
+ recipe's append file to override or set the
+ recipe to point to the local directory on your disk to pull
+ in the whole source tree.
+ </para>
+ </section>
+
+ <section id='scms'>
+ <title>Source Control Managers (Optional)</title>
+
+ <para>
+ Another place from which the build system can get source
+ files is with
+ <ulink url='&YOCTO_DOCS_BB_URL;#bb-fetchers'>fetchers</ulink>
+ employing various Source Control Managers (SCMs) such as
+ Git or Subversion.
+ In such cases, a repository is cloned or checked out.
+ The
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-fetch'><filename>do_fetch</filename></ulink>
+ task inside BitBake uses
+ the <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
+ variable and the argument's prefix to determine the correct
+ fetcher module.
+ <note>
+ For information on how to have the OpenEmbedded build
+ system generate tarballs for Git repositories and place
+ them in the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink>
+ directory, see the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></ulink>
+ variable in the Yocto Project Reference Manual.
+ </note>
+ </para>
+
+ <para>
+ When fetching a repository, BitBake uses the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink>
+ variable to determine the specific revision from which to
+ build.
+ </para>
+ </section>
+
+ <section id='source-mirrors'>
+ <title>Source Mirror(s)</title>
+
+ <para>
+ Two kinds of mirrors exist: pre-mirrors and regular
+ mirrors.
+ The
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PREMIRRORS'><filename>PREMIRRORS</filename></ulink>
+ and
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-MIRRORS'><filename>MIRRORS</filename></ulink>
+ variables point to these, respectively.
+ BitBake checks pre-mirrors before looking upstream for any
+ source files.
+ Pre-mirrors are appropriate when you have a shared
+ directory that is not a directory defined by the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink>
+ variable.
+ A Pre-mirror typically points to a shared directory that is
+ local to your organization.
+ </para>
+
+ <para>
+ Regular mirrors can be any site across the Internet
+ that is used as an alternative location for source
+ code should the primary site not be functioning for
+ some reason or another.
+ </para>
+ </section>
+ </section>
+
+ <section id="package-feeds-dev-environment">
+ <title>Package Feeds</title>
+
+ <para>
+ When the OpenEmbedded build system generates an image or an
+ SDK, it gets the packages from a package feed area located
+ in the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>.
+ The
+ <link linkend='general-workflow-figure'>general workflow figure</link>
+ shows this package feeds area in the upper-right corner.
+ </para>
+
+ <para>
+ This section looks a little closer into the package feeds
+ area used by the build system.
+ Here is a more detailed look at the area:
+ <imagedata fileref="figures/package-feeds.png" align="center" width="7in" depth="6in" />
+ </para>
+
+ <para>
+ Package feeds are an intermediary step in the build process.
+ The OpenEmbedded build system provides classes to generate
+ different package types, and you specify which classes to
+ enable through the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink>
+ variable.
+ Before placing the packages into package feeds,
+ the build process validates them with generated output quality
+ assurance checks through the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-insane'><filename>insane</filename></ulink>
+ class.
+ </para>
+
+ <para>
+ The package feed area resides in the Build Directory.
+ The directory the build system uses to temporarily store
+ packages is determined by a combination of variables and the
+ particular package manager in use.
+ See the "Package Feeds" box in the illustration and note the
+ information to the right of that area.
+ In particular, the following defines where package files are
+ kept:
+ <itemizedlist>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></ulink>:
+ Defined as <filename>tmp/deploy</filename> in the Build
+ Directory.
+ </para></listitem>
+ <listitem><para>
+ <filename>DEPLOY_DIR_*</filename>:
+ Depending on the package manager used, the package type
+ sub-folder.
+ Given RPM, IPK, or DEB packaging and tarball creation,
+ the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR_RPM'><filename>DEPLOY_DIR_RPM</filename></ulink>,
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR_IPK'><filename>DEPLOY_DIR_IPK</filename></ulink>,
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR_DEB'><filename>DEPLOY_DIR_DEB</filename></ulink>,
+ or
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR_TAR'><filename>DEPLOY_DIR_TAR</filename></ulink>,
+ variables are used, respectively.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink>:
+ Defines architecture-specific sub-folders.
+ For example, packages could exist for the i586 or
+ qemux86 architectures.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ BitBake uses the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_deb'><filename>do_package_write_*</filename></ulink>
+ tasks to generate packages and place them into the package
+ holding area (e.g. <filename>do_package_write_ipk</filename>
+ for IPK packages).
+ See the
+ "<ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_deb'><filename>do_package_write_deb</filename></ulink>",
+ "<ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_ipk'><filename>do_package_write_ipk</filename></ulink>",
+ "<ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_rpm'><filename>do_package_write_rpm</filename></ulink>",
+ and
+ "<ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_tar'><filename>do_package_write_tar</filename></ulink>"
+ sections in the Yocto Project Reference Manual
+ for additional information.
+ As an example, consider a scenario where an IPK packaging
+ manager is being used and package architecture support for
+ both i586 and qemux86 exist.
+ Packages for the i586 architecture are placed in
+ <filename>build/tmp/deploy/ipk/i586</filename>, while packages
+ for the qemux86 architecture are placed in
+ <filename>build/tmp/deploy/ipk/qemux86</filename>.
+ </para>
+ </section>
+
+ <section id='bitbake-dev-environment'>
+ <title>BitBake</title>
+
+ <para>
+ The OpenEmbedded build system uses
+ <ulink url='&YOCTO_DOCS_REF_URL;#bitbake-term'>BitBake</ulink>
+ to produce images and Software Development Kits (SDKs).
+ You can see from the
+ <link linkend='general-workflow-figure'>general workflow figure</link>,
+ the BitBake area consists of several functional areas.
+ This section takes a closer look at each of those areas.
+ <note>
+ Separate documentation exists for the BitBake tool.
+ See the
+ <ulink url='&YOCTO_DOCS_BB_URL;'>BitBake User Manual</ulink>
+ for reference material on BitBake.
+ </note>
+ </para>
+
+ <section id='source-fetching-dev-environment'>
+ <title>Source Fetching</title>
+
+ <para>
+ The first stages of building a recipe are to fetch and
+ unpack the source code:
+ <imagedata fileref="figures/source-fetching.png" align="center" width="6.5in" depth="5in" />
+ </para>
+
+ <para>
+ The
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-fetch'><filename>do_fetch</filename></ulink>
+ and
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-unpack'><filename>do_unpack</filename></ulink>
+ tasks fetch the source files and unpack them into the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>.
+ <note>
+ For every local file (e.g. <filename>file://</filename>)
+ that is part of a recipe's
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
+ statement, the OpenEmbedded build system takes a
+ checksum of the file for the recipe and inserts the
+ checksum into the signature for the
+ <filename>do_fetch</filename> task.
+ If any local file has been modified, the
+ <filename>do_fetch</filename> task and all tasks that
+ depend on it are re-executed.
+ </note>
+ By default, everything is accomplished in the Build
+ Directory, which has a defined structure.
+ For additional general information on the Build Directory,
+ see the
+ "<ulink url='&YOCTO_DOCS_REF_URL;#structure-core-build'><filename>build/</filename></ulink>"
+ section in the Yocto Project Reference Manual.
+ </para>
+
+ <para>
+ Each recipe has an area in the Build Directory where the
+ unpacked source code resides.
+ The
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink>
+ variable points to this area for a recipe's unpacked source
+ code.
+ The name of that directory for any given recipe is defined
+ from several different variables.
+ The preceding figure and the following list describe
+ the Build Directory's hierarchy:
+ <itemizedlist>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>:
+ The base directory where the OpenEmbedded build
+ system performs all its work during the build.
+ The default base directory is the
+ <filename>tmp</filename> directory.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink>:
+ The architecture of the built package or packages.
+ Depending on the eventual destination of the
+ package or packages (i.e. machine architecture,
+ <ulink url='&YOCTO_DOCS_REF_URL;#hardware-build-system-term'>build host</ulink>,
+ SDK, or specific machine),
+ <filename>PACKAGE_ARCH</filename> varies.
+ See the variable's description for details.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-TARGET_OS'><filename>TARGET_OS</filename></ulink>:
+ The operating system of the target device.
+ A typical value would be "linux" (e.g.
+ "qemux86-poky-linux").
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink>:
+ The name of the recipe used to build the package.
+ This variable can have multiple meanings.
+ However, when used in the context of input files,
+ <filename>PN</filename> represents the the name
+ of the recipe.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>:
+ The location where the OpenEmbedded build system
+ builds a recipe (i.e. does the work to create the
+ package).
+ <itemizedlist>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>:
+ The version of the recipe used to build the
+ package.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>:
+ The revision of the recipe used to build the
+ package.
+ </para></listitem>
+ </itemizedlist>
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink>:
+ Contains the unpacked source files for a given
+ recipe.
+ <itemizedlist>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-BPN'><filename>BPN</filename></ulink>:
+ The name of the recipe used to build the
+ package.
+ The <filename>BPN</filename> variable is
+ a version of the <filename>PN</filename>
+ variable but with common prefixes and
+ suffixes removed.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>:
+ The version of the recipe used to build the
+ package.
+ </para></listitem>
+ </itemizedlist>
+ </para></listitem>
+ </itemizedlist>
+ <note>
+ In the previous figure, notice that two sample
+ hierarchies exist: one based on package architecture (i.e.
+ <filename>PACKAGE_ARCH</filename>) and one based on a
+ machine (i.e. <filename>MACHINE</filename>).
+ The underlying structures are identical.
+ The differentiator being what the OpenEmbedded build
+ system is using as a build target (e.g. general
+ architecture, a build host, an SDK, or a specific
+ machine).
+ </note>
+ </para>
+ </section>
+
+ <section id='patching-dev-environment'>
+ <title>Patching</title>
+
+ <para>
+ Once source code is fetched and unpacked, BitBake locates
+ patch files and applies them to the source files:
+ <imagedata fileref="figures/patching.png" align="center" width="7in" depth="6in" />
+ </para>
+
+ <para>
+ The
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink>
+ task uses a recipe's
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
+ statements and the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink>
+ variable to locate applicable patch files.
+ </para>
+
+ <para>
+ Default processing for patch files assumes the files have
+ either <filename>*.patch</filename> or
+ <filename>*.diff</filename> file types.
+ You can use <filename>SRC_URI</filename> parameters to
+ change the way the build system recognizes patch files.
+ See the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink>
+ task for more information.
+ </para>
+
+ <para>
+ BitBake finds and applies multiple patches for a single
+ recipe in the order in which it locates the patches.
+ The <filename>FILESPATH</filename> variable defines the
+ default set of directories that the build system uses to
+ search for patch files.
+ Once found, patches are applied to the recipe's source
+ files, which are located in the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink>
+ directory.
+ </para>
+
+ <para>
+ For more information on how the source directories are
+ created, see the
+ "<link linkend='source-fetching-dev-environment'>Source Fetching</link>"
+ section.
+ For more information on how to create patches and how the
+ build system processes patches, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#new-recipe-patching-code'>Patching Code</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ You can also see the
+ "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-devtool-use-devtool-modify-to-modify-the-source-of-an-existing-component'>Use <filename>devtool modify</filename> to Modify the Source of an Existing Component</ulink>"
+ section in the Yocto Project Application Development and
+ the Extensible Software Development Kit (SDK) manual and
+ the
+ "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#using-traditional-kernel-development-to-patch-the-kernel'>Using Traditional Kernel Development to Patch the Kernel</ulink>"
+ section in the Yocto Project Linux Kernel Development
+ Manual.
+ </para>
+ </section>
+
+ <section id='configuration-compilation-and-staging-dev-environment'>
+ <title>Configuration, Compilation, and Staging</title>
+
+ <para>
+ After source code is patched, BitBake executes tasks that
+ configure and compile the source code.
+ Once compilation occurs, the files are copied to a holding
+ area (staged) in preparation for packaging:
+ <imagedata fileref="figures/configuration-compile-autoreconf.png" align="center" width="7in" depth="5in" />
+ </para>
+
+ <para>
+ This step in the build process consists of the following
+ tasks:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis><ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-prepare_recipe_sysroot'><filename>do_prepare_recipe_sysroot</filename></ulink></emphasis>:
+ This task sets up the two sysroots in
+ <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}</filename>
+ (i.e. <filename>recipe-sysroot</filename> and
+ <filename>recipe-sysroot-native</filename>) so that
+ during the packaging phase the sysroots can contain
+ the contents of the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></ulink>
+ tasks of the recipes on which the recipe
+ containing the tasks depends.
+ A sysroot exists for both the target and for the
+ native binaries, which run on the host system.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>do_configure</filename></emphasis>:
+ This task configures the source by enabling and
+ disabling any build-time and configuration options
+ for the software being built.
+ Configurations can come from the recipe itself as
+ well as from an inherited class.
+ Additionally, the software itself might configure
+ itself depending on the target for which it is
+ being built.</para>
+
+ <para>The configurations handled by the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink>
+ task are specific to configurations for the source
+ code being built by the recipe.</para>
+
+ <para>If you are using the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-autotools'><filename>autotools</filename></ulink>
+ class, you can add additional configuration options
+ by using the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OECONF'><filename>EXTRA_OECONF</filename></ulink>
+ or
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG_CONFARGS'><filename>PACKAGECONFIG_CONFARGS</filename></ulink>
+ variables.
+ For information on how this variable works within
+ that class, see the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-autotools'><filename>autotools</filename></ulink>
+ class
+ <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta/classes/autotools.bbclass'>here</ulink>.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>do_compile</filename></emphasis>:
+ Once a configuration task has been satisfied,
+ BitBake compiles the source using the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink>
+ task.
+ Compilation occurs in the directory pointed to by
+ the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-B'><filename>B</filename></ulink>
+ variable.
+ Realize that the <filename>B</filename> directory
+ is, by default, the same as the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink>
+ directory.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>do_install</filename></emphasis>:
+ After compilation completes, BitBake executes the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>
+ task.
+ This task copies files from the
+ <filename>B</filename> directory and places them
+ in a holding area pointed to by the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink>
+ variable.
+ Packaging occurs later using files from this
+ holding directory.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </section>
+
+ <section id='package-splitting-dev-environment'>
+ <title>Package Splitting</title>
+
+ <para>
+ After source code is configured, compiled, and staged, the
+ build system analyzes the results and splits the output
+ into packages:
+ <imagedata fileref="figures/analysis-for-package-splitting.png" align="center" width="7in" depth="7in" />
+ </para>
+
+ <para>
+ The
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package'><filename>do_package</filename></ulink>
+ and
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-packagedata'><filename>do_packagedata</filename></ulink>
+ tasks combine to analyze the files found in the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink>
+ directory and split them into subsets based on available
+ packages and files.
+ Analysis involves the following as well as other items:
+ splitting out debugging symbols, looking at shared library
+ dependencies between packages, and looking at package
+ relationships.
+ </para>
+
+ <para>
+ The <filename>do_packagedata</filename> task creates
+ package metadata based on the analysis such that the
+ build system can generate the final packages.
+ The
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></ulink>
+ task stages (copies) a subset of the files installed by
+ the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>
+ task into the appropriate sysroot.
+ Working, staged, and intermediate results of the analysis
+ and package splitting process use several areas:
+ <itemizedlist>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PKGD'><filename>PKGD</filename></ulink>:
+ The destination directory
+ (i.e. <filename>package</filename>) for packages
+ before they are split into individual packages.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PKGDESTWORK'><filename>PKGDESTWORK</filename></ulink>:
+ A temporary work area (i.e.
+ <filename>pkgdata</filename>) used by the
+ <filename>do_package</filename> task to save
+ package metadata.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PKGDEST'><filename>PKGDEST</filename></ulink>:
+ The parent directory (i.e.
+ <filename>packages-split</filename>) for packages
+ after they have been split.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></ulink>:
+ A shared, global-state directory that holds
+ packaging metadata generated during the packaging
+ process.
+ The packaging process copies metadata from
+ <filename>PKGDESTWORK</filename> to the
+ <filename>PKGDATA_DIR</filename> area where it
+ becomes globally available.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_DIR_HOST'><filename>STAGING_DIR_HOST</filename></ulink>:
+ The path for the sysroot for the system on which
+ a component is built to run (i.e.
+ <filename>recipe-sysroot</filename>).
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_DIR_NATIVE'><filename>STAGING_DIR_NATIVE</filename></ulink>:
+ The path for the sysroot used when building
+ components for the build host (i.e.
+ <filename>recipe-sysroot-native</filename>).
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_DIR_TARGET'><filename>STAGING_DIR_TARGET</filename></ulink>:
+ The path for the sysroot used when a component that
+ is built to execute on a system and it generates
+ code for yet another machine (e.g. cross-canadian
+ recipes).
+ </para></listitem>
+ </itemizedlist>
+ The
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink>
+ variable defines the files that go into each package in
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>.
+ If you want details on how this is accomplished, you can
+ look at
+ <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta/classes/package.bbclass'><filename>package.bbclass</filename></ulink>.
+ </para>
+
+ <para>
+ Depending on the type of packages being created (RPM, DEB,
+ or IPK), the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_deb'><filename>do_package_write_*</filename></ulink>
+ task creates the actual packages and places them in the
+ Package Feed area, which is
+ <filename>${TMPDIR}/deploy</filename>.
+ You can see the
+ "<link linkend='package-feeds-dev-environment'>Package Feeds</link>"
+ section for more detail on that part of the build process.
+ <note>
+ Support for creating feeds directly from the
+ <filename>deploy/*</filename> directories does not
+ exist.
+ Creating such feeds usually requires some kind of feed
+ maintenance mechanism that would upload the new
+ packages into an official package feed (e.g. the
+ Ångström distribution).
+ This functionality is highly distribution-specific
+ and thus is not provided out of the box.
+ </note>
+ </para>
+ </section>
+
+ <section id='image-generation-dev-environment'>
+ <title>Image Generation</title>
+
+ <para>
+ Once packages are split and stored in the Package Feeds
+ area, the build system uses BitBake to generate the root
+ filesystem image:
+ <imagedata fileref="figures/image-generation.png" align="center" width="7.5in" depth="7.5in" />
+ </para>
+
+ <para>
+ The image generation process consists of several stages and
+ depends on several tasks and variables.
+ The
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-rootfs'><filename>do_rootfs</filename></ulink>
+ task creates the root filesystem (file and directory
+ structure) for an image.
+ This task uses several key variables to help create the
+ list of packages to actually install:
+ <itemizedlist>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink>:
+ Lists out the base set of packages from which to
+ install from the Package Feeds area.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_EXCLUDE'><filename>PACKAGE_EXCLUDE</filename></ulink>:
+ Specifies packages that should not be installed
+ into the image.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>:
+ Specifies features to include in the image.
+ Most of these features map to additional packages
+ for installation.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink>:
+ Specifies the package backend (e.g. RPM, DEB, or
+ IPK) to use and consequently helps determine where
+ to locate packages within the Package Feeds area.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_LINGUAS'><filename>IMAGE_LINGUAS</filename></ulink>:
+ Determines the language(s) for which additional
+ language support packages are installed.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_INSTALL'><filename>PACKAGE_INSTALL</filename></ulink>:
+ The final list of packages passed to the package
+ manager for installation into the image.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ With
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_ROOTFS'><filename>IMAGE_ROOTFS</filename></ulink>
+ pointing to the location of the filesystem under
+ construction and the <filename>PACKAGE_INSTALL</filename>
+ variable providing the final list of packages to install,
+ the root file system is created.
+ </para>
+
+ <para>
+ Package installation is under control of the package
+ manager (e.g. dnf/rpm, opkg, or apt/dpkg) regardless of
+ whether or not package management is enabled for the
+ target.
+ At the end of the process, if package management is not
+ enabled for the target, the package manager's data files
+ are deleted from the root filesystem.
+ As part of the final stage of package installation,
+ post installation scripts that are part of the packages
+ are run.
+ Any scripts that fail to run on the build host are run on
+ the target when the target system is first booted.
+ If you are using a
+ <ulink url='&YOCTO_DOCS_DEV_URL;#creating-a-read-only-root-filesystem'>read-only root filesystem</ulink>,
+ all the post installation scripts must succeed on the
+ build host during the package installation phase since the
+ root filesystem on the target is read-only.
+ </para>
+
+ <para>
+ The final stages of the <filename>do_rootfs</filename> task
+ handle post processing.
+ Post processing includes creation of a manifest file and
+ optimizations.
+ </para>
+
+ <para>
+ The manifest file (<filename>.manifest</filename>) resides
+ in the same directory as the root filesystem image.
+ This file lists out, line-by-line, the installed packages.
+ The manifest file is useful for the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage*'><filename>testimage</filename></ulink>
+ class, for example, to determine whether or not to run
+ specific tests.
+ See the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_MANIFEST'><filename>IMAGE_MANIFEST</filename></ulink>
+ variable for additional information.
+ </para>
+
+ <para>
+ Optimizing processes that are run across the image include
+ <filename>mklibs</filename>, <filename>prelink</filename>,
+ and any other post-processing commands as defined by the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-ROOTFS_POSTPROCESS_COMMAND'><filename>ROOTFS_POSTPROCESS_COMMAND</filename></ulink>
+ variable.
+ The <filename>mklibs</filename> process optimizes the size
+ of the libraries, while the <filename>prelink</filename>
+ process optimizes the dynamic linking of shared libraries
+ to reduce start up time of executables.
+ </para>
+
+ <para>
+ After the root filesystem is built, processing begins on
+ the image through the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-image'><filename>do_image</filename></ulink>
+ task.
+ The build system runs any pre-processing commands as
+ defined by the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_PREPROCESS_COMMAND'><filename>IMAGE_PREPROCESS_COMMAND</filename></ulink>
+ variable.
+ This variable specifies a list of functions to call before
+ the build system creates the final image output files.
+ </para>
+
+ <para>
+ The build system dynamically creates
+ <filename>do_image_*</filename> tasks as needed, based
+ on the image types specified in the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></ulink>
+ variable.
+ The process turns everything into an image file or a set of
+ image files and can compress the root filesystem image to
+ reduce the overall size of the image.
+ The formats used for the root filesystem depend on the
+ <filename>IMAGE_FSTYPES</filename> variable.
+ Compression depends on whether the formats support
+ compression.
+ </para>
+
+ <para>
+ As an example, a dynamically created task when creating a
+ particular image <replaceable>type</replaceable> would
+ take the following form:
+ <literallayout class='monospaced'>
+ do_image_<replaceable>type</replaceable>
+ </literallayout>
+ So, if the <replaceable>type</replaceable> as specified by
+ the <filename>IMAGE_FSTYPES</filename> were
+ <filename>ext4</filename>, the dynamically generated task
+ would be as follows:
+ <literallayout class='monospaced'>
+ do_image_ext4
+ </literallayout>
+ </para>
+
+ <para>
+ The final task involved in image creation is the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-image-complete'><filename>do_image_complete</filename></ulink>
+ task.
+ This task completes the image by applying any image
+ post processing as defined through the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_POSTPROCESS_COMMAND'><filename>IMAGE_POSTPROCESS_COMMAND</filename></ulink>
+ variable.
+ The variable specifies a list of functions to call once the
+ build system has created the final image output files.
+ <note>
+ The entire image generation process is run under
+ <link linkend='fakeroot-and-pseudo'>Pseudo</link>.
+ Running under Pseudo ensures that the files in the
+ root filesystem have correct ownership.
+ </note>
+ </para>
+ </section>
+
+ <section id='sdk-generation-dev-environment'>
+ <title>SDK Generation</title>
+
+ <para>
+ The OpenEmbedded build system uses BitBake to generate the
+ Software Development Kit (SDK) installer scripts for both
+ the standard SDK and the extensible SDK (eSDK):
+ </para>
+
+ <para>
+ <imagedata fileref="figures/sdk-generation.png" width="9in" align="center" />
+ <note>
+ For more information on the cross-development toolchain
+ generation, see the
+ "<link linkend='cross-development-toolchain-generation'>Cross-Development Toolchain Generation</link>"
+ section.
+ For information on advantages gained when building a
+ cross-development toolchain using the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sdk'><filename>do_populate_sdk</filename></ulink>
+ task, see the
+ "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-building-an-sdk-installer'>Building an SDK Installer</ulink>"
+ section in the Yocto Project Application Development
+ and the Extensible Software Development Kit (eSDK)
+ manual.
+ </note>
+ </para>
+
+ <para>
+ Like image generation, the SDK script process consists of
+ several stages and depends on many variables.
+ The
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sdk'><filename>do_populate_sdk</filename></ulink>
+ and
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sdk_ext'><filename>do_populate_sdk_ext</filename></ulink>
+ tasks use these key variables to help create the list of
+ packages to actually install.
+ For information on the variables listed in the figure,
+ see the
+ "<link linkend='sdk-dev-environment'>Application Development SDK</link>"
+ section.
+ </para>
+
+ <para>
+ The <filename>do_populate_sdk</filename> task helps create
+ the standard SDK and handles two parts: a target part and a
+ host part.
+ The target part is the part built for the target hardware
+ and includes libraries and headers.
+ The host part is the part of the SDK that runs on the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SDKMACHINE'><filename>SDKMACHINE</filename></ulink>.
+ </para>
+
+ <para>
+ The <filename>do_populate_sdk_ext</filename> task helps
+ create the extensible SDK and handles host and target parts
+ differently than its counter part does for the standard SDK.
+ For the extensible SDK, the task encapsulates the build
+ system, which includes everything needed (host and target)
+ for the SDK.
+ </para>
+
+ <para>
+ Regardless of the type of SDK being constructed, the
+ tasks perform some cleanup after which a cross-development
+ environment setup script and any needed configuration files
+ are created.
+ The final output is the Cross-development
+ toolchain installation script (<filename>.sh</filename>
+ file), which includes the environment setup script.
+ </para>
+ </section>
+
+ <section id='stamp-files-and-the-rerunning-of-tasks'>
+ <title>Stamp Files and the Rerunning of Tasks</title>
+
+ <para>
+ For each task that completes successfully, BitBake writes a
+ stamp file into the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-STAMPS_DIR'><filename>STAMPS_DIR</filename></ulink>
+ directory.
+ The beginning of the stamp file's filename is determined
+ by the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-STAMP'><filename>STAMP</filename></ulink>
+ variable, and the end of the name consists of the task's
+ name and current
+ <link linkend='overview-checksums'>input checksum</link>.
+ <note>
+ This naming scheme assumes that
+ <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_SIGNATURE_HANDLER'><filename>BB_SIGNATURE_HANDLER</filename></ulink>
+ is "OEBasicHash", which is almost always the case in
+ current OpenEmbedded.
+ </note>
+ To determine if a task needs to be rerun, BitBake checks
+ if a stamp file with a matching input checksum exists
+ for the task.
+ If such a stamp file exists, the task's output is
+ assumed to exist and still be valid.
+ If the file does not exist, the task is rerun.
+ <note>
+ <para>The stamp mechanism is more general than the
+ shared state (sstate) cache mechanism described in the
+ "<link linkend='setscene-tasks-and-shared-state'>Setscene Tasks and Shared State</link>"
+ section.
+ BitBake avoids rerunning any task that has a valid
+ stamp file, not just tasks that can be accelerated
+ through the sstate cache.</para>
+
+ <para>However, you should realize that stamp files only
+ serve as a marker that some work has been done and that
+ these files do not record task output.
+ The actual task output would usually be somewhere in
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>
+ (e.g. in some recipe's
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>.)
+ What the sstate cache mechanism adds is a way to cache
+ task output that can then be shared between build
+ machines.</para>
+ </note>
+ Since <filename>STAMPS_DIR</filename> is usually a
+ subdirectory of <filename>TMPDIR</filename>, removing
+ <filename>TMPDIR</filename> will also remove
+ <filename>STAMPS_DIR</filename>, which means tasks will
+ properly be rerun to repopulate
+ <filename>TMPDIR</filename>.
+ </para>
+
+ <para>
+ If you want some task to always be considered "out of
+ date", you can mark it with the
+ <ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'><filename>nostamp</filename></ulink>
+ varflag.
+ If some other task depends on such a task, then that
+ task will also always be considered out of date, which
+ might not be what you want.
+ </para>
+
+ <para>
+ For details on how to view information about a task's
+ signature, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#dev-viewing-task-variable-dependencies'>Viewing Task Variable Dependencies</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para>
+ </section>
+
+ <section id='setscene-tasks-and-shared-state'>
+ <title>Setscene Tasks and Shared State</title>
+
+ <para>
+ The description of tasks so far assumes that BitBake needs
+ to build everything and no available prebuilt objects
+ exist.
+ BitBake does support skipping tasks if prebuilt objects are
+ available.
+ These objects are usually made available in the form of a
+ shared state (sstate) cache.
+ <note>
+ For information on variables affecting sstate, see the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink>
+ and
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_MIRRORS'><filename>SSTATE_MIRRORS</filename></ulink>
+ variables.
+ </note>
+ </para>
+
+ <para>
+ The idea of a setscene task (i.e
+ <filename>do_</filename><replaceable>taskname</replaceable><filename>_setscene</filename>)
+ is a version of the task where
+ instead of building something, BitBake can skip to the end
+ result and simply place a set of files into specific
+ locations as needed.
+ In some cases, it makes sense to have a setscene task
+ variant (e.g. generating package files in the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_deb'><filename>do_package_write_*</filename></ulink>
+ task).
+ In other cases, it does not make sense (e.g. a
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink>
+ task or a
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-unpack'><filename>do_unpack</filename></ulink>
+ task) since the work involved would be equal to or greater
+ than the underlying task.
+ </para>
+
+ <para>
+ In the build system, the common tasks that have setscene
+ variants are
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package'><filename>do_package</filename></ulink>,
+ <filename>do_package_write_*</filename>,
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-deploy'><filename>do_deploy</filename></ulink>,
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-packagedata'><filename>do_packagedata</filename></ulink>,
+ and
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></ulink>.
+ Notice that these tasks represent most of the tasks whose
+ output is an end result.
+ </para>
+
+ <para>
+ The build system has knowledge of the relationship between
+ these tasks and other preceding tasks.
+ For example, if BitBake runs
+ <filename>do_populate_sysroot_setscene</filename> for
+ something, it does not make sense to run any of the
+ <filename>do_fetch</filename>,
+ <filename>do_unpack</filename>,
+ <filename>do_patch</filename>,
+ <filename>do_configure</filename>,
+ <filename>do_compile</filename>, and
+ <filename>do_install</filename> tasks.
+ However, if <filename>do_package</filename> needs to be
+ run, BitBake needs to run those other tasks.
+ </para>
+
+ <para>
+ It becomes more complicated if everything can come
+ from an sstate cache because some objects are simply
+ not required at all.
+ For example, you do not need a compiler or native tools,
+ such as quilt, if nothing exists to compile or patch.
+ If the <filename>do_package_write_*</filename> packages
+ are available from sstate, BitBake does not need the
+ <filename>do_package</filename> task data.
+ </para>
+
+ <para>
+ To handle all these complexities, BitBake runs in two
+ phases.
+ The first is the "setscene" stage.
+ During this stage, BitBake first checks the sstate cache
+ for any targets it is planning to build.
+ BitBake does a fast check to see if the object exists
+ rather than a complete download.
+ If nothing exists, the second phase, which is the setscene
+ stage, completes and the main build proceeds.
+ </para>
+
+ <para>
+ If objects are found in the sstate cache, the build system
+ works backwards from the end targets specified by the user.
+ For example, if an image is being built, the build system
+ first looks for the packages needed for that image and the
+ tools needed to construct an image.
+ If those are available, the compiler is not needed.
+ Thus, the compiler is not even downloaded.
+ If something was found to be unavailable, or the
+ download or setscene task fails, the build system then
+ tries to install dependencies, such as the compiler, from
+ the cache.
+ </para>
+
+ <para>
+ The availability of objects in the sstate cache is
+ handled by the function specified by the
+ <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_HASHCHECK_FUNCTION'><filename>BB_HASHCHECK_FUNCTION</filename></ulink>
+ variable and returns a list of available objects.
+ The function specified by the
+ <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_SETSCENE_DEPVALID'><filename>BB_SETSCENE_DEPVALID</filename></ulink>
+ variable is the function that determines whether a given
+ dependency needs to be followed, and whether for any given
+ relationship the function needs to be passed.
+ The function returns a True or False value.
+ </para>
+ </section>
+ </section>
+
+ <section id='images-dev-environment'>
+ <title>Images</title>
+
+ <para>
+ The images produced by the build system are compressed forms
+ of the root filesystem and are ready to boot on a target
+ device.
+ You can see from the
+ <link linkend='general-workflow-figure'>general workflow figure</link>
+ that BitBake output, in part, consists of images.
+ This section takes a closer look at this output:
+ <imagedata fileref="figures/images.png" align="center" width="5.5in" depth="5.5in" />
+ </para>
+
+ <note>
+ For a list of example images that the Yocto Project provides,
+ see the
+ "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>"
+ chapter in the Yocto Project Reference Manual.
+ </note>
+
+ <para>
+ The build process writes images out to the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>
+ inside the
+ <filename>tmp/deploy/images/<replaceable>machine</replaceable>/</filename>
+ folder as shown in the figure.
+ This folder contains any files expected to be loaded on the
+ target device.
+ The
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></ulink>
+ variable points to the <filename>deploy</filename> directory,
+ while the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR_IMAGE'><filename>DEPLOY_DIR_IMAGE</filename></ulink>
+ variable points to the appropriate directory containing images
+ for the current configuration.
+ <itemizedlist>
+ <listitem><para>
+ <replaceable>kernel-image</replaceable>:
+ A kernel binary file.
+ The
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-KERNEL_IMAGETYPE'><filename>KERNEL_IMAGETYPE</filename></ulink>
+ variable determines the naming scheme for the
+ kernel image file.
+ Depending on this variable, the file could begin with
+ a variety of naming strings.
+ The
+ <filename>deploy/images/</filename><replaceable>machine</replaceable>
+ directory can contain multiple image files for the
+ machine.
+ </para></listitem>
+ <listitem><para>
+ <replaceable>root-filesystem-image</replaceable>:
+ Root filesystems for the target device (e.g.
+ <filename>*.ext3</filename> or
+ <filename>*.bz2</filename> files).
+ The
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></ulink>
+ variable determines the root filesystem image type.
+ The
+ <filename>deploy/images/</filename><replaceable>machine</replaceable>
+ directory can contain multiple root filesystems for the
+ machine.
+ </para></listitem>
+ <listitem><para>
+ <replaceable>kernel-modules</replaceable>:
+ Tarballs that contain all the modules built for the
+ kernel.
+ Kernel module tarballs exist for legacy purposes and
+ can be suppressed by setting the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-MODULE_TARBALL_DEPLOY'><filename>MODULE_TARBALL_DEPLOY</filename></ulink>
+ variable to "0".
+ The
+ <filename>deploy/images/</filename><replaceable>machine</replaceable>
+ directory can contain multiple kernel module tarballs
+ for the machine.
+ </para></listitem>
+ <listitem><para>
+ <replaceable>bootloaders</replaceable>:
+ If applicable to the target machine, bootloaders
+ supporting the image.
+ The <filename>deploy/images/</filename><replaceable>machine</replaceable>
+ directory can contain multiple bootloaders for the
+ machine.
+ </para></listitem>
+ <listitem><para>
+ <replaceable>symlinks</replaceable>:
+ The
+ <filename>deploy/images/</filename><replaceable>machine</replaceable>
+ folder contains a symbolic link that points to the
+ most recently built file for each machine.
+ These links might be useful for external scripts that
+ need to obtain the latest version of each file.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </section>
+
+ <section id='sdk-dev-environment'>
+ <title>Application Development SDK</title>
+
+ <para>
+ In the
+ <link linkend='general-workflow-figure'>general workflow figure</link>,
+ the output labeled "Application Development SDK" represents an
+ SDK.
+ The SDK generation process differs depending on whether you
+ build an extensible SDK (e.g.
+ <filename>bitbake -c populate_sdk_ext</filename> <replaceable>imagename</replaceable>)
+ or a standard SDK (e.g.
+ <filename>bitbake -c populate_sdk</filename> <replaceable>imagename</replaceable>).
+ This section takes a closer look at this output:
+ <imagedata fileref="figures/sdk.png" align="center" width="9in" depth="7.25in" />
+ </para>
+
+ <para>
+ The specific form of this output is a set of files that
+ includes a self-extracting SDK installer
+ (<filename>*.sh</filename>), host and target manifest files,
+ and files used for SDK testing.
+ When the SDK installer file is run, it installs the SDK.
+ The SDK consists of a cross-development toolchain, a set of
+ libraries and headers, and an SDK environment setup script.
+ Running this installer essentially sets up your
+ cross-development environment.
+ You can think of the cross-toolchain as the "host"
+ part because it runs on the SDK machine.
+ You can think of the libraries and headers as the "target"
+ part because they are built for the target hardware.
+ The environment setup script is added so that you can
+ initialize the environment before using the tools.
+ </para>
+
+ <note><title>Notes</title>
+ <itemizedlist>
+ <listitem><para>
+ The Yocto Project supports several methods by which
+ you can set up this cross-development environment.
+ These methods include downloading pre-built SDK
+ installers or building and installing your own SDK
+ installer.
+ </para></listitem>
+ <listitem><para>
+ For background information on cross-development
+ toolchains in the Yocto Project development
+ environment, see the
+ "<link linkend='cross-development-toolchain-generation'>Cross-Development Toolchain Generation</link>"
+ section.
+ </para></listitem>
+ <listitem><para>
+ For information on setting up a cross-development
+ environment, see the
+ <ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Application Development and the Extensible Software Development Kit (eSDK)</ulink>
+ manual.
+ </para></listitem>
+ </itemizedlist>
+ </note>
+
+ <para>
+ All the output files for an SDK are written to the
+ <filename>deploy/sdk</filename> folder inside the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>
+ as shown in the previous figure.
+ Depending on the type of SDK, several variables exist that help
+ configure these files.
+ The following list shows the variables associated with an
+ extensible SDK:
+ <itemizedlist>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></ulink>:
+ Points to the <filename>deploy</filename> directory.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SDK_EXT_TYPE'><filename>SDK_EXT_TYPE</filename></ulink>:
+ Controls whether or not shared state artifacts are
+ copied into the extensible SDK.
+ By default, all required shared state artifacts are
+ copied into the SDK.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SDK_INCLUDE_PKGDATA'><filename>SDK_INCLUDE_PKGDATA</filename></ulink>:
+ Specifies whether or not packagedata is included in the
+ extensible SDK for all recipes in the "world" target.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SDK_INCLUDE_TOOLCHAIN'><filename>SDK_INCLUDE_TOOLCHAIN</filename></ulink>:
+ Specifies whether or not the toolchain is included
+ when building the extensible SDK.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SDK_LOCAL_CONF_WHITELIST'><filename>SDK_LOCAL_CONF_WHITELIST</filename></ulink>:
+ A list of variables allowed through from the build
+ system configuration into the extensible SDK
+ configuration.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SDK_LOCAL_CONF_BLACKLIST'><filename>SDK_LOCAL_CONF_BLACKLIST</filename></ulink>:
+ A list of variables not allowed through from the build
+ system configuration into the extensible SDK
+ configuration.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SDK_INHERIT_BLACKLIST'><filename>SDK_INHERIT_BLACKLIST</filename></ulink>:
+ A list of classes to remove from the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-INHERIT'><filename>INHERIT</filename></ulink>
+ value globally within the extensible SDK configuration.
+ </para></listitem>
+ </itemizedlist>
+ This next list, shows the variables associated with a standard
+ SDK:
+ <itemizedlist>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></ulink>:
+ Points to the <filename>deploy</filename> directory.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SDKMACHINE'><filename>SDKMACHINE</filename></ulink>:
+ Specifies the architecture of the machine on which the
+ cross-development tools are run to create packages for
+ the target hardware.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SDKIMAGE_FEATURES'><filename>SDKIMAGE_FEATURES</filename></ulink>:
+ Lists the features to include in the "target" part
+ of the SDK.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-TOOLCHAIN_HOST_TASK'><filename>TOOLCHAIN_HOST_TASK</filename></ulink>:
+ Lists packages that make up the host part of the SDK
+ (i.e. the part that runs on the
+ <filename>SDKMACHINE</filename>).
+ When you use
+ <filename>bitbake -c populate_sdk <replaceable>imagename</replaceable></filename>
+ to create the SDK, a set of default packages apply.
+ This variable allows you to add more packages.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-TOOLCHAIN_TARGET_TASK'><filename>TOOLCHAIN_TARGET_TASK</filename></ulink>:
+ Lists packages that make up the target part of the SDK
+ (i.e. the part built for the target hardware).
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SDKPATH'><filename>SDKPATH</filename></ulink>:
+ Defines the default SDK installation path offered by
+ the installation script.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SDK_HOST_MANIFEST'><filename>SDK_HOST_MANIFEST</filename></ulink>:
+ Lists all the installed packages that make up the host
+ part of the SDK.
+ This variable also plays a minor role for extensible
+ SDK development as well.
+ However, it is mainly used for the standard SDK.
+ </para></listitem>
+ <listitem><para>
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SDK_TARGET_MANIFEST'><filename>SDK_TARGET_MANIFEST</filename></ulink>:
+ Lists all the installed packages that make up the
+ target part of the SDK.
+ This variable also plays a minor role for extensible
+ SDK development as well.
+ However, it is mainly used for the standard SDK.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </section>
+ </section>
+
+ <section id="cross-development-toolchain-generation">
+ <title>Cross-Development Toolchain Generation</title>
+
+ <para>
+ The Yocto Project does most of the work for you when it comes to
+ creating
+ <ulink url='&YOCTO_DOCS_REF_URL;#cross-development-toolchain'>cross-development toolchains</ulink>.
+ This section provides some technical background on how
+ cross-development toolchains are created and used.
+ For more information on toolchains, you can also see the
+ <ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Application Development and the Extensible Software Development Kit (eSDK)</ulink>
+ manual.
+ </para>
+
+ <para>
+ In the Yocto Project development environment, cross-development
+ toolchains are used to build images and applications that run
+ on the target hardware.
+ With just a few commands, the OpenEmbedded build system creates
+ these necessary toolchains for you.
+ </para>
+
+ <para>
+ The following figure shows a high-level build environment regarding
+ toolchain construction and use.
+ </para>
+
+ <para>
+ <imagedata fileref="figures/cross-development-toolchains.png" width="8in" depth="6in" align="center" />
+ </para>
+
+ <para>
+ Most of the work occurs on the Build Host.
+ This is the machine used to build images and generally work within
+ the the Yocto Project environment.
+ When you run
+ <ulink url='&YOCTO_DOCS_REF_URL;#bitbake-term'>BitBake</ulink>
+ to create an image, the OpenEmbedded build system
+ uses the host <filename>gcc</filename> compiler to bootstrap a
+ cross-compiler named <filename>gcc-cross</filename>.
+ The <filename>gcc-cross</filename> compiler is what BitBake uses to
+ compile source files when creating the target image.
+ You can think of <filename>gcc-cross</filename> simply as an
+ automatically generated cross-compiler that is used internally
+ within BitBake only.
+ <note>
+ The extensible SDK does not use
+ <filename>gcc-cross-canadian</filename> since this SDK
+ ships a copy of the OpenEmbedded build system and the sysroot
+ within it contains <filename>gcc-cross</filename>.
+ </note>
+ </para>
+
+ <para>
+ The chain of events that occurs when <filename>gcc-cross</filename> is
+ bootstrapped is as follows:
+ <literallayout class='monospaced'>
+ gcc -> binutils-cross -> gcc-cross-initial -> linux-libc-headers -> glibc-initial -> glibc -> gcc-cross -> gcc-runtime
+ </literallayout>
+ <itemizedlist>
+ <listitem><para>
+ <filename>gcc</filename>:
+ The build host's GNU Compiler Collection (GCC).
+ </para></listitem>
+ <listitem><para>
+ <filename>binutils-cross</filename>:
+ The bare minimum binary utilities needed in order to run
+ the <filename>gcc-cross-initial</filename> phase of the
+ bootstrap operation.
+ </para></listitem>
+ <listitem><para>
+ <filename>gcc-cross-initial</filename>:
+ An early stage of the bootstrap process for creating
+ the cross-compiler.
+ This stage builds enough of the <filename>gcc-cross</filename>,
+ the C library, and other pieces needed to finish building the
+ final cross-compiler in later stages.
+ This tool is a "native" package (i.e. it is designed to run on
+ the build host).
+ </para></listitem>
+ <listitem><para>
+ <filename>linux-libc-headers</filename>:
+ Headers needed for the cross-compiler.
+ </para></listitem>
+ <listitem><para>
+ <filename>glibc-initial</filename>:
+ An initial version of the Embedded GNU C Library
+ (GLIBC) needed to bootstrap <filename>glibc</filename>.
+ </para></listitem>
+ <listitem><para>
+ <filename>glibc</filename>:
+ The GNU C Library.
+ </para></listitem>
+ <listitem><para>
+ <filename>gcc-cross</filename>:
+ The final stage of the bootstrap process for the
+ cross-compiler.
+ This stage results in the actual cross-compiler that
+ BitBake uses when it builds an image for a targeted
+ device.
+ <note>
+ If you are replacing this cross compiler toolchain
+ with a custom version, you must replace
+ <filename>gcc-cross</filename>.
+ </note>
+ This tool is also a "native" package (i.e. it is
+ designed to run on the build host).
+ </para></listitem>
+ <listitem><para>
+ <filename>gcc-runtime</filename>:
+ Runtime libraries resulting from the toolchain bootstrapping
+ process.
+ This tool produces a binary that consists of the
+ runtime libraries need for the targeted device.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ You can use the OpenEmbedded build system to build an installer for
+ the relocatable SDK used to develop applications.
+ When you run the installer, it installs the toolchain, which
+ contains the development tools (e.g.,
+ <filename>gcc-cross-canadian</filename>,
+ <filename>binutils-cross-canadian</filename>, and other
+ <filename>nativesdk-*</filename> tools),
+ which are tools native to the SDK (i.e. native to
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SDK_ARCH'><filename>SDK_ARCH</filename></ulink>),
+ you need to cross-compile and test your software.
+ The figure shows the commands you use to easily build out this
+ toolchain.
+ This cross-development toolchain is built to execute on the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SDKMACHINE'><filename>SDKMACHINE</filename></ulink>,
+ which might or might not be the same
+ machine as the Build Host.
+ <note>
+ If your target architecture is supported by the Yocto Project,
+ you can take advantage of pre-built images that ship with the
+ Yocto Project and already contain cross-development toolchain
+ installers.
+ </note>
+ </para>
+
+ <para>
+ Here is the bootstrap process for the relocatable toolchain:
+ <literallayout class='monospaced'>
+ gcc -> binutils-crosssdk -> gcc-crosssdk-initial -> linux-libc-headers ->
+ glibc-initial -> nativesdk-glibc -> gcc-crosssdk -> gcc-cross-canadian
+ </literallayout>
+ <itemizedlist>
+ <listitem><para>
+ <filename>gcc</filename>:
+ The build host's GNU Compiler Collection (GCC).
+ </para></listitem>
+ <listitem><para>
+ <filename>binutils-crosssdk</filename>:
+ The bare minimum binary utilities needed in order to run
+ the <filename>gcc-crosssdk-initial</filename> phase of the
+ bootstrap operation.
+ </para></listitem>
+ <listitem><para>
+ <filename>gcc-crosssdk-initial</filename>:
+ An early stage of the bootstrap process for creating
+ the cross-compiler.
+ This stage builds enough of the
+ <filename>gcc-crosssdk</filename> and supporting pieces so that
+ the final stage of the bootstrap process can produce the
+ finished cross-compiler.
+ This tool is a "native" binary that runs on the build host.
+ </para></listitem>
+ <listitem><para>
+ <filename>linux-libc-headers</filename>:
+ Headers needed for the cross-compiler.
+ </para></listitem>
+ <listitem><para>
+ <filename>glibc-initial</filename>:
+ An initial version of the Embedded GLIBC needed to bootstrap
+ <filename>nativesdk-glibc</filename>.
+ </para></listitem>
+ <listitem><para>
+ <filename>nativesdk-glibc</filename>:
+ The Embedded GLIBC needed to bootstrap the
+ <filename>gcc-crosssdk</filename>.
+ </para></listitem>
+ <listitem><para>
+ <filename>gcc-crosssdk</filename>:
+ The final stage of the bootstrap process for the
+ relocatable cross-compiler.
+ The <filename>gcc-crosssdk</filename> is a transitory
+ compiler and never leaves the build host.
+ Its purpose is to help in the bootstrap process to create
+ the eventual <filename>gcc-cross-canadian</filename>
+ compiler, which is relocatable.
+ This tool is also a "native" package (i.e. it is
+ designed to run on the build host).
+ </para></listitem>
+ <listitem><para>
+ <filename>gcc-cross-canadian</filename>:
+ The final relocatable cross-compiler.
+ When run on the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SDKMACHINE'><filename>SDKMACHINE</filename></ulink>,
+ this tool
+ produces executable code that runs on the target device.
+ Only one cross-canadian compiler is produced per architecture
+ since they can be targeted at different processor optimizations
+ using configurations passed to the compiler through the
+ compile commands.
+ This circumvents the need for multiple compilers and thus
+ reduces the size of the toolchains.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <note>
+ For information on advantages gained when building a
+ cross-development toolchain installer, see the
+ "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-building-an-sdk-installer'>Building an SDK Installer</ulink>"
+ appendix in the Yocto Project Application Development and the
+ Extensible Software Development Kit (eSDK) manual.
+ </note>
+ </section>
+
+ <section id="shared-state-cache">
+ <title>Shared State Cache</title>
+
+ <para>
+ By design, the OpenEmbedded build system builds everything from
+ scratch unless
+ <ulink url='&YOCTO_DOCS_REF_URL;#bitbake-term'>BitBake</ulink>
+ can determine that parts do not need to be rebuilt.
+ Fundamentally, building from scratch is attractive as it means all
+ parts are built fresh and no possibility of stale data exists that
+ can cause problems.
+ When developers hit problems, they typically default back to
+ building from scratch so they have a know state from the
+ start.
+ </para>
+
+ <para>
+ Building an image from scratch is both an advantage and a
+ disadvantage to the process.
+ As mentioned in the previous paragraph, building from scratch
+ ensures that everything is current and starts from a known state.
+ However, building from scratch also takes much longer as it
+ generally means rebuilding things that do not necessarily need
+ to be rebuilt.
+ </para>
+
+ <para>
+ The Yocto Project implements shared state code that supports
+ incremental builds.
+ The implementation of the shared state code answers the following
+ questions that were fundamental roadblocks within the OpenEmbedded
+ incremental build support system:
+ <itemizedlist>
+ <listitem><para>
+ What pieces of the system have changed and what pieces have
+ not changed?
+ </para></listitem>
+ <listitem><para>
+ How are changed pieces of software removed and replaced?
+ </para></listitem>
+ <listitem><para>
+ How are pre-built components that do not need to be rebuilt
+ from scratch used when they are available?
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ For the first question, the build system detects changes in the
+ "inputs" to a given task by creating a checksum (or signature) of
+ the task's inputs.
+ If the checksum changes, the system assumes the inputs have changed
+ and the task needs to be rerun.
+ For the second question, the shared state (sstate) code tracks
+ which tasks add which output to the build process.
+ This means the output from a given task can be removed, upgraded
+ or otherwise manipulated.
+ The third question is partly addressed by the solution for the
+ second question assuming the build system can fetch the sstate
+ objects from remote locations and install them if they are deemed
+ to be valid.
+ <note><title>Notes</title>
+ <itemizedlist>
+ <listitem><para>
+ The build system does not maintain
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>
+ information as part of the shared state packages.
+ Consequently, considerations exist that affect
+ maintaining shared state feeds.
+ For information on how the build system works with
+ packages and can track incrementing
+ <filename>PR</filename> information, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#automatically-incrementing-a-binary-package-revision-number'>Automatically Incrementing a Binary Package Revision Number</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para></listitem>
+ <listitem><para>
+ The code in the build system that supports incremental
+ builds is not simple code.
+ For techniques that help you work around issues related
+ to shared state code, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#dev-viewing-metadata-used-to-create-the-input-signature-of-a-shared-state-task'>Viewing Metadata Used to Create the Input Signature of a Shared State Task</ulink>"
+ and
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#dev-invalidating-shared-state-to-force-a-task-to-run'>Invalidating Shared State to Force a Task to Run</ulink>"
+ sections both in the Yocto Project Development Tasks
+ Manual.
+ </para></listitem>
+ </itemizedlist>
+ </note>
+ </para>
+
+ <para>
+ The rest of this section goes into detail about the overall
+ incremental build architecture, the checksums (signatures), and
+ shared state.
+ </para>
+
+ <section id='concepts-overall-architecture'>
+ <title>Overall Architecture</title>
+
+ <para>
+ When determining what parts of the system need to be built,
+ BitBake works on a per-task basis rather than a per-recipe
+ basis.
+ You might wonder why using a per-task basis is preferred over
+ a per-recipe basis.
+ To help explain, consider having the IPK packaging backend
+ enabled and then switching to DEB.
+ In this case, the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>
+ and
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package'><filename>do_package</filename></ulink>
+ task outputs are still valid.
+ However, with a per-recipe approach, the build would not
+ include the <filename>.deb</filename> files.
+ Consequently, you would have to invalidate the whole build and
+ rerun it.
+ Rerunning everything is not the best solution.
+ Also, in this case, the core must be "taught" much about
+ specific tasks.
+ This methodology does not scale well and does not allow users
+ to easily add new tasks in layers or as external recipes
+ without touching the packaged-staging core.
+ </para>
+ </section>
+
+ <section id='overview-checksums'>
+ <title>Checksums (Signatures)</title>
+
+ <para>
+ The shared state code uses a checksum, which is a unique
+ signature of a task's inputs, to determine if a task needs to
+ be run again.
+ Because it is a change in a task's inputs that triggers a
+ rerun, the process needs to detect all the inputs to a given
+ task.
+ For shell tasks, this turns out to be fairly easy because
+ the build process generates a "run" shell script for each task
+ and it is possible to create a checksum that gives you a good
+ idea of when the task's data changes.
+ </para>
+
+ <para>
+ To complicate the problem, there are things that should not be
+ included in the checksum.
+ First, there is the actual specific build path of a given
+ task - the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>.
+ It does not matter if the work directory changes because it
+ should not affect the output for target packages.
+ Also, the build process has the objective of making native
+ or cross packages relocatable.
+ <note>
+ Both native and cross packages run on the
+ <ulink url='&YOCTO_DOCS_REF_URL;#hardware-build-system-term'>build host</ulink>.
+ However, cross packages generate output for the target
+ architecture.
+ </note>
+ The checksum therefore needs to exclude
+ <filename>WORKDIR</filename>.
+ The simplistic approach for excluding the work directory is to
+ set <filename>WORKDIR</filename> to some fixed value and
+ create the checksum for the "run" script.
+ </para>
+
+ <para>
+ Another problem results from the "run" scripts containing
+ functions that might or might not get called.
+ The incremental build solution contains code that figures out
+ dependencies between shell functions.
+ This code is used to prune the "run" scripts down to the
+ minimum set, thereby alleviating this problem and making the
+ "run" scripts much more readable as a bonus.
+ </para>
+
+ <para>
+ So far, solutions for shell scripts exist.
+ What about Python tasks?
+ The same approach applies even though these tasks are more
+ difficult.
+ The process needs to figure out what variables a Python
+ function accesses and what functions it calls.
+ Again, the incremental build solution contains code that first
+ figures out the variable and function dependencies, and then
+ creates a checksum for the data used as the input to the task.
+ </para>
+
+ <para>
+ Like the <filename>WORKDIR</filename> case, situations exist
+ where dependencies should be ignored.
+ For these situations, you can instruct the build process to
+ ignore a dependency by using a line like the following:
+ <literallayout class='monospaced'>
+ PACKAGE_ARCHS[vardepsexclude] = "MACHINE"
+ </literallayout>
+ This example ensures that the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCHS'><filename>PACKAGE_ARCHS</filename></ulink>
+ variable does not depend on the value of
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>,
+ even if it does reference it.
+ </para>
+
+ <para>
+ Equally, there are cases where you need to add dependencies
+ BitBake is not able to find.
+ You can accomplish this by using a line like the following:
+ <literallayout class='monospaced'>
+ PACKAGE_ARCHS[vardeps] = "MACHINE"
+ </literallayout>
+ This example explicitly adds the <filename>MACHINE</filename>
+ variable as a dependency for
+ <filename>PACKAGE_ARCHS</filename>.
+ </para>
+
+ <para>
+ As an example, consider a case with in-line Python where
+ BitBake is not able to figure out dependencies.
+ When running in debug mode (i.e. using
+ <filename>-DDD</filename>), BitBake produces output when it
+ discovers something for which it cannot figure out dependencies.
+ The Yocto Project team has currently not managed to cover
+ those dependencies in detail and is aware of the need to fix
+ this situation.
+ </para>
+
+ <para>
+ Thus far, this section has limited discussion to the direct
+ inputs into a task.
+ Information based on direct inputs is referred to as the
+ "basehash" in the code.
+ However, the question of a task's indirect inputs still
+ exits - items already built and present in the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>.
+ The checksum (or signature) for a particular task needs to add
+ the hashes of all the tasks on which the particular task
+ depends.
+ Choosing which dependencies to add is a policy decision.
+ However, the effect is to generate a master checksum that
+ combines the basehash and the hashes of the task's
+ dependencies.
+ </para>
+
+ <para>
+ At the code level, a variety of ways exist by which both the
+ basehash and the dependent task hashes can be influenced.
+ Within the BitBake configuration file, you can give BitBake
+ some extra information to help it construct the basehash.
+ The following statement effectively results in a list of
+ global variable dependency excludes (i.e. variables never
+ included in any checksum):
+ <literallayout class='monospaced'>
+ BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR \
+ SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM \
+ USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \
+ PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \
+ CCACHE_DIR EXTERNAL_TOOLCHAIN CCACHE CCACHE_DISABLE LICENSE_PATH SDKPKGSUFFIX"
+ </literallayout>
+ The previous example excludes
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>
+ since that variable is actually constructed as a path within
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>,
+ which is on the whitelist.
+ </para>
+
+ <para>
+ The rules for deciding which hashes of dependent tasks to
+ include through dependency chains are more complex and are
+ generally accomplished with a Python function.
+ The code in <filename>meta/lib/oe/sstatesig.py</filename> shows
+ two examples of this and also illustrates how you can insert
+ your own policy into the system if so desired.
+ This file defines the two basic signature generators
+ <ulink url='&YOCTO_DOCS_REF_URL;#oe-core'>OE-Core</ulink>
+ uses: "OEBasic" and "OEBasicHash".
+ By default, a dummy "noop" signature handler is enabled
+ in BitBake.
+ This means that behavior is unchanged from previous versions.
+ OE-Core uses the "OEBasicHash" signature handler by default
+ through this setting in the <filename>bitbake.conf</filename>
+ file:
+ <literallayout class='monospaced'>
+ BB_SIGNATURE_HANDLER ?= "OEBasicHash"
+ </literallayout>
+ The "OEBasicHash" <filename>BB_SIGNATURE_HANDLER</filename>
+ is the same as the "OEBasic" version but adds the task hash to
+ the
+ <link linkend='stamp-files-and-the-rerunning-of-tasks'>stamp files</link>.
+ This results in any metadata change that changes the task hash,
+ automatically causing the task to be run again.
+ This removes the need to bump
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>
+ values, and changes to metadata automatically ripple across
+ the build.
+ </para>
+
+ <para>
+ It is also worth noting that the end result of these
+ signature generators is to make some dependency and hash
+ information available to the build.
+ This information includes:
+ <itemizedlist>
+ <listitem><para>
+ <filename>BB_BASEHASH_task-</filename><replaceable>taskname</replaceable>:
+ The base hashes for each task in the recipe.
+ </para></listitem>
+ <listitem><para>
+ <filename>BB_BASEHASH_</filename><replaceable>filename</replaceable><filename>:</filename><replaceable>taskname</replaceable>:
+ The base hashes for each dependent task.
+ </para></listitem>
+ <listitem><para>
+ <filename>BBHASHDEPS_</filename><replaceable>filename</replaceable><filename>:</filename><replaceable>taskname</replaceable>:
+ The task dependencies for each task.
+ </para></listitem>
+ <listitem><para>
+ <filename>BB_TASKHASH</filename>:
+ The hash of the currently running task.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </section>
+
+ <section id='shared-state'>
+ <title>Shared State</title>
+
+ <para>
+ Checksums and dependencies, as discussed in the previous
+ section, solve half the problem of supporting a shared state.
+ The other half of the problem is being able to use checksum
+ information during the build and being able to reuse or rebuild
+ specific components.
+ </para>
+
+ <para>
+ The
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-sstate'><filename>sstate</filename></ulink>
+ class is a relatively generic implementation of how to
+ "capture" a snapshot of a given task.
+ The idea is that the build process does not care about the
+ source of a task's output.
+ Output could be freshly built or it could be downloaded and
+ unpacked from somewhere.
+ In other words, the build process does not need to worry about
+ its origin.
+ </para>
+
+ <para>
+ Two types of output exist.
+ One type is just about creating a directory in
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>.
+ A good example is the output of either
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>
+ or
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package'><filename>do_package</filename></ulink>.
+ The other type of output occurs when a set of data is merged
+ into a shared directory tree such as the sysroot.
+ </para>
+
+ <para>
+ The Yocto Project team has tried to keep the details of the
+ implementation hidden in <filename>sstate</filename> class.
+ From a user's perspective, adding shared state wrapping to a
+ task is as simple as this
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-deploy'><filename>do_deploy</filename></ulink>
+ example taken from the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-deploy'><filename>deploy</filename></ulink>
+ class:
+ <literallayout class='monospaced'>
+ DEPLOYDIR = "${WORKDIR}/deploy-${PN}"
+ SSTATETASKS += "do_deploy"
+ do_deploy[sstate-inputdirs] = "${DEPLOYDIR}"
+ do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
+
+ python do_deploy_setscene () {
+ sstate_setscene(d)
+ }
+ addtask do_deploy_setscene
+ do_deploy[dirs] = "${DEPLOYDIR} ${B}"
+ do_deploy[stamp-extra-info] = "${MACHINE_ARCH}"
+ </literallayout>
+ The following list explains the previous example:
+ <itemizedlist>
+ <listitem><para>
+ Adding "do_deploy" to <filename>SSTATETASKS</filename>
+ adds some required sstate-related processing, which is
+ implemented in the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-sstate'><filename>sstate</filename></ulink>
+ class, to before and after the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-deploy'><filename>do_deploy</filename></ulink>
+ task.
+ </para></listitem>
+ <listitem><para>
+ The
+ <filename>do_deploy[sstate-inputdirs] = "${DEPLOYDIR}"</filename>
+ declares that <filename>do_deploy</filename> places its
+ output in <filename>${DEPLOYDIR}</filename> when run
+ normally (i.e. when not using the sstate cache).
+ This output becomes the input to the shared state cache.
+ </para></listitem>
+ <listitem><para>
+ The
+ <filename>do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"</filename>
+ line causes the contents of the shared state cache to be
+ copied to <filename>${DEPLOY_DIR_IMAGE}</filename>.
+ <note>
+ If <filename>do_deploy</filename> is not already in
+ the shared state cache or if its input checksum
+ (signature) has changed from when the output was
+ cached, the task runs to populate the shared
+ state cache, after which the contents of the shared
+ state cache is copied to
+ <filename>${DEPLOY_DIR_IMAGE}</filename>.
+ If <filename>do_deploy</filename> is in the shared
+ state cache and its signature indicates that the
+ cached output is still valid (i.e. if no
+ relevant task inputs have changed), then the
+ contents of the shared state cache copies
+ directly to
+ <filename>${DEPLOY_DIR_IMAGE}</filename> by the
+ <filename>do_deploy_setscene</filename> task
+ instead, skipping the
+ <filename>do_deploy</filename> task.
+ </note>
+ </para></listitem>
+ <listitem><para>
+ The following task definition is glue logic needed to
+ make the previous settings effective:
+ <literallayout class='monospaced'>
+ python do_deploy_setscene () {
+ sstate_setscene(d)
+ }
+ addtask do_deploy_setscene
+ </literallayout>
+ <filename>sstate_setscene()</filename> takes the flags
+ above as input and accelerates the
+ <filename>do_deploy</filename> task through the
+ shared state cache if possible.
+ If the task was accelerated,
+ <filename>sstate_setscene()</filename> returns True.
+ Otherwise, it returns False, and the normal
+ <filename>do_deploy</filename> task runs.
+ For more information, see the
+ "<ulink url='&YOCTO_DOCS_BB_URL;#setscene'>setscene</ulink>"
+ section in the BitBake User Manual.
+ </para></listitem>
+ <listitem><para>
+ The <filename>do_deploy[dirs] = "${DEPLOYDIR} ${B}"</filename>
+ line creates <filename>${DEPLOYDIR}</filename> and
+ <filename>${B}</filename> before the
+ <filename>do_deploy</filename> task runs, and also sets
+ the current working directory of
+ <filename>do_deploy</filename> to
+ <filename>${B}</filename>.
+ For more information, see the
+ "<ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'>Variable Flags</ulink>"
+ section in the BitBake User Manual.
+ <note>
+ In cases where
+ <filename>sstate-inputdirs</filename> and
+ <filename>sstate-outputdirs</filename> would be the
+ same, you can use
+ <filename>sstate-plaindirs</filename>.
+ For example, to preserve the
+ <filename>${PKGD}</filename> and
+ <filename>${PKGDEST}</filename> output from the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package'><filename>do_package</filename></ulink>
+ task, use the following:
+ <literallayout class='monospaced'>
+ do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}"
+ </literallayout>
+ </note>
+ </para></listitem>
+ <listitem><para>
+ The <filename>do_deploy[stamp-extra-info] = "${MACHINE_ARCH}"</filename>
+ line appends extra metadata to the
+ <link linkend='stamp-files-and-the-rerunning-of-tasks'>stamp file</link>.
+ In this case, the metadata makes the task specific
+ to a machine's architecture.
+ See
+ "<ulink url='&YOCTO_DOCS_BB_URL;#ref-bitbake-tasklist'>The Task List</ulink>"
+ section in the BitBake User Manual for more
+ information on the <filename>stamp-extra-info</filename>
+ flag.
+ </para></listitem>
+ <listitem><para>
+ <filename>sstate-inputdirs</filename> and
+ <filename>sstate-outputdirs</filename> can also be used
+ with multiple directories.
+ For example, the following declares
+ <filename>PKGDESTWORK</filename> and
+ <filename>SHLIBWORK</filename> as shared state
+ input directories, which populates the shared state
+ cache, and <filename>PKGDATA_DIR</filename> and
+ <filename>SHLIBSDIR</filename> as the corresponding
+ shared state output directories:
+ <literallayout class='monospaced'>
+ do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}"
+ do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}"
+ </literallayout>
+ </para></listitem>
+ <listitem><para>
+ These methods also include the ability to take a
+ lockfile when manipulating shared state directory
+ structures, for cases where file additions or removals
+ are sensitive:
+ <literallayout class='monospaced'>
+ do_package[sstate-lockfile] = "${PACKAGELOCK}"
+ </literallayout>
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ Behind the scenes, the shared state code works by looking in
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink>
+ and
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_MIRRORS'><filename>SSTATE_MIRRORS</filename></ulink>
+ for shared state files.
+ Here is an example:
+ <literallayout class='monospaced'>
+ SSTATE_MIRRORS ?= "\
+ file://.* http://someserver.tld/share/sstate/PATH;downloadfilename=PATH \n \
+ file://.* file:///some/local/dir/sstate/PATH"
+ </literallayout>
+ <note>
+ The shared state directory
+ (<filename>SSTATE_DIR</filename>) is organized into
+ two-character subdirectories, where the subdirectory
+ names are based on the first two characters of the hash.
+ If the shared state directory structure for a mirror has the
+ same structure as <filename>SSTATE_DIR</filename>, you must
+ specify "PATH" as part of the URI to enable the build system
+ to map to the appropriate subdirectory.
+ </note>
+ </para>
+
+ <para>
+ The shared state package validity can be detected just by
+ looking at the filename since the filename contains the task
+ checksum (or signature) as described earlier in this section.
+ If a valid shared state package is found, the build process
+ downloads it and uses it to accelerate the task.
+ </para>
+
+ <para>
+ The build processes use the <filename>*_setscene</filename>
+ tasks for the task acceleration phase.
+ BitBake goes through this phase before the main execution
+ code and tries to accelerate any tasks for which it can find
+ shared state packages.
+ If a shared state package for a task is available, the
+ shared state package is used.
+ This means the task and any tasks on which it is dependent
+ are not executed.
+ </para>
+
+ <para>
+ As a real world example, the aim is when building an IPK-based
+ image, only the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_ipk'><filename>do_package_write_ipk</filename></ulink>
+ tasks would have their shared state packages fetched and
+ extracted.
+ Since the sysroot is not used, it would never get extracted.
+ This is another reason why a task-based approach is preferred
+ over a recipe-based approach, which would have to install the
+ output from every task.
+ </para>
+ </section>
+ </section>
+
+ <section id='automatically-added-runtime-dependencies'>
+ <title>Automatically Added Runtime Dependencies</title>
+
+ <para>
+ The OpenEmbedded build system automatically adds common types of
+ runtime dependencies between packages, which means that you do not
+ need to explicitly declare the packages using
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>.
+ Three automatic mechanisms exist (<filename>shlibdeps</filename>,
+ <filename>pcdeps</filename>, and <filename>depchains</filename>)
+ that handle shared libraries, package configuration (pkg-config)
+ modules, and <filename>-dev</filename> and
+ <filename>-dbg</filename> packages, respectively.
+ For other types of runtime dependencies, you must manually declare
+ the dependencies.
+ <itemizedlist>
+ <listitem><para>
+ <filename>shlibdeps</filename>:
+ During the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package'><filename>do_package</filename></ulink>
+ task of each recipe, all shared libraries installed by the
+ recipe are located.
+ For each shared library, the package that contains the
+ shared library is registered as providing the shared
+ library.
+ More specifically, the package is registered as providing
+ the
+ <ulink url='https://en.wikipedia.org/wiki/Soname'>soname</ulink>
+ of the library.
+ The resulting shared-library-to-package mapping
+ is saved globally in
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></ulink>
+ by the
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-packagedata'><filename>do_packagedata</filename></ulink>
+ task.</para>
+
+ <para>Simultaneously, all executables and shared libraries
+ installed by the recipe are inspected to see what shared
+ libraries they link against.
+ For each shared library dependency that is found,
+ <filename>PKGDATA_DIR</filename> is queried to
+ see if some package (likely from a different recipe)
+ contains the shared library.
+ If such a package is found, a runtime dependency is added
+ from the package that depends on the shared library to the
+ package that contains the library.</para>
+
+ <para>The automatically added runtime dependency also
+ includes a version restriction.
+ This version restriction specifies that at least the
+ current version of the package that provides the shared
+ library must be used, as if
+ "<replaceable>package</replaceable> (>= <replaceable>version</replaceable>)"
+ had been added to <filename>RDEPENDS</filename>.
+ This forces an upgrade of the package containing the shared
+ library when installing the package that depends on the
+ library, if needed.</para>
+
+ <para>If you want to avoid a package being registered as
+ providing a particular shared library (e.g. because the library
+ is for internal use only), then add the library to
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PRIVATE_LIBS'><filename>PRIVATE_LIBS</filename></ulink>
+ inside the package's recipe.
+ </para></listitem>
+ <listitem><para>
+ <filename>pcdeps</filename>:
+ During the <filename>do_package</filename> task of each
+ recipe, all pkg-config modules
+ (<filename>*.pc</filename> files) installed by the recipe
+ are located.
+ For each module, the package that contains the module is
+ registered as providing the module.
+ The resulting module-to-package mapping is saved globally in
+ <filename>PKGDATA_DIR</filename> by the
+ <filename>do_packagedata</filename> task.</para>
+
+ <para>Simultaneously, all pkg-config modules installed by
+ the recipe are inspected to see what other pkg-config
+ modules they depend on.
+ A module is seen as depending on another module if it
+ contains a "Requires:" line that specifies the other module.
+ For each module dependency,
+ <filename>PKGDATA_DIR</filename> is queried to see if some
+ package contains the module.
+ If such a package is found, a runtime dependency is added
+ from the package that depends on the module to the package
+ that contains the module.
+ <note>
+ The <filename>pcdeps</filename> mechanism most often
+ infers dependencies between <filename>-dev</filename>
+ packages.
+ </note>
+ </para></listitem>
+ <listitem><para>
+ <filename>depchains</filename>:
+ If a package <filename>foo</filename> depends on a package
+ <filename>bar</filename>, then <filename>foo-dev</filename>
+ and <filename>foo-dbg</filename> are also made to depend on
+ <filename>bar-dev</filename> and
+ <filename>bar-dbg</filename>, respectively.
+ Taking the <filename>-dev</filename> packages as an
+ example, the <filename>bar-dev</filename> package might
+ provide headers and shared library symlinks needed by
+ <filename>foo-dev</filename>, which shows the need
+ for a dependency between the packages.</para>
+
+ <para>The dependencies added by
+ <filename>depchains</filename> are in the form of
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'><filename>RRECOMMENDS</filename></ulink>.
+ <note>
+ By default, <filename>foo-dev</filename> also has an
+ <filename>RDEPENDS</filename>-style dependency on
+ <filename>foo</filename>, because the default value of
+ <filename>RDEPENDS_${PN}-dev</filename> (set in
+ <filename>bitbake.conf</filename>) includes
+ "${PN}".
+ </note></para>
+
+ <para>To ensure that the dependency chain is never broken,
+ <filename>-dev</filename> and <filename>-dbg</filename>
+ packages are always generated by default, even if the
+ packages turn out to be empty.
+ See the
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-ALLOW_EMPTY'><filename>ALLOW_EMPTY</filename></ulink>
+ variable for more information.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ The <filename>do_package</filename> task depends on the
+ <filename>do_packagedata</filename> task of each recipe in
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
+ through use of a
+ <filename>[</filename><ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'><filename>deptask</filename></ulink><filename>]</filename>
+ declaration, which guarantees that the required
+ shared-library/module-to-package mapping information will be available
+ when needed as long as <filename>DEPENDS</filename> has been
+ correctly set.
+ </para>
+ </section>
+
+ <section id='fakeroot-and-pseudo'>
+ <title>Fakeroot and Pseudo</title>
+
+ <para>
+ Some tasks are easier to implement when allowed to perform certain
+ operations that are normally reserved for the root user (e.g.
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>,
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_deb'><filename>do_package_write*</filename></ulink>,
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-rootfs'><filename>do_rootfs</filename></ulink>,
+ and
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-image'><filename>do_image*</filename></ulink>).
+ For example, the <filename>do_install</filename> task benefits
+ from being able to set the UID and GID of installed files to
+ arbitrary values.
+ </para>
+
+ <para>
+ One approach to allowing tasks to perform root-only operations
+ would be to require
+ <ulink url='&YOCTO_DOCS_REF_URL;#bitbake-term'>BitBake</ulink>
+ to run as root.
+ However, this method is cumbersome and has security issues.
+ The approach that is actually used is to run tasks that benefit
+ from root privileges in a "fake" root environment.
+ Within this environment, the task and its child processes believe
+ that they are running as the root user, and see an internally
+ consistent view of the filesystem.
+ As long as generating the final output (e.g. a package or an image)
+ does not require root privileges, the fact that some earlier
+ steps ran in a fake root environment does not cause problems.
+ </para>
+
+ <para>
+ The capability to run tasks in a fake root environment is known as
+ "<ulink url='http://man.he.net/man1/fakeroot'>fakeroot</ulink>",
+ which is derived from the BitBake keyword/variable
+ flag that requests a fake root environment for a task.
+ </para>
+
+ <para>
+ In the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>OpenEmbedded build system</ulink>,
+ the program that implements fakeroot is known as
+ <ulink url='https://www.yoctoproject.org/software-item/pseudo/'>Pseudo</ulink>.
+ Pseudo overrides system calls by using the environment variable
+ <filename>LD_PRELOAD</filename>, which results in the illusion
+ of running as root.
+ To keep track of "fake" file ownership and permissions resulting
+ from operations that require root permissions, Pseudo uses
+ an SQLite 3 database.
+ This database is stored in
+ <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}/pseudo/files.db</filename>
+ for individual recipes.
+ Storing the database in a file as opposed to in memory
+ gives persistence between tasks and builds, which is not
+ accomplished using fakeroot.
+ <note><title>Caution</title>
+ If you add your own task that manipulates the same files or
+ directories as a fakeroot task, then that task also needs to
+ run under fakeroot.
+ Otherwise, the task cannot run root-only operations, and
+ cannot see the fake file ownership and permissions set by the
+ other task.
+ You need to also add a dependency on
+ <filename>virtual/fakeroot-native:do_populate_sysroot</filename>,
+ giving the following:
+ <literallayout class='monospaced'>
+ fakeroot do_mytask () {
+ ...
+ }
+ do_mytask[depends] += "virtual/fakeroot-native:do_populate_sysroot"
+ </literallayout>
+ </note>
+ For more information, see the
+ <ulink url='&YOCTO_DOCS_BB_URL;#var-FAKEROOT'><filename>FAKEROOT*</filename></ulink>
+ variables in the BitBake User Manual.
+ You can also reference the
+ "<ulink url='https://github.com/wrpseudo/pseudo/wiki/WhyNotFakeroot'>Why Not Fakeroot?</ulink>"
+ article for background information on Fakeroot and Pseudo.
+ </para>
+ </section>
+</chapter>
+<!--
+vim: expandtab tw=80 ts=4
+-->
diff --git a/poky/documentation/overview-manual/overview-manual-customization.xsl b/poky/documentation/overview-manual/overview-manual-customization.xsl
new file mode 100644
index 0000000..1dd91bd
--- /dev/null
+++ b/poky/documentation/overview-manual/overview-manual-customization.xsl
@@ -0,0 +1,29 @@
+<?xml version='1.0'?>
+<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
+
+<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
+
+ <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
+
+<!--
+
+ <xsl:import href="../template/1.76.1/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
+
+ <xsl:import href="http://docbook.sourceforge.net/release/xsl/1.76.1/xhtml/docbook.xsl" />
+
+-->
+
+ <xsl:include href="../template/permalinks.xsl"/>
+ <xsl:include href="../template/section.title.xsl"/>
+ <xsl:include href="../template/component.title.xsl"/>
+ <xsl:include href="../template/division.title.xsl"/>
+ <xsl:include href="../template/formal.object.heading.xsl"/>
+
+ <xsl:param name="html.stylesheet" select="'overview-manual-style.css'" />
+ <xsl:param name="chapter.autolabel" select="1" />
+ <xsl:param name="appendix.autolabel" select="A" />
+ <xsl:param name="section.autolabel" select="1" />
+ <xsl:param name="section.label.includes.component.label" select="1" />
+ <xsl:param name="generate.id.attributes" select="1" />
+
+</xsl:stylesheet>
diff --git a/poky/documentation/overview-manual/overview-manual-development-environment.rst b/poky/documentation/overview-manual/overview-manual-development-environment.rst
index 4bedd6d..bb2c8e7 100644
--- a/poky/documentation/overview-manual/overview-manual-development-environment.rst
+++ b/poky/documentation/overview-manual/overview-manual-development-environment.rst
@@ -1,4 +1,4 @@
-.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
+.. SPDX-License-Identifier: CC-BY-2.0-UK
*****************************************
The Yocto Project Development Environment
diff --git a/poky/documentation/overview-manual/overview-manual-development-environment.xml b/poky/documentation/overview-manual/overview-manual-development-environment.xml
new file mode 100644
index 0000000..08ad071
--- /dev/null
+++ b/poky/documentation/overview-manual/overview-manual-development-environment.xml
@@ -0,0 +1,954 @@
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
+"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
+[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
+<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
+
+<chapter id='overview-development-environment'>
+<title>The Yocto Project Development Environment</title>
+
+<para>
+ This chapter takes a look at the Yocto Project development
+ environment.
+ The chapter provides Yocto Project Development environment concepts that
+ help you understand how work is accomplished in an open source environment,
+ which is very different as compared to work accomplished in a closed,
+ proprietary environment.
+</para>
+
+<para>
+ Specifically, this chapter addresses open source philosophy, source
+ repositories, workflows, Git, and licensing.
+</para>
+
+<section id='open-source-philosophy'>
+ <title>Open Source Philosophy</title>
+
+ <para>
+ Open source philosophy is characterized by software development
+ directed by peer production and collaboration through an active
+ community of developers.
+ Contrast this to the more standard centralized development models
+ used by commercial software companies where a finite set of developers
+ produces a product for sale using a defined set of procedures that
+ ultimately result in an end product whose architecture and source
+ material are closed to the public.
+ </para>
+
+ <para>
+ Open source projects conceptually have differing concurrent agendas,
+ approaches, and production.
+ These facets of the development process can come from anyone in the
+ public (community) who has a stake in the software project.
+ The open source environment contains new copyright, licensing, domain,
+ and consumer issues that differ from the more traditional development
+ environment.
+ In an open source environment, the end product, source material,
+ and documentation are all available to the public at no cost.
+ </para>
+
+ <para>
+ A benchmark example of an open source project is the Linux kernel,
+ which was initially conceived and created by Finnish computer science
+ student Linus Torvalds in 1991.
+ Conversely, a good example of a non-open source project is the
+ <trademark class='registered'>Windows</trademark> family of operating
+ systems developed by
+ <trademark class='registered'>Microsoft</trademark> Corporation.
+ </para>
+
+ <para>
+ Wikipedia has a good historical description of the Open Source
+ Philosophy
+ <ulink url='http://en.wikipedia.org/wiki/Open_source'>here</ulink>.
+ You can also find helpful information on how to participate in the
+ Linux Community
+ <ulink url='http://ldn.linuxfoundation.org/book/how-participate-linux-community'>here</ulink>.
+ </para>
+</section>
+
+<section id='gs-the-development-host'>
+ <title>The Development Host</title>
+
+ <para>
+ A development host or
+ <ulink url='&YOCTO_DOCS_REF_URL;#hardware-build-system-term'>build host</ulink>
+ is key to using the Yocto Project.
+ Because the goal of the Yocto Project is to develop images or
+ applications that run on embedded hardware, development of those
+ images and applications generally takes place on a system not
+ intended to run the software - the development host.
+ </para>
+
+ <para>
+ You need to set up a development host in order to use it with the
+ Yocto Project.
+ Most find that it is best to have a native Linux machine function as
+ the development host.
+ However, it is possible to use a system that does not run Linux
+ as its operating system as your development host.
+ When you have a Mac or Windows-based system, you can set it up
+ as the development host by using
+ <ulink url='https://github.com/crops/poky-container'>CROPS</ulink>,
+ which leverages
+ <ulink url='https://www.docker.com/'>Docker Containers</ulink>.
+ Once you take the steps to set up a CROPS machine, you effectively
+ have access to a shell environment that is similar to what you see
+ when using a Linux-based development host.
+ For the steps needed to set up a system using CROPS, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#setting-up-to-use-crops'>Setting Up to Use CROss PlatformS (CROPS)</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para>
+
+ <para>
+ If your development host is going to be a system that runs a Linux
+ distribution, steps still exist that you must take to prepare the
+ system for use with the Yocto Project.
+ You need to be sure that the Linux distribution on the system is
+ one that supports the Yocto Project.
+ You also need to be sure that the correct set of host packages are
+ installed that allow development using the Yocto Project.
+ For the steps needed to set up a development host that runs Linux,
+ see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#setting-up-a-native-linux-host'>Setting Up a Native Linux Host</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para>
+
+ <para>
+ Once your development host is set up to use the Yocto Project,
+ several methods exist for you to do work in the Yocto Project
+ environment:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis>Command Lines, BitBake, and Shells:</emphasis>
+ Traditional development in the Yocto Project involves using the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>OpenEmbedded build system</ulink>,
+ which uses BitBake, in a command-line environment from a shell
+ on your development host.
+ You can accomplish this from a host that is a native Linux
+ machine or from a host that has been set up with CROPS.
+ Either way, you create, modify, and build images and
+ applications all within a shell-based environment using
+ components and tools available through your Linux distribution
+ and the Yocto Project.</para>
+
+ <para>For a general flow of the build procedures, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#dev-building-a-simple-image'>Building a Simple Image</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Board Support Package (BSP) Development:</emphasis>
+ Development of BSPs involves using the Yocto Project to
+ create and test layers that allow easy development of
+ images and applications targeted for specific hardware.
+ To development BSPs, you need to take some additional steps
+ beyond what was described in setting up a development host.
+ </para>
+
+ <para>The
+ <ulink url='&YOCTO_DOCS_BSP_URL;'>Yocto Project Board Support Package (BSP) Developer's Guide</ulink>
+ provides BSP-related development information.
+ For specifics on development host preparation, see the
+ "<ulink url='&YOCTO_DOCS_BSP_URL;#preparing-your-build-host-to-work-with-bsp-layers'>Preparing Your Build Host to Work With BSP Layers</ulink>"
+ section in the Yocto Project Board Support Package (BSP)
+ Developer's Guide.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Kernel Development:</emphasis>
+ If you are going to be developing kernels using the Yocto
+ Project you likely will be using <filename>devtool</filename>.
+ A workflow using <filename>devtool</filename> makes kernel
+ development quicker by reducing iteration cycle times.</para>
+
+ <para>The
+ <ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;'>Yocto Project Linux Kernel Development Manual</ulink>
+ provides kernel-related development information.
+ For specifics on development host preparation, see the
+ "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#preparing-the-build-host-to-work-on-the-kernel'>Preparing the Build Host to Work on the Kernel</ulink>"
+ section in the Yocto Project Linux Kernel Development Manual.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Using Toaster:</emphasis>
+ The other Yocto Project development method that involves an
+ interface that effectively puts the Yocto Project into the
+ background is Toaster.
+ Toaster provides an interface to the OpenEmbedded build system.
+ The interface enables you to configure and run your builds.
+ Information about builds is collected and stored in a database.
+ You can use Toaster to configure and start builds on multiple
+ remote build servers.</para>
+
+ <para>For steps that show you how to set up your development
+ host to use Toaster and on how to use Toaster in general,
+ see the
+ <ulink url='&YOCTO_DOCS_TOAST_URL;'>Toaster User Manual</ulink>.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+</section>
+
+<section id='yocto-project-repositories'>
+ <title>Yocto Project Source Repositories</title>
+
+ <para>
+ The Yocto Project team maintains complete source repositories for all
+ Yocto Project files at
+ <ulink url='&YOCTO_GIT_URL;'></ulink>.
+ This web-based source code browser is organized into categories by
+ function such as IDE Plugins, Matchbox, Poky, Yocto Linux Kernel, and
+ so forth.
+ From the interface, you can click on any particular item in the "Name"
+ column and see the URL at the bottom of the page that you need to clone
+ a Git repository for that particular item.
+ Having a local Git repository of the
+ <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>,
+ which is usually named "poky", allows
+ you to make changes, contribute to the history, and ultimately enhance
+ the Yocto Project's tools, Board Support Packages, and so forth.
+ </para>
+
+ <para>
+ For any supported release of Yocto Project, you can also go to the
+ <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink> and
+ select the "DOWNLOADS" item from the "SOFTWARE" menu and get a
+ released tarball of the <filename>poky</filename> repository, any
+ supported BSP tarball, or Yocto Project tools.
+ Unpacking these tarballs gives you a snapshot of the released
+ files.
+ <note><title>Notes</title>
+ <itemizedlist>
+ <listitem><para>
+ The recommended method for setting up the Yocto Project
+ <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>
+ and the files for supported BSPs
+ (e.g., <filename>meta-intel</filename>) is to use
+ <link linkend='git'>Git</link> to create a local copy of
+ the upstream repositories.
+ </para></listitem>
+ <listitem><para>
+ Be sure to always work in matching branches for both
+ the selected BSP repository and the Source Directory
+ (i.e. <filename>poky</filename>) repository.
+ For example, if you have checked out the "master" branch
+ of <filename>poky</filename> and you are going to use
+ <filename>meta-intel</filename>, be sure to checkout the
+ "master" branch of <filename>meta-intel</filename>.
+ </para></listitem>
+ </itemizedlist>
+ </note>
+ </para>
+
+ <para>
+ In summary, here is where you can get the project files needed for
+ development:
+ <itemizedlist>
+ <listitem><para id='source-repositories'>
+ <emphasis>
+ <ulink url='&YOCTO_GIT_URL;'>Source Repositories:</ulink>
+ </emphasis>
+ This area contains IDE Plugins, Matchbox, Poky, Poky Support,
+ Tools, Yocto Linux Kernel, and Yocto Metadata Layers.
+ You can create local copies of Git repositories for each of
+ these areas.</para>
+
+ <para>
+ <imagedata fileref="figures/source-repos.png" align="center" width="6in" depth="4in" />
+ For steps on how to view and access these upstream Git
+ repositories, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#accessing-source-repositories'>Accessing Source Repositories</ulink>"
+ Section in the Yocto Project Development Tasks Manual.
+ </para></listitem>
+ <listitem><para><anchor id='index-downloads' />
+ <emphasis>
+ <ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink>
+ </emphasis>
+ This is an index of releases such as Poky, Pseudo, installers
+ for cross-development toolchains, miscellaneous support
+ and all released versions of Yocto Project in the form of
+ images or tarballs.
+ Downloading and extracting these files does not produce a local
+ copy of the Git repository but rather a snapshot of a
+ particular release or image.</para>
+
+ <para>
+ <imagedata fileref="figures/index-downloads.png" align="center" width="6in" depth="3.5in" />
+ For steps on how to view and access these files, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#accessing-index-of-releases'>Accessing Index of Releases</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para></listitem>
+ <listitem><para id='downloads-page'>
+ <emphasis>"DOWNLOADS" page for the
+ <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>:
+ </emphasis></para>
+
+ <para>The Yocto Project website includes a "DOWNLOADS" page
+ accessible through the "SOFTWARE" menu that allows you to
+ download any Yocto Project release, tool, and Board Support
+ Package (BSP) in tarball form.
+ The tarballs are similar to those found in the
+ <ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink>
+ area.</para>
+
+ <para>
+ <imagedata fileref="figures/yp-download.png" align="center" width="6in" depth="4in" />
+ For steps on how to use the "DOWNLOADS" page, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#using-the-downloads-page'>Using the Downloads Page</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+</section>
+
+<section id='gs-git-workflows-and-the-yocto-project'>
+ <title>Git Workflows and the Yocto Project</title>
+
+ <para>
+ Developing using the Yocto Project likely requires the use of
+ <link linkend='git'>Git</link>.
+ Git is a free, open source distributed version control system
+ used as part of many collaborative design environments.
+ This section provides workflow concepts using the Yocto Project and
+ Git.
+ In particular, the information covers basic practices that describe
+ roles and actions in a collaborative development environment.
+ <note>
+ If you are familiar with this type of development environment, you
+ might not want to read this section.
+ </note>
+ </para>
+
+ <para>
+ The Yocto Project files are maintained using Git in "branches"
+ whose Git histories track every change and whose structures
+ provide branches for all diverging functionality.
+ Although there is no need to use Git, many open source projects do so.
+ <para>
+
+ </para>
+ For the Yocto Project, a key individual called the "maintainer" is
+ responsible for the integrity of the "master" branch of a given Git
+ repository.
+ The "master" branch is the "upstream" repository from which final or
+ most recent builds of a project occur.
+ The maintainer is responsible for accepting changes from other
+ developers and for organizing the underlying branch structure to
+ reflect release strategies and so forth.
+ <note>
+ For information on finding out who is responsible for (maintains)
+ a particular area of code in the Yocto Project, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change'>Submitting a Change to the Yocto Project</ulink>"
+ section of the Yocto Project Development Tasks Manual.
+ </note>
+ </para>
+
+ <para>
+ The Yocto Project <filename>poky</filename> Git repository also has an
+ upstream contribution Git repository named
+ <filename>poky-contrib</filename>.
+ You can see all the branches in this repository using the web interface
+ of the
+ <ulink url='&YOCTO_GIT_URL;'>Source Repositories</ulink> organized
+ within the "Poky Support" area.
+ These branches hold changes (commits) to the project that have been
+ submitted or committed by the Yocto Project development team and by
+ community members who contribute to the project.
+ The maintainer determines if the changes are qualified to be moved
+ from the "contrib" branches into the "master" branch of the Git
+ repository.
+ </para>
+
+ <para>
+ Developers (including contributing community members) create and
+ maintain cloned repositories of upstream branches.
+ The cloned repositories are local to their development platforms and
+ are used to develop changes.
+ When a developer is satisfied with a particular feature or change,
+ they "push" the change to the appropriate "contrib" repository.
+ </para>
+
+ <para>
+ Developers are responsible for keeping their local repository
+ up-to-date with whatever upstream branch they are working against.
+ They are also responsible for straightening out any conflicts that
+ might arise within files that are being worked on simultaneously by
+ more than one person.
+ All this work is done locally on the development host before
+ anything is pushed to a "contrib" area and examined at the maintainer's
+ level.
+ </para>
+
+ <para>
+ A somewhat formal method exists by which developers commit changes
+ and push them into the "contrib" area and subsequently request that
+ the maintainer include them into an upstream branch.
+ This process is called "submitting a patch" or "submitting a change."
+ For information on submitting patches and changes, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change'>Submitting a Change to the Yocto Project</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para>
+
+ <para>
+ In summary, a single point of entry
+ exists for changes into a "master" or development branch of the
+ Git repository, which is controlled by the project's maintainer.
+ And, a set of developers exist who independently develop, test, and
+ submit changes to "contrib" areas for the maintainer to examine.
+ The maintainer then chooses which changes are going to become a
+ permanent part of the project.
+ </para>
+
+ <para>
+ <imagedata fileref="figures/git-workflow.png" width="6in" depth="3in" align="left" scalefit="1" />
+ </para>
+
+ <para>
+ While each development environment is unique, there are some best
+ practices or methods that help development run smoothly.
+ The following list describes some of these practices.
+ For more information about Git workflows, see the workflow topics in
+ the
+ <ulink url='http://book.git-scm.com'>Git Community Book</ulink>.
+ <itemizedlist>
+ <listitem><para>
+ <emphasis>Make Small Changes:</emphasis>
+ It is best to keep the changes you commit small as compared to
+ bundling many disparate changes into a single commit.
+ This practice not only keeps things manageable but also allows
+ the maintainer to more easily include or refuse changes.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Make Complete Changes:</emphasis>
+ It is also good practice to leave the repository in a
+ state that allows you to still successfully build your project.
+ In other words, do not commit half of a feature,
+ then add the other half as a separate, later commit.
+ Each commit should take you from one buildable project state
+ to another buildable state.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Use Branches Liberally:</emphasis>
+ It is very easy to create, use, and delete local branches in
+ your working Git repository on the development host.
+ You can name these branches anything you like.
+ It is helpful to give them names associated with the particular
+ feature or change on which you are working.
+ Once you are done with a feature or change and have merged it
+ into your local master branch, simply discard the temporary
+ branch.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Merge Changes:</emphasis>
+ The <filename>git merge</filename> command allows you to take
+ the changes from one branch and fold them into another branch.
+ This process is especially helpful when more than a single
+ developer might be working on different parts of the same
+ feature.
+ Merging changes also automatically identifies any collisions
+ or "conflicts" that might happen as a result of the same lines
+ of code being altered by two different developers.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Manage Branches:</emphasis>
+ Because branches are easy to use, you should use a system
+ where branches indicate varying levels of code readiness.
+ For example, you can have a "work" branch to develop in, a
+ "test" branch where the code or change is tested, a "stage"
+ branch where changes are ready to be committed, and so forth.
+ As your project develops, you can merge code across the
+ branches to reflect ever-increasing stable states of the
+ development.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Use Push and Pull:</emphasis>
+ The push-pull workflow is based on the concept of developers
+ "pushing" local commits to a remote repository, which is
+ usually a contribution repository.
+ This workflow is also based on developers "pulling" known
+ states of the project down into their local development
+ repositories.
+ The workflow easily allows you to pull changes submitted by
+ other developers from the upstream repository into your
+ work area ensuring that you have the most recent software
+ on which to develop.
+ The Yocto Project has two scripts named
+ <filename>create-pull-request</filename> and
+ <filename>send-pull-request</filename> that ship with the
+ release to facilitate this workflow.
+ You can find these scripts in the <filename>scripts</filename>
+ folder of the
+ <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>.
+ For information on how to use these scripts, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#pushing-a-change-upstream'>Using Scripts to Push a Change Upstream and Request a Pull</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Patch Workflow:</emphasis>
+ This workflow allows you to notify the maintainer through an
+ email that you have a change (or patch) you would like
+ considered for the "master" branch of the Git repository.
+ To send this type of change, you format the patch and then
+ send the email using the Git commands
+ <filename>git format-patch</filename> and
+ <filename>git send-email</filename>.
+ For information on how to use these scripts, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change'>Submitting a Change to the Yocto Project</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+</section>
+
+<section id='git'>
+ <title>Git</title>
+
+ <para>
+ The Yocto Project makes extensive use of Git, which is a
+ free, open source distributed version control system.
+ Git supports distributed development, non-linear development,
+ and can handle large projects.
+ It is best that you have some fundamental understanding
+ of how Git tracks projects and how to work with Git if
+ you are going to use the Yocto Project for development.
+ This section provides a quick overview of how Git works and
+ provides you with a summary of some essential Git commands.
+ <note><title>Notes</title>
+ <itemizedlist>
+ <listitem><para>
+ For more information on Git, see
+ <ulink url='http://git-scm.com/documentation'></ulink>.
+ </para></listitem>
+ <listitem><para>
+ If you need to download Git, it is recommended that you add
+ Git to your system through your distribution's "software
+ store" (e.g. for Ubuntu, use the Ubuntu Software feature).
+ For the Git download page, see
+ <ulink url='http://git-scm.com/download'></ulink>.
+ </para></listitem>
+ <listitem><para>
+ For information beyond the introductory nature in this
+ section, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#locating-yocto-project-source-files'>Locating Yocto Project Source Files</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para></listitem>
+ </itemizedlist>
+ </note>
+ </para>
+
+ <section id='repositories-tags-and-branches'>
+ <title>Repositories, Tags, and Branches</title>
+
+ <para>
+ As mentioned briefly in the previous section and also in the
+ "<link linkend='gs-git-workflows-and-the-yocto-project'>Git Workflows and the Yocto Project</link>"
+ section, the Yocto Project maintains source repositories at
+ <ulink url='&YOCTO_GIT_URL;'></ulink>.
+ If you look at this web-interface of the repositories, each item
+ is a separate Git repository.
+ </para>
+
+ <para>
+ Git repositories use branching techniques that track content
+ change (not files) within a project (e.g. a new feature or updated
+ documentation).
+ Creating a tree-like structure based on project divergence allows
+ for excellent historical information over the life of a project.
+ This methodology also allows for an environment from which you can
+ do lots of local experimentation on projects as you develop
+ changes or new features.
+ </para>
+
+ <para>
+ A Git repository represents all development efforts for a given
+ project.
+ For example, the Git repository <filename>poky</filename> contains
+ all changes and developments for that repository over the course
+ of its entire life.
+ That means that all changes that make up all releases are captured.
+ The repository maintains a complete history of changes.
+ </para>
+
+ <para>
+ You can create a local copy of any repository by "cloning" it
+ with the <filename>git clone</filename> command.
+ When you clone a Git repository, you end up with an identical
+ copy of the repository on your development system.
+ Once you have a local copy of a repository, you can take steps to
+ develop locally.
+ For examples on how to clone Git repositories, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#locating-yocto-project-source-files'>Locating Yocto Project Source Files</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para>
+
+ <para>
+ It is important to understand that Git tracks content change and
+ not files.
+ Git uses "branches" to organize different development efforts.
+ For example, the <filename>poky</filename> repository has
+ several branches that include the current "&DISTRO_NAME_NO_CAP;"
+ branch, the "master" branch, and many branches for past
+ Yocto Project releases.
+ You can see all the branches by going to
+ <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and
+ clicking on the
+ <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/heads'>[...]</ulink></filename>
+ link beneath the "Branch" heading.
+ </para>
+
+ <para>
+ Each of these branches represents a specific area of development.
+ The "master" branch represents the current or most recent
+ development.
+ All other branches represent offshoots of the "master" branch.
+ </para>
+
+ <para>
+ When you create a local copy of a Git repository, the copy has
+ the same set of branches as the original.
+ This means you can use Git to create a local working area
+ (also called a branch) that tracks a specific development branch
+ from the upstream source Git repository.
+ in other words, you can define your local Git environment to
+ work on any development branch in the repository.
+ To help illustrate, consider the following example Git commands:
+ <literallayout class='monospaced'>
+ $ cd ~
+ $ git clone git://git.yoctoproject.org/poky
+ $ cd poky
+ $ git checkout -b &DISTRO_NAME_NO_CAP; origin/&DISTRO_NAME_NO_CAP;
+ </literallayout>
+ In the previous example after moving to the home directory, the
+ <filename>git clone</filename> command creates a
+ local copy of the upstream <filename>poky</filename> Git repository.
+ By default, Git checks out the "master" branch for your work.
+ After changing the working directory to the new local repository
+ (i.e. <filename>poky</filename>), the
+ <filename>git checkout</filename> command creates
+ and checks out a local branch named "&DISTRO_NAME_NO_CAP;", which
+ tracks the upstream "origin/&DISTRO_NAME_NO_CAP;" branch.
+ Changes you make while in this branch would ultimately affect
+ the upstream "&DISTRO_NAME_NO_CAP;" branch of the
+ <filename>poky</filename> repository.
+ </para>
+
+ <para>
+ It is important to understand that when you create and checkout a
+ local working branch based on a branch name,
+ your local environment matches the "tip" of that particular
+ development branch at the time you created your local branch,
+ which could be different from the files in the "master" branch
+ of the upstream repository.
+ In other words, creating and checking out a local branch based on
+ the "&DISTRO_NAME_NO_CAP;" branch name is not the same as
+ checking out the "master" branch in the repository.
+ Keep reading to see how you create a local snapshot of a Yocto
+ Project Release.
+ </para>
+
+ <para>
+ Git uses "tags" to mark specific changes in a repository branch
+ structure.
+ Typically, a tag is used to mark a special point such as the final
+ change (or commit) before a project is released.
+ You can see the tags used with the <filename>poky</filename> Git
+ repository by going to
+ <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and
+ clicking on the
+ <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/tags'>[...]</ulink></filename>
+ link beneath the "Tag" heading.
+ </para>
+
+ <para>
+ Some key tags for the <filename>poky</filename> repository are
+ <filename>jethro-14.0.3</filename>,
+ <filename>morty-16.0.1</filename>,
+ <filename>pyro-17.0.0</filename>, and
+ <filename>&DISTRO_NAME_NO_CAP;-&POKYVERSION;</filename>.
+ These tags represent Yocto Project releases.
+ </para>
+
+ <para>
+ When you create a local copy of the Git repository, you also
+ have access to all the tags in the upstream repository.
+ Similar to branches, you can create and checkout a local working
+ Git branch based on a tag name.
+ When you do this, you get a snapshot of the Git repository that
+ reflects the state of the files when the change was made associated
+ with that tag.
+ The most common use is to checkout a working branch that matches
+ a specific Yocto Project release.
+ Here is an example:
+ <literallayout class='monospaced'>
+ $ cd ~
+ $ git clone git://git.yoctoproject.org/poky
+ $ cd poky
+ $ git fetch --tags
+ $ git checkout tags/rocko-18.0.0 -b my_rocko-18.0.0
+ </literallayout>
+ In this example, the name of the top-level directory of your
+ local Yocto Project repository is <filename>poky</filename>.
+ After moving to the <filename>poky</filename> directory, the
+ <filename>git fetch</filename> command makes all the upstream
+ tags available locally in your repository.
+ Finally, the <filename>git checkout</filename> command
+ creates and checks out a branch named "my-rocko-18.0.0" that is
+ based on the upstream branch whose "HEAD" matches the
+ commit in the repository associated with the "rocko-18.0.0" tag.
+ The files in your repository now exactly match that particular
+ Yocto Project release as it is tagged in the upstream Git
+ repository.
+ It is important to understand that when you create and
+ checkout a local working branch based on a tag, your environment
+ matches a specific point in time and not the entire development
+ branch (i.e. from the "tip" of the branch backwards).
+ </para>
+ </section>
+
+ <section id='basic-commands'>
+ <title>Basic Commands</title>
+
+ <para>
+ Git has an extensive set of commands that lets you manage changes
+ and perform collaboration over the life of a project.
+ Conveniently though, you can manage with a small set of basic
+ operations and workflows once you understand the basic
+ philosophy behind Git.
+ You do not have to be an expert in Git to be functional.
+ A good place to look for instruction on a minimal set of Git
+ commands is
+ <ulink url='http://git-scm.com/documentation'>here</ulink>.
+ </para>
+
+ <para>
+ The following list of Git commands briefly describes some basic
+ Git operations as a way to get started.
+ As with any set of commands, this list (in most cases) simply shows
+ the base command and omits the many arguments it supports.
+ See the Git documentation for complete descriptions and strategies
+ on how to use these commands:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis><filename>git init</filename>:</emphasis>
+ Initializes an empty Git repository.
+ You cannot use Git commands unless you have a
+ <filename>.git</filename> repository.
+ </para></listitem>
+ <listitem><para id='git-commands-clone'>
+ <emphasis><filename>git clone</filename>:</emphasis>
+ Creates a local clone of a Git repository that is on
+ equal footing with a fellow developer's Git repository
+ or an upstream repository.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>git add</filename>:</emphasis>
+ Locally stages updated file contents to the index that
+ Git uses to track changes.
+ You must stage all files that have changed before you
+ can commit them.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>git commit</filename>:</emphasis>
+ Creates a local "commit" that documents the changes you
+ made.
+ Only changes that have been staged can be committed.
+ Commits are used for historical purposes, for determining
+ if a maintainer of a project will allow the change,
+ and for ultimately pushing the change from your local
+ Git repository into the project's upstream repository.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>git status</filename>:</emphasis>
+ Reports any modified files that possibly need to be
+ staged and gives you a status of where you stand regarding
+ local commits as compared to the upstream repository.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>git checkout</filename> <replaceable>branch-name</replaceable>:</emphasis>
+ Changes your local working branch and in this form
+ assumes the local branch already exists.
+ This command is analogous to "cd".
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>git checkout –b</filename> <replaceable>working-branch</replaceable> <replaceable>upstream-branch</replaceable>:</emphasis>
+ Creates and checks out a working branch on your local
+ machine.
+ The local branch tracks the upstream branch.
+ You can use your local branch to isolate your work.
+ It is a good idea to use local branches when adding
+ specific features or changes.
+ Using isolated branches facilitates easy removal of
+ changes if they do not work out.
+ </para></listitem>
+ <listitem><para><emphasis><filename>git branch</filename>:</emphasis>
+ Displays the existing local branches associated with your
+ local repository.
+ The branch that you have currently checked out is noted
+ with an asterisk character.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>git branch -D</filename> <replaceable>branch-name</replaceable>:</emphasis>
+ Deletes an existing local branch.
+ You need to be in a local branch other than the one you
+ are deleting in order to delete
+ <replaceable>branch-name</replaceable>.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>git pull --rebase</filename>:</emphasis>
+ Retrieves information from an upstream Git repository
+ and places it in your local Git repository.
+ You use this command to make sure you are synchronized with
+ the repository from which you are basing changes
+ (.e.g. the "master" branch).
+ The "--rebase" option ensures that any local commits you
+ have in your branch are preserved at the top of your
+ local branch.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>git push</filename> <replaceable>repo-name</replaceable> <replaceable>local-branch</replaceable><filename>:</filename><replaceable>upstream-branch</replaceable>:</emphasis>
+ Sends all your committed local changes to the upstream Git
+ repository that your local repository is tracking
+ (e.g. a contribution repository).
+ The maintainer of the project draws from these repositories
+ to merge changes (commits) into the appropriate branch
+ of project's upstream repository.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>git merge</filename>:</emphasis>
+ Combines or adds changes from one
+ local branch of your repository with another branch.
+ When you create a local Git repository, the default branch
+ is named "master".
+ A typical workflow is to create a temporary branch that is
+ based off "master" that you would use for isolated work.
+ You would make your changes in that isolated branch,
+ stage and commit them locally, switch to the "master"
+ branch, and then use the <filename>git merge</filename>
+ command to apply the changes from your isolated branch
+ into the currently checked out branch (e.g. "master").
+ After the merge is complete and if you are done with
+ working in that isolated branch, you can safely delete
+ the isolated branch.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>git cherry-pick</filename> <replaceable>commits</replaceable>:</emphasis>
+ Choose and apply specific commits from one branch
+ into another branch.
+ There are times when you might not be able to merge
+ all the changes in one branch with
+ another but need to pick out certain ones.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>gitk</filename>:</emphasis>
+ Provides a GUI view of the branches and changes in your
+ local Git repository.
+ This command is a good way to graphically see where things
+ have diverged in your local repository.
+ <note>
+ You need to install the <filename>gitk</filename>
+ package on your development system to use this
+ command.
+ </note>
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>git log</filename>:</emphasis>
+ Reports a history of your commits to the repository.
+ This report lists all commits regardless of whether you
+ have pushed them upstream or not.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>git diff</filename>:</emphasis>
+ Displays line-by-line differences between a local
+ working file and the same file as understood by Git.
+ This command is useful to see what you have changed
+ in any given file.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </section>
+</section>
+
+<section id='licensing'>
+ <title>Licensing</title>
+
+ <para>
+ Because open source projects are open to the public, they have
+ different licensing structures in place.
+ License evolution for both Open Source and Free Software has an
+ interesting history.
+ If you are interested in this history, you can find basic information
+ here:
+ <itemizedlist>
+ <listitem><para>
+ <ulink url='http://en.wikipedia.org/wiki/Open-source_license'>Open source license history</ulink>
+ </para></listitem>
+ <listitem><para>
+ <ulink url='http://en.wikipedia.org/wiki/Free_software_license'>Free software license history</ulink>
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ In general, the Yocto Project is broadly licensed under the
+ Massachusetts Institute of Technology (MIT) License.
+ MIT licensing permits the reuse of software within proprietary
+ software as long as the license is distributed with that software.
+ MIT is also compatible with the GNU General Public License (GPL).
+ Patches to the Yocto Project follow the upstream licensing scheme.
+ You can find information on the MIT license
+ <ulink url='http://www.opensource.org/licenses/mit-license.php'>here</ulink>.
+ You can find information on the GNU GPL
+ <ulink url='http://www.opensource.org/licenses/LGPL-3.0'>here</ulink>.
+ </para>
+
+ <para>
+ When you build an image using the Yocto Project, the build process
+ uses a known list of licenses to ensure compliance.
+ You can find this list in the
+ <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>
+ at <filename>meta/files/common-licenses</filename>.
+ Once the build completes, the list of all licenses found and used
+ during that build are kept in the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>
+ at <filename>tmp/deploy/licenses</filename>.
+ </para>
+
+ <para>
+ If a module requires a license that is not in the base list, the
+ build process generates a warning during the build.
+ These tools make it easier for a developer to be certain of the
+ licenses with which their shipped products must comply.
+ However, even with these tools it is still up to the developer to
+ resolve potential licensing issues.
+ </para>
+
+ <para>
+ The base list of licenses used by the build process is a combination
+ of the Software Package Data Exchange (SPDX) list and the Open
+ Source Initiative (OSI) projects.
+ <ulink url='http://spdx.org'>SPDX Group</ulink> is a working group of
+ the Linux Foundation that maintains a specification for a standard
+ format for communicating the components, licenses, and copyrights
+ associated with a software package.
+ <ulink url='http://opensource.org'>OSI</ulink> is a corporation
+ dedicated to the Open Source Definition and the effort for reviewing
+ and approving licenses that conform to the Open Source Definition
+ (OSD).
+ </para>
+
+ <para>
+ You can find a list of the combined SPDX and OSI licenses that the
+ Yocto Project uses in the
+ <filename>meta/files/common-licenses</filename> directory in your
+ <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>.
+ </para>
+
+ <para>
+ For information that can help you maintain compliance with various
+ open source licensing during the lifecycle of a product created using
+ the Yocto Project, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Product's Lifecycle</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para>
+</section>
+</chapter>
+<!--
+vim: expandtab tw=80 ts=4
+-->
diff --git a/poky/documentation/overview-manual/overview-manual-intro.rst b/poky/documentation/overview-manual/overview-manual-intro.rst
index 8885eb8..3f206fd 100644
--- a/poky/documentation/overview-manual/overview-manual-intro.rst
+++ b/poky/documentation/overview-manual/overview-manual-intro.rst
@@ -1,4 +1,4 @@
-.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
+.. SPDX-License-Identifier: CC-BY-2.0-UK
**********************************************
The Yocto Project Overview and Concepts Manual
diff --git a/poky/documentation/overview-manual/overview-manual-intro.xml b/poky/documentation/overview-manual/overview-manual-intro.xml
new file mode 100644
index 0000000..0e0bfed
--- /dev/null
+++ b/poky/documentation/overview-manual/overview-manual-intro.xml
@@ -0,0 +1,113 @@
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
+"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
+[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
+<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
+
+<chapter id='overview-manual-intro'>
+
+<title>The Yocto Project Overview and Concepts Manual</title>
+ <section id='overview-manual-welcome'>
+ <title>Welcome</title>
+
+ <para>
+ Welcome to the Yocto Project Overview and Concepts Manual!
+ This manual introduces the Yocto Project by providing concepts,
+ software overviews, best-known-methods (BKMs), and any other
+ high-level introductory information suitable for a new Yocto
+ Project user.
+ </para>
+
+ <para>
+ The following list describes what you can get from this manual:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis><link linkend='overview-yp'>Introducing the Yocto Project</link>:</emphasis>
+ This chapter provides an introduction to the Yocto
+ Project.
+ You will learn about features and challenges of the
+ Yocto Project, the layer model, components and tools,
+ development methods, the
+ <ulink url='&YOCTO_DOCS_REF_URL;#poky'>Poky</ulink>
+ reference distribution, the OpenEmbedded build system
+ workflow, and some basic Yocto terms.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><link linkend='overview-development-environment'>The Yocto Project Development Environment</link>:</emphasis>
+ This chapter helps you get started understanding the
+ Yocto Project development environment.
+ You will learn about open source, development hosts,
+ Yocto Project source repositories, workflows using Git
+ and the Yocto Project, a Git primer, and information
+ about licensing.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><link linkend='overview-manual-concepts'>Yocto Project Concepts</link>:</emphasis>
+ This chapter presents various concepts regarding the
+ Yocto Project.
+ You can find conceptual information about components,
+ development, cross-toolchains, and so forth.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ This manual does not give you the following:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis>Step-by-step Instructions for Development Tasks:</emphasis>
+ Instructional procedures reside in other manuals within
+ the Yocto Project documentation set.
+ For example, the
+ <ulink url='&YOCTO_DOCS_DEV_URL;'>Yocto Project Development Tasks Manual</ulink>
+ provides examples on how to perform various development
+ tasks.
+ As another example, the
+ <ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Application Development and the Extensible Software Development Kit (eSDK)</ulink>
+ manual contains detailed instructions on how to install an
+ SDK, which is used to develop applications for target
+ hardware.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Reference Material:</emphasis>
+ This type of material resides in an appropriate reference
+ manual.
+ For example, system variables are documented in the
+ <ulink url='&YOCTO_DOCS_REF_URL;'>Yocto Project Reference Manual</ulink>.
+ As another example, the
+ <ulink url='&YOCTO_DOCS_BSP_URL;'>Yocto Project Board Support Package (BSP) Developer's Guide</ulink>
+ contains reference information on BSPs.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Detailed Public Information Not Specific to the
+ Yocto Project:</emphasis>
+ For example, exhaustive information on how to use the
+ Source Control Manager Git is better covered with Internet
+ searches and official Git Documentation than through the
+ Yocto Project documentation.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </section>
+
+ <section id='overview-manual-other-information'>
+ <title>Other Information</title>
+
+ <para>
+ Because this manual presents information for many different
+ topics, supplemental information is recommended for full
+ comprehension.
+ For additional introductory information on the Yocto Project, see
+ the <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>.
+ If you want to build an image with no knowledge of Yocto Project
+ as a way of quickly testing it out, see the
+ <ulink url='&YOCTO_DOCS_BRIEF_URL;'>Yocto Project Quick Build</ulink>
+ document.
+ For a comprehensive list of links and other documentation, see the
+ "<ulink url='&YOCTO_DOCS_REF_URL;#resources-links-and-related-documentation'>Links and Related Documentation</ulink>"
+ section in the Yocto Project Reference Manual.
+ </para>
+ </section>
+</chapter>
+<!--
+vim: expandtab tw=80 ts=4
+-->
diff --git a/poky/documentation/overview-manual/overview-manual-style.css b/poky/documentation/overview-manual/overview-manual-style.css
new file mode 100644
index 0000000..eec9341
--- /dev/null
+++ b/poky/documentation/overview-manual/overview-manual-style.css
@@ -0,0 +1,990 @@
+/*
+ SPDX-License-Identifier: CC-BY-2.0-UK
+
+ Generic XHTML / DocBook XHTML CSS Stylesheet.
+
+ Browser wrangling and typographic design by
+ Oyvind Kolas / pippin@gimp.org
+
+ Customised for Poky by
+ Matthew Allum / mallum@o-hand.com
+
+ Thanks to:
+ Liam R. E. Quin
+ William Skaggs
+ Jakub Steiner
+
+ Structure
+ ---------
+
+ The stylesheet is divided into the following sections:
+
+ Positioning
+ Margins, paddings, width, font-size, clearing.
+ Decorations
+ Borders, style
+ Colors
+ Colors
+ Graphics
+ Graphical backgrounds
+ Nasty IE tweaks
+ Workarounds needed to make it work in internet explorer,
+ currently makes the stylesheet non validating, but up until
+ this point it is validating.
+ Mozilla extensions
+ Transparency for footer
+ Rounded corners on boxes
+
+*/
+
+
+ /*************** /
+ / Positioning /
+/ ***************/
+
+body {
+ font-family: Verdana, Sans, sans-serif;
+
+ min-width: 640px;
+ width: 80%;
+ margin: 0em auto;
+ padding: 2em 5em 5em 5em;
+ color: #333;
+}
+
+h1,h2,h3,h4,h5,h6,h7 {
+ font-family: Arial, Sans;
+ color: #00557D;
+ clear: both;
+}
+
+h1 {
+ font-size: 2em;
+ text-align: left;
+ padding: 0em 0em 0em 0em;
+ margin: 2em 0em 0em 0em;
+}
+
+h2.subtitle {
+ margin: 0.10em 0em 3.0em 0em;
+ padding: 0em 0em 0em 0em;
+ font-size: 1.8em;
+ padding-left: 20%;
+ font-weight: normal;
+ font-style: italic;
+}
+
+h2 {
+ margin: 2em 0em 0.66em 0em;
+ padding: 0.5em 0em 0em 0em;
+ font-size: 1.5em;
+ font-weight: bold;
+}
+
+h3.subtitle {
+ margin: 0em 0em 1em 0em;
+ padding: 0em 0em 0em 0em;
+ font-size: 142.14%;
+ text-align: right;
+}
+
+h3 {
+ margin: 1em 0em 0.5em 0em;
+ padding: 1em 0em 0em 0em;
+ font-size: 140%;
+ font-weight: bold;
+}
+
+h4 {
+ margin: 1em 0em 0.5em 0em;
+ padding: 1em 0em 0em 0em;
+ font-size: 120%;
+ font-weight: bold;
+}
+
+h5 {
+ margin: 1em 0em 0.5em 0em;
+ padding: 1em 0em 0em 0em;
+ font-size: 110%;
+ font-weight: bold;
+}
+
+h6 {
+ margin: 1em 0em 0em 0em;
+ padding: 1em 0em 0em 0em;
+ font-size: 110%;
+ font-weight: bold;
+}
+
+.authorgroup {
+ background-color: transparent;
+ background-repeat: no-repeat;
+ padding-top: 256px;
+ background-image: url("figures/overview-manual-title.png");
+ background-position: left top;
+ margin-top: -256px;
+ padding-right: 50px;
+ margin-left: 0px;
+ text-align: right;
+ width: 740px;
+}
+
+h3.author {
+ margin: 0em 0me 0em 0em;
+ padding: 0em 0em 0em 0em;
+ font-weight: normal;
+ font-size: 100%;
+ color: #333;
+ clear: both;
+}
+
+.author tt.email {
+ font-size: 66%;
+}
+
+.titlepage hr {
+ width: 0em;
+ clear: both;
+}
+
+.revhistory {
+ padding-top: 2em;
+ clear: both;
+}
+
+.toc,
+.list-of-tables,
+.list-of-examples,
+.list-of-figures {
+ padding: 1.33em 0em 2.5em 0em;
+ color: #00557D;
+}
+
+.toc p,
+.list-of-tables p,
+.list-of-figures p,
+.list-of-examples p {
+ padding: 0em 0em 0em 0em;
+ padding: 0em 0em 0.3em;
+ margin: 1.5em 0em 0em 0em;
+}
+
+.toc p b,
+.list-of-tables p b,
+.list-of-figures p b,
+.list-of-examples p b{
+ font-size: 100.0%;
+ font-weight: bold;
+}
+
+.toc dl,
+.list-of-tables dl,
+.list-of-figures dl,
+.list-of-examples dl {
+ margin: 0em 0em 0.5em 0em;
+ padding: 0em 0em 0em 0em;
+}
+
+.toc dt {
+ margin: 0em 0em 0em 0em;
+ padding: 0em 0em 0em 0em;
+}
+
+.toc dd {
+ margin: 0em 0em 0em 2.6em;
+ padding: 0em 0em 0em 0em;
+}
+
+div.glossary dl,
+div.variablelist dl {
+}
+
+.glossary dl dt,
+.variablelist dl dt,
+.variablelist dl dt span.term {
+ font-weight: normal;
+ width: 20em;
+ text-align: right;
+}
+
+.variablelist dl dt {
+ margin-top: 0.5em;
+}
+
+.glossary dl dd,
+.variablelist dl dd {
+ margin-top: -1em;
+ margin-left: 25.5em;
+}
+
+.glossary dd p,
+.variablelist dd p {
+ margin-top: 0em;
+ margin-bottom: 1em;
+}
+
+
+div.calloutlist table td {
+ padding: 0em 0em 0em 0em;
+ margin: 0em 0em 0em 0em;
+}
+
+div.calloutlist table td p {
+ margin-top: 0em;
+ margin-bottom: 1em;
+}
+
+div p.copyright {
+ text-align: left;
+}
+
+div.legalnotice p.legalnotice-title {
+ margin-bottom: 0em;
+}
+
+p {
+ line-height: 1.5em;
+ margin-top: 0em;
+
+}
+
+dl {
+ padding-top: 0em;
+}
+
+hr {
+ border: solid 1px;
+}
+
+
+.mediaobject,
+.mediaobjectco {
+ text-align: center;
+}
+
+img {
+ border: none;
+}
+
+ul {
+ padding: 0em 0em 0em 1.5em;
+}
+
+ul li {
+ padding: 0em 0em 0em 0em;
+}
+
+ul li p {
+ text-align: left;
+}
+
+table {
+ width :100%;
+}
+
+th {
+ padding: 0.25em;
+ text-align: left;
+ font-weight: normal;
+ vertical-align: top;
+}
+
+td {
+ padding: 0.25em;
+ vertical-align: top;
+}
+
+p a[id] {
+ margin: 0px;
+ padding: 0px;
+ display: inline;
+ background-image: none;
+}
+
+a {
+ text-decoration: underline;
+ color: #444;
+}
+
+pre {
+ overflow: auto;
+}
+
+a:hover {
+ text-decoration: underline;
+ /*font-weight: bold;*/
+}
+
+/* This style defines how the permalink character
+ appears by itself and when hovered over with
+ the mouse. */
+
+[alt='Permalink'] { color: #eee; }
+[alt='Permalink']:hover { color: black; }
+
+
+div.informalfigure,
+div.informalexample,
+div.informaltable,
+div.figure,
+div.table,
+div.example {
+ margin: 1em 0em;
+ padding: 1em;
+ page-break-inside: avoid;
+}
+
+
+div.informalfigure p.title b,
+div.informalexample p.title b,
+div.informaltable p.title b,
+div.figure p.title b,
+div.example p.title b,
+div.table p.title b{
+ padding-top: 0em;
+ margin-top: 0em;
+ font-size: 100%;
+ font-weight: normal;
+}
+
+.mediaobject .caption,
+.mediaobject .caption p {
+ text-align: center;
+ font-size: 80%;
+ padding-top: 0.5em;
+ padding-bottom: 0.5em;
+}
+
+.epigraph {
+ padding-left: 55%;
+ margin-bottom: 1em;
+}
+
+.epigraph p {
+ text-align: left;
+}
+
+.epigraph .quote {
+ font-style: italic;
+}
+.epigraph .attribution {
+ font-style: normal;
+ text-align: right;
+}
+
+span.application {
+ font-style: italic;
+}
+
+.programlisting {
+ font-family: monospace;
+ font-size: 80%;
+ white-space: pre;
+ margin: 1.33em 0em;
+ padding: 1.33em;
+}
+
+.tip,
+.warning,
+.caution,
+.note {
+ margin-top: 1em;
+ margin-bottom: 1em;
+
+}
+
+/* force full width of table within div */
+.tip table,
+.warning table,
+.caution table,
+.note table {
+ border: none;
+ width: 100%;
+}
+
+
+.tip table th,
+.warning table th,
+.caution table th,
+.note table th {
+ padding: 0.8em 0.0em 0.0em 0.0em;
+ margin : 0em 0em 0em 0em;
+}
+
+.tip p,
+.warning p,
+.caution p,
+.note p {
+ margin-top: 0.5em;
+ margin-bottom: 0.5em;
+ padding-right: 1em;
+ text-align: left;
+}
+
+.acronym {
+ text-transform: uppercase;
+}
+
+b.keycap,
+.keycap {
+ padding: 0.09em 0.3em;
+ margin: 0em;
+}
+
+.itemizedlist li {
+ clear: none;
+}
+
+.filename {
+ font-size: medium;
+ font-family: Courier, monospace;
+}
+
+
+div.navheader, div.heading{
+ position: absolute;
+ left: 0em;
+ top: 0em;
+ width: 100%;
+ background-color: #cdf;
+ width: 100%;
+}
+
+div.navfooter, div.footing{
+ position: fixed;
+ left: 0em;
+ bottom: 0em;
+ background-color: #eee;
+ width: 100%;
+}
+
+
+div.navheader td,
+div.navfooter td {
+ font-size: 66%;
+}
+
+div.navheader table th {
+ /*font-family: Georgia, Times, serif;*/
+ /*font-size: x-large;*/
+ font-size: 80%;
+}
+
+div.navheader table {
+ border-left: 0em;
+ border-right: 0em;
+ border-top: 0em;
+ width: 100%;
+}
+
+div.navfooter table {
+ border-left: 0em;
+ border-right: 0em;
+ border-bottom: 0em;
+ width: 100%;
+}
+
+div.navheader table td a,
+div.navfooter table td a {
+ color: #777;
+ text-decoration: none;
+}
+
+/* normal text in the footer */
+div.navfooter table td {
+ color: black;
+}
+
+div.navheader table td a:visited,
+div.navfooter table td a:visited {
+ color: #444;
+}
+
+
+/* links in header and footer */
+div.navheader table td a:hover,
+div.navfooter table td a:hover {
+ text-decoration: underline;
+ background-color: transparent;
+ color: #33a;
+}
+
+div.navheader hr,
+div.navfooter hr {
+ display: none;
+}
+
+
+.qandaset tr.question td p {
+ margin: 0em 0em 1em 0em;
+ padding: 0em 0em 0em 0em;
+}
+
+.qandaset tr.answer td p {
+ margin: 0em 0em 1em 0em;
+ padding: 0em 0em 0em 0em;
+}
+.answer td {
+ padding-bottom: 1.5em;
+}
+
+.emphasis {
+ font-weight: bold;
+}
+
+
+ /************* /
+ / decorations /
+/ *************/
+
+.titlepage {
+}
+
+.part .title {
+}
+
+.subtitle {
+ border: none;
+}
+
+/*
+h1 {
+ border: none;
+}
+
+h2 {
+ border-top: solid 0.2em;
+ border-bottom: solid 0.06em;
+}
+
+h3 {
+ border-top: 0em;
+ border-bottom: solid 0.06em;
+}
+
+h4 {
+ border: 0em;
+ border-bottom: solid 0.06em;
+}
+
+h5 {
+ border: 0em;
+}
+*/
+
+.programlisting {
+ border: solid 1px;
+}
+
+div.figure,
+div.table,
+div.informalfigure,
+div.informaltable,
+div.informalexample,
+div.example {
+ border: 1px solid;
+}
+
+
+
+.tip,
+.warning,
+.caution,
+.note {
+ border: 1px solid;
+}
+
+.tip table th,
+.warning table th,
+.caution table th,
+.note table th {
+ border-bottom: 1px solid;
+}
+
+.question td {
+ border-top: 1px solid black;
+}
+
+.answer {
+}
+
+
+b.keycap,
+.keycap {
+ border: 1px solid;
+}
+
+
+div.navheader, div.heading{
+ border-bottom: 1px solid;
+}
+
+
+div.navfooter, div.footing{
+ border-top: 1px solid;
+}
+
+ /********* /
+ / colors /
+/ *********/
+
+body {
+ color: #333;
+ background: white;
+}
+
+a {
+ background: transparent;
+}
+
+a:hover {
+ background-color: #dedede;
+}
+
+
+h1,
+h2,
+h3,
+h4,
+h5,
+h6,
+h7,
+h8 {
+ background-color: transparent;
+}
+
+hr {
+ border-color: #aaa;
+}
+
+
+.tip, .warning, .caution, .note {
+ border-color: #fff;
+}
+
+
+.tip table th,
+.warning table th,
+.caution table th,
+.note table th {
+ border-bottom-color: #fff;
+}
+
+
+.warning {
+ background-color: #f0f0f2;
+}
+
+.caution {
+ background-color: #f0f0f2;
+}
+
+.tip {
+ background-color: #f0f0f2;
+}
+
+.note {
+ background-color: #f0f0f2;
+}
+
+.glossary dl dt,
+.variablelist dl dt,
+.variablelist dl dt span.term {
+ color: #044;
+}
+
+div.figure,
+div.table,
+div.example,
+div.informalfigure,
+div.informaltable,
+div.informalexample {
+ border-color: #aaa;
+}
+
+pre.programlisting {
+ color: black;
+ background-color: #fff;
+ border-color: #aaa;
+ border-width: 2px;
+}
+
+.guimenu,
+.guilabel,
+.guimenuitem {
+ background-color: #eee;
+}
+
+
+b.keycap,
+.keycap {
+ background-color: #eee;
+ border-color: #999;
+}
+
+
+div.navheader {
+ border-color: black;
+}
+
+
+div.navfooter {
+ border-color: black;
+}
+
+.writernotes {
+ color: red;
+}
+
+
+ /*********** /
+ / graphics /
+/ ***********/
+
+/*
+body {
+ background-image: url("images/body_bg.jpg");
+ background-attachment: fixed;
+}
+
+.navheader,
+.note,
+.tip {
+ background-image: url("images/note_bg.jpg");
+ background-attachment: fixed;
+}
+
+.warning,
+.caution {
+ background-image: url("images/warning_bg.jpg");
+ background-attachment: fixed;
+}
+
+.figure,
+.informalfigure,
+.example,
+.informalexample,
+.table,
+.informaltable {
+ background-image: url("images/figure_bg.jpg");
+ background-attachment: fixed;
+}
+
+*/
+h1,
+h2,
+h3,
+h4,
+h5,
+h6,
+h7{
+}
+
+/*
+Example of how to stick an image as part of the title.
+
+div.article .titlepage .title
+{
+ background-image: url("figures/white-on-black.png");
+ background-position: center;
+ background-repeat: repeat-x;
+}
+*/
+
+div.preface .titlepage .title,
+div.colophon .title,
+div.chapter .titlepage .title,
+div.article .titlepage .title
+{
+}
+
+div.section div.section .titlepage .title,
+div.sect2 .titlepage .title {
+ background: none;
+}
+
+
+h1.title {
+ background-color: transparent;
+ background-repeat: no-repeat;
+ height: 256px;
+ text-indent: -9000px;
+ overflow:hidden;
+}
+
+h2.subtitle {
+ background-color: transparent;
+ text-indent: -9000px;
+ overflow:hidden;
+ width: 0px;
+ display: none;
+}
+
+ /*************************************** /
+ / pippin.gimp.org specific alterations /
+/ ***************************************/
+
+/*
+div.heading, div.navheader {
+ color: #777;
+ font-size: 80%;
+ padding: 0;
+ margin: 0;
+ text-align: left;
+ position: absolute;
+ top: 0px;
+ left: 0px;
+ width: 100%;
+ height: 50px;
+ background: url('/gfx/heading_bg.png') transparent;
+ background-repeat: repeat-x;
+ background-attachment: fixed;
+ border: none;
+}
+
+div.heading a {
+ color: #444;
+}
+
+div.footing, div.navfooter {
+ border: none;
+ color: #ddd;
+ font-size: 80%;
+ text-align:right;
+
+ width: 100%;
+ padding-top: 10px;
+ position: absolute;
+ bottom: 0px;
+ left: 0px;
+
+ background: url('/gfx/footing_bg.png') transparent;
+}
+*/
+
+
+
+ /****************** /
+ / nasty ie tweaks /
+/ ******************/
+
+/*
+div.heading, div.navheader {
+ width:expression(document.body.clientWidth + "px");
+}
+
+div.footing, div.navfooter {
+ width:expression(document.body.clientWidth + "px");
+ margin-left:expression("-5em");
+}
+body {
+ padding:expression("4em 5em 0em 5em");
+}
+*/
+
+ /**************************************** /
+ / mozilla vendor specific css extensions /
+/ ****************************************/
+/*
+div.navfooter, div.footing{
+ -moz-opacity: 0.8em;
+}
+
+div.figure,
+div.table,
+div.informalfigure,
+div.informaltable,
+div.informalexample,
+div.example,
+.tip,
+.warning,
+.caution,
+.note {
+ -moz-border-radius: 0.5em;
+}
+
+b.keycap,
+.keycap {
+ -moz-border-radius: 0.3em;
+}
+*/
+
+table tr td table tr td {
+ display: none;
+}
+
+
+hr {
+ display: none;
+}
+
+table {
+ border: 0em;
+}
+
+ .photo {
+ float: right;
+ margin-left: 1.5em;
+ margin-bottom: 1.5em;
+ margin-top: 0em;
+ max-width: 17em;
+ border: 1px solid gray;
+ padding: 3px;
+ background: white;
+}
+ .seperator {
+ padding-top: 2em;
+ clear: both;
+ }
+
+ #validators {
+ margin-top: 5em;
+ text-align: right;
+ color: #777;
+ }
+ @media print {
+ body {
+ font-size: 8pt;
+ }
+ .noprint {
+ display: none;
+ }
+ }
+
+
+.tip,
+.note {
+ background: #f0f0f2;
+ color: #333;
+ padding: 20px;
+ margin: 20px;
+}
+
+.tip h3,
+.note h3 {
+ padding: 0em;
+ margin: 0em;
+ font-size: 2em;
+ font-weight: bold;
+ color: #333;
+}
+
+.tip a,
+.note a {
+ color: #333;
+ text-decoration: underline;
+}
+
+.footnote {
+ font-size: small;
+ color: #333;
+}
+
+/* Changes the announcement text */
+.tip h3,
+.warning h3,
+.caution h3,
+.note h3 {
+ font-size:large;
+ color: #00557D;
+}
diff --git a/poky/documentation/overview-manual/overview-manual-yp-intro.rst b/poky/documentation/overview-manual/overview-manual-yp-intro.rst
index 9073582..5cdab7c 100644
--- a/poky/documentation/overview-manual/overview-manual-yp-intro.rst
+++ b/poky/documentation/overview-manual/overview-manual-yp-intro.rst
@@ -1,4 +1,4 @@
-.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
+.. SPDX-License-Identifier: CC-BY-2.0-UK
*****************************
Introducing the Yocto Project
diff --git a/poky/documentation/overview-manual/overview-manual-yp-intro.xml b/poky/documentation/overview-manual/overview-manual-yp-intro.xml
new file mode 100644
index 0000000..a2a1f49
--- /dev/null
+++ b/poky/documentation/overview-manual/overview-manual-yp-intro.xml
@@ -0,0 +1,1333 @@
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
+"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
+[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
+<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
+
+<chapter id='overview-yp'>
+ <title>Introducing the Yocto Project</title>
+
+ <section id='what-is-the-yocto-project'>
+ <title>What is the Yocto Project?</title>
+
+ <para>
+ The Yocto Project is an open source collaboration project
+ that helps developers create custom Linux-based systems that are
+ designed for embedded products regardless of the product's hardware
+ architecture.
+ Yocto Project provides a flexible toolset and a development
+ environment that allows embedded device developers across the
+ world to collaborate through shared technologies, software stacks,
+ configurations, and best practices used to create these tailored
+ Linux images.
+ </para>
+
+ <para>
+ Thousands of developers worldwide have discovered that Yocto
+ Project provides advantages in both systems and applications
+ development, archival and management benefits, and customizations
+ used for speed, footprint, and memory utilization.
+ The project is a standard when it comes to delivering embedded
+ software stacks.
+ The project allows software customizations and build interchange
+ for multiple hardware platforms as well as software stacks that
+ can be maintained and scaled.
+ </para>
+
+ <para id='yp-key-dev-elements'>
+ <imagedata fileref="figures/key-dev-elements.png" format="PNG" align='center' width="8in"/>
+ </para>
+
+ <para>
+ For further introductory information on the Yocto Project, you
+ might be interested in this
+ <ulink url='https://www.embedded.com/electronics-blogs/say-what-/4458600/Why-the-Yocto-Project-for-my-IoT-Project-'>article</ulink>
+ by Drew Moseley and in this short introductory
+ <ulink url='https://www.youtube.com/watch?v=utZpKM7i5Z4'>video</ulink>.
+ </para>
+
+ <para>
+ The remainder of this section overviews advantages and challenges
+ tied to the Yocto Project.
+ </para>
+
+ <section id='gs-features'>
+ <title>Features</title>
+
+ <para>
+ The following list describes features and advantages of the
+ Yocto Project:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis>Widely Adopted Across the Industry:</emphasis>
+ Semiconductor, operating system, software, and
+ service vendors exist whose products and services
+ adopt and support the Yocto Project.
+ For a look at the Yocto Project community and
+ the companies involved with the Yocto
+ Project, see the "COMMUNITY" and "ECOSYSTEM" tabs
+ on the
+ <ulink url='&YOCTO_HOME_URL;'>Yocto Project</ulink>
+ home page.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Architecture Agnostic:</emphasis>
+ Yocto Project supports Intel, ARM, MIPS, AMD, PPC
+ and other architectures.
+ Most ODMs, OSVs, and chip vendors create and supply
+ BSPs that support their hardware.
+ If you have custom silicon, you can create a BSP
+ that supports that architecture.</para>
+
+ <para>Aside from lots of architecture support, the
+ Yocto Project fully supports a wide range of device
+ emulation through the Quick EMUlator (QEMU).
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Images and Code Transfer Easily:</emphasis>
+ Yocto Project output can easily move between
+ architectures without moving to new development
+ environments.
+ Additionally, if you have used the Yocto Project to
+ create an image or application and you find yourself
+ not able to support it, commercial Linux vendors such
+ as Wind River, Mentor Graphics, Timesys, and ENEA could
+ take it and provide ongoing support.
+ These vendors have offerings that are built using
+ the Yocto Project.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Flexibility:</emphasis>
+ Corporations use the Yocto Project many different ways.
+ One example is to create an internal Linux distribution
+ as a code base the corporation can use across multiple
+ product groups.
+ Through customization and layering, a project group
+ can leverage the base Linux distribution to create
+ a distribution that works for their product needs.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Ideal for Constrained Embedded and IoT devices:</emphasis>
+ Unlike a full Linux distribution, you can use the
+ Yocto Project to create exactly what you need for
+ embedded devices.
+ You only add the feature support or packages that you
+ absolutely need for the device.
+ For devices that have display hardware, you can use
+ available system components such as X11, GTK+, Qt,
+ Clutter, and SDL (among others) to create a rich user
+ experience.
+ For devices that do not have a display or where you
+ want to use alternative UI frameworks, you can choose
+ to not install these components.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Comprehensive Toolchain Capabilities:</emphasis>
+ Toolchains for supported architectures satisfy most
+ use cases.
+ However, if your hardware supports features that are
+ not part of a standard toolchain, you can easily
+ customize that toolchain through specification of
+ platform-specific tuning parameters.
+ And, should you need to use a third-party toolchain,
+ mechanisms built into the Yocto Project allow for that.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Mechanism Rules Over Policy:</emphasis>
+ Focusing on mechanism rather than policy ensures that
+ you are free to set policies based on the needs of your
+ design instead of adopting decisions enforced by some
+ system software provider.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Uses a Layer Model:</emphasis>
+ The Yocto Project
+ <link linkend='the-yocto-project-layer-model'>layer infrastructure</link>
+ groups related functionality into separate bundles.
+ You can incrementally add these grouped functionalities
+ to your project as needed.
+ Using layers to isolate and group functionality
+ reduces project complexity and redundancy, allows you
+ to easily extend the system, make customizations,
+ and keep functionality organized.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Supports Partial Builds:</emphasis>
+ You can build and rebuild individual packages as
+ needed.
+ Yocto Project accomplishes this through its
+ <link linkend='shared-state-cache'>shared-state cache</link>
+ (sstate) scheme.
+ Being able to build and debug components individually
+ eases project development.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Releases According to a Strict Schedule:</emphasis>
+ Major releases occur on a
+ <ulink url='&YOCTO_DOCS_REF_URL;#ref-release-process'>six-month cycle</ulink>
+ predictably in October and April.
+ The most recent two releases support point releases
+ to address common vulnerabilities and exposures.
+ This predictability is crucial for projects based on
+ the Yocto Project and allows development teams to
+ plan activities.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Rich Ecosystem of Individuals and Organizations:</emphasis>
+ For open source projects, the value of community is
+ very important.
+ Support forums, expertise, and active developers who
+ continue to push the Yocto Project forward are readily
+ available.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Binary Reproducibility:</emphasis>
+ The Yocto Project allows you to be very specific about
+ dependencies and achieves very high percentages of
+ binary reproducibility (e.g. 99.8% for
+ <filename>core-image-minimal</filename>).
+ When distributions are not specific about which
+ packages are pulled in and in what order to support
+ dependencies, other build systems can arbitrarily
+ include packages.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>License Manifest:</emphasis>
+ The Yocto Project provides a
+ <ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>license manifest</ulink>
+ for review by people who need to track the use of open
+ source licenses (e.g.legal teams).
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </section>
+
+ <section id='gs-challenges'>
+ <title>Challenges</title>
+
+ <para>
+ The following list presents challenges you might encounter
+ when developing using the Yocto Project:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis>Steep Learning Curve:</emphasis>
+ The Yocto Project has a steep learning curve and has
+ many different ways to accomplish similar tasks.
+ It can be difficult to choose how to proceed when
+ varying methods exist by which to accomplish a given
+ task.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Understanding What Changes You Need to Make
+ For Your Design Requires Some Research:</emphasis>
+ Beyond the simple tutorial stage, understanding what
+ changes need to be made for your particular design
+ can require a significant amount of research and
+ investigation.
+ For information that helps you transition from
+ trying out the Yocto Project to using it for your
+ project, see the
+ "<ulink url='&YOCTO_DOCS_URL;/what-i-wish-id-known/'>What I wish I'd Known</ulink>"
+ and
+ "<ulink url='&YOCTO_DOCS_URL;/transitioning-to-a-custom-environment/'>Transitioning to a Custom Environment for Systems Development</ulink>"
+ documents on the Yocto Project website.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Project Workflow Could Be Confusing:</emphasis>
+ The
+ <link linkend='overview-development-environment'>Yocto Project workflow</link>
+ could be confusing if you are used to traditional
+ desktop and server software development.
+ In a desktop development environment, mechanisms exist
+ to easily pull and install new packages, which are
+ typically pre-compiled binaries from servers accessible
+ over the Internet.
+ Using the Yocto Project, you must modify your
+ configuration and rebuild to add additional packages.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Working in a Cross-Build Environment Can
+ Feel Unfamiliar:</emphasis>
+ When developing code to run on a target, compilation,
+ execution, and testing done on the actual target
+ can be faster than running a BitBake build on a
+ development host and then deploying binaries to the
+ target for test.
+ While the Yocto Project does support development tools
+ on the target, the additional step of integrating your
+ changes back into the Yocto Project build environment
+ would be required.
+ Yocto Project supports an intermediate approach that
+ involves making changes on the development system
+ within the BitBake environment and then deploying only
+ the updated packages to the target.</para>
+
+ <para>The Yocto Project
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>OpenEmbedded build system</ulink>
+ produces packages in standard formats (i.e. RPM,
+ DEB, IPK, and TAR).
+ You can deploy these packages into the running system
+ on the target by using utilities on the target such
+ as <filename>rpm</filename> or
+ <filename>ipk</filename>.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Initial Build Times Can be Significant:</emphasis>
+ Long initial build times are unfortunately unavoidable
+ due to the large number of packages initially built
+ from scratch for a fully functioning Linux system.
+ Once that initial build is completed, however, the
+ shared-state (sstate) cache mechanism Yocto Project
+ uses keeps the system from rebuilding packages that
+ have not been "touched" since the last build.
+ The sstate mechanism significantly reduces times
+ for successive builds.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </section>
+ </section>
+
+ <section id='the-yocto-project-layer-model'>
+ <title>The Yocto Project Layer Model</title>
+
+ <para>
+ The Yocto Project's "Layer Model" is a development model for
+ embedded and IoT Linux creation that distinguishes the
+ Yocto Project from other simple build systems.
+ The Layer Model simultaneously supports collaboration and
+ customization.
+ Layers are repositories that contain related sets of instructions
+ that tell the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>OpenEmbedded build system</ulink>
+ what to do.
+ You can collaborate, share, and reuse layers.
+ </para>
+
+ <para>
+ Layers can contain changes to previous instructions or settings
+ at any time.
+ This powerful override capability is what allows you to customize
+ previously supplied collaborative or community layers to suit your
+ product requirements.
+ </para>
+
+ <para>
+ You use different layers to logically separate information in your
+ build.
+ As an example, you could have BSP, GUI, distro configuration,
+ middleware, or application layers.
+ Putting your entire build into one layer limits and complicates
+ future customization and reuse.
+ Isolating information into layers, on the other hand, helps
+ simplify future customizations and reuse.
+ You might find it tempting to keep everything in one layer when
+ working on a single project.
+ However, the more modular your Metadata, the easier
+ it is to cope with future changes.
+ <note><title>Notes</title>
+ <itemizedlist>
+ <listitem><para>
+ Use Board Support Package (BSP) layers from silicon
+ vendors when possible.
+ </para></listitem>
+ <listitem><para>
+ Familiarize yourself with the
+ <ulink url='https://caffelli-staging.yoctoproject.org/software-overview/layers/'>Yocto Project curated layer index</ulink>
+ or the
+ <ulink url='http://layers.openembedded.org/layerindex/branch/master/layers/'>OpenEmbedded layer index</ulink>.
+ The latter contains more layers but they are less
+ universally validated.
+ </para></listitem>
+ <listitem><para>
+ Layers support the inclusion of technologies, hardware
+ components, and software components.
+ The
+ <ulink url='&YOCTO_DOCS_DEV_URL;#making-sure-your-layer-is-compatible-with-yocto-project'>Yocto Project Compatible</ulink>
+ designation provides a minimum level of standardization
+ that contributes to a strong ecosystem.
+ "YP Compatible" is applied to appropriate products and
+ software components such as BSPs, other OE-compatible
+ layers, and related open-source projects, allowing the
+ producer to use Yocto Project badges and branding
+ assets.
+ </para></listitem>
+ </itemizedlist>
+ </note>
+ </para>
+
+ <para>
+ To illustrate how layers are used to keep things modular, consider
+ machine customizations.
+ These types of customizations typically reside in a special layer,
+ rather than a general layer, called a BSP Layer.
+ Furthermore, the machine customizations should be isolated from
+ recipes and Metadata that support a new GUI environment,
+ for example.
+ This situation gives you a couple of layers: one for the machine
+ configurations, and one for the GUI environment.
+ It is important to understand, however, that the BSP layer can
+ still make machine-specific additions to recipes within the GUI
+ environment layer without polluting the GUI layer itself
+ with those machine-specific changes.
+ You can accomplish this through a recipe that is a BitBake append
+ (<filename>.bbappend</filename>) file, which is described later
+ in this section.
+ <note>
+ For general information on BSP layer structure, see the
+ <ulink url='&YOCTO_DOCS_BSP_URL;'>Yocto Project Board Support Packages (BSP) Developer's Guide</ulink>.
+ </note>
+ </para>
+
+ <para>
+ The
+ <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>
+ contains both general layers and BSP layers right out of the box.
+ You can easily identify layers that ship with a Yocto Project
+ release in the Source Directory by their names.
+ Layers typically have names that begin with the string
+ <filename>meta-</filename>.
+ <note>
+ It is not a requirement that a layer name begin with the
+ prefix <filename>meta-</filename>, but it is a commonly
+ accepted standard in the Yocto Project community.
+ </note>
+ For example, if you were to examine the
+ <ulink url='https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/'>tree view</ulink>
+ of the <filename>poky</filename> repository, you will see several
+ layers: <filename>meta</filename>,
+ <filename>meta-skeleton</filename>,
+ <filename>meta-selftest</filename>,
+ <filename>meta-poky</filename>, and
+ <filename>meta-yocto-bsp</filename>.
+ Each of these repositories represents a distinct layer.
+ </para>
+
+ <para>
+ For procedures on how to create layers, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and Creating Layers</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para>
+ </section>
+
+ <section id='components-and-tools'>
+ <title>Components and Tools</title>
+
+ <para>
+ The Yocto Project employs a collection of components and
+ tools used by the project itself, by project developers,
+ and by those using the Yocto Project.
+ These components and tools are open source projects and
+ metadata that are separate from the reference distribution
+ (<ulink url='&YOCTO_DOCS_REF_URL;#poky'>Poky</ulink>)
+ and the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>OpenEmbedded build system</ulink>.
+ Most of the components and tools are downloaded separately.
+ </para>
+
+ <para>
+ This section provides brief overviews of the components and
+ tools associated with the Yocto Project.
+ </para>
+
+ <section id='gs-development-tools'>
+ <title>Development Tools</title>
+
+ <para>
+ The following list consists of tools that help you develop
+ images and applications using the Yocto Project:
+ <itemizedlist>
+ <listitem><para id='gs-crops-overview'>
+ <emphasis>CROPS:</emphasis>
+ <ulink url='https://github.com/crops/poky-container/'>CROPS</ulink>
+ is an open source, cross-platform development framework
+ that leverages
+ <ulink url='https://www.docker.com/'>Docker Containers</ulink>.
+ CROPS provides an easily managed, extensible environment
+ that allows you to build binaries for a variety of
+ architectures on Windows, Linux and Mac OS X hosts.
+ </para></listitem>
+ <listitem><para>
+ <emphasis><filename>devtool</filename>:</emphasis>
+ This command-line tool is available as part of the
+ extensible SDK (eSDK) and is its cornerstone.
+ You can use <filename>devtool</filename> to help build,
+ test, and package software within the eSDK.
+ You can use the tool to optionally integrate what you
+ build into an image built by the OpenEmbedded build
+ system.</para>
+
+ <para>The <filename>devtool</filename> command employs
+ a number of sub-commands that allow you to add, modify,
+ and upgrade recipes.
+ As with the OpenEmbedded build system, "recipes"
+ represent software packages within
+ <filename>devtool</filename>.
+ When you use <filename>devtool add</filename>, a recipe
+ is automatically created.
+ When you use <filename>devtool modify</filename>, the
+ specified existing recipe is used in order to determine
+ where to get the source code and how to patch it.
+ In both cases, an environment is set up so that when
+ you build the recipe a source tree that is under your
+ control is used in order to allow you to make changes
+ to the source as desired.
+ By default, both new recipes and the source go into
+ a "workspace" directory under the eSDK.
+ The <filename>devtool upgrade</filename> command
+ updates an existing recipe so that you can build it
+ for an updated set of source files.</para>
+
+ <para>You can read about the
+ <filename>devtool</filename> workflow in the Yocto
+ Project Application Development and Extensible
+ Software Development Kit (eSDK) Manual in the
+ "<ulink url='&YOCTO_DOCS_SDK_URL;#using-devtool-in-your-sdk-workflow'>Using <filename>devtool</filename> in Your SDK Workflow'</ulink>"
+ section.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Extensible Software Development Kit (eSDK):</emphasis>
+ The eSDK provides a cross-development toolchain and
+ libraries tailored to the contents of a specific image.
+ The eSDK makes it easy to add new applications and
+ libraries to an image, modify the source for an
+ existing component, test changes on the target
+ hardware, and integrate into the rest of the
+ OpenEmbedded build system.
+ The eSDK gives you a toolchain experience supplemented
+ with the powerful set of <filename>devtool</filename>
+ commands tailored for the Yocto Project environment.
+ </para>
+
+ <para>For information on the eSDK, see the
+ <ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Application Development and the Extensible Software Development Kit (eSDK)</ulink>
+ Manual.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Toaster:</emphasis>
+ Toaster is a web interface to the Yocto Project
+ OpenEmbedded build system.
+ Toaster allows you to configure, run, and view
+ information about builds.
+ For information on Toaster, see the
+ <ulink url='&YOCTO_DOCS_TOAST_URL;'>Toaster User Manual</ulink>.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </section>
+
+ <section id='gs-production-tools'>
+ <title>Production Tools</title>
+
+ <para>
+ The following list consists of tools that help production
+ related activities using the Yocto Project:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis>Auto Upgrade Helper:</emphasis>
+ This utility when used in conjunction with the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>OpenEmbedded build system</ulink>
+ (BitBake and OE-Core) automatically generates upgrades
+ for recipes that are based on new versions of the
+ recipes published upstream.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Recipe Reporting System:</emphasis>
+ The Recipe Reporting System tracks recipe versions
+ available for Yocto Project.
+ The main purpose of the system is to help you
+ manage the recipes you maintain and to offer a dynamic
+ overview of the project.
+ The Recipe Reporting System is built on top of the
+ <ulink url="http://layers.openembedded.org/layerindex/layers/">OpenEmbedded Layer Index</ulink>,
+ which is a website that indexes OpenEmbedded-Core
+ layers.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Patchwork:</emphasis>
+ <ulink url='http://jk.ozlabs.org/projects/patchwork/'>Patchwork</ulink>
+ is a fork of a project originally started by
+ <ulink url='http://ozlabs.org/'>OzLabs</ulink>.
+ The project is a web-based tracking system designed
+ to streamline the process of bringing contributions
+ into a project.
+ The Yocto Project uses Patchwork as an organizational
+ tool to handle patches, which number in the thousands
+ for every release.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>AutoBuilder:</emphasis>
+ AutoBuilder is a project that automates build tests
+ and quality assurance (QA).
+ By using the public AutoBuilder, anyone can determine
+ the status of the current "master" branch of Poky.
+ <note>
+ AutoBuilder is based on
+ <ulink url='https://buildbot.net/'>buildbot</ulink>.
+ </note></para>
+
+ <para>A goal of the Yocto Project is to lead the
+ open source industry with a project that automates
+ testing and QA procedures.
+ In doing so, the project encourages a development
+ community that publishes QA and test plans, publicly
+ demonstrates QA and test plans, and encourages
+ development of tools that automate and test and QA
+ procedures for the benefit of the development
+ community.</para>
+
+ <para>You can learn more about the AutoBuilder used
+ by the Yocto Project
+ <ulink url='&YOCTO_AB_URL;'>here</ulink>.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Cross-Prelink:</emphasis>
+ Prelinking is the process of pre-computing the load
+ addresses and link tables generated by the dynamic
+ linker as compared to doing this at runtime.
+ Doing this ahead of time results in performance
+ improvements when the application is launched and
+ reduced memory usage for libraries shared by many
+ applications.</para>
+
+ <para>Historically, cross-prelink is a variant of
+ prelink, which was conceived by
+ <ulink url='http://people.redhat.com/jakub/prelink.pdf'>Jakub Jelínek</ulink>
+ a number of years ago.
+ Both prelink and cross-prelink are maintained in the
+ same repository albeit on separate branches.
+ By providing an emulated runtime dynamic linker
+ (i.e. <filename>glibc</filename>-derived
+ <filename>ld.so</filename> emulation), the
+ cross-prelink project extends the prelink software's
+ ability to prelink a sysroot environment.
+ Additionally, the cross-prelink software enables the
+ ability to work in sysroot style environments.</para>
+
+ <para>The dynamic linker determines standard load
+ address calculations based on a variety of factors
+ such as mapping addresses, library usage, and library
+ function conflicts.
+ The prelink tool uses this information, from the
+ dynamic linker, to determine unique load addresses
+ for executable and linkable format (ELF) binaries
+ that are shared libraries and dynamically linked.
+ The prelink tool modifies these ELF binaries with the
+ pre-computed information.
+ The result is faster loading and often lower memory
+ consumption because more of the library code can
+ be re-used from shared Copy-On-Write (COW) pages.
+ </para>
+
+ <para>The original upstream prelink project only
+ supports running prelink on the end target device
+ due to the reliance on the target device's dynamic
+ linker.
+ This restriction causes issues when developing a
+ cross-compiled system.
+ The cross-prelink adds a synthesized dynamic loader
+ that runs on the host, thus permitting cross-prelinking
+ without ever having to run on a read-write target
+ filesystem.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Pseudo:</emphasis>
+ Pseudo is the Yocto Project implementation of
+ <ulink url='http://man.he.net/man1/fakeroot'>fakeroot</ulink>,
+ which is used to run commands in an environment
+ that seemingly has root privileges.</para>
+
+ <para>During a build, it can be necessary to perform
+ operations that require system administrator
+ privileges.
+ For example, file ownership or permissions might need
+ definition.
+ Pseudo is a tool that you can either use directly or
+ through the environment variable
+ <filename>LD_PRELOAD</filename>.
+ Either method allows these operations to succeed as
+ if system administrator privileges exist even
+ when they do not.</para>
+
+ <para>You can read more about Pseudo in the
+ "<link linkend='fakeroot-and-pseudo'>Fakeroot and Pseudo</link>"
+ section.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </section>
+
+ <section id='gs-openembedded-build-system'>
+ <title>Open-Embedded Build System Components</title>
+
+ <para>
+ The following list consists of components associated with the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>OpenEmbedded build system</ulink>:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis>BitBake:</emphasis>
+ BitBake is a core component of the Yocto Project and is
+ used by the OpenEmbedded build system to build images.
+ While BitBake is key to the build system, BitBake
+ is maintained separately from the Yocto Project.</para>
+
+ <para>BitBake is a generic task execution engine that
+ allows shell and Python tasks to be run efficiently
+ and in parallel while working within complex inter-task
+ dependency constraints.
+ In short, BitBake is a build engine that works
+ through recipes written in a specific format in order
+ to perform sets of tasks.</para>
+
+ <para>You can learn more about BitBake in the
+ <ulink url='&YOCTO_DOCS_BB_URL;'>BitBake User Manual</ulink>.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>OpenEmbedded-Core:</emphasis>
+ OpenEmbedded-Core (OE-Core) is a common layer of
+ metadata (i.e. recipes, classes, and associated files)
+ used by OpenEmbedded-derived systems, which includes
+ the Yocto Project.
+ The Yocto Project and the OpenEmbedded Project both
+ maintain the OpenEmbedded-Core.
+ You can find the OE-Core metadata in the Yocto Project
+ <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta'>Source Repositories</ulink>.
+ </para>
+
+ <para>Historically, the Yocto Project integrated the
+ OE-Core metadata throughout the Yocto Project
+ source repository reference system (Poky).
+ After Yocto Project Version 1.0, the Yocto Project
+ and OpenEmbedded agreed to work together and share a
+ common core set of metadata (OE-Core), which contained
+ much of the functionality previously found in Poky.
+ This collaboration achieved a long-standing
+ OpenEmbedded objective for having a more tightly
+ controlled and quality-assured core.
+ The results also fit well with the Yocto Project
+ objective of achieving a smaller number of fully
+ featured tools as compared to many different ones.
+ </para>
+
+ <para>Sharing a core set of metadata results in Poky
+ as an integration layer on top of OE-Core.
+ You can see that in this
+ <link linkend='yp-key-dev-elements'>figure</link>.
+ The Yocto Project combines various components such as
+ BitBake, OE-Core, script "glue", and documentation
+ for its build system.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </section>
+
+ <section id='gs-reference-distribution-poky'>
+ <title>Reference Distribution (Poky)</title>
+
+ <para>
+ Poky is the Yocto Project reference distribution.
+ It contains the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>Open-Embedded build system</ulink>
+ (BitBake and OE-Core) as well as a set of metadata to get you
+ started building your own distribution.
+ See the
+ <link linkend='what-is-the-yocto-project'>figure</link> in
+ "What is the Yocto Project?" section for an illustration
+ that shows Poky and its relationship with other parts of the
+ Yocto Project.</para>
+
+ <para>To use the Yocto Project tools and components, you
+ can download (<filename>clone</filename>) Poky and use it
+ to bootstrap your own distribution.
+ <note>
+ Poky does not contain binary files.
+ It is a working example of how to build your own custom
+ Linux distribution from source.
+ </note>
+ You can read more about Poky in the
+ "<link linkend='reference-embedded-distribution'>Reference Embedded Distribution (Poky)</link>"
+ section.
+ </para>
+ </section>
+
+ <section id='gs-packages-for-finished-targets'>
+ <title>Packages for Finished Targets</title>
+
+ <para>
+ The following lists components associated with packages
+ for finished targets:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis>Matchbox:</emphasis>
+ Matchbox is an Open Source, base environment for the
+ X Window System running on non-desktop, embedded
+ platforms such as handhelds, set-top boxes, kiosks,
+ and anything else for which screen space, input
+ mechanisms, or system resources are limited.</para>
+
+ <para>Matchbox consists of a number of interchangeable
+ and optional applications that you can tailor to a
+ specific, non-desktop platform to enhance usability
+ in constrained environments.</para>
+
+ <para>You can find the Matchbox source in the Yocto
+ Project
+ <ulink url='&YOCTO_GIT_URL;'>Source Repositories</ulink>.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Opkg</emphasis>
+ Open PacKaGe management (opkg) is a lightweight
+ package management system based on the itsy package
+ (ipkg) management system.
+ Opkg is written in C and resembles Advanced Package
+ Tool (APT) and Debian Package (dpkg) in operation.
+ </para>
+
+ <para>Opkg is intended for use on embedded Linux
+ devices and is used in this capacity in the
+ <ulink url='http://www.openembedded.org/wiki/Main_Page'>OpenEmbedded</ulink>
+ and
+ <ulink url='https://openwrt.org/'>OpenWrt</ulink>
+ projects, as well as the Yocto Project.
+ <note>
+ As best it can, opkg maintains backwards
+ compatibility with ipkg and conforms to a subset
+ of Debian's policy manual regarding control files.
+ </note>
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </section>
+
+ <section id='gs-archived-components'>
+ <title>Archived Components</title>
+
+ <para>
+ The Build Appliance is a virtual machine image that enables
+ you to build and boot a custom embedded Linux image with
+ the Yocto Project using a non-Linux development system.
+ </para>
+
+ <para>
+ Historically, the Build Appliance was the second of three
+ methods by which you could use the Yocto Project on a system
+ that was not native to Linux.
+ <orderedlist>
+ <listitem><para>
+ <emphasis>Hob:</emphasis>
+ Hob, which is now deprecated and is no longer available
+ since the 2.1 release of the Yocto Project provided
+ a rudimentary, GUI-based interface to the Yocto
+ Project.
+ Toaster has fully replaced Hob.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Build Appliance:</emphasis>
+ Post Hob, the Build Appliance became available.
+ It was never recommended that you use the Build
+ Appliance as a day-to-day production development
+ environment with the Yocto Project.
+ Build Appliance was useful as a way to try out
+ development in the Yocto Project environment.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>CROPS:</emphasis>
+ The final and best solution available now for
+ developing using the Yocto Project on a system
+ not native to Linux is with
+ <link linkend='gs-crops-overview'>CROPS</link>.
+ </para></listitem>
+ </orderedlist>
+ </para>
+ </section>
+ </section>
+
+ <section id='gs-development-methods'>
+ <title>Development Methods</title>
+
+ <para>
+ The Yocto Project development environment usually involves a
+ <ulink url='&YOCTO_DOCS_REF_URL;#hardware-build-system-term'>Build Host</ulink>
+ and target hardware.
+ You use the Build Host to build images and develop applications,
+ while you use the target hardware to test deployed software.
+ </para>
+
+ <para>
+ This section provides an introduction to the choices or
+ development methods you have when setting up your Build Host.
+ Depending on the your particular workflow preference and the
+ type of operating system your Build Host runs, several choices
+ exist that allow you to use the Yocto Project.
+ <note>
+ For additional detail about the Yocto Project development
+ environment, see the
+ "<link linkend='overview-development-environment'>The Yocto Project Development Environment</link>"
+ chapter.
+ </note>
+ <itemizedlist>
+ <listitem><para>
+ <emphasis>Native Linux Host:</emphasis>
+ By far the best option for a Build Host.
+ A system running Linux as its native operating system
+ allows you to develop software by directly using the
+ <ulink url='&YOCTO_DOCS_REF_URL;#bitbake-term'>BitBake</ulink>
+ tool.
+ You can accomplish all aspects of development from a
+ familiar shell of a supported Linux distribution.</para>
+
+ <para>For information on how to set up a Build Host on
+ a system running Linux as its native operating system,
+ see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#setting-up-a-native-linux-host'>Setting Up a Native Linux Host</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>CROss PlatformS (CROPS):</emphasis>
+ Typically, you use
+ <ulink url='https://github.com/crops/poky-container/'>CROPS</ulink>,
+ which leverages
+ <ulink url='https://www.docker.com/'>Docker Containers</ulink>,
+ to set up a Build Host that is not running Linux (e.g.
+ <trademark class='registered'>Microsoft</trademark>
+ <trademark class='trademark'>Windows</trademark>
+ or
+ <trademark class='registered'>macOS</trademark>).
+ <note>
+ You can, however, use CROPS on a Linux-based system.
+ </note>
+ CROPS is an open source, cross-platform development
+ framework that provides an easily managed, extensible
+ environment for building binaries targeted for a variety
+ of architectures on Windows, macOS, or Linux hosts.
+ Once the Build Host is set up using CROPS, you can prepare
+ a shell environment to mimic that of a shell being used
+ on a system natively running Linux.</para>
+
+ <para>For information on how to set up a Build Host with
+ CROPS, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#setting-up-to-use-crops'>Setting Up to Use CROss PlatformS (CROPS)</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Windows Subsystem For Linux (WSLv2):</emphasis>
+ You may use Windows Subsystem For Linux v2 to set up a build
+ host using Windows 10.
+ <note>
+ The Yocto Project is not compatible with WSLv1, it is
+ compatible but not officially supported nor validated
+ with WSLv2, if you still decide to use WSL please upgrade
+ to WSLv2.
+ </note>
+ The Windows Subsystem For Linux allows Windows 10 to run a real
+ Linux kernel inside of a lightweight utility virtual
+ machine (VM) using virtualization technology.</para>
+ <para>For information on how to set up a Build Host with
+ WSLv2, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#setting-up-to-use-wsl'>Setting Up to Use Windows Subsystem For Linux</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Toaster:</emphasis>
+ Regardless of what your Build Host is running, you can
+ use Toaster to develop software using the Yocto Project.
+ Toaster is a web interface to the Yocto Project's
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>Open-Embedded build system</ulink>.
+ The interface enables you to configure and run your
+ builds.
+ Information about builds is collected and stored in a
+ database.
+ You can use Toaster to configure and start builds on
+ multiple remote build servers.</para>
+
+ <para>For information about and how to use Toaster,
+ see the
+ <ulink url='&YOCTO_DOCS_TOAST_URL;'>Toaster User Manual</ulink>.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </section>
+
+ <section id='reference-embedded-distribution'>
+ <title>Reference Embedded Distribution (Poky)</title>
+
+ <para>
+ "Poky", which is pronounced <emphasis>Pock</emphasis>-ee, is the
+ name of the Yocto Project's reference distribution or Reference OS
+ Kit.
+ Poky contains the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>OpenEmbedded Build System</ulink>
+ (<ulink url='&YOCTO_DOCS_REF_URL;#bitbake-term'>BitBake</ulink> and
+ <ulink url='&YOCTO_DOCS_REF_URL;#oe-core'>OpenEmbedded-Core</ulink>)
+ as well as a set of
+ <ulink url='&YOCTO_DOCS_REF_URL;#metadata'>metadata</ulink> to get
+ you started building your own distro.
+ In other words, Poky is a base specification of the functionality
+ needed for a typical embedded system as well as the components
+ from the Yocto Project that allow you to build a distribution into
+ a usable binary image.
+ </para>
+
+ <para>
+ Poky is a combined repository of BitBake, OpenEmbedded-Core
+ (which is found in <filename>meta</filename>),
+ <filename>meta-poky</filename>,
+ <filename>meta-yocto-bsp</filename>, and documentation provided
+ all together and known to work well together.
+ You can view these items that make up the Poky repository in the
+ <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/'>Source Repositories</ulink>.
+ <note>
+ If you are interested in all the contents of the
+ <filename>poky</filename> Git repository, see the
+ "<ulink url='&YOCTO_DOCS_REF_URL;#structure-core'>Top-Level Core Components</ulink>"
+ section in the Yocto Project Reference Manual.
+ </note>
+ </para>
+
+ <para id='gs-poky-reference-distribution'>
+ The following figure illustrates what generally comprises Poky:
+ <imagedata fileref="figures/poky-reference-distribution.png" format="PNG" align='center' width="8in"/>
+ <itemizedlist>
+ <listitem><para>
+ BitBake is a task executor and scheduler that is the heart of
+ the OpenEmbedded build system.
+ </para></listitem>
+ <listitem><para>
+ <filename>meta-poky</filename>, which is Poky-specific
+ metadata.
+ </para></listitem>
+ <listitem><para>
+ <filename>meta-yocto-bsp</filename>, which are Yocto
+ Project-specific Board Support Packages (BSPs).
+ </para></listitem>
+ <listitem><para>
+ OpenEmbedded-Core (OE-Core) metadata, which includes
+ shared configurations, global variable definitions,
+ shared classes, packaging, and recipes.
+ Classes define the encapsulation and inheritance of build
+ logic.
+ Recipes are the logical units of software and images
+ to be built.
+ </para></listitem>
+ <listitem><para>
+ Documentation, which contains the Yocto Project source
+ files used to make the set of user manuals.
+ </para></listitem>
+ </itemizedlist>
+ <note>
+ While Poky is a "complete" distribution specification and is
+ tested and put through QA, you cannot use it as a product
+ "out of the box" in its current form.
+ </note>
+ </para>
+
+ <para>
+ To use the Yocto Project tools, you can use Git to clone (download)
+ the Poky repository then use your local copy of the reference
+ distribution to bootstrap your own distribution.
+ <note>
+ Poky does not contain binary files.
+ It is a working example of how to build your own custom Linux distribution
+ from source.
+ </note>
+ </para>
+
+ <para>
+ Poky has a regular, well established, six-month release cycle
+ under its own version.
+ Major releases occur at the same time major releases (point
+ releases) occur for the Yocto Project, which are typically in the
+ Spring and Fall.
+ For more information on the Yocto Project release schedule and
+ cadence, see the
+ "<ulink url='&YOCTO_DOCS_REF_URL;#ref-release-process'>Yocto Project Releases and the Stable Release Process</ulink>"
+ chapter in the Yocto Project Reference Manual.
+ </para>
+
+ <para>
+ Much has been said about Poky being a "default configuration."
+ A default configuration provides a starting image footprint.
+ You can use Poky out of the box to create an image ranging from a
+ shell-accessible minimal image all the way up to a Linux
+ Standard Base-compliant image that uses a GNOME Mobile and
+ Embedded (GMAE) based reference user interface called Sato.
+ </para>
+
+ <para>
+ One of the most powerful properties of Poky is that every aspect
+ of a build is controlled by the metadata.
+ You can use metadata to augment these base image types by
+ adding metadata
+ <link linkend='the-yocto-project-layer-model'>layers</link>
+ that extend functionality.
+ These layers can provide, for example, an additional software
+ stack for an image type, add a board support package (BSP) for
+ additional hardware, or even create a new image type.
+ </para>
+
+ <para>
+ Metadata is loosely grouped into configuration files or package
+ recipes.
+ A recipe is a collection of non-executable metadata used by
+ BitBake to set variables or define additional build-time tasks.
+ A recipe contains fields such as the recipe description, the recipe
+ version, the license of the package and the upstream source
+ repository.
+ A recipe might also indicate that the build process uses autotools,
+ make, distutils or any other build process, in which case the basic
+ functionality can be defined by the classes it inherits from
+ the OE-Core layer's class definitions in
+ <filename>./meta/classes</filename>.
+ Within a recipe you can also define additional tasks as well as
+ task prerequisites.
+ Recipe syntax through BitBake also supports both
+ <filename>_prepend</filename> and <filename>_append</filename>
+ operators as a method of extending task functionality.
+ These operators inject code into the beginning or end of a task.
+ For information on these BitBake operators, see the
+ "<ulink url='&YOCTO_DOCS_BB_URL;#appending-and-prepending-override-style-syntax'>Appending and Prepending (Override Style Syntax)</ulink>"
+ section in the BitBake User's Manual.
+ </para>
+ </section>
+
+ <section id='openembedded-build-system-workflow'>
+ <title>The OpenEmbedded Build System Workflow</title>
+
+ <para>
+ The
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>OpenEmbedded build system</ulink>
+ uses a "workflow" to accomplish image and SDK generation.
+ The following figure overviews that workflow:
+ <imagedata fileref="figures/YP-flow-diagram.png"
+ format="PNG" align='center' width="8in"/>
+ Following is a brief summary of the "workflow":
+ <orderedlist>
+ <listitem><para>
+ Developers specify architecture, policies, patches and
+ configuration details.
+ </para></listitem>
+ <listitem><para>
+ The build system fetches and downloads the source code
+ from the specified location.
+ The build system supports standard methods such as tarballs
+ or source code repositories systems such as Git.
+ </para></listitem>
+ <listitem><para>
+ Once source code is downloaded, the build system extracts
+ the sources into a local work area where patches are
+ applied and common steps for configuring and compiling
+ the software are run.
+ </para></listitem>
+ <listitem><para>
+ The build system then installs the software into a
+ temporary staging area where the binary package format you
+ select (DEB, RPM, or IPK) is used to roll up the software.
+ </para></listitem>
+ <listitem><para>
+ Different QA and sanity checks run throughout entire
+ build process.
+ </para></listitem>
+ <listitem><para>
+ After the binaries are created, the build system
+ generates a binary package feed that is used to create
+ the final root file image.
+ </para></listitem>
+ <listitem><para>
+ The build system generates the file system image and a
+ customized Extensible SDK (eSDK) for application
+ development in parallel.
+ </para></listitem>
+ </orderedlist>
+ </para>
+
+ <para>
+ For a very detailed look at this workflow, see the
+ "<link linkend='openembedded-build-system-build-concepts'>OpenEmbedded Build System Concepts</link>"
+ section.
+ </para>
+ </section>
+
+
+ <section id='some-basic-terms'>
+ <title>Some Basic Terms</title>
+
+ <para>
+ It helps to understand some basic fundamental terms when
+ learning the Yocto Project.
+ Although a list of terms exists in the
+ "<ulink url='&YOCTO_DOCS_REF_URL;#ref-terms'>Yocto Project Terms</ulink>"
+ section of the Yocto Project Reference Manual, this section
+ provides the definitions of some terms helpful for getting started:
+ <itemizedlist>
+ <listitem><para>
+ <emphasis>Configuration Files:</emphasis>
+ Files that hold global definitions of variables,
+ user-defined variables, and hardware configuration
+ information.
+ These files tell the
+ <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>Open-Embedded build system</ulink>
+ what to build and what to put into the image to support a
+ particular platform.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Extensible Software Development Kit (eSDK):</emphasis>
+ A custom SDK for application developers.
+ This eSDK allows developers to incorporate their library
+ and programming changes back into the image to make
+ their code available to other application developers.
+ For information on the eSDK, see the
+ <ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Application Development and the Extensible Software Development Kit (eSDK)</ulink>
+ manual.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Layer:</emphasis>
+ A collection of related recipes.
+ Layers allow you to consolidate related metadata to
+ customize your build.
+ Layers also isolate information used when building
+ for multiple architectures.
+ Layers are hierarchical in their ability to override
+ previous specifications.
+ You can include any number of available layers from the
+ Yocto Project and customize the build by adding your
+ layers after them.
+ You can search the Layer Index for layers used within
+ Yocto Project.</para>
+
+ <para>For more detailed information on layers, see the
+ "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and Creating Layers</ulink>"
+ section in the Yocto Project Development Tasks Manual.
+ For a discussion specifically on BSP Layers, see the
+ "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>"
+ section in the Yocto Project Board Support Packages (BSP)
+ Developer's Guide.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Metadata:</emphasis>
+ A key element of the Yocto Project is the Metadata that
+ is used to construct a Linux distribution and is contained
+ in the files that the OpenEmbedded build system parses
+ when building an image.
+ In general, Metadata includes recipes, configuration
+ files, and other information that refers to the build
+ instructions themselves, as well as the data used to
+ control what things get built and the effects of the
+ build.
+ Metadata also includes commands and data used to
+ indicate what versions of software are used, from
+ where they are obtained, and changes or additions to the
+ software itself (patches or auxiliary files) that
+ are used to fix bugs or customize the software for use
+ in a particular situation.
+ OpenEmbedded-Core is an important set of validated
+ metadata.
+ </para></listitem>
+ <listitem><para id='gs-term-openembedded-build-system'>
+ <emphasis>OpenEmbedded Build System:</emphasis>
+ The terms "BitBake" and "build system" are sometimes
+ used for the OpenEmbedded Build System.</para>
+
+ <para>BitBake is a task scheduler and execution engine
+ that parses instructions (i.e. recipes) and configuration
+ data.
+ After a parsing phase, BitBake creates a dependency tree
+ to order the compilation, schedules the compilation of
+ the included code, and finally executes the building
+ of the specified custom Linux image (distribution).
+ BitBake is similar to the <filename>make</filename>
+ tool.</para>
+
+ <para>During a build process, the build system tracks
+ dependencies and performs a native or cross-compilation
+ of the package.
+ As a first step in a cross-build setup, the framework
+ attempts to create a cross-compiler toolchain
+ (i.e. Extensible SDK) suited for the target platform.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>OpenEmbedded-Core (OE-Core):</emphasis>
+ OE-Core is metadata comprised of foundation recipes,
+ classes, and associated files that are meant to be
+ common among many different OpenEmbedded-derived systems,
+ including the Yocto Project.
+ OE-Core is a curated subset of an original repository
+ developed by the OpenEmbedded community that has been
+ pared down into a smaller, core set of continuously
+ validated recipes.
+ The result is a tightly controlled and quality-assured
+ core set of recipes.</para>
+
+ <para>You can see the Metadata in the
+ <filename>meta</filename> directory of the Yocto Project
+ <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi'>Source Repositories</ulink>.
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Packages:</emphasis>
+ In the context of the Yocto Project, this term refers to a
+ recipe's packaged output produced by BitBake (i.e. a
+ "baked recipe").
+ A package is generally the compiled binaries produced from the
+ recipe's sources.
+ You "bake" something by running it through BitBake.</para>
+
+ <para>It is worth noting that the term "package" can,
+ in general, have subtle meanings.
+ For example, the packages referred to in the
+ "<ulink url='&YOCTO_DOCS_REF_URL;#required-packages-for-the-build-host'>Required Packages for the Build Host</ulink>"
+ section in the Yocto Project Reference Manual are compiled
+ binaries that, when installed, add functionality to your
+ Linux distribution.</para>
+
+ <para>Another point worth noting is that historically within
+ the Yocto Project, recipes were referred to as packages - thus,
+ the existence of several BitBake variables that are seemingly
+ mis-named,
+ (e.g. <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>,
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>,
+ and
+ <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>).
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Poky:</emphasis>
+ Poky is a reference embedded distribution and a reference
+ test configuration.
+ Poky provides the following:
+ <itemizedlist>
+ <listitem><para>
+ A base-level functional distro used to illustrate
+ how to customize a distribution.
+ </para></listitem>
+ <listitem><para>
+ A means by which to test the Yocto Project
+ components (i.e. Poky is used to validate
+ the Yocto Project).
+ </para></listitem>
+ <listitem><para>
+ A vehicle through which you can download
+ the Yocto Project.
+ </para></listitem>
+ </itemizedlist>
+ Poky is not a product level distro.
+ Rather, it is a good starting point for customization.
+ <note>
+ Poky is an integration layer on top of OE-Core.
+ </note>
+ </para></listitem>
+ <listitem><para>
+ <emphasis>Recipe:</emphasis>
+ The most common form of metadata.
+ A recipe contains a list of settings and tasks
+ (i.e. instructions) for building packages that are then
+ used to build the binary image.
+ A recipe describes where you get source code and which
+ patches to apply.
+ Recipes describe dependencies for libraries or for other
+ recipes as well as configuration and compilation options.
+ Related recipes are consolidated into a layer.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </section>
+</chapter>
+<!--
+vim: expandtab tw=80 ts=4
+-->
diff --git a/poky/documentation/overview-manual/overview-manual.rst b/poky/documentation/overview-manual/overview-manual.rst
index f20b20e..80ce9aa 100644
--- a/poky/documentation/overview-manual/overview-manual.rst
+++ b/poky/documentation/overview-manual/overview-manual.rst
@@ -1,4 +1,4 @@
-.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
+.. SPDX-License-Identifier: CC-BY-2.0-UK
==========================================
Yocto Project Overview and Concepts Manual
diff --git a/poky/documentation/overview-manual/overview-manual.xml b/poky/documentation/overview-manual/overview-manual.xml
new file mode 100755
index 0000000..8021a2e
--- /dev/null
+++ b/poky/documentation/overview-manual/overview-manual.xml
@@ -0,0 +1,130 @@
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
+"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
+[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
+<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
+
+<book id='overview-manual' lang='en'
+ xmlns:xi="http://www.w3.org/2003/XInclude"
+ xmlns="http://docbook.org/ns/docbook"
+ >
+ <bookinfo>
+
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref='figures/overview-manual-title.png'
+ format='SVG'
+ align='left' scalefit='1' width='100%'/>
+ </imageobject>
+ </mediaobject>
+
+ <title>
+ Yocto Project Overview and Concepts Manual
+ </title>
+
+ <authorgroup>
+ <author>
+ <affiliation>
+ <orgname>&ORGNAME;</orgname>
+ </affiliation>
+ <email>&ORGEMAIL;</email>
+ </author>
+ </authorgroup>
+
+ <revhistory>
+ <revision>
+ <revnumber>2.5</revnumber>
+ <date>May 2018</date>
+ <revremark>The initial document released with the Yocto Project 2.5 Release.</revremark>
+ </revision>
+ <revision>
+ <revnumber>2.6</revnumber>
+ <date>November 2018</date>
+ <revremark>Released with the Yocto Project 2.6 Release.</revremark>
+ </revision>
+ <revision>
+ <revnumber>2.7</revnumber>
+ <date>May 2019</date>
+ <revremark>Released with the Yocto Project 2.7 Release.</revremark>
+ </revision>
+ <revision>
+ <revnumber>3.0</revnumber>
+ <date>October 2019</date>
+ <revremark>Released with the Yocto Project 3.0 Release.</revremark>
+ </revision>
+ <revision>
+ <revnumber>3.1</revnumber>
+ <date>&REL_MONTH_YEAR;</date>
+ <revremark>Released with the Yocto Project 3.1 Release.</revremark>
+ </revision>
+ </revhistory>
+
+ <copyright>
+ <year>©RIGHT_YEAR;</year>
+ <holder>Linux Foundation</holder>
+ </copyright>
+
+ <legalnotice>
+ <para>
+ Permission is granted to copy, distribute and/or modify this document under
+ the terms of the <ulink type="http" url="http://creativecommons.org/licenses/by-sa/2.0/uk/">
+ Creative Commons Attribution-Share Alike 2.0 UK: England & Wales</ulink> as published by
+ Creative Commons.
+ </para>
+ <note><title>Manual Notes</title>
+ <itemizedlist>
+ <listitem><para>
+ This version of the
+ <emphasis>Yocto Project Overview and Concepts Manual</emphasis>
+ is for the &YOCTO_DOC_VERSION; release of the
+ Yocto Project.
+ To be sure you have the latest version of the manual
+ for this release, go to the
+ <ulink url='&YOCTO_DOCS_URL;'>Yocto Project documentation page</ulink>
+ and select the manual from that site.
+ Manuals from the site are more up-to-date than manuals
+ derived from the Yocto Project released TAR files.
+ </para></listitem>
+ <listitem><para>
+ If you located this manual through a web search, the
+ version of the manual might not be the one you want
+ (e.g. the search might have returned a manual much
+ older than the Yocto Project version with which you
+ are working).
+ You can see all Yocto Project major releases by
+ visiting the
+ <ulink url='&YOCTO_WIKI_URL;/wiki/Releases'>Releases</ulink>
+ page.
+ If you need a version of this manual for a different
+ Yocto Project release, visit the
+ <ulink url='&YOCTO_DOCS_URL;'>Yocto Project documentation page</ulink>
+ and select the manual set by using the
+ "ACTIVE RELEASES DOCUMENTATION" or "DOCUMENTS ARCHIVE"
+ pull-down menus.
+ </para></listitem>
+ <listitem>
+ <para>
+ To report any inaccuracies or problems with this
+ (or any other Yocto Project) manual, send an email to
+ the Yocto Project documentation mailing list at
+ <filename>docs@lists.yoctoproject.org</filename> or
+ log into the freenode <filename>#yocto</filename> channel.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </note>
+ </legalnotice>
+
+ </bookinfo>
+
+ <xi:include href="overview-manual-intro.xml"/>
+
+ <xi:include href="overview-manual-yp-intro.xml"/>
+
+ <xi:include href="overview-manual-development-environment.xml"/>
+
+ <xi:include href="overview-manual-concepts.xml" />
+
+</book>
+<!--
+vim: expandtab tw=80 ts=4
+-->