| <!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" |
| "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" |
| [<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > |
| |
| <chapter id='closer-look'> |
| <title>A Closer Look at the Yocto Project Development Environment</title> |
| |
| <para> |
| This chapter takes a more detailed look at the Yocto Project |
| development environment. |
| The following diagram represents the development environment at a |
| high level. |
| The remainder of this chapter expands on the fundamental input, output, |
| process, and |
| <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>) blocks |
| in the Yocto Project development environment. |
| </para> |
| |
| <para id='general-yocto-environment-figure'> |
| <imagedata fileref="figures/yocto-environment-ref.png" align="center" width="8in" depth="4.25in" /> |
| </para> |
| |
| <para> |
| The generalized Yocto Project Development Environment consists of |
| several functional areas: |
| <itemizedlist> |
| <listitem><para><emphasis>User Configuration:</emphasis> |
| Metadata you can use to control the build process. |
| </para></listitem> |
| <listitem><para><emphasis>Metadata Layers:</emphasis> |
| Various layers that provide software, machine, and |
| distro Metadata.</para></listitem> |
| <listitem><para><emphasis>Source Files:</emphasis> |
| Upstream releases, local projects, and SCMs.</para></listitem> |
| <listitem><para><emphasis>Build System:</emphasis> |
| Processes under the control of |
| <ulink url='&YOCTO_DOCS_DEV_URL;#bitbake-term'>BitBake</ulink>. |
| This block expands on how BitBake fetches source, applies |
| patches, completes compilation, analyzes output for package |
| generation, creates and tests packages, generates images, and |
| generates cross-development tools.</para></listitem> |
| <listitem><para><emphasis>Package Feeds:</emphasis> |
| Directories containing output packages (RPM, DEB or IPK), |
| which are subsequently used in the construction of an image or |
| SDK, produced by the build system. |
| These feeds can also be copied and shared using a web server or |
| other means to facilitate extending or updating existing |
| images on devices at runtime if runtime package management is |
| enabled.</para></listitem> |
| <listitem><para><emphasis>Images:</emphasis> |
| Images produced by the development process. |
| </para></listitem> |
| <listitem><para><emphasis>Application Development SDK:</emphasis> |
| Cross-development tools that are produced along with an image |
| or separately with BitBake.</para></listitem> |
| </itemizedlist> |
| </para> |
| |
| <section id="user-configuration"> |
| <title>User Configuration</title> |
| |
| <para> |
| User configuration helps define the build. |
| Through user configuration, you can tell BitBake the |
| target architecture for which you are building the image, |
| where to store downloaded source, and other build properties. |
| </para> |
| |
| <para> |
| The following figure shows an expanded representation of the |
| "User Configuration" box of the |
| <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>: |
| </para> |
| |
| <para> |
| <imagedata fileref="figures/user-configuration.png" align="center" /> |
| </para> |
| |
| <para> |
| BitBake needs some basic configuration files in order to complete |
| a build. |
| These files are <filename>*.conf</filename> files. |
| The minimally necessary ones reside as example files in the |
| <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>. |
| For simplicity, this section refers to the Source Directory as |
| the "Poky Directory." |
| </para> |
| |
| <para> |
| When you clone the <filename>poky</filename> Git repository or you |
| download and unpack a Yocto Project release, you can set up the |
| Source Directory to be named anything you want. |
| For this discussion, the cloned repository uses the default |
| name <filename>poky</filename>. |
| <note> |
| The Poky repository is primarily an aggregation of existing |
| repositories. |
| It is not a canonical upstream source. |
| </note> |
| </para> |
| |
| <para> |
| The <filename>meta-poky</filename> layer inside Poky contains |
| a <filename>conf</filename> directory that has example |
| configuration files. |
| These example files are used as a basis for creating actual |
| configuration files when you source the build environment |
| script |
| (i.e. |
| <link linkend='structure-core-script'><filename>&OE_INIT_FILE;</filename></link> |
| or |
| <link linkend='structure-memres-core-script'><filename>oe-init-build-env-memres</filename></link>). |
| </para> |
| |
| <para> |
| Sourcing the build environment script creates a |
| <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink> |
| if one does not already exist. |
| BitBake uses the Build Directory for all its work during builds. |
| The Build Directory has a <filename>conf</filename> directory that |
| contains default versions of your <filename>local.conf</filename> |
| and <filename>bblayers.conf</filename> configuration files. |
| These default configuration files are created only if versions |
| do not already exist in the Build Directory at the time you |
| source the build environment setup script. |
| </para> |
| |
| <para> |
| Because the Poky repository is fundamentally an aggregation of |
| existing repositories, some users might be familiar with running |
| the <filename>&OE_INIT_FILE;</filename> or |
| <filename>oe-init-build-env-memres</filename> script in the context |
| of separate OpenEmbedded-Core and BitBake repositories rather than a |
| single Poky repository. |
| This discussion assumes the script is executed from within a cloned |
| or unpacked version of Poky. |
| </para> |
| |
| <para> |
| Depending on where the script is sourced, different sub-scripts |
| are called to set up the Build Directory (Yocto or OpenEmbedded). |
| Specifically, the script |
| <filename>scripts/oe-setup-builddir</filename> inside the |
| poky directory sets up the Build Directory and seeds the directory |
| (if necessary) with configuration files appropriate for the |
| Yocto Project development environment. |
| <note> |
| The <filename>scripts/oe-setup-builddir</filename> script |
| uses the <filename>$TEMPLATECONF</filename> variable to |
| determine which sample configuration files to locate. |
| </note> |
| </para> |
| |
| <para> |
| The <filename>local.conf</filename> file provides many |
| basic variables that define a build environment. |
| Here is a list of a few. |
| To see the default configurations in a <filename>local.conf</filename> |
| file created by the build environment script, see the |
| <filename>local.conf.sample</filename> in the |
| <filename>meta-poky</filename> layer: |
| <itemizedlist> |
| <listitem><para><emphasis>Parallelism Options:</emphasis> |
| Controlled by the |
| <link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename></link>, |
| <link linkend='var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></link>, |
| and |
| <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_NUMBER_PARSE_THREADS'><filename>BB_NUMBER_PARSE_THREADS</filename></ulink> |
| variables.</para></listitem> |
| <listitem><para><emphasis>Target Machine Selection:</emphasis> |
| Controlled by the |
| <link linkend='var-MACHINE'><filename>MACHINE</filename></link> |
| variable.</para></listitem> |
| <listitem><para><emphasis>Download Directory:</emphasis> |
| Controlled by the |
| <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link> |
| variable.</para></listitem> |
| <listitem><para><emphasis>Shared State Directory:</emphasis> |
| Controlled by the |
| <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> |
| variable.</para></listitem> |
| <listitem><para><emphasis>Build Output:</emphasis> |
| Controlled by the |
| <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link> |
| variable.</para></listitem> |
| </itemizedlist> |
| <note> |
| Configurations set in the <filename>conf/local.conf</filename> |
| file can also be set in the |
| <filename>conf/site.conf</filename> and |
| <filename>conf/auto.conf</filename> configuration files. |
| </note> |
| </para> |
| |
| <para> |
| The <filename>bblayers.conf</filename> file tells BitBake what |
| layers you want considered during the build. |
| By default, the layers listed in this file include layers |
| minimally needed by the build system. |
| However, you must manually add any custom layers you have created. |
| You can find more information on working with the |
| <filename>bblayers.conf</filename> file in the |
| "<ulink url='&YOCTO_DOCS_DEV_URL;#enabling-your-layer'>Enabling Your Layer</ulink>" |
| section in the Yocto Project Development Manual. |
| </para> |
| |
| <para> |
| The files <filename>site.conf</filename> and |
| <filename>auto.conf</filename> are not created by the environment |
| initialization script. |
| If you want the <filename>site.conf</filename> file, you need to |
| create that yourself. |
| The <filename>auto.conf</filename> file is typically created by |
| an autobuilder: |
| <itemizedlist> |
| <listitem><para><emphasis><filename>site.conf</filename>:</emphasis> |
| You can use the <filename>conf/site.conf</filename> |
| configuration file to configure multiple build directories. |
| For example, suppose you had several build environments and |
| they shared some common features. |
| You can set these default build properties here. |
| A good example is perhaps the packaging format to use |
| through the |
| <link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link> |
| variable.</para> |
| <para>One useful scenario for using the |
| <filename>conf/site.conf</filename> file is to extend your |
| <link linkend='var-BBPATH'><filename>BBPATH</filename></link> |
| variable to include the path to a |
| <filename>conf/site.conf</filename>. |
| Then, when BitBake looks for Metadata using |
| <filename>BBPATH</filename>, it finds the |
| <filename>conf/site.conf</filename> file and applies your |
| common configurations found in the file. |
| To override configurations in a particular build directory, |
| alter the similar configurations within that build |
| directory's <filename>conf/local.conf</filename> file. |
| </para></listitem> |
| <listitem><para><emphasis><filename>auto.conf</filename>:</emphasis> |
| The file is usually created and written to by |
| an autobuilder. |
| The settings put into the file are typically the same as |
| you would find in the <filename>conf/local.conf</filename> |
| or the <filename>conf/site.conf</filename> files. |
| </para></listitem> |
| </itemizedlist> |
| </para> |
| |
| <para> |
| You can edit all configuration files to further define |
| any particular build environment. |
| This process is represented by the "User Configuration Edits" |
| box in the figure. |
| </para> |
| |
| <para> |
| When you launch your build with the |
| <filename>bitbake <replaceable>target</replaceable></filename> |
| command, BitBake sorts out the configurations to ultimately |
| define your build environment. |
| It is important to understand that the OpenEmbedded build system |
| reads the configuration files in a specific order: |
| <filename>site.conf</filename>, <filename>auto.conf</filename>, |
| and <filename>local.conf</filename>. |
| And, the build system applies the normal assignment statement |
| rules. |
| Because the files are parsed in a specific order, variable |
| assignments for the same variable could be affected. |
| For example, if the <filename>auto.conf</filename> file and |
| the <filename>local.conf</filename> set |
| <replaceable>variable1</replaceable> to different values, because |
| the build system parses <filename>local.conf</filename> after |
| <filename>auto.conf</filename>, |
| <replaceable>variable1</replaceable> is assigned the value from |
| the <filename>local.conf</filename> file. |
| </para> |
| </section> |
| |
| <section id="metadata-machine-configuration-and-policy-configuration"> |
| <title>Metadata, Machine Configuration, and Policy Configuration</title> |
| |
| <para> |
| The previous section described the user configurations that |
| define BitBake's global behavior. |
| This section takes a closer look at the layers the build system |
| uses to further control the build. |
| These layers provide Metadata for the software, machine, and |
| policy. |
| </para> |
| |
| <para> |
| In general, three types of layer input exist: |
| <itemizedlist> |
| <listitem><para><emphasis>Policy Configuration:</emphasis> |
| Distribution Layers provide top-level or general |
| policies for the image or SDK being built. |
| For example, this layer would dictate whether BitBake |
| produces RPM or IPK packages.</para></listitem> |
| <listitem><para><emphasis>Machine Configuration:</emphasis> |
| Board Support Package (BSP) layers provide machine |
| configurations. |
| This type of information is specific to a particular |
| target architecture.</para></listitem> |
| <listitem><para><emphasis>Metadata:</emphasis> |
| Software layers contain user-supplied recipe files, |
| patches, and append files. |
| </para></listitem> |
| </itemizedlist> |
| </para> |
| |
| <para> |
| The following figure shows an expanded representation of the |
| Metadata, Machine Configuration, and Policy Configuration input |
| (layers) boxes of the |
| <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>: |
| </para> |
| |
| <para> |
| <imagedata fileref="figures/layer-input.png" align="center" width="8in" depth="7.5in" /> |
| </para> |
| |
| <para> |
| In general, all layers have a similar structure. |
| They all contain a licensing file |
| (e.g. <filename>COPYING</filename>) if the layer is to be |
| distributed, a <filename>README</filename> file as good practice |
| and especially if the layer is to be distributed, a |
| configuration directory, and recipe directories. |
| </para> |
| |
| <para> |
| The Yocto Project has many layers that can be used. |
| You can see a web-interface listing of them on the |
| <ulink url="http://git.yoctoproject.org/">Source Repositories</ulink> |
| page. |
| The layers are shown at the bottom categorized under |
| "Yocto Metadata Layers." |
| These layers are fundamentally a subset of the |
| <ulink url="http://layers.openembedded.org/layerindex/layers/">OpenEmbedded Metadata Index</ulink>, |
| which lists all layers provided by the OpenEmbedded community. |
| <note> |
| Layers exist in the Yocto Project Source Repositories that |
| cannot be found in the OpenEmbedded Metadata Index. |
| These layers are either deprecated or experimental in nature. |
| </note> |
| </para> |
| |
| <para> |
| BitBake uses the <filename>conf/bblayers.conf</filename> file, |
| which is part of the user configuration, to find what layers it |
| should be using as part of the build. |
| </para> |
| |
| <para> |
| For more information on layers, see the |
| "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and Creating Layers</ulink>" |
| section in the Yocto Project Development Manual. |
| </para> |
| |
| <section id="distro-layer"> |
| <title>Distro Layer</title> |
| |
| <para> |
| The distribution layer provides policy configurations for your |
| distribution. |
| Best practices dictate that you isolate these types of |
| configurations into their own layer. |
| Settings you provide in |
| <filename>conf/distro/<replaceable>distro</replaceable>.conf</filename> override |
| similar |
| settings that BitBake finds in your |
| <filename>conf/local.conf</filename> file in the Build |
| Directory. |
| </para> |
| |
| <para> |
| The following list provides some explanation and references |
| for what you typically find in the distribution layer: |
| <itemizedlist> |
| <listitem><para><emphasis>classes:</emphasis> |
| Class files (<filename>.bbclass</filename>) hold |
| common functionality that can be shared among |
| recipes in the distribution. |
| When your recipes inherit a class, they take on the |
| settings and functions for that class. |
| You can read more about class files in the |
| "<link linkend='ref-classes'>Classes</link>" section. |
| </para></listitem> |
| <listitem><para><emphasis>conf:</emphasis> |
| This area holds configuration files for the |
| layer (<filename>conf/layer.conf</filename>), |
| the distribution |
| (<filename>conf/distro/<replaceable>distro</replaceable>.conf</filename>), |
| and any distribution-wide include files. |
| </para></listitem> |
| <listitem><para><emphasis>recipes-*:</emphasis> |
| Recipes and append files that affect common |
| functionality across the distribution. |
| This area could include recipes and append files |
| to add distribution-specific configuration, |
| initialization scripts, custom image recipes, |
| and so forth.</para></listitem> |
| </itemizedlist> |
| </para> |
| </section> |
| |
| <section id="bsp-layer"> |
| <title>BSP Layer</title> |
| |
| <para> |
| The BSP Layer provides machine configurations. |
| Everything in this layer is specific to the machine for which |
| you are building the image or the SDK. |
| A common structure or form is defined for BSP layers. |
| You can learn more about this structure in the |
| <ulink url='&YOCTO_DOCS_BSP_URL;'>Yocto Project Board Support Package (BSP) Developer's Guide</ulink>. |
| <note> |
| In order for a BSP layer to be considered compliant with the |
| Yocto Project, it must meet some structural requirements. |
| </note> |
| </para> |
| |
| <para> |
| The BSP Layer's configuration directory contains |
| configuration files for the machine |
| (<filename>conf/machine/<replaceable>machine</replaceable>.conf</filename>) and, |
| of course, the layer (<filename>conf/layer.conf</filename>). |
| </para> |
| |
| <para> |
| The remainder of the layer is dedicated to specific recipes |
| by function: <filename>recipes-bsp</filename>, |
| <filename>recipes-core</filename>, |
| <filename>recipes-graphics</filename>, and |
| <filename>recipes-kernel</filename>. |
| Metadata can exist for multiple formfactors, graphics |
| support systems, and so forth. |
| <note> |
| While the figure shows several <filename>recipes-*</filename> |
| directories, not all these directories appear in all |
| BSP layers. |
| </note> |
| </para> |
| </section> |
| |
| <section id="software-layer"> |
| <title>Software Layer</title> |
| |
| <para> |
| The software layer provides the Metadata for additional |
| software packages used during the build. |
| This layer does not include Metadata that is specific to the |
| distribution or the machine, which are found in their |
| respective layers. |
| </para> |
| |
| <para> |
| This layer contains any new recipes that your project needs |
| in the form of recipe files. |
| </para> |
| </section> |
| </section> |
| |
| <section id="sources-dev-environment"> |
| <title>Sources</title> |
| |
| <para> |
| In order for the OpenEmbedded build system to create an image or |
| any target, it must be able to access source files. |
| The |
| <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link> |
| represents source files using the "Upstream Project Releases", |
| "Local Projects", and "SCMs (optional)" boxes. |
| The figure represents mirrors, which also play a role in locating |
| source files, with the "Source Mirror(s)" box. |
| </para> |
| |
| <para> |
| The method by which source files are ultimately organized is |
| a function of the project. |
| For example, for released software, projects tend to use tarballs |
| or other archived files that can capture the state of a release |
| guaranteeing that it is statically represented. |
| On the other hand, for a project that is more dynamic or |
| experimental in nature, a project might keep source files in a |
| repository controlled by a Source Control Manager (SCM) such as |
| Git. |
| Pulling source from a repository allows you to control |
| the point in the repository (the revision) from which you want to |
| build software. |
| Finally, a combination of the two might exist, which would give the |
| consumer a choice when deciding where to get source files. |
| </para> |
| |
| <para> |
| BitBake uses the |
| <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link> |
| variable to point to source files regardless of their location. |
| Each recipe must have a <filename>SRC_URI</filename> variable |
| that points to the source. |
| </para> |
| |
| <para> |
| Another area that plays a significant role in where source files |
| come from is pointed to by the |
| <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link> |
| variable. |
| This area is a cache that can hold previously downloaded source. |
| You can also instruct the OpenEmbedded build system to create |
| tarballs from Git repositories, which is not the default behavior, |
| and store them in the <filename>DL_DIR</filename> by using the |
| <link linkend='var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></link> |
| variable. |
| </para> |
| |
| <para> |
| Judicious use of a <filename>DL_DIR</filename> directory can |
| save the build system a trip across the Internet when looking |
| for files. |
| A good method for using a download directory is to have |
| <filename>DL_DIR</filename> point to an area outside of your |
| Build Directory. |
| Doing so allows you to safely delete the Build Directory |
| if needed without fear of removing any downloaded source file. |
| </para> |
| |
| <para> |
| The remainder of this section provides a deeper look into the |
| source files and the mirrors. |
| Here is a more detailed look at the source file area of the |
| base figure: |
| <imagedata fileref="figures/source-input.png" align="center" width="7in" depth="7.5in" /> |
| </para> |
| |
| <section id='upstream-project-releases'> |
| <title>Upstream Project Releases</title> |
| |
| <para> |
| Upstream project releases exist anywhere in the form of an |
| archived file (e.g. tarball or zip file). |
| These files correspond to individual recipes. |
| For example, the figure uses specific releases each for |
| BusyBox, Qt, and Dbus. |
| An archive file can be for any released product that can be |
| built using a recipe. |
| </para> |
| </section> |
| |
| <section id='local-projects'> |
| <title>Local Projects</title> |
| |
| <para> |
| Local projects are custom bits of software the user provides. |
| These bits reside somewhere local to a project - perhaps |
| a directory into which the user checks in items (e.g. |
| a local directory containing a development source tree |
| used by the group). |
| </para> |
| |
| <para> |
| The canonical method through which to include a local project |
| is to use the |
| <link linkend='ref-classes-externalsrc'><filename>externalsrc</filename></link> |
| class to include that local project. |
| You use either the <filename>local.conf</filename> or a |
| recipe's append file to override or set the |
| recipe to point to the local directory on your disk to pull |
| in the whole source tree. |
| </para> |
| |
| <para> |
| For information on how to use the |
| <filename>externalsrc</filename> class, see the |
| "<link linkend='ref-classes-externalsrc'><filename>externalsrc.bbclass</filename></link>" |
| section. |
| </para> |
| </section> |
| |
| <section id='scms'> |
| <title>Source Control Managers (Optional)</title> |
| |
| <para> |
| Another place the build system can get source files from is |
| through an SCM such as Git or Subversion. |
| In this case, a repository is cloned or checked out. |
| The |
| <link linkend='ref-tasks-fetch'><filename>do_fetch</filename></link> |
| task inside BitBake uses |
| the <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link> |
| variable and the argument's prefix to determine the correct |
| fetcher module. |
| </para> |
| |
| <note> |
| For information on how to have the OpenEmbedded build system |
| generate tarballs for Git repositories and place them in the |
| <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link> |
| directory, see the |
| <link linkend='var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></link> |
| variable. |
| </note> |
| |
| <para> |
| When fetching a repository, BitBake uses the |
| <link linkend='var-SRCREV'><filename>SRCREV</filename></link> |
| variable to determine the specific revision from which to |
| build. |
| </para> |
| </section> |
| |
| <section id='source-mirrors'> |
| <title>Source Mirror(s)</title> |
| |
| <para> |
| Two kinds of mirrors exist: pre-mirrors and regular mirrors. |
| The <link linkend='var-PREMIRRORS'><filename>PREMIRRORS</filename></link> |
| and |
| <link linkend='var-MIRRORS'><filename>MIRRORS</filename></link> |
| variables point to these, respectively. |
| BitBake checks pre-mirrors before looking upstream for any |
| source files. |
| Pre-mirrors are appropriate when you have a shared directory |
| that is not a directory defined by the |
| <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link> |
| variable. |
| A Pre-mirror typically points to a shared directory that is |
| local to your organization. |
| </para> |
| |
| <para> |
| Regular mirrors can be any site across the Internet that is |
| used as an alternative location for source code should the |
| primary site not be functioning for some reason or another. |
| </para> |
| </section> |
| </section> |
| |
| <section id="package-feeds-dev-environment"> |
| <title>Package Feeds</title> |
| |
| <para> |
| When the OpenEmbedded build system generates an image or an SDK, |
| it gets the packages from a package feed area located in the |
| <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>. |
| The |
| <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link> |
| shows this package feeds area in the upper-right corner. |
| </para> |
| |
| <para> |
| This section looks a little closer into the package feeds area used |
| by the build system. |
| Here is a more detailed look at the area: |
| <imagedata fileref="figures/package-feeds.png" align="center" width="7in" depth="6in" /> |
| </para> |
| |
| <para> |
| Package feeds are an intermediary step in the build process. |
| The OpenEmbedded build system provides classes to generate |
| different package types, and you specify which classes to enable |
| through the |
| <link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link> |
| variable. |
| Before placing the packages into package feeds, |
| the build process validates them with generated output quality |
| assurance checks through the |
| <link linkend='ref-classes-insane'><filename>insane</filename></link> |
| class. |
| </para> |
| |
| <para> |
| The package feed area resides in the Build Directory. |
| The directory the build system uses to temporarily store packages |
| is determined by a combination of variables and the particular |
| package manager in use. |
| See the "Package Feeds" box in the illustration and note the |
| information to the right of that area. |
| In particular, the following defines where package files are |
| kept: |
| <itemizedlist> |
| <listitem><para><link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>: |
| Defined as <filename>tmp/deploy</filename> in the Build |
| Directory. |
| </para></listitem> |
| <listitem><para><filename>DEPLOY_DIR_*</filename>: |
| Depending on the package manager used, the package type |
| sub-folder. |
| Given RPM, IPK, or DEB packaging and tarball creation, the |
| <link linkend='var-DEPLOY_DIR_RPM'><filename>DEPLOY_DIR_RPM</filename></link>, |
| <link linkend='var-DEPLOY_DIR_IPK'><filename>DEPLOY_DIR_IPK</filename></link>, |
| <link linkend='var-DEPLOY_DIR_DEB'><filename>DEPLOY_DIR_DEB</filename></link>, |
| or |
| <link linkend='var-DEPLOY_DIR_TAR'><filename>DEPLOY_DIR_TAR</filename></link>, |
| variables are used, respectively. |
| </para></listitem> |
| <listitem><para><link linkend='var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></link>: |
| Defines architecture-specific sub-folders. |
| For example, packages could exist for the i586 or qemux86 |
| architectures. |
| </para></listitem> |
| </itemizedlist> |
| </para> |
| |
| <para> |
| BitBake uses the <filename>do_package_write_*</filename> tasks to |
| generate packages and place them into the package holding area (e.g. |
| <filename>do_package_write_ipk</filename> for IPK packages). |
| See the |
| "<link linkend='ref-tasks-package_write_deb'><filename>do_package_write_deb</filename></link>", |
| "<link linkend='ref-tasks-package_write_ipk'><filename>do_package_write_ipk</filename></link>", |
| "<link linkend='ref-tasks-package_write_rpm'><filename>do_package_write_rpm</filename></link>", |
| and |
| "<link linkend='ref-tasks-package_write_tar'><filename>do_package_write_tar</filename></link>" |
| sections for additional information. |
| As an example, consider a scenario where an IPK packaging manager |
| is being used and package architecture support for both i586 |
| and qemux86 exist. |
| Packages for the i586 architecture are placed in |
| <filename>build/tmp/deploy/ipk/i586</filename>, while packages for |
| the qemux86 architecture are placed in |
| <filename>build/tmp/deploy/ipk/qemux86</filename>. |
| </para> |
| </section> |
| |
| <section id='bitbake-dev-environment'> |
| <title>BitBake</title> |
| |
| <para> |
| The OpenEmbedded build system uses |
| <ulink url='&YOCTO_DOCS_DEV_URL;#bitbake-term'>BitBake</ulink> |
| to produce images. |
| You can see from the |
| <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>, |
| the BitBake area consists of several functional areas. |
| This section takes a closer look at each of those areas. |
| </para> |
| |
| <para> |
| Separate documentation exists for the BitBake tool. |
| See the |
| <ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual'>BitBake User Manual</ulink> |
| for reference material on BitBake. |
| </para> |
| |
| <section id='source-fetching-dev-environment'> |
| <title>Source Fetching</title> |
| |
| <para> |
| The first stages of building a recipe are to fetch and unpack |
| the source code: |
| <imagedata fileref="figures/source-fetching.png" align="center" width="6.5in" depth="5in" /> |
| </para> |
| |
| <para> |
| The |
| <link linkend='ref-tasks-fetch'><filename>do_fetch</filename></link> |
| and |
| <link linkend='ref-tasks-unpack'><filename>do_unpack</filename></link> |
| tasks fetch the source files and unpack them into the work |
| directory. |
| <note> |
| For every local file (e.g. <filename>file://</filename>) |
| that is part of a recipe's |
| <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link> |
| statement, the OpenEmbedded build system takes a checksum |
| of the file for the recipe and inserts the checksum into |
| the signature for the <filename>do_fetch</filename>. |
| If any local file has been modified, the |
| <filename>do_fetch</filename> task and all tasks that |
| depend on it are re-executed. |
| </note> |
| By default, everything is accomplished in the |
| <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>, |
| which has a defined structure. |
| For additional general information on the Build Directory, |
| see the |
| "<link linkend='structure-core-build'><filename>build/</filename></link>" |
| section. |
| </para> |
| |
| <para> |
| Unpacked source files are pointed to by the |
| <link linkend='var-S'><filename>S</filename></link> variable. |
| Each recipe has an area in the Build Directory where the |
| unpacked source code resides. |
| The name of that directory for any given recipe is defined from |
| several different variables. |
| You can see the variables that define these directories |
| by looking at the figure: |
| <itemizedlist> |
| <listitem><para><link linkend='var-TMPDIR'><filename>TMPDIR</filename></link> - |
| The base directory where the OpenEmbedded build system |
| performs all its work during the build. |
| </para></listitem> |
| <listitem><para><link linkend='var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></link> - |
| The architecture of the built package or packages. |
| </para></listitem> |
| <listitem><para><link linkend='var-TARGET_OS'><filename>TARGET_OS</filename></link> - |
| The operating system of the target device. |
| </para></listitem> |
| <listitem><para><link linkend='var-PN'><filename>PN</filename></link> - |
| The name of the built package. |
| </para></listitem> |
| <listitem><para><link linkend='var-PV'><filename>PV</filename></link> - |
| The version of the recipe used to build the package. |
| </para></listitem> |
| <listitem><para><link linkend='var-PR'><filename>PR</filename></link> - |
| The revision of the recipe used to build the package. |
| </para></listitem> |
| <listitem><para><link linkend='var-WORKDIR'><filename>WORKDIR</filename></link> - |
| The location within <filename>TMPDIR</filename> where |
| a specific package is built. |
| </para></listitem> |
| <listitem><para><link linkend='var-S'><filename>S</filename></link> - |
| Contains the unpacked source files for a given recipe. |
| </para></listitem> |
| </itemizedlist> |
| </para> |
| </section> |
| |
| <section id='patching-dev-environment'> |
| <title>Patching</title> |
| |
| <para> |
| Once source code is fetched and unpacked, BitBake locates |
| patch files and applies them to the source files: |
| <imagedata fileref="figures/patching.png" align="center" width="6in" depth="5in" /> |
| </para> |
| |
| <para> |
| The |
| <link linkend='ref-tasks-patch'><filename>do_patch</filename></link> |
| task processes recipes by |
| using the |
| <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link> |
| variable to locate applicable patch files, which by default |
| are <filename>*.patch</filename> or |
| <filename>*.diff</filename> files, or any file if |
| "apply=yes" is specified for the file in |
| <filename>SRC_URI</filename>. |
| </para> |
| |
| <para> |
| BitBake finds and applies multiple patches for a single recipe |
| in the order in which it finds the patches. |
| Patches are applied to the recipe's source files located in the |
| <link linkend='var-S'><filename>S</filename></link> directory. |
| </para> |
| |
| <para> |
| For more information on how the source directories are |
| created, see the |
| "<link linkend='source-fetching-dev-environment'>Source Fetching</link>" |
| section. |
| </para> |
| </section> |
| |
| <section id='configuration-and-compilation-dev-environment'> |
| <title>Configuration and Compilation</title> |
| |
| <para> |
| After source code is patched, BitBake executes tasks that |
| configure and compile the source code: |
| <imagedata fileref="figures/configuration-compile-autoreconf.png" align="center" width="7in" depth="5in" /> |
| </para> |
| |
| <para> |
| This step in the build process consists of three tasks: |
| <itemizedlist> |
| <listitem><para> |
| <emphasis><link linkend='ref-tasks-prepare_recipe_sysroot'><filename>do_prepare_recipe_sysroot</filename></link>:</emphasis> |
| This task sets up the two sysroots in |
| <filename>${</filename><link linkend='var-WORKDIR'><filename>WORKDIR</filename></link><filename>}</filename> |
| (i.e. <filename>recipe-sysroot</filename> and |
| <filename>recipe-sysroot-native</filename>) so that |
| the sysroots contain the contents of the |
| <link linkend='ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></link> |
| tasks of the recipes on which the recipe |
| containing the tasks depends. |
| A sysroot exists for both the target and for the native |
| binaries, which run on the host system. |
| </para></listitem> |
| <listitem><para><emphasis><filename>do_configure</filename>:</emphasis> |
| This task configures the source by enabling and |
| disabling any build-time and configuration options for |
| the software being built. |
| Configurations can come from the recipe itself as well |
| as from an inherited class. |
| Additionally, the software itself might configure itself |
| depending on the target for which it is being built. |
| </para> |
| |
| <para>The configurations handled by the |
| <link linkend='ref-tasks-configure'><filename>do_configure</filename></link> |
| task are specific |
| to source code configuration for the source code |
| being built by the recipe.</para> |
| |
| <para>If you are using the |
| <link linkend='ref-classes-autotools'><filename>autotools</filename></link> |
| class, |
| you can add additional configuration options by using |
| the <link linkend='var-EXTRA_OECONF'><filename>EXTRA_OECONF</filename></link> |
| or |
| <link linkend='var-PACKAGECONFIG_CONFARGS'><filename>PACKAGECONFIG_CONFARGS</filename></link> |
| variables. |
| For information on how this variable works within |
| that class, see the |
| <filename>meta/classes/autotools.bbclass</filename> file. |
| </para></listitem> |
| <listitem><para><emphasis><filename>do_compile</filename>:</emphasis> |
| Once a configuration task has been satisfied, BitBake |
| compiles the source using the |
| <link linkend='ref-tasks-compile'><filename>do_compile</filename></link> |
| task. |
| Compilation occurs in the directory pointed to by the |
| <link linkend='var-B'><filename>B</filename></link> |
| variable. |
| Realize that the <filename>B</filename> directory is, by |
| default, the same as the |
| <link linkend='var-S'><filename>S</filename></link> |
| directory.</para></listitem> |
| <listitem><para><emphasis><filename>do_install</filename>:</emphasis> |
| Once compilation is done, BitBake executes the |
| <link linkend='ref-tasks-install'><filename>do_install</filename></link> |
| task. |
| This task copies files from the <filename>B</filename> |
| directory and places them in a holding area pointed to |
| by the |
| <link linkend='var-D'><filename>D</filename></link> |
| variable.</para></listitem> |
| </itemizedlist> |
| </para> |
| </section> |
| |
| <section id='package-splitting-dev-environment'> |
| <title>Package Splitting</title> |
| |
| <para> |
| After source code is configured and compiled, the |
| OpenEmbedded build system analyzes |
| the results and splits the output into packages: |
| <imagedata fileref="figures/analysis-for-package-splitting.png" align="center" width="7in" depth="7in" /> |
| </para> |
| |
| <para> |
| The |
| <link linkend='ref-tasks-package'><filename>do_package</filename></link> |
| and |
| <link linkend='ref-tasks-packagedata'><filename>do_packagedata</filename></link> |
| tasks combine to analyze |
| the files found in the |
| <link linkend='var-D'><filename>D</filename></link> directory |
| and split them into subsets based on available packages and |
| files. |
| The analyzing process involves the following as well as other |
| items: splitting out debugging symbols, |
| looking at shared library dependencies between packages, |
| and looking at package relationships. |
| The <filename>do_packagedata</filename> task creates package |
| metadata based on the analysis such that the |
| OpenEmbedded build system can generate the final packages. |
| Working, staged, and intermediate results of the analysis |
| and package splitting process use these areas: |
| <itemizedlist> |
| <listitem><para><link linkend='var-PKGD'><filename>PKGD</filename></link> - |
| The destination directory for packages before they are |
| split. |
| </para></listitem> |
| <listitem><para><link linkend='var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></link> - |
| A shared, global-state directory that holds data |
| generated during the packaging process. |
| </para></listitem> |
| <listitem><para><link linkend='var-PKGDESTWORK'><filename>PKGDESTWORK</filename></link> - |
| A temporary work area used by the |
| <filename>do_package</filename> task. |
| </para></listitem> |
| <listitem><para><link linkend='var-PKGDEST'><filename>PKGDEST</filename></link> - |
| The parent directory for packages after they have |
| been split. |
| </para></listitem> |
| </itemizedlist> |
| The <link linkend='var-FILES'><filename>FILES</filename></link> |
| variable defines the files that go into each package in |
| <link linkend='var-PACKAGES'><filename>PACKAGES</filename></link>. |
| If you want details on how this is accomplished, you can |
| look at the |
| <link linkend='ref-classes-package'><filename>package</filename></link> |
| class. |
| </para> |
| |
| <para> |
| Depending on the type of packages being created (RPM, DEB, or |
| IPK), the <filename>do_package_write_*</filename> task |
| creates the actual packages and places them in the |
| Package Feed area, which is |
| <filename>${TMPDIR}/deploy</filename>. |
| You can see the |
| "<link linkend='package-feeds-dev-environment'>Package Feeds</link>" |
| section for more detail on that part of the build process. |
| <note> |
| Support for creating feeds directly from the |
| <filename>deploy/*</filename> directories does not exist. |
| Creating such feeds usually requires some kind of feed |
| maintenance mechanism that would upload the new packages |
| into an official package feed (e.g. the |
| Ångström distribution). |
| This functionality is highly distribution-specific |
| and thus is not provided out of the box. |
| </note> |
| </para> |
| </section> |
| |
| <section id='image-generation-dev-environment'> |
| <title>Image Generation</title> |
| |
| <para> |
| Once packages are split and stored in the Package Feeds area, |
| the OpenEmbedded build system uses BitBake to generate the |
| root filesystem image: |
| <imagedata fileref="figures/image-generation.png" align="center" width="6in" depth="7in" /> |
| </para> |
| |
| <para> |
| The image generation process consists of several stages and |
| depends on several tasks and variables. |
| The |
| <link linkend='ref-tasks-rootfs'><filename>do_rootfs</filename></link> |
| task creates the root filesystem (file and directory structure) |
| for an image. |
| This task uses several key variables to help create the list |
| of packages to actually install: |
| <itemizedlist> |
| <listitem><para><link linkend='var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></link>: |
| Lists out the base set of packages to install from |
| the Package Feeds area.</para></listitem> |
| <listitem><para><link linkend='var-PACKAGE_EXCLUDE'><filename>PACKAGE_EXCLUDE</filename></link>: |
| Specifies packages that should not be installed. |
| </para></listitem> |
| <listitem><para><link linkend='var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></link>: |
| Specifies features to include in the image. |
| Most of these features map to additional packages for |
| installation.</para></listitem> |
| <listitem><para><link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link>: |
| Specifies the package backend to use and consequently |
| helps determine where to locate packages within the |
| Package Feeds area.</para></listitem> |
| <listitem><para><link linkend='var-IMAGE_LINGUAS'><filename>IMAGE_LINGUAS</filename></link>: |
| Determines the language(s) for which additional |
| language support packages are installed. |
| </para></listitem> |
| <listitem><para><link linkend='var-PACKAGE_INSTALL'><filename>PACKAGE_INSTALL</filename></link>: |
| The final list of packages passed to the package manager |
| for installation into the image. |
| </para></listitem> |
| </itemizedlist> |
| </para> |
| |
| <para> |
| With |
| <link linkend='var-IMAGE_ROOTFS'><filename>IMAGE_ROOTFS</filename></link> |
| pointing to the location of the filesystem under construction and |
| the <filename>PACKAGE_INSTALL</filename> variable providing the |
| final list of packages to install, the root file system is |
| created. |
| </para> |
| |
| <para> |
| Package installation is under control of the package manager |
| (e.g. dnf/rpm, opkg, or apt/dpkg) regardless of whether or |
| not package management is enabled for the target. |
| At the end of the process, if package management is not |
| enabled for the target, the package manager's data files |
| are deleted from the root filesystem. |
| As part of the final stage of package installation, postinstall |
| scripts that are part of the packages are run. |
| Any scripts that fail to run |
| on the build host are run on the target when the target system |
| is first booted. |
| If you are using a |
| <ulink url='&YOCTO_DOCS_DEV_URL;#creating-a-read-only-root-filesystem'>read-only root filesystem</ulink>, |
| all the post installation scripts must succeed during the |
| package installation phase since the root filesystem is |
| read-only. |
| </para> |
| |
| <para> |
| The final stages of the <filename>do_rootfs</filename> task |
| handle post processing. |
| Post processing includes creation of a manifest file and |
| optimizations. |
| </para> |
| |
| <para> |
| The manifest file (<filename>.manifest</filename>) resides |
| in the same directory as the root filesystem image. |
| This file lists out, line-by-line, the installed packages. |
| The manifest file is useful for the |
| <link linkend='ref-classes-testimage*'><filename>testimage</filename></link> |
| class, for example, to determine whether or not to run |
| specific tests. |
| See the |
| <link linkend='var-IMAGE_MANIFEST'><filename>IMAGE_MANIFEST</filename></link> |
| variable for additional information. |
| </para> |
| |
| <para> |
| Optimizing processes run across the image include |
| <filename>mklibs</filename>, <filename>prelink</filename>, |
| and any other post-processing commands as defined by the |
| <link linkend='var-ROOTFS_POSTPROCESS_COMMAND'><filename>ROOTFS_POSTPROCESS_COMMAND</filename></link> |
| variable. |
| The <filename>mklibs</filename> process optimizes the size |
| of the libraries, while the |
| <filename>prelink</filename> process optimizes the dynamic |
| linking of shared libraries to reduce start up time of |
| executables. |
| </para> |
| |
| <para> |
| After the root filesystem is built, processing begins on |
| the image through the <filename>do_image</filename> task. |
| The build system runs any pre-processing commands as defined |
| by the |
| <link linkend='var-IMAGE_PREPROCESS_COMMAND'><filename>IMAGE_PREPROCESS_COMMAND</filename></link> |
| variable. |
| This variable specifies a list of functions to call before |
| the OpenEmbedded build system creates the final image output |
| files. |
| </para> |
| |
| <para> |
| The <filename>do_image</filename> task dynamically creates |
| other <filename>do_image_*</filename> tasks as needed, which |
| include compressing the root filesystem image to reduce the |
| overall size of the image. |
| The process turns everything into an image file or a set of |
| image files. |
| The formats used for the root filesystem depend on the |
| <link linkend='var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></link> |
| variable. |
| </para> |
| |
| <para> |
| The final task involved in image creation is the |
| <filename>do_image_complete</filename> task. |
| This task completes the image by applying any image |
| post processing as defined through the |
| <link linkend='var-IMAGE_POSTPROCESS_COMMAND'><filename>IMAGE_POSTPROCESS_COMMAND</filename></link> |
| variable. |
| The variable specifies a list of functions to call once the |
| OpenEmbedded build system has created the final image output |
| files. |
| </para> |
| |
| <note> |
| The entire image generation process is run under Pseudo. |
| Running under Pseudo ensures that the files in the root |
| filesystem have correct ownership. |
| </note> |
| </section> |
| |
| <section id='sdk-generation-dev-environment'> |
| <title>SDK Generation</title> |
| |
| <para> |
| The OpenEmbedded build system uses BitBake to generate the |
| Software Development Kit (SDK) installer script for both the |
| standard and extensible SDKs: |
| <imagedata fileref="figures/sdk-generation.png" align="center" /> |
| </para> |
| |
| <note> |
| For more information on the cross-development toolchain |
| generation, see the |
| "<link linkend='cross-development-toolchain-generation'>Cross-Development Toolchain Generation</link>" |
| section. |
| For information on advantages gained when building a |
| cross-development toolchain using the |
| <link linkend='ref-tasks-populate_sdk'><filename>do_populate_sdk</filename></link> |
| task, see the |
| "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-building-an-sdk-installer'>Building an SDK Installer</ulink>" |
| section in the Yocto Project Software Development Kit (SDK) |
| Developer's Guide. |
| </note> |
| |
| <para> |
| Like image generation, the SDK script process consists of |
| several stages and depends on many variables. |
| The <filename>do_populate_sdk</filename> and |
| <filename>do_populate_sdk_ext</filename> tasks use these |
| key variables to help create the list of packages to actually |
| install. |
| For information on the variables listed in the figure, see the |
| "<link linkend='sdk-dev-environment'>Application Development SDK</link>" |
| section. |
| </para> |
| |
| <para> |
| The <filename>do_populate_sdk</filename> task helps create |
| the standard SDK and handles two parts: a target part and a |
| host part. |
| The target part is the part built for the target hardware and |
| includes libraries and headers. |
| The host part is the part of the SDK that runs on the |
| <link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>. |
| </para> |
| |
| <para> |
| The <filename>do_populate_sdk_ext</filename> task helps create |
| the extensible SDK and handles host and target parts |
| differently than its counter part does for the standard SDK. |
| For the extensible SDK, the task encapsulates the build system, |
| which includes everything needed (host and target) for the SDK. |
| </para> |
| |
| <para> |
| Regardless of the type of SDK being constructed, the |
| tasks perform some cleanup after which a cross-development |
| environment setup script and any needed configuration files |
| are created. |
| The final output is the Cross-development |
| toolchain installation script (<filename>.sh</filename> file), |
| which includes the environment setup script. |
| </para> |
| </section> |
| |
| <section id='stamp-files-and-the-rerunning-of-tasks'> |
| <title>Stamp Files and the Rerunning of Tasks</title> |
| |
| <para> |
| For each task that completes successfully, BitBake writes a |
| stamp file into the |
| <link linkend='var-STAMPS_DIR'><filename>STAMPS_DIR</filename></link> |
| directory. |
| The beginning of the stamp file's filename is determined by the |
| <link linkend='var-STAMP'><filename>STAMP</filename></link> |
| variable, and the end of the name consists of the task's name |
| and current |
| <ulink url='&YOCTO_DOCS_BB_URL;#checksums'>input checksum</ulink>. |
| <note> |
| This naming scheme assumes that |
| <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_SIGNATURE_HANDLER'><filename>BB_SIGNATURE_HANDLER</filename></ulink> |
| is "OEBasicHash", which is almost always the case in |
| current OpenEmbedded. |
| </note> |
| To determine if a task needs to be rerun, BitBake checks if a |
| stamp file with a matching input checksum exists for the task. |
| If such a stamp file exists, the task's output is assumed to |
| exist and still be valid. |
| If the file does not exist, the task is rerun. |
| <note> |
| <para>The stamp mechanism is more general than the shared |
| state (sstate) cache mechanism described in the |
| "<link linkend='setscene-tasks-and-shared-state'>Setscene Tasks and Shared State</link>" |
| section. |
| BitBake avoids rerunning any task that has a valid |
| stamp file, not just tasks that can be accelerated through |
| the sstate cache.</para> |
| <para>However, you should realize that stamp files only |
| serve as a marker that some work has been done and that |
| these files do not record task output. |
| The actual task output would usually be somewhere in |
| <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link> |
| (e.g. in some recipe's |
| <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>.) |
| What the sstate cache mechanism adds is a way to cache task |
| output that can then be shared between build machines. |
| </para> |
| </note> |
| Since <filename>STAMPS_DIR</filename> is usually a subdirectory |
| of <filename>TMPDIR</filename>, removing |
| <filename>TMPDIR</filename> will also remove |
| <filename>STAMPS_DIR</filename>, which means tasks will |
| properly be rerun to repopulate <filename>TMPDIR</filename>. |
| </para> |
| |
| <para> |
| If you want some task to always be considered "out of date", |
| you can mark it with the |
| <ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'><filename>nostamp</filename></ulink> |
| varflag. |
| If some other task depends on such a task, then that task will |
| also always be considered out of date, which might not be what |
| you want. |
| </para> |
| |
| <para> |
| For details on how to view information about a task's |
| signature, see the |
| "<link linkend='usingpoky-viewing-task-variable-dependencies'>Viewing Task Variable Dependencies</link>" |
| section. |
| </para> |
| </section> |
| |
| <section id='setscene-tasks-and-shared-state'> |
| <title>Setscene Tasks and Shared State</title> |
| |
| <para> |
| The description of tasks so far assumes that BitBake needs to |
| build everything and there are no prebuilt objects available. |
| BitBake does support skipping tasks if prebuilt objects are |
| available. |
| These objects are usually made available in the form of a |
| shared state (sstate) cache. |
| <note> |
| For information on variables affecting sstate, see the |
| <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> |
| and |
| <link linkend='var-SSTATE_MIRRORS'><filename>SSTATE_MIRRORS</filename></link> |
| variables. |
| </note> |
| </para> |
| |
| <para> |
| The idea of a setscene task (i.e |
| <filename>do_</filename><replaceable>taskname</replaceable><filename>_setscene</filename>) |
| is a version of the task where |
| instead of building something, BitBake can skip to the end |
| result and simply place a set of files into specific locations |
| as needed. |
| In some cases, it makes sense to have a setscene task variant |
| (e.g. generating package files in the |
| <filename>do_package_write_*</filename> task). |
| In other cases, it does not make sense, (e.g. a |
| <link linkend='ref-tasks-patch'><filename>do_patch</filename></link> |
| task or |
| <link linkend='ref-tasks-unpack'><filename>do_unpack</filename></link> |
| task) since the work involved would be equal to or greater than |
| the underlying task. |
| </para> |
| |
| <para> |
| In the OpenEmbedded build system, the common tasks that have |
| setscene variants are <link linkend='ref-tasks-package'><filename>do_package</filename></link>, |
| <filename>do_package_write_*</filename>, |
| <link linkend='ref-tasks-deploy'><filename>do_deploy</filename></link>, |
| <link linkend='ref-tasks-packagedata'><filename>do_packagedata</filename></link>, |
| and |
| <link linkend='ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></link>. |
| Notice that these are most of the tasks whose output is an |
| end result. |
| </para> |
| |
| <para> |
| The OpenEmbedded build system has knowledge of the relationship |
| between these tasks and other tasks that precede them. |
| For example, if BitBake runs |
| <filename>do_populate_sysroot_setscene</filename> for |
| something, there is little point in running any of the |
| <filename>do_fetch</filename>, <filename>do_unpack</filename>, |
| <filename>do_patch</filename>, |
| <filename>do_configure</filename>, |
| <filename>do_compile</filename>, and |
| <filename>do_install</filename> tasks. |
| However, if <filename>do_package</filename> needs to be run, |
| BitBake would need to run those other tasks. |
| </para> |
| |
| <para> |
| It becomes more complicated if everything can come from an |
| sstate cache because some objects are simply not required at |
| all. |
| For example, you do not need a compiler or native tools, such |
| as quilt, if there is nothing to compile or patch. |
| If the <filename>do_package_write_*</filename> packages are |
| available from sstate, BitBake does not need the |
| <filename>do_package</filename> task data. |
| </para> |
| |
| <para> |
| To handle all these complexities, BitBake runs in two phases. |
| The first is the "setscene" stage. |
| During this stage, BitBake first checks the sstate cache for |
| any targets it is planning to build. |
| BitBake does a fast check to see if the object exists rather |
| than a complete download. |
| If nothing exists, the second phase, which is the setscene |
| stage, completes and the main build proceeds. |
| </para> |
| |
| <para> |
| If objects are found in the sstate cache, the OpenEmbedded |
| build system works backwards from the end targets specified |
| by the user. |
| For example, if an image is being built, the OpenEmbedded build |
| system first looks for the packages needed for that image and |
| the tools needed to construct an image. |
| If those are available, the compiler is not needed. |
| Thus, the compiler is not even downloaded. |
| If something was found to be unavailable, or the download or |
| setscene task fails, the OpenEmbedded build system then tries |
| to install dependencies, such as the compiler, from the cache. |
| </para> |
| |
| <para> |
| The availability of objects in the sstate cache is handled by |
| the function specified by the |
| <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_HASHCHECK_FUNCTION'><filename>BB_HASHCHECK_FUNCTION</filename></ulink> |
| variable and returns a list of the objects that are available. |
| The function specified by the |
| <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_SETSCENE_DEPVALID'><filename>BB_SETSCENE_DEPVALID</filename></ulink> |
| variable is the function that determines whether a given |
| dependency needs to be followed, and whether for any given |
| relationship the function needs to be passed. |
| The function returns a True or False value. |
| </para> |
| </section> |
| </section> |
| |
| <section id='images-dev-environment'> |
| <title>Images</title> |
| |
| <para> |
| The images produced by the OpenEmbedded build system |
| are compressed forms of the |
| root filesystem that are ready to boot on a target device. |
| You can see from the |
| <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link> |
| that BitBake output, in part, consists of images. |
| This section is going to look more closely at this output: |
| <imagedata fileref="figures/images.png" align="center" width="5.5in" depth="5.5in" /> |
| </para> |
| |
| <para> |
| For a list of example images that the Yocto Project provides, |
| see the |
| "<link linkend='ref-images'>Images</link>" chapter. |
| </para> |
| |
| <para> |
| Images are written out to the |
| <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink> |
| inside the <filename>tmp/deploy/images/<replaceable>machine</replaceable>/</filename> |
| folder as shown in the figure. |
| This folder contains any files expected to be loaded on the |
| target device. |
| The |
| <link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link> |
| variable points to the <filename>deploy</filename> directory, |
| while the |
| <link linkend='var-DEPLOY_DIR_IMAGE'><filename>DEPLOY_DIR_IMAGE</filename></link> |
| variable points to the appropriate directory containing images for |
| the current configuration. |
| <itemizedlist> |
| <listitem><para><filename><replaceable>kernel-image</replaceable></filename>: |
| A kernel binary file. |
| The <link linkend='var-KERNEL_IMAGETYPE'><filename>KERNEL_IMAGETYPE</filename></link> |
| variable setting determines the naming scheme for the |
| kernel image file. |
| Depending on that variable, the file could begin with |
| a variety of naming strings. |
| The <filename>deploy/images/<replaceable>machine</replaceable></filename> |
| directory can contain multiple image files for the |
| machine.</para></listitem> |
| <listitem><para><filename><replaceable>root-filesystem-image</replaceable></filename>: |
| Root filesystems for the target device (e.g. |
| <filename>*.ext3</filename> or <filename>*.bz2</filename> |
| files). |
| The <link linkend='var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></link> |
| variable setting determines the root filesystem image |
| type. |
| The <filename>deploy/images/<replaceable>machine</replaceable></filename> |
| directory can contain multiple root filesystems for the |
| machine.</para></listitem> |
| <listitem><para><filename><replaceable>kernel-modules</replaceable></filename>: |
| Tarballs that contain all the modules built for the kernel. |
| Kernel module tarballs exist for legacy purposes and |
| can be suppressed by setting the |
| <link linkend='var-MODULE_TARBALL_DEPLOY'><filename>MODULE_TARBALL_DEPLOY</filename></link> |
| variable to "0". |
| The <filename>deploy/images/<replaceable>machine</replaceable></filename> |
| directory can contain multiple kernel module tarballs |
| for the machine.</para></listitem> |
| <listitem><para><filename><replaceable>bootloaders</replaceable></filename>: |
| Bootloaders supporting the image, if applicable to the |
| target machine. |
| The <filename>deploy/images/<replaceable>machine</replaceable></filename> |
| directory can contain multiple bootloaders for the |
| machine.</para></listitem> |
| <listitem><para><filename><replaceable>symlinks</replaceable></filename>: |
| The <filename>deploy/images/<replaceable>machine</replaceable></filename> |
| folder contains |
| a symbolic link that points to the most recently built file |
| for each machine. |
| These links might be useful for external scripts that |
| need to obtain the latest version of each file. |
| </para></listitem> |
| </itemizedlist> |
| </para> |
| </section> |
| |
| <section id='sdk-dev-environment'> |
| <title>Application Development SDK</title> |
| |
| <para> |
| In the |
| <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>, |
| the output labeled "Application Development SDK" represents an |
| SDK. |
| The SDK generation process differs depending on whether you build |
| a standard SDK |
| (e.g. <filename>bitbake -c populate_sdk</filename> <replaceable>imagename</replaceable>) |
| or an extensible SDK |
| (e.g. <filename>bitbake -c populate_sdk_ext</filename> <replaceable>imagename</replaceable>). |
| This section is going to take a closer look at this output: |
| <imagedata fileref="figures/sdk.png" align="center" width="9in" depth="7.25in" /> |
| </para> |
| |
| <para> |
| The specific form of this output is a self-extracting |
| SDK installer (<filename>*.sh</filename>) that, when run, |
| installs the SDK, which consists of a cross-development |
| toolchain, a set of libraries and headers, and an SDK |
| environment setup script. |
| Running this installer essentially sets up your |
| cross-development environment. |
| You can think of the cross-toolchain as the "host" |
| part because it runs on the SDK machine. |
| You can think of the libraries and headers as the "target" |
| part because they are built for the target hardware. |
| The environment setup script is added so that you can initialize |
| the environment before using the tools. |
| </para> |
| |
| <note><title>Notes</title> |
| <itemizedlist> |
| <listitem><para> |
| The Yocto Project supports several methods by which you can |
| set up this cross-development environment. |
| These methods include downloading pre-built SDK installers |
| or building and installing your own SDK installer. |
| </para></listitem> |
| <listitem><para> |
| For background information on cross-development toolchains |
| in the Yocto Project development environment, see the |
| "<link linkend='cross-development-toolchain-generation'>Cross-Development Toolchain Generation</link>" |
| section. |
| </para></listitem> |
| <listitem><para> |
| For information on setting up a cross-development |
| environment, see the |
| <ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>. |
| </para></listitem> |
| </itemizedlist> |
| </note> |
| |
| <para> |
| Once built, the SDK installers are written out to the |
| <filename>deploy/sdk</filename> folder inside the |
| <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink> |
| as shown in the figure at the beginning of this section. |
| Depending on the type of SDK, several variables exist that help |
| configure these files. |
| The following list shows the variables associated with a standard |
| SDK: |
| <itemizedlist> |
| <listitem><para><link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>: |
| Points to the <filename>deploy</filename> |
| directory.</para></listitem> |
| <listitem><para><link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>: |
| Specifies the architecture of the machine |
| on which the cross-development tools are run to |
| create packages for the target hardware. |
| </para></listitem> |
| <listitem><para><link linkend='var-SDKIMAGE_FEATURES'><filename>SDKIMAGE_FEATURES</filename></link>: |
| Lists the features to include in the "target" part |
| of the SDK. |
| </para></listitem> |
| <listitem><para><link linkend='var-TOOLCHAIN_HOST_TASK'><filename>TOOLCHAIN_HOST_TASK</filename></link>: |
| Lists packages that make up the host |
| part of the SDK (i.e. the part that runs on |
| the <filename>SDKMACHINE</filename>). |
| When you use |
| <filename>bitbake -c populate_sdk <replaceable>imagename</replaceable></filename> |
| to create the SDK, a set of default packages |
| apply. |
| This variable allows you to add more packages. |
| </para></listitem> |
| <listitem><para><link linkend='var-TOOLCHAIN_TARGET_TASK'><filename>TOOLCHAIN_TARGET_TASK</filename></link>: |
| Lists packages that make up the target part |
| of the SDK (i.e. the part built for the |
| target hardware). |
| </para></listitem> |
| <listitem><para><link linkend='var-SDKPATH'><filename>SDKPATH</filename></link>: |
| Defines the default SDK installation path offered by the |
| installation script. |
| </para></listitem> |
| </itemizedlist> |
| This next list, shows the variables associated with an extensible |
| SDK: |
| <itemizedlist> |
| <listitem><para><link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>: |
| Points to the <filename>deploy</filename> directory. |
| </para></listitem> |
| <listitem><para><link linkend='var-SDK_EXT_TYPE'><filename>SDK_EXT_TYPE</filename></link>: |
| Controls whether or not shared state artifacts are copied |
| into the extensible SDK. |
| By default, all required shared state artifacts are copied |
| into the SDK. |
| </para></listitem> |
| <listitem><para><link linkend='var-SDK_INCLUDE_PKGDATA'><filename>SDK_INCLUDE_PKGDATA</filename></link>: |
| Specifies whether or not packagedata will be included in |
| the extensible SDK for all recipes in the "world" target. |
| </para></listitem> |
| <listitem><para><link linkend='var-SDK_INCLUDE_TOOLCHAIN'><filename>SDK_INCLUDE_TOOLCHAIN</filename></link>: |
| Specifies whether or not the toolchain will be included |
| when building the extensible SDK. |
| </para></listitem> |
| <listitem><para><link linkend='var-SDK_LOCAL_CONF_WHITELIST'><filename>SDK_LOCAL_CONF_WHITELIST</filename></link>: |
| A list of variables allowed through from the build system |
| configuration into the extensible SDK configuration. |
| </para></listitem> |
| <listitem><para><link linkend='var-SDK_LOCAL_CONF_BLACKLIST'><filename>SDK_LOCAL_CONF_BLACKLIST</filename></link>: |
| A list of variables not allowed through from the build |
| system configuration into the extensible SDK configuration. |
| </para></listitem> |
| <listitem><para><link linkend='var-SDK_INHERIT_BLACKLIST'><filename>SDK_INHERIT_BLACKLIST</filename></link>: |
| A list of classes to remove from the |
| <link linkend='var-INHERIT'><filename>INHERIT</filename></link> |
| value globally within the extensible SDK configuration. |
| </para></listitem> |
| </itemizedlist> |
| </para> |
| </section> |
| |
| </chapter> |
| <!-- |
| vim: expandtab tw=80 ts=4 |
| --> |