| <!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" |
| "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" |
| [<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > |
| |
| <chapter id='technical-details'> |
| <title>Technical Details</title> |
| |
| <para> |
| This chapter provides technical details for various parts of the |
| Yocto Project. |
| Currently, topics include Yocto Project components, |
| cross-toolchain generation, shared state (sstate) cache, |
| x32, Wayland support, and Licenses. |
| </para> |
| |
| <section id='usingpoky-components'> |
| <title>Yocto Project Components</title> |
| |
| <para> |
| The |
| <ulink url='&YOCTO_DOCS_DEV_URL;#bitbake-term'>BitBake</ulink> |
| task executor together with various types of configuration files form |
| the OpenEmbedded Core. |
| This section overviews these components by describing their use and |
| how they interact. |
| </para> |
| |
| <para> |
| BitBake handles the parsing and execution of the data files. |
| The data itself is of various types: |
| <itemizedlist> |
| <listitem><para><emphasis>Recipes:</emphasis> Provides details |
| about particular pieces of software. |
| </para></listitem> |
| <listitem><para><emphasis>Class Data:</emphasis> Abstracts |
| common build information (e.g. how to build a Linux kernel). |
| </para></listitem> |
| <listitem><para><emphasis>Configuration Data:</emphasis> Defines |
| machine-specific settings, policy decisions, and so forth. |
| Configuration data acts as the glue to bind everything |
| together. |
| </para></listitem> |
| </itemizedlist> |
| </para> |
| |
| <para> |
| BitBake knows how to combine multiple data sources together and refers |
| to each data source as a layer. |
| For information on layers, see the |
| "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and |
| Creating Layers</ulink>" section of the Yocto Project Development Manual. |
| </para> |
| |
| <para> |
| Following are some brief details on these core components. |
| For additional information on how these components interact during |
| a build, see the |
| "<link linkend='closer-look'>A Closer Look at the Yocto Project Development Environment</link>" |
| Chapter. |
| </para> |
| |
| <section id='usingpoky-components-bitbake'> |
| <title>BitBake</title> |
| |
| <para> |
| BitBake is the tool at the heart of the OpenEmbedded build system |
| and is responsible for parsing the |
| <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>, |
| generating a list of tasks from it, and then executing those tasks. |
| </para> |
| |
| <para> |
| This section briefly introduces BitBake. |
| If you want more information on BitBake, see the |
| <ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual'>BitBake User Manual</ulink>. |
| </para> |
| |
| <para> |
| To see a list of the options BitBake supports, use either of |
| the following commands: |
| <literallayout class='monospaced'> |
| $ bitbake -h |
| $ bitbake --help |
| </literallayout> |
| </para> |
| |
| <para> |
| The most common usage for BitBake is <filename>bitbake <replaceable>packagename</replaceable></filename>, where |
| <filename>packagename</filename> is the name of the package you want to build |
| (referred to as the "target" in this manual). |
| The target often equates to the first part of a recipe's filename |
| (e.g. "foo" for a recipe named |
| <filename>foo_1.3.0-r0.bb</filename>). |
| So, to process the <filename>matchbox-desktop_1.2.3.bb</filename> recipe file, you |
| might type the following: |
| <literallayout class='monospaced'> |
| $ bitbake matchbox-desktop |
| </literallayout> |
| Several different versions of <filename>matchbox-desktop</filename> might exist. |
| BitBake chooses the one selected by the distribution configuration. |
| You can get more details about how BitBake chooses between different |
| target versions and providers in the |
| "<ulink url='&YOCTO_DOCS_BB_URL;#bb-bitbake-preferences'>Preferences</ulink>" |
| section of the BitBake User Manual. |
| </para> |
| |
| <para> |
| BitBake also tries to execute any dependent tasks first. |
| So for example, before building <filename>matchbox-desktop</filename>, BitBake |
| would build a cross compiler and <filename>glibc</filename> if they had not already |
| been built. |
| </para> |
| |
| <para> |
| A useful BitBake option to consider is the <filename>-k</filename> or |
| <filename>--continue</filename> option. |
| This option instructs BitBake to try and continue processing the job |
| as long as possible even after encountering an error. |
| When an error occurs, the target that |
| failed and those that depend on it cannot be remade. |
| However, when you use this option other dependencies can still be |
| processed. |
| </para> |
| </section> |
| |
| <section id='usingpoky-components-metadata'> |
| <title>Metadata (Recipes)</title> |
| |
| <para> |
| Files that have the <filename>.bb</filename> suffix are "recipes" |
| files. |
| In general, a recipe contains information about a single piece of |
| software. |
| This information includes the location from which to download the |
| unaltered source, any source patches to be applied to that source |
| (if needed), which special configuration options to apply, |
| how to compile the source files, and how to package the compiled |
| output. |
| </para> |
| |
| <para> |
| The term "package" is sometimes used to refer to recipes. However, |
| since the word "package" is used for the packaged output from the OpenEmbedded |
| build system (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files), |
| this document avoids using the term "package" when referring to recipes. |
| </para> |
| </section> |
| |
| <section id='usingpoky-components-classes'> |
| <title>Classes</title> |
| |
| <para> |
| Class files (<filename>.bbclass</filename>) contain information that |
| is useful to share between |
| <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink> files. |
| An example is the |
| <link linkend='ref-classes-autotools'><filename>autotools</filename></link> |
| class, which contains common settings for any application that |
| Autotools uses. |
| The "<link linkend='ref-classes'>Classes</link>" chapter provides |
| details about classes and how to use them. |
| </para> |
| </section> |
| |
| <section id='usingpoky-components-configuration'> |
| <title>Configuration</title> |
| |
| <para> |
| The configuration files (<filename>.conf</filename>) define various configuration variables |
| that govern the OpenEmbedded build process. |
| These files fall into several areas that define machine configuration options, |
| distribution configuration options, compiler tuning options, general common configuration |
| options, and user configuration options in <filename>local.conf</filename>, which is found |
| in the |
| <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>. |
| </para> |
| </section> |
| </section> |
| |
| <section id="cross-development-toolchain-generation"> |
| <title>Cross-Development Toolchain Generation</title> |
| |
| <para> |
| The Yocto Project does most of the work for you when it comes to |
| creating |
| <ulink url='&YOCTO_DOCS_DEV_URL;#cross-development-toolchain'>cross-development toolchains</ulink>. |
| This section provides some technical background on how |
| cross-development toolchains are created and used. |
| For more information on toolchains, you can also see the |
| <ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>. |
| </para> |
| |
| <para> |
| In the Yocto Project development environment, cross-development |
| toolchains are used to build the image and applications that run on the |
| target hardware. |
| With just a few commands, the OpenEmbedded build system creates |
| these necessary toolchains for you. |
| </para> |
| |
| <para> |
| The following figure shows a high-level build environment regarding |
| toolchain construction and use. |
| </para> |
| |
| <para> |
| <imagedata fileref="figures/cross-development-toolchains.png" width="8in" depth="6in" align="center" /> |
| </para> |
| |
| <para> |
| Most of the work occurs on the Build Host. |
| This is the machine used to build images and generally work within the |
| the Yocto Project environment. |
| When you run BitBake to create an image, the OpenEmbedded build system |
| uses the host <filename>gcc</filename> compiler to bootstrap a |
| cross-compiler named <filename>gcc-cross</filename>. |
| The <filename>gcc-cross</filename> compiler is what BitBake uses to |
| compile source files when creating the target image. |
| You can think of <filename>gcc-cross</filename> simply as an |
| automatically generated cross-compiler that is used internally within |
| BitBake only. |
| <note> |
| The extensible SDK does not use |
| <filename>gcc-cross-canadian</filename> since this SDK |
| ships a copy of the OpenEmbedded build system and the sysroot |
| within it contains <filename>gcc-cross</filename>. |
| </note> |
| </para> |
| |
| <para> |
| The chain of events that occurs when <filename>gcc-cross</filename> is |
| bootstrapped is as follows: |
| <literallayout class='monospaced'> |
| gcc -> binutils-cross -> gcc-cross-initial -> linux-libc-headers -> glibc-initial -> glibc -> gcc-cross -> gcc-runtime |
| </literallayout> |
| <itemizedlist> |
| <listitem><para><filename>gcc</filename>: |
| The build host's GNU Compiler Collection (GCC). |
| </para></listitem> |
| <listitem><para><filename>binutils-cross</filename>: |
| The bare minimum binary utilities needed in order to run |
| the <filename>gcc-cross-initial</filename> phase of the |
| bootstrap operation. |
| </para></listitem> |
| <listitem><para><filename>gcc-cross-initial</filename>: |
| An early stage of the bootstrap process for creating |
| the cross-compiler. |
| This stage builds enough of the <filename>gcc-cross</filename>, |
| the C library, and other pieces needed to finish building the |
| final cross-compiler in later stages. |
| This tool is a "native" package (i.e. it is designed to run on |
| the build host). |
| </para></listitem> |
| <listitem><para><filename>linux-libc-headers</filename>: |
| Headers needed for the cross-compiler. |
| </para></listitem> |
| <listitem><para><filename>glibc-initial</filename>: |
| An initial version of the Embedded GLIBC needed to bootstrap |
| <filename>glibc</filename>. |
| </para></listitem> |
| <listitem><para><filename>gcc-cross</filename>: |
| The final stage of the bootstrap process for the |
| cross-compiler. |
| This stage results in the actual cross-compiler that |
| BitBake uses when it builds an image for a targeted |
| device. |
| <note> |
| If you are replacing this cross compiler toolchain |
| with a custom version, you must replace |
| <filename>gcc-cross</filename>. |
| </note> |
| This tool is also a "native" package (i.e. it is |
| designed to run on the build host). |
| </para></listitem> |
| <listitem><para><filename>gcc-runtime</filename>: |
| Runtime libraries resulting from the toolchain bootstrapping |
| process. |
| This tool produces a binary that consists of the |
| runtime libraries need for the targeted device. |
| </para></listitem> |
| </itemizedlist> |
| </para> |
| |
| <para> |
| You can use the OpenEmbedded build system to build an installer for |
| the relocatable SDK used to develop applications. |
| When you run the installer, it installs the toolchain, which contains |
| the development tools (e.g., the |
| <filename>gcc-cross-canadian</filename>), |
| <filename>binutils-cross-canadian</filename>, and other |
| <filename>nativesdk-*</filename> tools, |
| which are tools native to the SDK (i.e. native to |
| <link linkend='var-SDK_ARCH'><filename>SDK_ARCH</filename></link>), |
| you need to cross-compile and test your software. |
| The figure shows the commands you use to easily build out this |
| toolchain. |
| This cross-development toolchain is built to execute on the |
| <link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>, |
| which might or might not be the same |
| machine as the Build Host. |
| <note> |
| If your target architecture is supported by the Yocto Project, |
| you can take advantage of pre-built images that ship with the |
| Yocto Project and already contain cross-development toolchain |
| installers. |
| </note> |
| </para> |
| |
| <para> |
| Here is the bootstrap process for the relocatable toolchain: |
| <literallayout class='monospaced'> |
| gcc -> binutils-crosssdk -> gcc-crosssdk-initial -> linux-libc-headers -> |
| glibc-initial -> nativesdk-glibc -> gcc-crosssdk -> gcc-cross-canadian |
| </literallayout> |
| <itemizedlist> |
| <listitem><para><filename>gcc</filename>: |
| The build host's GNU Compiler Collection (GCC). |
| </para></listitem> |
| <listitem><para><filename>binutils-crosssdk</filename>: |
| The bare minimum binary utilities needed in order to run |
| the <filename>gcc-crosssdk-initial</filename> phase of the |
| bootstrap operation. |
| </para></listitem> |
| <listitem><para><filename>gcc-crosssdk-initial</filename>: |
| An early stage of the bootstrap process for creating |
| the cross-compiler. |
| This stage builds enough of the |
| <filename>gcc-crosssdk</filename> and supporting pieces so that |
| the final stage of the bootstrap process can produce the |
| finished cross-compiler. |
| This tool is a "native" binary that runs on the build host. |
| </para></listitem> |
| <listitem><para><filename>linux-libc-headers</filename>: |
| Headers needed for the cross-compiler. |
| </para></listitem> |
| <listitem><para><filename>glibc-initial</filename>: |
| An initial version of the Embedded GLIBC needed to bootstrap |
| <filename>nativesdk-glibc</filename>. |
| </para></listitem> |
| <listitem><para><filename>nativesdk-glibc</filename>: |
| The Embedded GLIBC needed to bootstrap the |
| <filename>gcc-crosssdk</filename>. |
| </para></listitem> |
| <listitem><para><filename>gcc-crosssdk</filename>: |
| The final stage of the bootstrap process for the |
| relocatable cross-compiler. |
| The <filename>gcc-crosssdk</filename> is a transitory compiler |
| and never leaves the build host. |
| Its purpose is to help in the bootstrap process to create the |
| eventual relocatable <filename>gcc-cross-canadian</filename> |
| compiler, which is relocatable. |
| This tool is also a "native" package (i.e. it is |
| designed to run on the build host). |
| </para></listitem> |
| <listitem><para><filename>gcc-cross-canadian</filename>: |
| The final relocatable cross-compiler. |
| When run on the |
| <link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>, |
| this tool |
| produces executable code that runs on the target device. |
| Only one cross-canadian compiler is produced per architecture |
| since they can be targeted at different processor optimizations |
| using configurations passed to the compiler through the |
| compile commands. |
| This circumvents the need for multiple compilers and thus |
| reduces the size of the toolchains. |
| </para></listitem> |
| </itemizedlist> |
| </para> |
| |
| <note> |
| For information on advantages gained when building a |
| cross-development toolchain installer, see the |
| "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-building-an-sdk-installer'>Building an SDK Installer</ulink>" |
| section in the Yocto Project Software Development Kit (SDK) Developer's |
| Guide. |
| </note> |
| </section> |
| |
| <section id="shared-state-cache"> |
| <title>Shared State Cache</title> |
| |
| <para> |
| By design, the OpenEmbedded build system builds everything from scratch unless |
| BitBake can determine that parts do not need to be rebuilt. |
| Fundamentally, building from scratch is attractive as it means all parts are |
| built fresh and there is no possibility of stale data causing problems. |
| When developers hit problems, they typically default back to building from scratch |
| so they know the state of things from the start. |
| </para> |
| |
| <para> |
| Building an image from scratch is both an advantage and a disadvantage to the process. |
| As mentioned in the previous paragraph, building from scratch ensures that |
| everything is current and starts from a known state. |
| However, building from scratch also takes much longer as it generally means |
| rebuilding things that do not necessarily need to be rebuilt. |
| </para> |
| |
| <para> |
| The Yocto Project implements shared state code that supports incremental builds. |
| The implementation of the shared state code answers the following questions that |
| were fundamental roadblocks within the OpenEmbedded incremental build support system: |
| <itemizedlist> |
| <listitem><para>What pieces of the system have changed and what pieces have |
| not changed?</para></listitem> |
| <listitem><para>How are changed pieces of software removed and replaced?</para></listitem> |
| <listitem><para>How are pre-built components that do not need to be rebuilt from scratch |
| used when they are available?</para></listitem> |
| </itemizedlist> |
| </para> |
| |
| <para> |
| For the first question, the build system detects changes in the "inputs" to a given task by |
| creating a checksum (or signature) of the task's inputs. |
| If the checksum changes, the system assumes the inputs have changed and the task needs to be |
| rerun. |
| For the second question, the shared state (sstate) code tracks which tasks add which output |
| to the build process. |
| This means the output from a given task can be removed, upgraded or otherwise manipulated. |
| The third question is partly addressed by the solution for the second question |
| assuming the build system can fetch the sstate objects from remote locations and |
| install them if they are deemed to be valid. |
| </para> |
| |
| <note> |
| The OpenEmbedded build system does not maintain |
| <link linkend='var-PR'><filename>PR</filename></link> information |
| as part of the shared state packages. |
| Consequently, considerations exist that affect maintaining shared |
| state feeds. |
| For information on how the OpenEmbedded build system |
| works with packages and can |
| track incrementing <filename>PR</filename> information, see the |
| "<ulink url='&YOCTO_DOCS_DEV_URL;#incrementing-a-package-revision-number'>Incrementing a Package Revision Number</ulink>" |
| section. |
| </note> |
| |
| <para> |
| The rest of this section goes into detail about the overall incremental build |
| architecture, the checksums (signatures), shared state, and some tips and tricks. |
| </para> |
| |
| <section id='overall-architecture'> |
| <title>Overall Architecture</title> |
| |
| <para> |
| When determining what parts of the system need to be built, BitBake |
| works on a per-task basis rather than a per-recipe basis. |
| You might wonder why using a per-task basis is preferred over a per-recipe basis. |
| To help explain, consider having the IPK packaging backend enabled and then switching to DEB. |
| In this case, the |
| <link linkend='ref-tasks-install'><filename>do_install</filename></link> |
| and |
| <link linkend='ref-tasks-package'><filename>do_package</filename></link> |
| task outputs are still valid. |
| However, with a per-recipe approach, the build would not include the |
| <filename>.deb</filename> files. |
| Consequently, you would have to invalidate the whole build and rerun it. |
| Rerunning everything is not the best solution. |
| Also, in this case, the core must be "taught" much about specific tasks. |
| This methodology does not scale well and does not allow users to easily add new tasks |
| in layers or as external recipes without touching the packaged-staging core. |
| </para> |
| </section> |
| |
| <section id='checksums'> |
| <title>Checksums (Signatures)</title> |
| |
| <para> |
| The shared state code uses a checksum, which is a unique signature of a task's |
| inputs, to determine if a task needs to be run again. |
| Because it is a change in a task's inputs that triggers a rerun, the process |
| needs to detect all the inputs to a given task. |
| For shell tasks, this turns out to be fairly easy because |
| the build process generates a "run" shell script for each task and |
| it is possible to create a checksum that gives you a good idea of when |
| the task's data changes. |
| </para> |
| |
| <para> |
| To complicate the problem, there are things that should not be |
| included in the checksum. |
| First, there is the actual specific build path of a given task - |
| the <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>. |
| It does not matter if the work directory changes because it should |
| not affect the output for target packages. |
| Also, the build process has the objective of making native |
| or cross packages relocatable. |
| <note> |
| Both native and cross packages run on the build host. |
| However, cross packages generate output for the target |
| architecture. |
| </note> |
| The checksum therefore needs to exclude |
| <filename>WORKDIR</filename>. |
| The simplistic approach for excluding the work directory is to set |
| <filename>WORKDIR</filename> to some fixed value and create the |
| checksum for the "run" script. |
| </para> |
| |
| <para> |
| Another problem results from the "run" scripts containing functions that |
| might or might not get called. |
| The incremental build solution contains code that figures out dependencies |
| between shell functions. |
| This code is used to prune the "run" scripts down to the minimum set, |
| thereby alleviating this problem and making the "run" scripts much more |
| readable as a bonus. |
| </para> |
| |
| <para> |
| So far we have solutions for shell scripts. |
| What about Python tasks? |
| The same approach applies even though these tasks are more difficult. |
| The process needs to figure out what variables a Python function accesses |
| and what functions it calls. |
| Again, the incremental build solution contains code that first figures out |
| the variable and function dependencies, and then creates a checksum for the data |
| used as the input to the task. |
| </para> |
| |
| <para> |
| Like the <filename>WORKDIR</filename> case, situations exist where dependencies |
| should be ignored. |
| For these cases, you can instruct the build process to ignore a dependency |
| by using a line like the following: |
| <literallayout class='monospaced'> |
| PACKAGE_ARCHS[vardepsexclude] = "MACHINE" |
| </literallayout> |
| This example ensures that the |
| <link linkend='var-PACKAGE_ARCHS'><filename>PACKAGE_ARCHS</filename></link> |
| variable does not |
| depend on the value of |
| <link linkend='var-MACHINE'><filename>MACHINE</filename></link>, |
| even if it does reference it. |
| </para> |
| |
| <para> |
| Equally, there are cases where we need to add dependencies BitBake is not able to find. |
| You can accomplish this by using a line like the following: |
| <literallayout class='monospaced'> |
| PACKAGE_ARCHS[vardeps] = "MACHINE" |
| </literallayout> |
| This example explicitly adds the <filename>MACHINE</filename> variable as a |
| dependency for <filename>PACKAGE_ARCHS</filename>. |
| </para> |
| |
| <para> |
| Consider a case with in-line Python, for example, where BitBake is not |
| able to figure out dependencies. |
| When running in debug mode (i.e. using <filename>-DDD</filename>), BitBake |
| produces output when it discovers something for which it cannot figure out |
| dependencies. |
| The Yocto Project team has currently not managed to cover those dependencies |
| in detail and is aware of the need to fix this situation. |
| </para> |
| |
| <para> |
| Thus far, this section has limited discussion to the direct inputs into a task. |
| Information based on direct inputs is referred to as the "basehash" in the |
| code. |
| However, there is still the question of a task's indirect inputs - the |
| things that were already built and present in the |
| <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>. |
| The checksum (or signature) for a particular task needs to add the hashes |
| of all the tasks on which the particular task depends. |
| Choosing which dependencies to add is a policy decision. |
| However, the effect is to generate a master checksum that combines the basehash |
| and the hashes of the task's dependencies. |
| </para> |
| |
| <para> |
| At the code level, there are a variety of ways both the basehash and the |
| dependent task hashes can be influenced. |
| Within the BitBake configuration file, we can give BitBake some extra information |
| to help it construct the basehash. |
| The following statement effectively results in a list of global variable |
| dependency excludes - variables never included in any checksum: |
| <literallayout class='monospaced'> |
| BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR \ |
| SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM \ |
| USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \ |
| PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \ |
| CCACHE_DIR EXTERNAL_TOOLCHAIN CCACHE CCACHE_DISABLE LICENSE_PATH SDKPKGSUFFIX" |
| </literallayout> |
| The previous example excludes |
| <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link> |
| since that variable is actually constructed as a path within |
| <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>, which is on |
| the whitelist. |
| </para> |
| |
| <para> |
| The rules for deciding which hashes of dependent tasks to include through |
| dependency chains are more complex and are generally accomplished with a |
| Python function. |
| The code in <filename>meta/lib/oe/sstatesig.py</filename> shows two examples |
| of this and also illustrates how you can insert your own policy into the system |
| if so desired. |
| This file defines the two basic signature generators <filename>OE-Core</filename> |
| uses: "OEBasic" and "OEBasicHash". |
| By default, there is a dummy "noop" signature handler enabled in BitBake. |
| This means that behavior is unchanged from previous versions. |
| <filename>OE-Core</filename> uses the "OEBasicHash" signature handler by default |
| through this setting in the <filename>bitbake.conf</filename> file: |
| <literallayout class='monospaced'> |
| BB_SIGNATURE_HANDLER ?= "OEBasicHash" |
| </literallayout> |
| The "OEBasicHash" <filename>BB_SIGNATURE_HANDLER</filename> is the same as the |
| "OEBasic" version but adds the task hash to the stamp files. |
| This results in any |
| <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink> |
| change that changes the task hash, automatically |
| causing the task to be run again. |
| This removes the need to bump <link linkend='var-PR'><filename>PR</filename></link> |
| values, and changes to Metadata automatically ripple across the build. |
| </para> |
| |
| <para> |
| It is also worth noting that the end result of these signature generators is to |
| make some dependency and hash information available to the build. |
| This information includes: |
| <itemizedlist> |
| <listitem><para><filename>BB_BASEHASH_task-</filename><replaceable>taskname</replaceable>: |
| The base hashes for each task in the recipe. |
| </para></listitem> |
| <listitem><para><filename>BB_BASEHASH_</filename><replaceable>filename</replaceable><filename>:</filename><replaceable>taskname</replaceable>: |
| The base hashes for each dependent task. |
| </para></listitem> |
| <listitem><para><filename>BBHASHDEPS_</filename><replaceable>filename</replaceable><filename>:</filename><replaceable>taskname</replaceable>: |
| The task dependencies for each task. |
| </para></listitem> |
| <listitem><para><filename>BB_TASKHASH</filename>: |
| The hash of the currently running task. |
| </para></listitem> |
| </itemizedlist> |
| </para> |
| </section> |
| |
| <section id='shared-state'> |
| <title>Shared State</title> |
| |
| <para> |
| Checksums and dependencies, as discussed in the previous section, solve half the |
| problem of supporting a shared state. |
| The other part of the problem is being able to use checksum information during the build |
| and being able to reuse or rebuild specific components. |
| </para> |
| |
| <para> |
| The |
| <link linkend='ref-classes-sstate'><filename>sstate</filename></link> |
| class is a relatively generic implementation of how to "capture" |
| a snapshot of a given task. |
| The idea is that the build process does not care about the source of a task's output. |
| Output could be freshly built or it could be downloaded and unpacked from |
| somewhere - the build process does not need to worry about its origin. |
| </para> |
| |
| <para> |
| There are two types of output, one is just about creating a directory |
| in <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>. |
| A good example is the output of either |
| <link linkend='ref-tasks-install'><filename>do_install</filename></link> |
| or |
| <link linkend='ref-tasks-package'><filename>do_package</filename></link>. |
| The other type of output occurs when a set of data is merged into a shared directory |
| tree such as the sysroot. |
| </para> |
| |
| <para> |
| The Yocto Project team has tried to keep the details of the |
| implementation hidden in <filename>sstate</filename> class. |
| From a user's perspective, adding shared state wrapping to a task |
| is as simple as this |
| <link linkend='ref-tasks-deploy'><filename>do_deploy</filename></link> |
| example taken from the |
| <link linkend='ref-classes-deploy'><filename>deploy</filename></link> |
| class: |
| <literallayout class='monospaced'> |
| DEPLOYDIR = "${WORKDIR}/deploy-${PN}" |
| SSTATETASKS += "do_deploy" |
| do_deploy[sstate-inputdirs] = "${DEPLOYDIR}" |
| do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}" |
| |
| python do_deploy_setscene () { |
| sstate_setscene(d) |
| } |
| addtask do_deploy_setscene |
| do_deploy[dirs] = "${DEPLOYDIR} ${B}" |
| </literallayout> |
| In this example, we add some extra flags to the task, a name field ("deploy"), an |
| input directory where the task sends data, and the output |
| directory where the data from the task should eventually be copied. |
| We also add a <filename>_setscene</filename> variant of the task and add the task |
| name to the <filename>SSTATETASKS</filename> list. |
| </para> |
| |
| <para> |
| If you have a directory whose contents you need to preserve, you can do this with |
| a line like the following: |
| <literallayout class='monospaced'> |
| do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}" |
| </literallayout> |
| This method, as well as the following example, also works for multiple directories. |
| <literallayout class='monospaced'> |
| do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}" |
| do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}" |
| do_package[sstate-lockfile] = "${PACKAGELOCK}" |
| </literallayout> |
| These methods also include the ability to take a lockfile when manipulating |
| shared state directory structures since some cases are sensitive to file |
| additions or removals. |
| </para> |
| |
| <para> |
| Behind the scenes, the shared state code works by looking in |
| <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> and |
| <link linkend='var-SSTATE_MIRRORS'><filename>SSTATE_MIRRORS</filename></link> |
| for shared state files. |
| Here is an example: |
| <literallayout class='monospaced'> |
| SSTATE_MIRRORS ?= "\ |
| file://.* http://someserver.tld/share/sstate/PATH \n \ |
| file://.* file:///some/local/dir/sstate/PATH" |
| </literallayout> |
| <note> |
| The shared state directory (<filename>SSTATE_DIR</filename>) is |
| organized into two-character subdirectories, where the subdirectory |
| names are based on the first two characters of the hash. |
| If the shared state directory structure for a mirror has the |
| same structure as <filename>SSTATE_DIR</filename>, you must |
| specify "PATH" as part of the URI to enable the build system |
| to map to the appropriate subdirectory. |
| </note> |
| </para> |
| |
| <para> |
| The shared state package validity can be detected just by looking at the |
| filename since the filename contains the task checksum (or signature) as |
| described earlier in this section. |
| If a valid shared state package is found, the build process downloads it |
| and uses it to accelerate the task. |
| </para> |
| |
| <para> |
| The build processes use the <filename>*_setscene</filename> tasks |
| for the task acceleration phase. |
| BitBake goes through this phase before the main execution code and tries |
| to accelerate any tasks for which it can find shared state packages. |
| If a shared state package for a task is available, the shared state |
| package is used. |
| This means the task and any tasks on which it is dependent are not |
| executed. |
| </para> |
| |
| <para> |
| As a real world example, the aim is when building an IPK-based image, |
| only the |
| <link linkend='ref-tasks-package_write_ipk'><filename>do_package_write_ipk</filename></link> |
| tasks would have their |
| shared state packages fetched and extracted. |
| Since the sysroot is not used, it would never get extracted. |
| This is another reason why a task-based approach is preferred over a |
| recipe-based approach, which would have to install the output from every task. |
| </para> |
| </section> |
| |
| <section id='tips-and-tricks'> |
| <title>Tips and Tricks</title> |
| |
| <para> |
| The code in the build system that supports incremental builds is not |
| simple code. |
| This section presents some tips and tricks that help you work around |
| issues related to shared state code. |
| </para> |
| |
| <section id='debugging'> |
| <title>Debugging</title> |
| |
| <para> |
| When things go wrong, debugging needs to be straightforward. |
| Because of this, the Yocto Project includes strong debugging |
| tools: |
| <itemizedlist> |
| <listitem><para>Whenever a shared state package is written |
| out into the |
| <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link>, |
| a corresponding <filename>.siginfo</filename> file is |
| also written. |
| This file contains a pickled Python database of all |
| the Metadata that went into creating the hash for a |
| given shared state package. |
| Whenever a stamp is written into the stamp directory |
| <link linkend='var-STAMP'><filename>STAMP</filename></link>, |
| a corresponding <filename>.sigdata</filename> file |
| is created that contains the same hash data that |
| represented the executed task. |
| </para></listitem> |
| <listitem><para>You can use BitBake to dump out the |
| signature construction information without executing |
| tasks by using either of the following BitBake |
| command-line options: |
| <literallayout class='monospaced'> |
| ‐‐dump-signatures=<replaceable>SIGNATURE_HANDLER</replaceable> |
| -S <replaceable>SIGNATURE_HANDLER</replaceable> |
| </literallayout> |
| <note> |
| Two common values for |
| <replaceable>SIGNATURE_HANDLER</replaceable> are |
| "none" and "printdiff" to only dump the signature |
| or to compare the dumped signature with the |
| cached one, respectively. |
| </note> |
| Using BitBake with either of these options causes |
| BitBake to dump out <filename>.sigdata</filename> files |
| in the stamp directory for every task it would have |
| executed instead of building the specified target |
| package. |
| </para></listitem> |
| <listitem><para>There is a |
| <filename>bitbake-diffsigs</filename> command that |
| can process <filename>.sigdata</filename> and |
| <filename>.siginfo</filename> files. |
| If you specify one of these files, BitBake dumps out |
| the dependency information in the file. |
| If you specify two files, BitBake compares the two |
| files and dumps out the differences between the two. |
| This more easily helps answer the question of "What |
| changed between X and Y?"</para></listitem> |
| </itemizedlist> |
| </para> |
| </section> |
| |
| <section id='invalidating-shared-state'> |
| <title>Invalidating Shared State</title> |
| |
| <para> |
| The OpenEmbedded build system uses checksums and shared state |
| cache to avoid unnecessarily rebuilding tasks. |
| Collectively, this scheme is known as "shared state code." |
| </para> |
| |
| <para> |
| As with all schemes, this one has some drawbacks. |
| It is possible that you could make implicit changes to your |
| code that the checksum calculations do not take into |
| account. |
| These implicit changes affect a task's output but do not trigger |
| the shared state code into rebuilding a recipe. |
| Consider an example during which a tool changes its output. |
| Assume that the output of <filename>rpmdeps</filename> changes. |
| The result of the change should be that all the |
| <filename>package</filename> and |
| <filename>package_write_rpm</filename> shared state cache |
| items become invalid. |
| However, because the change to the output is |
| external to the code and therefore implicit, |
| the associated shared state cache items do not become |
| invalidated. |
| In this case, the build process uses the cached items rather |
| than running the task again. |
| Obviously, these types of implicit changes can cause problems. |
| </para> |
| |
| <para> |
| To avoid these problems during the build, you need to |
| understand the effects of any changes you make. |
| Realize that changes you make directly to a function |
| are automatically factored into the checksum calculation. |
| Thus, these explicit changes invalidate the associated area of |
| shared state cache. |
| However, you need to be aware of any implicit changes that |
| are not obvious changes to the code and could affect the output |
| of a given task. |
| </para> |
| |
| <para> |
| When you identify an implicit change, you can easily take steps |
| to invalidate the cache and force the tasks to run. |
| The steps you can take are as simple as changing a function's |
| comments in the source code. |
| For example, to invalidate package shared state files, change |
| the comment statements of |
| <link linkend='ref-tasks-package'><filename>do_package</filename></link> |
| or the comments of one of the functions it calls. |
| Even though the change is purely cosmetic, it causes the |
| checksum to be recalculated and forces the OpenEmbedded build |
| system to run the task again. |
| </para> |
| |
| <note> |
| For an example of a commit that makes a cosmetic change to |
| invalidate shared state, see this |
| <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>. |
| </note> |
| </section> |
| </section> |
| </section> |
| |
| <section id='x32'> |
| <title>x32</title> |
| |
| <para> |
| x32 is a processor-specific Application Binary Interface (psABI) for x86_64. |
| An ABI defines the calling conventions between functions in a processing environment. |
| The interface determines what registers are used and what the sizes are for various C data types. |
| </para> |
| |
| <para> |
| Some processing environments prefer using 32-bit applications even when running |
| on Intel 64-bit platforms. |
| Consider the i386 psABI, which is a very old 32-bit ABI for Intel 64-bit platforms. |
| The i386 psABI does not provide efficient use and access of the Intel 64-bit processor resources, |
| leaving the system underutilized. |
| Now consider the x86_64 psABI. |
| This ABI is newer and uses 64-bits for data sizes and program pointers. |
| The extra bits increase the footprint size of the programs, libraries, |
| and also increases the memory and file system size requirements. |
| Executing under the x32 psABI enables user programs to utilize CPU and system resources |
| more efficiently while keeping the memory footprint of the applications low. |
| Extra bits are used for registers but not for addressing mechanisms. |
| </para> |
| |
| <section id='support'> |
| <title>Support</title> |
| |
| <para> |
| This Yocto Project release supports the final specifications of x32 |
| psABI. |
| Support for x32 psABI exists as follows: |
| <itemizedlist> |
| <listitem><para>You can create packages and images in x32 psABI format on x86_64 architecture targets. |
| </para></listitem> |
| <listitem><para>You can successfully build many recipes with the x32 toolchain.</para></listitem> |
| <listitem><para>You can create and boot <filename>core-image-minimal</filename> and |
| <filename>core-image-sato</filename> images.</para></listitem> |
| </itemizedlist> |
| </para> |
| </section> |
| |
| <section id='completing-x32'> |
| <title>Completing x32</title> |
| |
| <para> |
| Future Plans for the x32 psABI in the Yocto Project include the following: |
| <itemizedlist> |
| <listitem><para>Enhance and fix the few remaining recipes so they |
| work with and support x32 toolchains.</para></listitem> |
| <listitem><para>Enhance RPM Package Manager (RPM) support for x32 binaries.</para></listitem> |
| <listitem><para>Support larger images.</para></listitem> |
| </itemizedlist> |
| </para> |
| </section> |
| |
| <section id='using-x32-right-now'> |
| <title>Using x32 Right Now</title> |
| |
| <para> |
| Follow these steps to use the x32 spABI: |
| <itemizedlist> |
| <listitem><para>Enable the x32 psABI tuning file for <filename>x86_64</filename> |
| machines by editing the <filename>conf/local.conf</filename> like this: |
| <literallayout class='monospaced'> |
| MACHINE = "qemux86-64" |
| DEFAULTTUNE = "x86-64-x32" |
| baselib = "${@d.getVar('BASE_LIB_tune-' + (d.getVar('DEFAULTTUNE', True) \ |
| or 'INVALID'), True) or 'lib'}" |
| #MACHINE = "genericx86" |
| #DEFAULTTUNE = "core2-64-x32" |
| </literallayout></para></listitem> |
| <listitem><para>As usual, use BitBake to build an image that supports the x32 psABI. |
| Here is an example: |
| <literallayout class='monospaced'> |
| $ bitbake core-image-sato |
| </literallayout></para></listitem> |
| <listitem><para>As usual, run your image using QEMU: |
| <literallayout class='monospaced'> |
| $ runqemu qemux86-64 core-image-sato |
| </literallayout></para></listitem> |
| </itemizedlist> |
| </para> |
| </section> |
| </section> |
| |
| <section id="wayland"> |
| <title>Wayland</title> |
| |
| <para> |
| <ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)'>Wayland</ulink> |
| is a computer display server protocol that |
| provides a method for compositing window managers to communicate |
| directly with applications and video hardware and expects them to |
| communicate with input hardware using other libraries. |
| Using Wayland with supporting targets can result in better control |
| over graphics frame rendering than an application might otherwise |
| achieve. |
| </para> |
| |
| <para> |
| The Yocto Project provides the Wayland protocol libraries and the |
| reference |
| <ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)#Weston'>Weston</ulink> |
| compositor as part of its release. |
| This section describes what you need to do to implement Wayland and |
| use the compositor when building an image for a supporting target. |
| </para> |
| |
| <section id="wayland-support"> |
| <title>Support</title> |
| |
| <para> |
| The Wayland protocol libraries and the reference Weston compositor |
| ship as integrated packages in the <filename>meta</filename> layer |
| of the |
| <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>. |
| Specifically, you can find the recipes that build both Wayland |
| and Weston at <filename>meta/recipes-graphics/wayland</filename>. |
| </para> |
| |
| <para> |
| You can build both the Wayland and Weston packages for use only |
| with targets that accept the |
| <ulink url='http://dri.freedesktop.org/wiki/'>Mesa 3D and Direct Rendering Infrastructure</ulink>, |
| which is also known as Mesa DRI. |
| This implies that you cannot build and use the packages if your |
| target uses, for example, the |
| <trademark class='registered'>Intel</trademark> Embedded Media and |
| Graphics Driver (<trademark class='registered'>Intel</trademark> |
| EMGD) that overrides Mesa DRI. |
| </para> |
| |
| <note> |
| Due to lack of EGL support, Weston 1.0.3 will not run directly on |
| the emulated QEMU hardware. |
| However, this version of Weston will run under X emulation without |
| issues. |
| </note> |
| </section> |
| |
| <section id="enabling-wayland-in-an-image"> |
| <title>Enabling Wayland in an Image</title> |
| |
| <para> |
| To enable Wayland, you need to enable it to be built and enable |
| it to be included in the image. |
| </para> |
| |
| <section id="enable-building"> |
| <title>Building</title> |
| |
| <para> |
| To cause Mesa to build the <filename>wayland-egl</filename> |
| platform and Weston to build Wayland with Kernel Mode |
| Setting |
| (<ulink url='https://wiki.archlinux.org/index.php/Kernel_Mode_Setting'>KMS</ulink>) |
| support, include the "wayland" flag in the |
| <link linkend="var-DISTRO_FEATURES"><filename>DISTRO_FEATURES</filename></link> |
| statement in your <filename>local.conf</filename> file: |
| <literallayout class='monospaced'> |
| DISTRO_FEATURES_append = " wayland" |
| </literallayout> |
| </para> |
| |
| <note> |
| If X11 has been enabled elsewhere, Weston will build Wayland |
| with X11 support |
| </note> |
| </section> |
| |
| <section id="enable-installation-in-an-image"> |
| <title>Installing</title> |
| |
| <para> |
| To install the Wayland feature into an image, you must |
| include the following |
| <link linkend='var-CORE_IMAGE_EXTRA_INSTALL'><filename>CORE_IMAGE_EXTRA_INSTALL</filename></link> |
| statement in your <filename>local.conf</filename> file: |
| <literallayout class='monospaced'> |
| CORE_IMAGE_EXTRA_INSTALL += "wayland weston" |
| </literallayout> |
| </para> |
| </section> |
| </section> |
| |
| <section id="running-weston"> |
| <title>Running Weston</title> |
| |
| <para> |
| To run Weston inside X11, enabling it as described earlier and |
| building a Sato image is sufficient. |
| If you are running your image under Sato, a Weston Launcher appears |
| in the "Utility" category. |
| </para> |
| |
| <para> |
| Alternatively, you can run Weston through the command-line |
| interpretor (CLI), which is better suited for development work. |
| To run Weston under the CLI, you need to do the following after |
| your image is built: |
| <orderedlist> |
| <listitem><para>Run these commands to export |
| <filename>XDG_RUNTIME_DIR</filename>: |
| <literallayout class='monospaced'> |
| mkdir -p /tmp/$USER-weston |
| chmod 0700 /tmp/$USER-weston |
| export XDG_RUNTIME_DIR=/tmp/$USER-weston |
| </literallayout></para></listitem> |
| <listitem><para>Launch Weston in the shell: |
| <literallayout class='monospaced'> |
| weston |
| </literallayout></para></listitem> |
| </orderedlist> |
| </para> |
| </section> |
| </section> |
| |
| <section id="licenses"> |
| <title>Licenses</title> |
| |
| <para> |
| This section describes the mechanism by which the OpenEmbedded build system |
| tracks changes to licensing text. |
| The section also describes how to enable commercially licensed recipes, |
| which by default are disabled. |
| </para> |
| |
| <para> |
| For information that can help you maintain compliance with various open |
| source licensing during the lifecycle of the product, see the |
| "<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Project's Lifecycle</ulink>" section |
| in the Yocto Project Development Manual. |
| </para> |
| |
| <section id="usingpoky-configuring-LIC_FILES_CHKSUM"> |
| <title>Tracking License Changes</title> |
| |
| <para> |
| The license of an upstream project might change in the future. |
| In order to prevent these changes going unnoticed, the |
| <filename><link linkend='var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</link></filename> |
| variable tracks changes to the license text. The checksums are validated at the end of the |
| configure step, and if the checksums do not match, the build will fail. |
| </para> |
| |
| <section id="usingpoky-specifying-LIC_FILES_CHKSUM"> |
| <title>Specifying the <filename>LIC_FILES_CHKSUM</filename> Variable</title> |
| |
| <para> |
| The <filename>LIC_FILES_CHKSUM</filename> |
| variable contains checksums of the license text in the source code for the recipe. |
| Following is an example of how to specify <filename>LIC_FILES_CHKSUM</filename>: |
| <literallayout class='monospaced'> |
| LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \ |
| file://licfile1.txt;beginline=5;endline=29;md5=yyyy \ |
| file://licfile2.txt;endline=50;md5=zzzz \ |
| ..." |
| </literallayout> |
| </para> |
| |
| <para> |
| The build system uses the |
| <filename><link linkend='var-S'>S</link></filename> variable as |
| the default directory when searching files listed in |
| <filename>LIC_FILES_CHKSUM</filename>. |
| The previous example employs the default directory. |
| </para> |
| |
| <para> |
| Consider this next example: |
| <literallayout class='monospaced'> |
| LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\ |
| md5=bb14ed3c4cda583abc85401304b5cd4e" |
| LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6" |
| </literallayout> |
| </para> |
| |
| <para> |
| The first line locates a file in |
| <filename>${S}/src/ls.c</filename>. |
| The second line refers to a file in |
| <filename><link linkend='var-WORKDIR'>WORKDIR</link></filename>. |
| </para> |
| <para> |
| Note that <filename>LIC_FILES_CHKSUM</filename> variable is |
| mandatory for all recipes, unless the |
| <filename>LICENSE</filename> variable is set to "CLOSED". |
| </para> |
| </section> |
| |
| <section id="usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax"> |
| <title>Explanation of Syntax</title> |
| <para> |
| As mentioned in the previous section, the |
| <filename>LIC_FILES_CHKSUM</filename> variable lists all the |
| important files that contain the license text for the source code. |
| It is possible to specify a checksum for an entire file, or a specific section of a |
| file (specified by beginning and ending line numbers with the "beginline" and "endline" |
| parameters, respectively). |
| The latter is useful for source files with a license notice header, |
| README documents, and so forth. |
| If you do not use the "beginline" parameter, then it is assumed that the text begins on the |
| first line of the file. |
| Similarly, if you do not use the "endline" parameter, it is assumed that the license text |
| ends with the last line of the file. |
| </para> |
| |
| <para> |
| The "md5" parameter stores the md5 checksum of the license text. |
| If the license text changes in any way as compared to this parameter |
| then a mismatch occurs. |
| This mismatch triggers a build failure and notifies the developer. |
| Notification allows the developer to review and address the license text changes. |
| Also note that if a mismatch occurs during the build, the correct md5 |
| checksum is placed in the build log and can be easily copied to the recipe. |
| </para> |
| |
| <para> |
| There is no limit to how many files you can specify using the |
| <filename>LIC_FILES_CHKSUM</filename> variable. |
| Generally, however, every project requires a few specifications for license tracking. |
| Many projects have a "COPYING" file that stores the license information for all the source |
| code files. |
| This practice allows you to just track the "COPYING" file as long as it is kept up to date. |
| </para> |
| |
| <tip> |
| If you specify an empty or invalid "md5" parameter, BitBake returns an md5 mis-match |
| error and displays the correct "md5" parameter value during the build. |
| The correct parameter is also captured in the build log. |
| </tip> |
| |
| <tip> |
| If the whole file contains only license text, you do not need to use the "beginline" and |
| "endline" parameters. |
| </tip> |
| </section> |
| </section> |
| |
| <section id="enabling-commercially-licensed-recipes"> |
| <title>Enabling Commercially Licensed Recipes</title> |
| |
| <para> |
| By default, the OpenEmbedded build system disables |
| components that have commercial or other special licensing |
| requirements. |
| Such requirements are defined on a |
| recipe-by-recipe basis through the |
| <link linkend='var-LICENSE_FLAGS'><filename>LICENSE_FLAGS</filename></link> |
| variable definition in the affected recipe. |
| For instance, the |
| <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename> |
| recipe contains the following statement: |
| <literallayout class='monospaced'> |
| LICENSE_FLAGS = "commercial" |
| </literallayout> |
| Here is a slightly more complicated example that contains both an |
| explicit recipe name and version (after variable expansion): |
| <literallayout class='monospaced'> |
| LICENSE_FLAGS = "license_${PN}_${PV}" |
| </literallayout> |
| In order for a component restricted by a <filename>LICENSE_FLAGS</filename> |
| definition to be enabled and included in an image, it |
| needs to have a matching entry in the global |
| <link linkend='var-LICENSE_FLAGS_WHITELIST'><filename>LICENSE_FLAGS_WHITELIST</filename></link> |
| variable, which is a variable |
| typically defined in your <filename>local.conf</filename> file. |
| For example, to enable |
| the <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename> |
| package, you could add either the string |
| "commercial_gst-plugins-ugly" or the more general string |
| "commercial" to <filename>LICENSE_FLAGS_WHITELIST</filename>. |
| See the |
| "<link linkend='license-flag-matching'>License Flag Matching</link>" section |
| for a full explanation of how <filename>LICENSE_FLAGS</filename> matching works. |
| Here is the example: |
| <literallayout class='monospaced'> |
| LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly" |
| </literallayout> |
| Likewise, to additionally enable the package built from the recipe containing |
| <filename>LICENSE_FLAGS = "license_${PN}_${PV}"</filename>, and assuming |
| that the actual recipe name was <filename>emgd_1.10.bb</filename>, |
| the following string would enable that package as well as |
| the original <filename>gst-plugins-ugly</filename> package: |
| <literallayout class='monospaced'> |
| LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly license_emgd_1.10" |
| </literallayout> |
| As a convenience, you do not need to specify the complete license string |
| in the whitelist for every package. |
| You can use an abbreviated form, which consists |
| of just the first portion or portions of the license string before |
| the initial underscore character or characters. |
| A partial string will match |
| any license that contains the given string as the first |
| portion of its license. |
| For example, the following |
| whitelist string will also match both of the packages |
| previously mentioned as well as any other packages that have |
| licenses starting with "commercial" or "license". |
| <literallayout class='monospaced'> |
| LICENSE_FLAGS_WHITELIST = "commercial license" |
| </literallayout> |
| </para> |
| |
| <section id="license-flag-matching"> |
| <title>License Flag Matching</title> |
| |
| <para> |
| License flag matching allows you to control what recipes the |
| OpenEmbedded build system includes in the build. |
| Fundamentally, the build system attempts to match |
| <link linkend='var-LICENSE_FLAGS'><filename>LICENSE_FLAGS</filename></link> |
| strings found in recipes against |
| <link linkend='var-LICENSE_FLAGS_WHITELIST'><filename>LICENSE_FLAGS_WHITELIST</filename></link> |
| strings found in the whitelist. |
| A match causes the build system to include a recipe in the |
| build, while failure to find a match causes the build system to |
| exclude a recipe. |
| </para> |
| |
| <para> |
| In general, license flag matching is simple. |
| However, understanding some concepts will help you |
| correctly and effectively use matching. |
| </para> |
| |
| <para> |
| Before a flag |
| defined by a particular recipe is tested against the |
| contents of the whitelist, the expanded string |
| <filename>_${PN}</filename> is appended to the flag. |
| This expansion makes each <filename>LICENSE_FLAGS</filename> |
| value recipe-specific. |
| After expansion, the string is then matched against the |
| whitelist. |
| Thus, specifying |
| <filename>LICENSE_FLAGS = "commercial"</filename> |
| in recipe "foo", for example, results in the string |
| <filename>"commercial_foo"</filename>. |
| And, to create a match, that string must appear in the |
| whitelist. |
| </para> |
| |
| <para> |
| Judicious use of the <filename>LICENSE_FLAGS</filename> |
| strings and the contents of the |
| <filename>LICENSE_FLAGS_WHITELIST</filename> variable |
| allows you a lot of flexibility for including or excluding |
| recipes based on licensing. |
| For example, you can broaden the matching capabilities by |
| using license flags string subsets in the whitelist. |
| <note>When using a string subset, be sure to use the part of |
| the expanded string that precedes the appended underscore |
| character (e.g. <filename>usethispart_1.3</filename>, |
| <filename>usethispart_1.4</filename>, and so forth). |
| </note> |
| For example, simply specifying the string "commercial" in |
| the whitelist matches any expanded |
| <filename>LICENSE_FLAGS</filename> definition that starts with |
| the string "commercial" such as "commercial_foo" and |
| "commercial_bar", which are the strings the build system |
| automatically generates for hypothetical recipes named |
| "foo" and "bar" assuming those recipes simply specify the |
| following: |
| <literallayout class='monospaced'> |
| LICENSE_FLAGS = "commercial" |
| </literallayout> |
| Thus, you can choose to exhaustively |
| enumerate each license flag in the whitelist and |
| allow only specific recipes into the image, or |
| you can use a string subset that causes a broader range of |
| matches to allow a range of recipes into the image. |
| </para> |
| |
| <para> |
| This scheme works even if the |
| <filename>LICENSE_FLAGS</filename> string already |
| has <filename>_${PN}</filename> appended. |
| For example, the build system turns the license flag |
| "commercial_1.2_foo" into "commercial_1.2_foo_foo" and would |
| match both the general "commercial" and the specific |
| "commercial_1.2_foo" strings found in the whitelist, as |
| expected. |
| </para> |
| |
| <para> |
| Here are some other scenarios: |
| <itemizedlist> |
| <listitem><para>You can specify a versioned string in the |
| recipe such as "commercial_foo_1.2" in a "foo" recipe. |
| The build system expands this string to |
| "commercial_foo_1.2_foo". |
| Combine this license flag with a whitelist that has |
| the string "commercial" and you match the flag along |
| with any other flag that starts with the string |
| "commercial".</para></listitem> |
| <listitem><para>Under the same circumstances, you can |
| use "commercial_foo" in the whitelist and the |
| build system not only matches "commercial_foo_1.2" but |
| also matches any license flag with the string |
| "commercial_foo", regardless of the version. |
| </para></listitem> |
| <listitem><para>You can be very specific and use both the |
| package and version parts in the whitelist (e.g. |
| "commercial_foo_1.2") to specifically match a |
| versioned recipe.</para></listitem> |
| </itemizedlist> |
| </para> |
| </section> |
| |
| <section id="other-variables-related-to-commercial-licenses"> |
| <title>Other Variables Related to Commercial Licenses</title> |
| |
| <para> |
| Other helpful variables related to commercial |
| license handling exist and are defined in the |
| <filename>poky/meta/conf/distro/include/default-distrovars.inc</filename> file: |
| <literallayout class='monospaced'> |
| COMMERCIAL_AUDIO_PLUGINS ?= "" |
| COMMERCIAL_VIDEO_PLUGINS ?= "" |
| </literallayout> |
| If you want to enable these components, you can do so by making sure you have |
| statements similar to the following |
| in your <filename>local.conf</filename> configuration file: |
| <literallayout class='monospaced'> |
| COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \ |
| gst-plugins-ugly-mpegaudioparse" |
| COMMERCIAL_VIDEO_PLUGINS = "gst-plugins-ugly-mpeg2dec \ |
| gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse" |
| LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp" |
| </literallayout> |
| Of course, you could also create a matching whitelist |
| for those components using the more general "commercial" |
| in the whitelist, but that would also enable all the |
| other packages with |
| <link linkend='var-LICENSE_FLAGS'><filename>LICENSE_FLAGS</filename></link> |
| containing "commercial", which you may or may not want: |
| <literallayout class='monospaced'> |
| LICENSE_FLAGS_WHITELIST = "commercial" |
| </literallayout> |
| </para> |
| |
| <para> |
| Specifying audio and video plug-ins as part of the |
| <filename>COMMERCIAL_AUDIO_PLUGINS</filename> and |
| <filename>COMMERCIAL_VIDEO_PLUGINS</filename> statements |
| (along with the enabling |
| <filename>LICENSE_FLAGS_WHITELIST</filename>) includes the |
| plug-ins or components into built images, thus adding |
| support for media formats or components. |
| </para> |
| </section> |
| </section> |
| </section> |
| </chapter> |
| <!-- |
| vim: expandtab tw=80 ts=4 |
| --> |