Squashed 'yocto-poky/' content from commit ea562de
git-subtree-dir: yocto-poky
git-subtree-split: ea562de57590c966cd5a75fda8defecd397e6436
diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.xml b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.xml
new file mode 100644
index 0000000..2188655
--- /dev/null
+++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.xml
@@ -0,0 +1,685 @@
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
+ "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
+
+<chapter id="bitbake-user-manual-intro">
+ <title>Overview</title>
+
+ <para>
+ Welcome to the BitBake User Manual.
+ This manual provides information on the BitBake tool.
+ The information attempts to be as independent as possible regarding
+ systems that use BitBake, such as OpenEmbedded and the
+ Yocto Project.
+ In some cases, scenarios or examples within the context of
+ a build system are used in the manual to help with understanding.
+ For these cases, the manual clearly states the context.
+ </para>
+
+ <section id="intro">
+ <title>Introduction</title>
+
+ <para>
+ Fundamentally, BitBake is a generic task execution
+ engine that allows shell and Python tasks to be run
+ efficiently and in parallel while working within
+ complex inter-task dependency constraints.
+ One of BitBake's main users, OpenEmbedded, takes this core
+ and builds embedded Linux software stacks using
+ a task-oriented approach.
+ </para>
+
+ <para>
+ Conceptually, BitBake is similar to GNU Make in
+ some regards but has significant differences:
+ <itemizedlist>
+ <listitem><para>
+ BitBake executes tasks according to provided
+ metadata that builds up the tasks.
+ Metadata is stored in recipe (<filename>.bb</filename>)
+ and related recipe "append" (<filename>.bbappend</filename>)
+ files, configuration (<filename>.conf</filename>) and
+ underlying include (<filename>.inc</filename>) files, and
+ in class (<filename>.bbclass</filename>) files.
+ The metadata provides
+ BitBake with instructions on what tasks to run and
+ the dependencies between those tasks.
+ </para></listitem>
+ <listitem><para>
+ BitBake includes a fetcher library for obtaining source
+ code from various places such as local files, source control
+ systems, or websites.
+ </para></listitem>
+ <listitem><para>
+ The instructions for each unit to be built (e.g. a piece
+ of software) are known as "recipe" files and
+ contain all the information about the unit
+ (dependencies, source file locations, checksums, description
+ and so on).
+ </para></listitem>
+ <listitem><para>
+ BitBake includes a client/server abstraction and can
+ be used from a command line or used as a service over
+ XML-RPC and has several different user interfaces.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </section>
+
+ <section id="history-and-goals">
+ <title>History and Goals</title>
+
+ <para>
+ BitBake was originally a part of the OpenEmbedded project.
+ It was inspired by the Portage package management system
+ used by the Gentoo Linux distribution.
+ On December 7, 2004, OpenEmbedded project team member
+ Chris Larson split the project into two distinct pieces:
+ <itemizedlist>
+ <listitem><para>BitBake, a generic task executor</para></listitem>
+ <listitem><para>OpenEmbedded, a metadata set utilized by
+ BitBake</para></listitem>
+ </itemizedlist>
+ Today, BitBake is the primary basis of the
+ <ulink url="http://www.openembedded.org/">OpenEmbedded</ulink>
+ project, which is being used to build and maintain Linux
+ distributions such as the
+ <ulink url='http://www.angstrom-distribution.org/'>Angstrom Distribution</ulink>,
+ and which is also being used as the build tool for Linux projects
+ such as the
+ <ulink url='http://www.yoctoproject.org'>Yocto Project</ulink>.
+ </para>
+
+ <para>
+ Prior to BitBake, no other build tool adequately met the needs of
+ an aspiring embedded Linux distribution.
+ All of the build systems used by traditional desktop Linux
+ distributions lacked important functionality, and none of the
+ ad hoc Buildroot-based systems, prevalent in the
+ embedded space, were scalable or maintainable.
+ </para>
+
+ <para>
+ Some important original goals for BitBake were:
+ <itemizedlist>
+ <listitem><para>
+ Handle cross-compilation.
+ </para></listitem>
+ <listitem><para>
+ Handle inter-package dependencies (build time on
+ target architecture, build time on native
+ architecture, and runtime).
+ </para></listitem>
+ <listitem><para>
+ Support running any number of tasks within a given
+ package, including, but not limited to, fetching
+ upstream sources, unpacking them, patching them,
+ configuring them, and so forth.
+ </para></listitem>
+ <listitem><para>
+ Be Linux distribution agnostic for both build and
+ target systems.
+ </para></listitem>
+ <listitem><para>
+ Be architecture agnostic.
+ </para></listitem>
+ <listitem><para>
+ Support multiple build and target operating systems
+ (e.g. Cygwin, the BSDs, and so forth).
+ </para></listitem>
+ <listitem><para>
+ Be self contained, rather than tightly
+ integrated into the build machine's root
+ filesystem.
+ </para></listitem>
+ <listitem><para>
+ Handle conditional metadata on the target architecture,
+ operating system, distribution, and machine.
+ </para></listitem>
+ <listitem><para>
+ Be easy to use the tools to supply local metadata and packages
+ against which to operate.
+ </para></listitem>
+ <listitem><para>
+ Be easy to use BitBake to collaborate between multiple
+ projects for their builds.
+ </para></listitem>
+ <listitem><para>
+ Provide an inheritance mechanism to share
+ common metadata between many packages.
+ </para></listitem>
+ </itemizedlist>
+ Over time it became apparent that some further requirements
+ were necessary:
+ <itemizedlist>
+ <listitem><para>
+ Handle variants of a base recipe (e.g. native, sdk,
+ and multilib).
+ </para></listitem>
+ <listitem><para>
+ Split metadata into layers and allow layers
+ to enhance or override other layers.
+ </para></listitem>
+ <listitem><para>
+ Allow representation of a given set of input variables
+ to a task as a checksum.
+ Based on that checksum, allow acceleration of builds
+ with prebuilt components.
+ </para></listitem>
+ </itemizedlist>
+ BitBake satisfies all the original requirements and many more
+ with extensions being made to the basic functionality to
+ reflect the additional requirements.
+ Flexibility and power have always been the priorities.
+ BitBake is highly extensible and supports embedded Python code and
+ execution of any arbitrary tasks.
+ </para>
+ </section>
+
+ <section id="Concepts">
+ <title>Concepts</title>
+
+ <para>
+ BitBake is a program written in the Python language.
+ At the highest level, BitBake interprets metadata, decides
+ what tasks are required to run, and executes those tasks.
+ Similar to GNU Make, BitBake controls how software is
+ built.
+ GNU Make achieves its control through "makefiles", while
+ BitBake uses "recipes".
+ </para>
+
+ <para>
+ BitBake extends the capabilities of a simple
+ tool like GNU Make by allowing for the definition of much more
+ complex tasks, such as assembling entire embedded Linux
+ distributions.
+ </para>
+
+ <para>
+ The remainder of this section introduces several concepts
+ that should be understood in order to better leverage
+ the power of BitBake.
+ </para>
+
+ <section id='recipes'>
+ <title>Recipes</title>
+
+ <para>
+ BitBake Recipes, which are denoted by the file extension
+ <filename>.bb</filename>, are the most basic metadata files.
+ These recipe files provide BitBake with the following:
+ <itemizedlist>
+ <listitem><para>Descriptive information about the
+ package (author, homepage, license, and so on)</para></listitem>
+ <listitem><para>The version of the recipe</para></listitem>
+ <listitem><para>Existing dependencies (both build
+ and runtime dependencies)</para></listitem>
+ <listitem><para>Where the source code resides and
+ how to fetch it</para></listitem>
+ <listitem><para>Whether the source code requires
+ any patches, where to find them, and how to apply
+ them</para></listitem>
+ <listitem><para>How to configure and compile the
+ source code</para></listitem>
+ <listitem><para>Where on the target machine to install the
+ package or packages created</para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ Within the context of BitBake, or any project utilizing BitBake
+ as its build system, files with the <filename>.bb</filename>
+ extension are referred to as recipes.
+ <note>
+ The term "package" is also commonly used to describe recipes.
+ However, since the same word is used to describe packaged
+ output from a project, it is best to maintain a single
+ descriptive term - "recipes".
+ Put another way, a single "recipe" file is quite capable
+ of generating a number of related but separately installable
+ "packages".
+ In fact, that ability is fairly common.
+ </note>
+ </para>
+ </section>
+
+ <section id='configuration-files'>
+ <title>Configuration Files</title>
+
+ <para>
+ Configuration files, which are denoted by the
+ <filename>.conf</filename> extension, define
+ various configuration variables that govern the project's build
+ process.
+ These files fall into several areas that define
+ machine configuration options, distribution configuration
+ options, compiler tuning options, general common
+ configuration options, and user configuration options.
+ The main configuration file is the sample
+ <filename>bitbake.conf</filename> file, which is
+ located within the BitBake source tree
+ <filename>conf</filename> directory.
+ </para>
+ </section>
+
+ <section id='classes'>
+ <title>Classes</title>
+
+ <para>
+ Class files, which are denoted by the
+ <filename>.bbclass</filename> extension, contain
+ information that is useful to share between metadata files.
+ The BitBake source tree currently comes with one class metadata file
+ called <filename>base.bbclass</filename>.
+ You can find this file in the
+ <filename>classes</filename> directory.
+ The <filename>base.bbclass</filename> class files is special since it
+ is always included automatically for all recipes
+ and classes.
+ This class contains definitions for standard basic tasks such
+ as fetching, unpacking, configuring (empty by default),
+ compiling (runs any Makefile present), installing (empty by
+ default) and packaging (empty by default).
+ These tasks are often overridden or extended by other classes
+ added during the project development process.
+ </para>
+ </section>
+
+ <section id='layers'>
+ <title>Layers</title>
+
+ <para>
+ Layers allow you to isolate different types of
+ customizations from each other.
+ While you might find it tempting to keep everything in one layer
+ when working on a single project, the more modular you organize
+ your metadata, the easier it is to cope with future changes.
+ </para>
+
+ <para>
+ To illustrate how you can use layers to keep things modular,
+ consider customizations you might make to support a specific target machine.
+ These types of customizations typically reside in a special layer,
+ rather than a general layer, called a Board Support Package (BSP)
+ Layer.
+ Furthermore, the machine customizations should be isolated from
+ recipes and metadata that support a new GUI environment, for
+ example.
+ This situation gives you a couple of layers: one for the machine
+ configurations and one for the GUI environment.
+ It is important to understand, however, that the BSP layer can still
+ make machine-specific additions to recipes within
+ the GUI environment layer without polluting the GUI layer itself
+ with those machine-specific changes.
+ You can accomplish this through a recipe that is a BitBake append
+ (<filename>.bbappend</filename>) file.
+ </para>
+ </section>
+
+ <section id='append-bbappend-files'>
+ <title>Append Files</title>
+
+ <para>
+ Append files, which are files that have the
+ <filename>.bbappend</filename> file extension, extend or
+ override information in an existing recipe file.
+ </para>
+
+ <para>
+ BitBake expects every append file to have a corresponding recipe file.
+ Furthermore, the append file and corresponding recipe file
+ must use the same root filename.
+ The filenames can differ only in the file type suffix used
+ (e.g. <filename>formfactor_0.0.bb</filename> and
+ <filename>formfactor_0.0.bbappend</filename>).
+ </para>
+
+ <para>
+ Information in append files extends or
+ overrides the information in the underlying,
+ similarly-named recipe files.
+ </para>
+
+ <para>
+ When you name an append file, you can use the
+ wildcard character (%) to allow for matching recipe names.
+ For example, suppose you have an append file named
+ as follows:
+ <literallayout class='monospaced'>
+ busybox_1.21.%.bbappend
+ </literallayout>
+ That append file would match any <filename>busybox_1.21.x.bb</filename>
+ version of the recipe.
+ So, the append file would match the following recipe names:
+ <literallayout class='monospaced'>
+ busybox_1.21.1.bb
+ busybox_1.21.2.bb
+ busybox_1.21.3.bb
+ </literallayout>
+ If the <filename>busybox</filename> recipe was updated to
+ <filename>busybox_1.3.0.bb</filename>, the append name would not
+ match.
+ However, if you named the append file
+ <filename>busybox_1.%.bbappend</filename>, then you would have a match.
+ </para>
+
+ <para>
+ In the most general case, you could name the append file something as
+ simple as <filename>busybox_%.bbappend</filename> to be entirely
+ version independent.
+ </para>
+ </section>
+ </section>
+
+ <section id='obtaining-bitbake'>
+ <title>Obtaining BitBake</title>
+
+ <para>
+ You can obtain BitBake several different ways:
+ <itemizedlist>
+ <listitem><para><emphasis>Cloning BitBake:</emphasis>
+ Using Git to clone the BitBake source code repository
+ is the recommended method for obtaining BitBake.
+ Cloning the repository makes it easy to get bug fixes
+ and have access to stable branches and the master
+ branch.
+ Once you have cloned BitBake, you should use
+ the latest stable
+ branch for development since the master branch is for
+ BitBake development and might contain less stable changes.
+ </para>
+ <para>You usually need a version of BitBake
+ that matches the metadata you are using.
+ The metadata is generally backwards compatible but
+ not forward compatible.</para>
+ <para>Here is an example that clones the BitBake repository:
+ <literallayout class='monospaced'>
+ $ git clone git://git.openembedded.org/bitbake
+ </literallayout>
+ This command clones the BitBake Git repository into a
+ directory called <filename>bitbake</filename>.
+ Alternatively, you can
+ designate a directory after the
+ <filename>git clone</filename> command
+ if you want to call the new directory something
+ other than <filename>bitbake</filename>.
+ Here is an example that names the directory
+ <filename>bbdev</filename>:
+ <literallayout class='monospaced'>
+ $ git clone git://git.openembedded.org/bitbake bbdev
+ </literallayout></para></listitem>
+ <listitem><para><emphasis>Installation using your Distribution
+ Package Management System:</emphasis>
+ This method is not
+ recommended because the BitBake version that is
+ provided by your distribution, in most cases,
+ is several
+ releases behind a snapshot of the BitBake repository.
+ </para></listitem>
+ <listitem><para><emphasis>Taking a snapshot of BitBake:</emphasis>
+ Downloading a snapshot of BitBake from the
+ source code repository gives you access to a known
+ branch or release of BitBake.
+ <note>
+ Cloning the Git repository, as described earlier,
+ is the preferred method for getting BitBake.
+ Cloning the repository makes it easier to update as
+ patches are added to the stable branches.
+ </note></para>
+ <para>The following example downloads a snapshot of
+ BitBake version 1.17.0:
+ <literallayout class='monospaced'>
+ $ wget http://git.openembedded.org/bitbake/snapshot/bitbake-1.17.0.tar.gz
+ $ tar zxpvf bitbake-1.17.0.tar.gz
+ </literallayout>
+ After extraction of the tarball using the tar utility,
+ you have a directory entitled
+ <filename>bitbake-1.17.0</filename>.
+ </para></listitem>
+ <listitem><para><emphasis>Using the BitBake that Comes With Your
+ Build Checkout:</emphasis>
+ A final possibility for getting a copy of BitBake is that it
+ already comes with your checkout of a larger Bitbake-based build
+ system, such as Poky or Yocto Project.
+ Rather than manually checking out individual layers and
+ gluing them together yourself, you can check
+ out an entire build system.
+ The checkout will already include a version of BitBake that
+ has been thoroughly tested for compatibility with the other
+ components.
+ For information on how to check out a particular BitBake-based
+ build system, consult that build system's supporting documentation.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </section>
+
+ <section id="bitbake-user-manual-command">
+ <title>The BitBake Command</title>
+
+ <para>
+ The <filename>bitbake</filename> command is the primary interface
+ to the BitBake tool.
+ This section presents the BitBake command syntax and provides
+ several execution examples.
+ </para>
+
+ <section id='usage-and-syntax'>
+ <title>Usage and syntax</title>
+
+ <para>
+ Following is the usage and syntax for BitBake:
+ <literallayout class='monospaced'>
+ $ bitbake -h
+ Usage: bitbake [options] [recipename/target ...]
+
+ Executes the specified task (default is 'build') for a given set of target recipes (.bb files).
+ It is assumed there is a conf/bblayers.conf available in cwd or in BBPATH which
+ will provide the layer, BBFILES and other configuration information.
+
+ Options:
+ --version show program's version number and exit
+ -h, --help show this help message and exit
+ -b BUILDFILE, --buildfile=BUILDFILE
+ Execute tasks from a specific .bb recipe directly.
+ WARNING: Does not handle any dependencies from other
+ recipes.
+ -k, --continue Continue as much as possible after an error. While the
+ target that failed and anything depending on it cannot
+ be built, as much as possible will be built before
+ stopping.
+ -a, --tryaltconfigs Continue with builds by trying to use alternative
+ providers where possible.
+ -f, --force Force the specified targets/task to run (invalidating
+ any existing stamp file).
+ -c CMD, --cmd=CMD Specify the task to execute. The exact options
+ available depend on the metadata. Some examples might
+ be 'compile' or 'populate_sysroot' or 'listtasks' may
+ give a list of the tasks available.
+ -C INVALIDATE_STAMP, --clear-stamp=INVALIDATE_STAMP
+ Invalidate the stamp for the specified task such as
+ 'compile' and then run the default task for the
+ specified target(s).
+ -r PREFILE, --read=PREFILE
+ Read the specified file before bitbake.conf.
+ -R POSTFILE, --postread=POSTFILE
+ Read the specified file after bitbake.conf.
+ -v, --verbose Output more log message data to the terminal.
+ -D, --debug Increase the debug level. You can specify this more
+ than once.
+ -n, --dry-run Don't execute, just go through the motions.
+ -S SIGNATURE_HANDLER, --dump-signatures=SIGNATURE_HANDLER
+ Dump out the signature construction information, with
+ no task execution. The SIGNATURE_HANDLER parameter is
+ passed to the handler. Two common values are none and
+ printdiff but the handler may define more/less. none
+ means only dump the signature, printdiff means compare
+ the dumped signature with the cached one.
+ -p, --parse-only Quit after parsing the BB recipes.
+ -s, --show-versions Show current and preferred versions of all recipes.
+ -e, --environment Show the global or per-recipe environment complete
+ with information about where variables were
+ set/changed.
+ -g, --graphviz Save dependency tree information for the specified
+ targets in the dot syntax.
+ -I EXTRA_ASSUME_PROVIDED, --ignore-deps=EXTRA_ASSUME_PROVIDED
+ Assume these dependencies don't exist and are already
+ provided (equivalent to ASSUME_PROVIDED). Useful to
+ make dependency graphs more appealing
+ -l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS
+ Show debug logging for the specified logging domains
+ -P, --profile Profile the command and save reports.
+ -u UI, --ui=UI The user interface to use (e.g. knotty, hob, depexp).
+ -t SERVERTYPE, --servertype=SERVERTYPE
+ Choose which server to use, process or xmlrpc.
+ --token=XMLRPCTOKEN Specify the connection token to be used when
+ connecting to a remote server.
+ --revisions-changed Set the exit code depending on whether upstream
+ floating revisions have changed or not.
+ --server-only Run bitbake without a UI, only starting a server
+ (cooker) process.
+ -B BIND, --bind=BIND The name/address for the bitbake server to bind to.
+ --no-setscene Do not run any setscene tasks. sstate will be ignored
+ and everything needed, built.
+ --remote-server=REMOTE_SERVER
+ Connect to the specified server.
+ -m, --kill-server Terminate the remote server.
+ --observe-only Connect to a server as an observing-only client.
+ --status-only Check the status of the remote bitbake server.
+ </literallayout>
+ </para>
+ </section>
+
+ <section id='bitbake-examples'>
+ <title>Examples</title>
+
+ <para>
+ This section presents some examples showing how to use BitBake.
+ </para>
+
+ <section id='example-executing-a-task-against-a-single-recipe'>
+ <title>Executing a Task Against a Single Recipe</title>
+
+ <para>
+ Executing tasks for a single recipe file is relatively simple.
+ You specify the file in question, and BitBake parses
+ it and executes the specified task.
+ If you do not specify a task, BitBake executes the default
+ task, which is "build”.
+ BitBake obeys inter-task dependencies when doing
+ so.
+ </para>
+
+ <para>
+ The following command runs the build task, which is
+ the default task, on the <filename>foo_1.0.bb</filename>
+ recipe file:
+ <literallayout class='monospaced'>
+ $ bitbake -b foo_1.0.bb
+ </literallayout>
+ The following command runs the clean task on the
+ <filename>foo.bb</filename> recipe file:
+ <literallayout class='monospaced'>
+ $ bitbake -b foo.bb -c clean
+ </literallayout>
+ <note>
+ The "-b" option explicitly does not handle recipe
+ dependencies.
+ Other than for debugging purposes, it is instead
+ recommended that you use the syntax presented in the
+ next section.
+ </note>
+ </para>
+ </section>
+
+ <section id='executing-tasks-against-a-set-of-recipe-files'>
+ <title>Executing Tasks Against a Set of Recipe Files</title>
+
+ <para>
+ There are a number of additional complexities introduced
+ when one wants to manage multiple <filename>.bb</filename>
+ files.
+ Clearly there needs to be a way to tell BitBake what
+ files are available and, of those, which you
+ want to execute.
+ There also needs to be a way for each recipe
+ to express its dependencies, both for build-time and
+ runtime.
+ There must be a way for you to express recipe preferences
+ when multiple recipes provide the same functionality, or when
+ there are multiple versions of a recipe.
+ </para>
+
+ <para>
+ The <filename>bitbake</filename> command, when not using
+ "--buildfile" or "-b" only accepts a "PROVIDES".
+ You cannot provide anything else.
+ By default, a recipe file generally "PROVIDES" its
+ "packagename" as shown in the following example:
+ <literallayout class='monospaced'>
+ $ bitbake foo
+ </literallayout>
+ This next example "PROVIDES" the package name and also uses
+ the "-c" option to tell BitBake to just execute the
+ <filename>do_clean</filename> task:
+ <literallayout class='monospaced'>
+ $ bitbake -c clean foo
+ </literallayout>
+ </para>
+ </section>
+
+ <section id='generating-dependency-graphs'>
+ <title>Generating Dependency Graphs</title>
+
+ <para>
+ BitBake is able to generate dependency graphs using
+ the <filename>dot</filename> syntax.
+ You can convert these graphs into images using the
+ <filename>dot</filename> tool from
+ <ulink url='http://www.graphviz.org'>Graphviz</ulink>.
+ </para>
+
+ <para>
+ When you generate a dependency graph, BitBake writes four files
+ to the current working directory:
+ <itemizedlist>
+ <listitem><para><emphasis><filename>package-depends.dot</filename>:</emphasis>
+ Shows BitBake's knowledge of dependencies between
+ runtime targets.
+ </para></listitem>
+ <listitem><para><emphasis><filename>pn-depends.dot</filename>:</emphasis>
+ Shows dependencies between build-time targets
+ (i.e. recipes).
+ </para></listitem>
+ <listitem><para><emphasis><filename>task-depends.dot</filename>:</emphasis>
+ Shows dependencies between tasks.
+ </para></listitem>
+ <listitem><para><emphasis><filename>pn-buildlist</filename>:</emphasis>
+ Shows a simple list of targets that are to be built.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ To stop depending on common depends, use the "-I" depend
+ option and BitBake omits them from the graph.
+ Leaving this information out can produce more readable graphs.
+ This way, you can remove from the graph
+ <filename>DEPENDS</filename> from inherited classes
+ such as <filename>base.bbclass</filename>.
+ </para>
+
+ <para>
+ Here are two examples that create dependency graphs.
+ The second example omits depends common in OpenEmbedded from
+ the graph:
+ <literallayout class='monospaced'>
+ $ bitbake -g foo
+
+ $ bitbake -g -I virtual/kernel -I eglibc foo
+ </literallayout>
+ </para>
+ </section>
+ </section>
+ </section>
+</chapter>