blob: 3d9c7c66fd31fedd99432e6817c7a9d974028688 [file] [log] [blame]
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
<appendix id='kernel-dev-maint-appx'>
<title>Kernel Maintenance</title>
<section id='tree-construction'>
<title>Tree Construction</title>
<para>
This section describes construction of the Yocto Project kernel
source repositories as accomplished by the Yocto Project team to
create Yocto Linux kernel repositories.
These kernel repositories are found under the heading "Yocto Linux
Kernel" at
<ulink url='&YOCTO_GIT_URL;'>&YOCTO_GIT_URL;</ulink>
and are shipped as part of a Yocto Project release.
The team creates these repositories by compiling and executing the
set of feature descriptions for every BSP and feature in the
product.
Those feature descriptions list all necessary patches,
configurations, branches, tags, and feature divisions found in a
Yocto Linux kernel.
Thus, the Yocto Project Linux kernel repository (or tree) and
accompanying Metadata in the
<filename>yocto-kernel-cache</filename> are built.
</para>
<para>
The existence of these repositories allow you to access and clone a
particular Yocto Project Linux kernel repository and use it to
build images based on their configurations and features.
</para>
<para>
You can find the files used to describe all the valid features and
BSPs in the Yocto Project Linux kernel in any clone of the Yocto
Project Linux kernel source repository and
<filename>yocto-kernel-cache</filename> Git trees.
For example, the following commands clone the Yocto Project
baseline Linux kernel that branches off
<filename>linux.org</filename> version 4.12 and the
<filename>yocto-kernel-cache</filename>, which contains stores of
kernel Metadata:
<literallayout class='monospaced'>
$ git clone git://git.yoctoproject.org/linux-yocto-4.12
$ git clone git://git.yoctoproject.org/linux-kernel-cache
</literallayout>
For more information on how to set up a local Git repository of
the Yocto Project Linux kernel files, see the
"<link linkend='preparing-the-build-host-to-work-on-the-kernel'>Preparing the Build Host to Work on the Kernel</link>"
section.
</para>
<para>
Once you have cloned the kernel Git repository and the
cache of Metadata on your local machine, you can discover the
branches that are available in the repository using the following
Git command:
<literallayout class='monospaced'>
$ git branch -a
</literallayout>
Checking out a branch allows you to work with a particular
Yocto Linux kernel.
For example, the following commands check out the
"standard/beagleboard" branch of the Yocto Linux kernel repository
and the "yocto-4.12" branch of the
<filename>yocto-kernel-cache</filename> repository:
<literallayout class='monospaced'>
$ cd ~/linux-yocto-4.12
$ git checkout -b my-kernel-4.12 remotes/origin/standard/beagleboard
$ cd ~/linux-kernel-cache
$ git checkout -b my-4.12-metadata remotes/origin/yocto-4.12
</literallayout>
<note>
Branches in the <filename>yocto-kernel-cache</filename>
repository correspond to Yocto Linux kernel versions
(e.g. "yocto-4.12", "yocto-4.10", "yocto-4.9", and so forth).
</note>
Once you have checked out and switched to appropriate branches,
you can see a snapshot of all the kernel source files used to
used to build that particular Yocto Linux kernel for a
particular board.
</para>
<para>
To see the features and configurations for a particular Yocto
Linux kernel, you need to examine the
<filename>yocto-kernel-cache</filename> Git repository.
As mentioned, branches in the
<filename>yocto-kernel-cache</filename> repository correspond to
Yocto Linux kernel versions (e.g. <filename>yocto-4.12</filename>).
Branches contain descriptions in the form of
<filename>.scc</filename> and <filename>.cfg</filename> files.
</para>
<para>
You should realize, however, that browsing your local
<filename>yocto-kernel-cache</filename> repository for feature
descriptions and patches is not an effective way to determine what
is in a particular kernel branch.
Instead, you should use Git directly to discover the changes in
a branch.
Using Git is an efficient and flexible way to inspect changes to
the kernel.
<note>
Ground up reconstruction of the complete kernel tree is an
action only taken by the Yocto Project team during an active
development cycle.
When you create a clone of the kernel Git repository, you are
simply making it efficiently available for building and
development.
</note>
</para>
<para>
The following steps describe what happens when the Yocto Project
Team constructs the Yocto Project kernel source Git repository
(or tree) found at
<ulink url='&YOCTO_GIT_URL;'></ulink> given the
introduction of a new top-level kernel feature or BSP.
The following actions effectively provide the Metadata
and create the tree that includes the new feature, patch, or BSP:
<orderedlist>
<listitem><para>
<emphasis>Pass Feature to the OpenEmbedded Build System:</emphasis>
A top-level kernel feature is passed to the kernel build
subsystem.
Normally, this feature is a BSP for a particular kernel
type.
</para></listitem>
<listitem><para>
<emphasis>Locate Feature:</emphasis>
The file that describes the top-level feature is located
by searching these system directories:
<itemizedlist>
<listitem><para>
The in-tree kernel-cache directories, which are
located in the
<ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/yocto-kernel-cache/tree/bsp'><filename>yocto-kernel-cache</filename></ulink>
repository organized under the "Yocto Linux Kernel"
heading in the
<ulink url='http://git.yoctoproject.org/cgit/cgit.cgi'>Yocto Project Source Repositories</ulink>.
</para></listitem>
<listitem><para>
Areas pointed to by <filename>SRC_URI</filename>
statements found in kernel recipes
</para></listitem>
</itemizedlist>
For a typical build, the target of the search is a
feature description in an <filename>.scc</filename> file
whose name follows this format (e.g.
<filename>beaglebone-standard.scc</filename> and
<filename>beaglebone-preempt-rt.scc</filename>):
<literallayout class='monospaced'>
<replaceable>bsp_root_name</replaceable>-<replaceable>kernel_type</replaceable>.scc
</literallayout>
</para></listitem>
<listitem><para>
<emphasis>Expand Feature:</emphasis>
Once located, the feature description is either expanded
into a simple script of actions, or into an existing
equivalent script that is already part of the shipped
kernel.
</para></listitem>
<listitem><para>
<emphasis>Append Extra Features:</emphasis>
Extra features are appended to the top-level feature
description.
These features can come from the
<ulink url='&YOCTO_DOCS_REF_URL;#var-KERNEL_FEATURES'><filename>KERNEL_FEATURES</filename></ulink>
variable in recipes.
</para></listitem>
<listitem><para>
<emphasis>Locate, Expand, and Append Each Feature:</emphasis>
Each extra feature is located, expanded and appended to
the script as described in step three.
</para></listitem>
<listitem><para>
<emphasis>Execute the Script:</emphasis>
The script is executed to produce files
<filename>.scc</filename> and <filename>.cfg</filename>
files in appropriate directories of the
<filename>yocto-kernel-cache</filename> repository.
These files are descriptions of all the branches, tags,
patches and configurations that need to be applied to the
base Git repository to completely create the
source (build) branch for the new BSP or feature.
</para></listitem>
<listitem><para>
<emphasis>Clone Base Repository:</emphasis>
The base repository is cloned, and the actions
listed in the <filename>yocto-kernel-cache</filename>
directories are applied to the tree.
</para></listitem>
<listitem><para>
<emphasis>Perform Cleanup:</emphasis>
The Git repositories are left with the desired branches
checked out and any required branching, patching and
tagging has been performed.
</para></listitem>
</orderedlist>
</para>
<para>
The kernel tree and cache are ready for developer consumption to
be locally cloned, configured, and built into a Yocto Project
kernel specific to some target hardware.
<note><title>Notes</title>
<itemizedlist>
<listitem><para>
The generated <filename>yocto-kernel-cache</filename>
repository adds to the kernel as shipped with the Yocto
Project release.
Any add-ons and configuration data are applied to the
end of an existing branch.
The full repository generation that is found in the
official Yocto Project kernel repositories at
<ulink url='&YOCTO_GIT_URL;'>http://git.yoctoproject.org</ulink>
is the combination of all supported boards and
configurations.
</para></listitem>
<listitem><para>
The technique the Yocto Project team uses is flexible
and allows for seamless blending of an immutable
history with additional patches specific to a
deployment.
Any additions to the kernel become an integrated part
of the branches.
</para></listitem>
<listitem><para>
The full kernel tree that you see on
<ulink url='&YOCTO_GIT_URL;'></ulink> is
generated through repeating the above steps for all
valid BSPs.
The end result is a branched, clean history tree that
makes up the kernel for a given release.
You can see the script (<filename>kgit-scc</filename>)
responsible for this in the
<ulink url='&YOCTO_GIT_URL;/cgit.cgi/yocto-kernel-tools/tree/tools'><filename>yocto-kernel-tools</filename></ulink>
repository.
</para></listitem>
<listitem><para>
The steps used to construct the full kernel tree are
the same steps that BitBake uses when it builds a
kernel image.
</para></listitem>
</itemizedlist>
</note>
</para>
</section>
<section id='build-strategy'>
<title>Build Strategy</title>
<para>
Once you have cloned a Yocto Linux kernel repository and the
cache repository (<filename>yocto-kernel-cache</filename>) onto
your development system, you can consider the compilation phase
of kernel development, which is building a kernel image.
Some prerequisites exist that are validated by the build process
before compilation starts:
</para>
<itemizedlist>
<listitem><para>
The
<ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
points to the kernel Git repository.
</para></listitem>
<listitem><para>
A BSP build branch with Metadata exists in the
<filename>yocto-kernel-cache</filename> repository.
The branch is based on the Yocto Linux kernel version and
has configurations and features grouped under the
<filename>yocto-kernel-cache/bsp</filename> directory.
For example, features and configurations for the
BeagleBone Board assuming a
<filename>linux-yocto_4.12</filename> kernel reside in the
following area of the <filename>yocto-kernel-cache</filename>
repository:
<literallayout class='monospaced'>
yocto-kernel-cache/bsp/beaglebone
</literallayout>
<note>
In the previous example, the "yocto-4.12" branch is
checked out in the <filename>yocto-kernel-cache</filename>
repository.
</note>
</para></listitem>
</itemizedlist>
<para>
The OpenEmbedded build system makes sure these conditions exist
before attempting compilation.
Other means, however, do exist, such as as bootstrapping a BSP.
</para>
<para>
Before building a kernel, the build process verifies the tree
and configures the kernel by processing all of the
configuration "fragments" specified by feature descriptions
in the <filename>.scc</filename> files.
As the features are compiled, associated kernel configuration
fragments are noted and recorded in the series of directories
in their compilation order.
The fragments are migrated, pre-processed and passed to the
Linux Kernel Configuration subsystem (<filename>lkc</filename>) as
raw input in the form of a <filename>.config</filename> file.
The <filename>lkc</filename> uses its own internal dependency
constraints to do the final processing of that information and
generates the final <filename>.config</filename> file that is used
during compilation.
</para>
<para>
Using the board's architecture and other relevant values from
the board's template, kernel compilation is started and a kernel
image is produced.
</para>
<para>
The other thing that you notice once you configure a kernel is that
the build process generates a build tree that is separate from
your kernel's local Git source repository tree.
This build tree has a name that uses the following form, where
<filename>${MACHINE}</filename> is the metadata name of the
machine (BSP) and "kernel_type" is one of the Yocto Project
supported kernel types (e.g. "standard"):
<literallayout class='monospaced'>
linux-${MACHINE}-<replaceable>kernel_type</replaceable>-build
</literallayout>
</para>
<para>
The existing support in the <filename>kernel.org</filename> tree
achieves this default functionality.
</para>
<para>
This behavior means that all the generated files for a particular
machine or BSP are now in the build tree directory.
The files include the final <filename>.config</filename> file,
all the <filename>.o</filename> files, the <filename>.a</filename>
files, and so forth.
Since each machine or BSP has its own separate
<ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>
in its own separate branch of the Git repository, you can easily
switch between different builds.
</para>
</section>
</appendix>
<!--
vim: expandtab tw=80 ts=4
-->