poky/documentation/kernel-dev/kernel-dev-maint-appx.xml

<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
<!--SPDX-License-Identifier: CC-BY-2.0-UK-->

<appendix id='kernel-dev-maint-appx'>
<title>Kernel Maintenance</title>

    <section id='tree-construction'>
        <title>Tree Construction</title>

        <para>
            This section describes construction of the Yocto Project kernel
            source repositories as accomplished by the Yocto Project team to
            create Yocto Linux kernel repositories.
            These kernel repositories are found under the heading "Yocto Linux
            Kernel" at
            <ulink url='&YOCTO_GIT_URL;'>&YOCTO_GIT_URL;</ulink>
            and are shipped as part of a Yocto Project release.
            The team creates these repositories by compiling and executing the
            set of feature descriptions for every BSP and feature in the
            product.
            Those feature descriptions list all necessary patches,
            configurations, branches, tags, and feature divisions found in a
            Yocto Linux kernel.
            Thus, the Yocto Project Linux kernel repository (or tree) and
            accompanying Metadata in the
            <filename>yocto-kernel-cache</filename> are built.
        </para>

        <para>
            The existence of these repositories allow you to access and clone a
            particular Yocto Project Linux kernel repository and use it to
            build images based on their configurations and features.
        </para>

        <para>
            You can find the files used to describe all the valid features and
            BSPs in the Yocto Project Linux kernel in any clone of the Yocto
            Project Linux kernel source repository and
            <filename>yocto-kernel-cache</filename> Git trees.
            For example, the following commands clone the Yocto Project
            baseline Linux kernel that branches off
            <filename>linux.org</filename> version 4.12 and the
            <filename>yocto-kernel-cache</filename>, which contains stores of
            kernel Metadata:
            <literallayout class='monospaced'>
     $ git clone git://git.yoctoproject.org/linux-yocto-4.12
     $ git clone git://git.yoctoproject.org/linux-kernel-cache
            </literallayout>
            For more information on how to set up a local Git repository of
            the Yocto Project Linux kernel files, see the
            "<link linkend='preparing-the-build-host-to-work-on-the-kernel'>Preparing the Build Host to Work on the Kernel</link>"
            section.
        </para>

        <para>
            Once you have cloned the kernel Git repository and the
            cache of Metadata on your local machine, you can discover the
            branches that are available in the repository using the following
            Git command:
            <literallayout class='monospaced'>
     $ git branch -a
            </literallayout>
            Checking out a branch allows you to work with a particular
            Yocto Linux kernel.
            For example, the following commands check out the
            "standard/beagleboard" branch of the Yocto Linux kernel repository
            and the "yocto-4.12" branch of the
            <filename>yocto-kernel-cache</filename> repository:
            <literallayout class='monospaced'>
     $ cd ~/linux-yocto-4.12
     $ git checkout -b my-kernel-4.12 remotes/origin/standard/beagleboard
     $ cd ~/linux-kernel-cache
     $ git checkout -b my-4.12-metadata remotes/origin/yocto-4.12
            </literallayout>
            <note>
                Branches in the <filename>yocto-kernel-cache</filename>
                repository correspond to Yocto Linux kernel versions
                (e.g. "yocto-4.12", "yocto-4.10", "yocto-4.9", and so forth).
            </note>
            Once you have checked out and switched to appropriate branches,
            you can see a snapshot of all the kernel source files used to
            used to build that particular Yocto Linux kernel for a
            particular board.
        </para>

        <para>
            To see the features and configurations for a particular Yocto
            Linux kernel, you need to examine the
            <filename>yocto-kernel-cache</filename> Git repository.
            As mentioned, branches in the
            <filename>yocto-kernel-cache</filename> repository correspond to
            Yocto Linux kernel versions (e.g. <filename>yocto-4.12</filename>).
            Branches contain descriptions in the form of
            <filename>.scc</filename> and <filename>.cfg</filename> files.
        </para>

        <para>
            You should realize, however, that browsing your local
            <filename>yocto-kernel-cache</filename> repository for feature
            descriptions and patches is not an effective way to determine what
            is in a particular kernel branch.
            Instead, you should use Git directly to discover the changes in
            a branch.
            Using Git is an efficient and flexible way to inspect changes to
            the kernel.
            <note>
                Ground up reconstruction of the complete kernel tree is an
                action only taken by the Yocto Project team during an active
                development cycle.
                When you create a clone of the kernel Git repository, you are
                simply making it efficiently available for building and
                development.
            </note>
        </para>

        <para>
            The following steps describe what happens when the Yocto Project
            Team constructs the Yocto Project kernel source Git repository
            (or tree) found at
            <ulink url='&YOCTO_GIT_URL;'></ulink> given the
            introduction of a new top-level kernel feature or BSP.
            The following actions effectively provide the Metadata
            and create the tree that includes the new feature, patch, or BSP:
            <orderedlist>
                <listitem><para>
                    <emphasis>Pass Feature to the OpenEmbedded Build System:</emphasis>
                    A top-level kernel feature is passed to the kernel build
                    subsystem.
                    Normally, this feature is a BSP for a particular kernel
                    type.
                    </para></listitem>
                <listitem><para>
                    <emphasis>Locate Feature:</emphasis>
                    The file that describes the top-level feature is located
                    by searching these system directories:
                    <itemizedlist>
                        <listitem><para>
                            The in-tree kernel-cache directories, which are
                            located in the
                            <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/yocto-kernel-cache/tree/bsp'><filename>yocto-kernel-cache</filename></ulink>
                            repository organized under the "Yocto Linux Kernel"
                            heading in the
                            <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi'>Yocto Project Source Repositories</ulink>.
                            </para></listitem>
                        <listitem><para>
                            Areas pointed to by <filename>SRC_URI</filename>
                            statements found in kernel recipes
                            </para></listitem>
                    </itemizedlist>
                    For a typical build, the target of the search is a
                    feature description in an <filename>.scc</filename> file
                    whose name follows this format (e.g.
                    <filename>beaglebone-standard.scc</filename> and
                    <filename>beaglebone-preempt-rt.scc</filename>):
                    <literallayout class='monospaced'>
     <replaceable>bsp_root_name</replaceable>-<replaceable>kernel_type</replaceable>.scc
                    </literallayout>
                </para></listitem>
                <listitem><para>
                    <emphasis>Expand Feature:</emphasis>
                    Once located, the feature description is either expanded
                    into a simple script of actions, or into an existing
                    equivalent script that is already part of the shipped
                    kernel.
                    </para></listitem>
                <listitem><para>
                    <emphasis>Append Extra Features:</emphasis>
                    Extra features are appended to the top-level feature
                    description.
                    These features can come from the
                    <ulink url='&YOCTO_DOCS_REF_URL;#var-KERNEL_FEATURES'><filename>KERNEL_FEATURES</filename></ulink>
                    variable in recipes.
                    </para></listitem>
                <listitem><para>
                    <emphasis>Locate, Expand, and Append Each Feature:</emphasis>
                    Each extra feature is located, expanded and appended to
                    the script as described in step three.
                    </para></listitem>
                <listitem><para>
                    <emphasis>Execute the Script:</emphasis>
                    The script is executed to produce files
                    <filename>.scc</filename> and <filename>.cfg</filename>
                    files in appropriate directories of the
                    <filename>yocto-kernel-cache</filename> repository.
                    These files are descriptions of all the branches, tags,
                    patches and configurations that need to be applied to the
                    base Git repository to completely create the
                    source (build) branch for the new BSP or feature.
                    </para></listitem>
                <listitem><para>
                    <emphasis>Clone Base Repository:</emphasis>
                    The base repository is cloned, and the actions
                    listed in the <filename>yocto-kernel-cache</filename>
                    directories are applied to the tree.
                    </para></listitem>
                <listitem><para>
                    <emphasis>Perform Cleanup:</emphasis>
                    The Git repositories are left with the desired branches
                    checked out and any required branching, patching and
                    tagging has been performed.
                    </para></listitem>
            </orderedlist>
        </para>

        <para>
            The kernel tree and cache are ready for developer consumption to
            be locally cloned, configured, and built into a Yocto Project
            kernel specific to some target hardware.
            <note><title>Notes</title>
                <itemizedlist>
                    <listitem><para>
                        The generated <filename>yocto-kernel-cache</filename>
                        repository adds to the kernel as shipped with the Yocto
                        Project release.
                        Any add-ons and configuration data are applied to the
                        end of an existing branch.
                        The full repository generation that is found in the
                        official Yocto Project kernel repositories at
                        <ulink url='&YOCTO_GIT_URL;'>http://git.yoctoproject.org</ulink>
                        is the combination of all supported boards and
                        configurations.
                        </para></listitem>
                    <listitem><para>
                        The technique the Yocto Project team uses is flexible
                        and allows for seamless blending of an immutable
                        history with additional patches specific to a
                        deployment.
                        Any additions to the kernel become an integrated part
                        of the branches.
                        </para></listitem>
                    <listitem><para>
                        The full kernel tree that you see on
                        <ulink url='&YOCTO_GIT_URL;'></ulink> is
                        generated through repeating the above steps for all
                        valid BSPs.
                        The end result is a branched, clean history tree that
                        makes up the kernel for a given release.
                        You can see the script (<filename>kgit-scc</filename>)
                        responsible for this in the
                        <ulink url='&YOCTO_GIT_URL;/cgit.cgi/yocto-kernel-tools/tree/tools'><filename>yocto-kernel-tools</filename></ulink>
                        repository.
                        </para></listitem>
                    <listitem><para>
                        The steps used to construct the full kernel tree are
                        the same steps that BitBake uses when it builds a
                        kernel image.
                        </para></listitem>
                </itemizedlist>
            </note>
        </para>
    </section>

    <section id='build-strategy'>
        <title>Build Strategy</title>

        <para>
            Once you have cloned a Yocto Linux kernel repository and the
            cache repository (<filename>yocto-kernel-cache</filename>) onto
            your development system, you can consider the compilation phase
            of kernel development, which is building a kernel image.
            Some prerequisites exist that are validated by the build process
            before compilation starts:
        </para>

        <itemizedlist>
            <listitem><para>
                The
                <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
                points to the kernel Git repository.
                </para></listitem>
            <listitem><para>
                A BSP build branch with Metadata exists in the
                <filename>yocto-kernel-cache</filename> repository.
                The branch is based on the Yocto Linux kernel version and
                has configurations and features grouped under the
                <filename>yocto-kernel-cache/bsp</filename> directory.
                For example, features and configurations for the
                BeagleBone Board assuming a
                <filename>linux-yocto_4.12</filename> kernel reside in the
                following area of the <filename>yocto-kernel-cache</filename>
                repository:
                <literallayout class='monospaced'>
     yocto-kernel-cache/bsp/beaglebone
                </literallayout>
                <note>
                    In the previous example, the "yocto-4.12" branch is
                    checked out in the <filename>yocto-kernel-cache</filename>
                    repository.
                </note>
                </para></listitem>
        </itemizedlist>

        <para>
            The OpenEmbedded build system makes sure these conditions exist
            before attempting compilation.
            Other means, however, do exist, such as as bootstrapping a BSP.
        </para>

        <para>
            Before building a kernel, the build process verifies the tree
            and configures the kernel by processing all of the
            configuration "fragments" specified by feature descriptions
            in the <filename>.scc</filename> files.
            As the features are compiled, associated kernel configuration
            fragments are noted and recorded in the series of directories
            in their compilation order.
            The fragments are migrated, pre-processed and passed to the
            Linux Kernel Configuration subsystem (<filename>lkc</filename>) as
            raw input in the form of a <filename>.config</filename> file.
            The <filename>lkc</filename> uses its own internal dependency
            constraints to do the final processing of that information and
            generates the final <filename>.config</filename> file that is used
            during compilation.
        </para>

        <para>
            Using the board's architecture and other relevant values from
            the board's template, kernel compilation is started and a kernel
            image is produced.
        </para>

        <para>
            The other thing that you notice once you configure a kernel is that
            the build process generates a build tree that is separate from
            your kernel's local Git source repository tree.
            This build tree has a name that uses the following form, where
            <filename>${MACHINE}</filename> is the metadata name of the
            machine (BSP) and "kernel_type" is one of the Yocto Project
            supported kernel types (e.g. "standard"):
        <literallayout class='monospaced'>
     linux-${MACHINE}-<replaceable>kernel_type</replaceable>-build
        </literallayout>
        </para>

        <para>
            The existing support in the <filename>kernel.org</filename> tree
            achieves this default functionality.
        </para>

        <para>
            This behavior means that all the generated files for a particular
            machine or BSP are now in the build tree directory.
            The files include the final <filename>.config</filename> file,
            all the <filename>.o</filename> files, the <filename>.a</filename>
            files, and so forth.
            Since each machine or BSP has its own separate
            <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>
            in its own separate branch of the Git repository, you can easily
            switch between different builds.
        </para>
    </section>
</appendix>
<!--
vim: expandtab tw=80 ts=4
-->