yocto-poky/documentation/kernel-dev/kernel-dev-concepts-appx.xml - openbmc/openbmc - Gitiles

 <!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
 "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
 [<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >

 <appendix id='kernel-dev-concepts-appx'>
 <title>Advanced Kernel Concepts</title>

     <section id='kernel-big-picture'>
         <title>Yocto Project Kernel Development and Maintenance</title>
         <para>
             Kernels available through the Yocto Project, like other kernels, are based off the Linux
             kernel releases from <ulink url='http://www.kernel.org'></ulink>.
             At the beginning of a major development cycle, the Yocto Project team
             chooses its kernel based on factors such as release timing, the anticipated release
             timing of final upstream <filename>kernel.org</filename> versions, and Yocto Project
             feature requirements.
             Typically, the kernel chosen is in the
             final stages of development by the community.
             In other words, the kernel is in the release
             candidate or "rc" phase and not yet a final release.
             But, by being in the final stages of external development, the team knows that the
             <filename>kernel.org</filename> final release will clearly be within the early stages of
             the Yocto Project development window.
         </para>
         <para>
             This balance allows the team to deliver the most up-to-date kernel
             possible, while still ensuring that the team has a stable official release for
             the baseline Linux kernel version.
         </para>
         <para>
             The ultimate source for kernels available through the Yocto Project are released kernels
             from <filename>kernel.org</filename>.
             In addition to a foundational kernel from <filename>kernel.org</filename>, the
             kernels available contain a mix of important new mainline
             developments, non-mainline developments (when there is no alternative),
             Board Support Package (BSP) developments,
             and custom features.
             These additions result in a commercially released Yocto Project Linux kernel that caters
             to specific embedded designer needs for targeted hardware.
         </para>
         <para>
             Once a kernel is officially released, the Yocto Project team goes into
             their next development cycle, or upward revision (uprev) cycle, while still
             continuing maintenance on the released kernel.
             It is important to note that the most sustainable and stable way
             to include feature development upstream is through a kernel uprev process.
             Back-porting hundreds of individual fixes and minor features from various
             kernel versions is not sustainable and can easily compromise quality.
         </para>
         <para>
             During the uprev cycle, the Yocto Project team uses an ongoing analysis of
             kernel development, BSP support, and release timing to select the best
             possible <filename>kernel.org</filename> version.
             The team continually monitors community kernel
             development to look for significant features of interest.
             The team does consider back-porting large features if they have a significant advantage.
             User or community demand can also trigger a back-port or creation of new
             functionality in the Yocto Project baseline kernel during the uprev cycle.
         </para>
         <para>
             Generally speaking, every new kernel both adds features and introduces new bugs.
             These consequences are the basic properties of upstream kernel development and are
             managed by the Yocto Project team's kernel strategy.
             It is the Yocto Project team's policy to not back-port minor features to the released kernel.
             They only consider back-porting significant technological jumps - and, that is done
             after a complete gap analysis.
             The reason for this policy is that back-porting any small to medium sized change
             from an evolving kernel can easily create mismatches, incompatibilities and very
             subtle errors.
         </para>
         <para>
             These policies result in both a stable and a cutting
             edge kernel that mixes forward ports of existing features and significant and critical
             new functionality.
             Forward porting functionality in the kernels available through the Yocto Project kernel
             can be thought of as a "micro uprev."
             The many “micro uprevs” produce a kernel version with a mix of
             important new mainline, non-mainline, BSP developments and feature integrations.
             This kernel gives insight into new features and allows focused
             amounts of testing to be done on the kernel, which prevents
             surprises when selecting the next major uprev.
             The quality of these cutting edge kernels is evolving and the kernels are used in leading edge
             feature and BSP development.
         </para>
     </section>

     <section id='kernel-architecture'>
         <title>Kernel Architecture</title>
         <para>
             This section describes the architecture of the kernels available through the
             Yocto Project and provides information
             on the mechanisms used to achieve that architecture.
         </para>

         <section id='architecture-overview'>
             <title>Overview</title>
             <para>
                 As mentioned earlier, a key goal of the Yocto Project is to present the
                 developer with
                 a kernel that has a clear and continuous history that is visible to the user.
                 The architecture and mechanisms used achieve that goal in a manner similar to the
                 upstream <filename>kernel.org</filename>.
             </para>
             <para>
                 You can think of a Yocto Project kernel as consisting of a baseline Linux kernel with
                 added features logically structured on top of the baseline.
                 The features are tagged and organized by way of a branching strategy implemented by the
                 source code manager (SCM) Git.
                 For information on Git as applied to the Yocto Project, see the
                 "<ulink url='&YOCTO_DOCS_DEV_URL;#git'>Git</ulink>" section in the
                 Yocto Project Development Manual.
             </para>
             <para>
                 The result is that the user has the ability to see the added features and
                 the commits that make up those features.
                 In addition to being able to see added features, the user can also view the history of what
                 made up the baseline kernel.
             </para>
             <para>
                 The following illustration shows the conceptual Yocto Project kernel.
             </para>
             <para>
                 <imagedata fileref="figures/kernel-architecture-overview.png" width="6in" depth="7in" align="center" scale="100" />
             </para>
             <para>
                 In the illustration, the "Kernel.org Branch Point"
                 marks the specific spot (or release) from
                 which the Yocto Project kernel is created.
                 From this point "up" in the tree, features and differences are organized and tagged.
             </para>
             <para>
                 The "Yocto Project Baseline Kernel" contains functionality that is common to every kernel
                 type and BSP that is organized further up the tree.
                 Placing these common features in the
                 tree this way means features do not have to be duplicated along individual branches of the
                 structure.
             </para>
             <para>
                 From the Yocto Project Baseline Kernel, branch points represent specific functionality
                 for individual BSPs as well as real-time kernels.
                 The illustration represents this through three BSP-specific branches and a real-time
                 kernel branch.
                 Each branch represents some unique functionality for the BSP or a real-time kernel.
             </para>
             <para>
                 In this example structure, the real-time kernel branch has common features for all
                 real-time kernels and contains
                 more branches for individual BSP-specific real-time kernels.
                 The illustration shows three branches as an example.
                 Each branch points the way to specific, unique features for a respective real-time
                 kernel as they apply to a given BSP.
             </para>
             <para>
                 The resulting tree structure presents a clear path of markers (or branches) to the
                 developer that, for all practical purposes, is the kernel needed for any given set
                 of requirements.
             </para>
         </section>

         <section id='branching-and-workflow'>
             <title>Branching Strategy and Workflow</title>
             <para>
                 The Yocto Project team creates kernel branches at points where functionality is
                 no longer shared and thus, needs to be isolated.
                 For example, board-specific incompatibilities would require different functionality
                 and would require a branch to separate the features.
                 Likewise, for specific kernel features, the same branching strategy is used.
             </para>
             <para>
                 This branching strategy results in a tree that has features organized to be specific
                 for particular functionality, single kernel types, or a subset of kernel types.
                 This strategy also results in not having to store the same feature twice
                 internally in the tree.
                 Rather, the kernel team stores the unique differences required to apply the
                 feature onto the kernel type in question.
                 <note>
                     The Yocto Project team strives to place features in the tree such that they can be
                     shared by all boards and kernel types where possible.
                     However, during development cycles or when large features are merged,
                     the team cannot always follow this practice.
                     In those cases, the team uses isolated branches to merge features.
                 </note>
             </para>
             <para>
                 BSP-specific code additions are handled in a similar manner to kernel-specific additions.
                 Some BSPs only make sense given certain kernel types.
                 So, for these types, the team creates branches off the end of that kernel type for all
                 of the BSPs that are supported on that kernel type.
                 From the perspective of the tools that create the BSP branch, the BSP is really no
                 different than a feature.
                 Consequently, the same branching strategy applies to BSPs as it does to features.
                 So again, rather than store the BSP twice, the team only stores the unique
                 differences for the BSP across the supported multiple kernels.
             </para>
             <para>
                 While this strategy can result in a tree with a significant number of branches, it is
                 important to realize that from the developer's point of view, there is a linear
                 path that travels from the baseline <filename>kernel.org</filename>, through a select
                 group of features and ends with their BSP-specific commits.
                 In other words, the divisions of the kernel are transparent and are not relevant
                 to the developer on a day-to-day basis.
                 From the developer's perspective, this path is the "master" branch.
                 The developer does not need to be aware of the existence of any other branches at all.
                 Of course, there is value in the existence of these branches
                 in the tree, should a person decide to explore them.
                 For example, a comparison between two BSPs at either the commit level or at the line-by-line
                 code <filename>diff</filename> level is now a trivial operation.
             </para>
             <para>
                 Working with the kernel as a structured tree follows recognized community best practices.
                 In particular, the kernel as shipped with the product, should be
                 considered an "upstream source" and viewed as a series of
                 historical and documented modifications (commits).
                 These modifications represent the development and stabilization done
                 by the Yocto Project kernel development team.
             </para>
             <para>
                 Because commits only change at significant release points in the product life cycle,
                 developers can work on a branch created
                 from the last relevant commit in the shipped Yocto Project kernel.
                 As mentioned previously, the structure is transparent to the developer
                 because the kernel tree is left in this state after cloning and building the kernel.
             </para>
         </section>

         <section id='source-code-manager-git'>
             <title>Source Code Manager - Git</title>
             <para>
                 The Source Code Manager (SCM) is Git.
                 This SCM is the obvious mechanism for meeting the previously mentioned goals.
                 Not only is it the SCM for <filename>kernel.org</filename> but,
                 Git continues to grow in popularity and supports many different work flows,
                 front-ends and management techniques.
             </para>
             <para>
                 You can find documentation on Git at <ulink url='http://git-scm.com/documentation'></ulink>.
                 You can also get an introduction to Git as it applies to the Yocto Project in the
                 "<ulink url='&YOCTO_DOCS_DEV_URL;#git'>Git</ulink>"
                 section in the Yocto Project Development Manual.
                 These referenced sections overview Git and describe a minimal set of
                 commands that allows you to be functional using Git.
                 <note>
                     You can use as much, or as little, of what Git has to offer to accomplish what
                     you need for your project.
                     You do not have to be a "Git Master" in order to use it with the Yocto Project.
                 </note>
             </para>
         </section>
     </section>
 </appendix>
 <!--
 vim: expandtab tw=80 ts=4
 -->
	<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
	"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
	[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >

	<appendix id='kernel-dev-concepts-appx'>
	<title>Advanced Kernel Concepts</title>

	<section id='kernel-big-picture'>
	<title>Yocto Project Kernel Development and Maintenance</title>
	<para>
	Kernels available through the Yocto Project, like other kernels, are based off the Linux
	kernel releases from <ulink url='http://www.kernel.org'></ulink>.
	At the beginning of a major development cycle, the Yocto Project team
	chooses its kernel based on factors such as release timing, the anticipated release
	timing of final upstream <filename>kernel.org</filename> versions, and Yocto Project
	feature requirements.
	Typically, the kernel chosen is in the
	final stages of development by the community.
	In other words, the kernel is in the release
	candidate or "rc" phase and not yet a final release.
	But, by being in the final stages of external development, the team knows that the
	<filename>kernel.org</filename> final release will clearly be within the early stages of
	the Yocto Project development window.
	</para>
	<para>
	This balance allows the team to deliver the most up-to-date kernel
	possible, while still ensuring that the team has a stable official release for
	the baseline Linux kernel version.
	</para>
	<para>
	The ultimate source for kernels available through the Yocto Project are released kernels
	from <filename>kernel.org</filename>.
	In addition to a foundational kernel from <filename>kernel.org</filename>, the
	kernels available contain a mix of important new mainline
	developments, non-mainline developments (when there is no alternative),
	Board Support Package (BSP) developments,
	and custom features.
	These additions result in a commercially released Yocto Project Linux kernel that caters
	to specific embedded designer needs for targeted hardware.
	</para>
	<para>
	Once a kernel is officially released, the Yocto Project team goes into
	their next development cycle, or upward revision (uprev) cycle, while still
	continuing maintenance on the released kernel.
	It is important to note that the most sustainable and stable way
	to include feature development upstream is through a kernel uprev process.
	Back-porting hundreds of individual fixes and minor features from various
	kernel versions is not sustainable and can easily compromise quality.
	</para>
	<para>
	During the uprev cycle, the Yocto Project team uses an ongoing analysis of
	kernel development, BSP support, and release timing to select the best
	possible <filename>kernel.org</filename> version.
	The team continually monitors community kernel
	development to look for significant features of interest.
	The team does consider back-porting large features if they have a significant advantage.
	User or community demand can also trigger a back-port or creation of new
	functionality in the Yocto Project baseline kernel during the uprev cycle.
	</para>
	<para>
	Generally speaking, every new kernel both adds features and introduces new bugs.
	These consequences are the basic properties of upstream kernel development and are
	managed by the Yocto Project team's kernel strategy.
	It is the Yocto Project team's policy to not back-port minor features to the released kernel.
	They only consider back-porting significant technological jumps - and, that is done
	after a complete gap analysis.
	The reason for this policy is that back-porting any small to medium sized change
	from an evolving kernel can easily create mismatches, incompatibilities and very
	subtle errors.
	</para>
	<para>
	These policies result in both a stable and a cutting
	edge kernel that mixes forward ports of existing features and significant and critical
	new functionality.
	Forward porting functionality in the kernels available through the Yocto Project kernel
	can be thought of as a "micro uprev."
	The many “micro uprevs” produce a kernel version with a mix of
	important new mainline, non-mainline, BSP developments and feature integrations.
	This kernel gives insight into new features and allows focused
	amounts of testing to be done on the kernel, which prevents
	surprises when selecting the next major uprev.
	The quality of these cutting edge kernels is evolving and the kernels are used in leading edge
	feature and BSP development.
	</para>
	</section>

	<section id='kernel-architecture'>
	<title>Kernel Architecture</title>
	<para>
	This section describes the architecture of the kernels available through the
	Yocto Project and provides information
	on the mechanisms used to achieve that architecture.
	</para>

	<section id='architecture-overview'>
	<title>Overview</title>
	<para>
	As mentioned earlier, a key goal of the Yocto Project is to present the
	developer with
	a kernel that has a clear and continuous history that is visible to the user.
	The architecture and mechanisms used achieve that goal in a manner similar to the
	upstream <filename>kernel.org</filename>.
	</para>
	<para>
	You can think of a Yocto Project kernel as consisting of a baseline Linux kernel with
	added features logically structured on top of the baseline.
	The features are tagged and organized by way of a branching strategy implemented by the
	source code manager (SCM) Git.
	For information on Git as applied to the Yocto Project, see the
	"<ulink url='&YOCTO_DOCS_DEV_URL;#git'>Git</ulink>" section in the
	Yocto Project Development Manual.
	</para>
	<para>
	The result is that the user has the ability to see the added features and
	the commits that make up those features.
	In addition to being able to see added features, the user can also view the history of what
	made up the baseline kernel.
	</para>
	<para>
	The following illustration shows the conceptual Yocto Project kernel.
	</para>
	<para>
	<imagedata fileref="figures/kernel-architecture-overview.png" width="6in" depth="7in" align="center" scale="100" />
	</para>
	<para>
	In the illustration, the "Kernel.org Branch Point"
	marks the specific spot (or release) from
	which the Yocto Project kernel is created.
	From this point "up" in the tree, features and differences are organized and tagged.
	</para>
	<para>
	The "Yocto Project Baseline Kernel" contains functionality that is common to every kernel
	type and BSP that is organized further up the tree.
	Placing these common features in the
	tree this way means features do not have to be duplicated along individual branches of the
	structure.
	</para>
	<para>
	From the Yocto Project Baseline Kernel, branch points represent specific functionality
	for individual BSPs as well as real-time kernels.
	The illustration represents this through three BSP-specific branches and a real-time
	kernel branch.
	Each branch represents some unique functionality for the BSP or a real-time kernel.
	</para>
	<para>
	In this example structure, the real-time kernel branch has common features for all
	real-time kernels and contains
	more branches for individual BSP-specific real-time kernels.
	The illustration shows three branches as an example.
	Each branch points the way to specific, unique features for a respective real-time
	kernel as they apply to a given BSP.
	</para>
	<para>
	The resulting tree structure presents a clear path of markers (or branches) to the
	developer that, for all practical purposes, is the kernel needed for any given set
	of requirements.
	</para>
	</section>

	<section id='branching-and-workflow'>
	<title>Branching Strategy and Workflow</title>
	<para>
	The Yocto Project team creates kernel branches at points where functionality is
	no longer shared and thus, needs to be isolated.
	For example, board-specific incompatibilities would require different functionality
	and would require a branch to separate the features.
	Likewise, for specific kernel features, the same branching strategy is used.
	</para>
	<para>
	This branching strategy results in a tree that has features organized to be specific
	for particular functionality, single kernel types, or a subset of kernel types.
	This strategy also results in not having to store the same feature twice
	internally in the tree.
	Rather, the kernel team stores the unique differences required to apply the
	feature onto the kernel type in question.
	<note>
	The Yocto Project team strives to place features in the tree such that they can be
	shared by all boards and kernel types where possible.
	However, during development cycles or when large features are merged,
	the team cannot always follow this practice.
	In those cases, the team uses isolated branches to merge features.
	</note>
	</para>
	<para>
	BSP-specific code additions are handled in a similar manner to kernel-specific additions.
	Some BSPs only make sense given certain kernel types.
	So, for these types, the team creates branches off the end of that kernel type for all
	of the BSPs that are supported on that kernel type.
	From the perspective of the tools that create the BSP branch, the BSP is really no
	different than a feature.
	Consequently, the same branching strategy applies to BSPs as it does to features.
	So again, rather than store the BSP twice, the team only stores the unique
	differences for the BSP across the supported multiple kernels.
	</para>
	<para>
	While this strategy can result in a tree with a significant number of branches, it is
	important to realize that from the developer's point of view, there is a linear
	path that travels from the baseline <filename>kernel.org</filename>, through a select
	group of features and ends with their BSP-specific commits.
	In other words, the divisions of the kernel are transparent and are not relevant
	to the developer on a day-to-day basis.
	From the developer's perspective, this path is the "master" branch.
	The developer does not need to be aware of the existence of any other branches at all.
	Of course, there is value in the existence of these branches
	in the tree, should a person decide to explore them.
	For example, a comparison between two BSPs at either the commit level or at the line-by-line
	code <filename>diff</filename> level is now a trivial operation.
	</para>
	<para>
	Working with the kernel as a structured tree follows recognized community best practices.
	In particular, the kernel as shipped with the product, should be
	considered an "upstream source" and viewed as a series of
	historical and documented modifications (commits).
	These modifications represent the development and stabilization done
	by the Yocto Project kernel development team.
	</para>
	<para>
	Because commits only change at significant release points in the product life cycle,
	developers can work on a branch created
	from the last relevant commit in the shipped Yocto Project kernel.
	As mentioned previously, the structure is transparent to the developer
	because the kernel tree is left in this state after cloning and building the kernel.
	</para>
	</section>

	<section id='source-code-manager-git'>
	<title>Source Code Manager - Git</title>
	<para>
	The Source Code Manager (SCM) is Git.
	This SCM is the obvious mechanism for meeting the previously mentioned goals.
	Not only is it the SCM for <filename>kernel.org</filename> but,
	Git continues to grow in popularity and supports many different work flows,
	front-ends and management techniques.
	</para>
	<para>
	You can find documentation on Git at <ulink url='http://git-scm.com/documentation'></ulink>.
	You can also get an introduction to Git as it applies to the Yocto Project in the
	"<ulink url='&YOCTO_DOCS_DEV_URL;#git'>Git</ulink>"
	section in the Yocto Project Development Manual.
	These referenced sections overview Git and describe a minimal set of
	commands that allows you to be functional using Git.
	<note>
	You can use as much, or as little, of what Git has to offer to accomplish what
	you need for your project.
	You do not have to be a "Git Master" in order to use it with the Yocto Project.
	</note>
	</para>
	</section>
	</section>
	</appendix>
	<!--
	vim: expandtab tw=80 ts=4
	-->