blob: fe79bc012994aef4327dd02f4f2c5224bbae534b [file] [log] [blame]
Andrew Geisslerf0343792020-11-18 10:42:21 -06001.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Andrew Geisslerc9f78652020-09-18 14:11:35 -05002
3=========================================
4What I wish I'd known about Yocto Project
5=========================================
6
7|
8
9.. note::
10
11 Before reading further, make sure you've taken a look at the
12 :yocto_home:`Software Overview</software-overview>` page which presents the
13 definitions for many of the terms referenced here. Also, know that some of the
14 information here won't make sense now, but as you start developing, it is the
15 information you'll want to keep close at hand. These are best known methods for
16 working with Yocto Project and they are updated regularly.
17
18Using the Yocto Project is fairly easy, *until something goes wrong*. Without an
19understanding of how the build process works, you'll find yourself trying to
20troubleshoot "a black box". Here are a few items that new users wished they had
21known before embarking on their first build with Yocto Project. Feel free to
22contact us with other suggestions.
23
24#. **Use Git, not the tarball download:**
25 If you use git the software will be automatically updated with bug updates
26 because of how git works. If you download the tarball instead, you will need
27 to be responsible for your own updates.
28
29#. **Get to know the layer index:**
Andrew Geisslerd1e89492021-02-12 15:35:20 -060030 All layers can be found in the :oe_layerindex:`layer index <>`. Layers which
31 have applied for Yocto Project Compatible status (structure continuity
Patrick Williamsac13d5f2023-11-24 18:59:46 -060032 assurance and testing) can be found in the :yocto_home:`Yocto Project
33 Compatible Layers </development/yocto-project-compatible-layers/>` page.
34 Generally check the Compatible layer index first,
Andrew Geisslerc9f78652020-09-18 14:11:35 -050035 and if you don't find the necessary layer check the general layer index. The
36 layer index is an original artifact from the Open Embedded Project. As such,
37 that index doesn't have the curating and testing that the Yocto Project
38 provides on Yocto Project Compatible layer list, but the latter has fewer
39 entries. Know that when you start searching in the layer index that not all
40 layers have the same level of maturity, validation, or usability. Nor do
41 searches prioritize displayed results. There is no easy way to help you
42 through the process of choosing the best layer to suit your needs.
43 Consequently, it is often trial and error, checking the mailing lists, or
44 working with other developers through collaboration rooms that can help you
45 make good choices.
46
47#. **Use existing BSP layers from silicon vendors when possible:**
48 Intel, TI, NXP and others have information on what BSP layers to use with
49 their silicon. These layers have names such as "meta-intel" or "meta-ti". Try
50 not to build layers from scratch. If you do have custom silicon, use one of
51 these layers as a guide or template and familiarize yourself with the
Andrew Geissler09209ee2020-12-13 08:44:15 -060052 :doc:`bsp-guide/index`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -050053
54#. **Do not put everything into one layer:**
55 Use different layers to logically separate information in your build. As an
56 example, you could have a BSP layer, a GUI layer, a distro configuration,
57 middleware, or an application (e.g. "meta-filesystems", "meta-python",
58 "meta-intel", and so forth). Putting your entire build into one layer limits
59 and complicates future customization and reuse. Isolating information into
60 layers, on the other hand, helps keep simplify future customizations and
61 reuse.
62
63#. **Never modify the POKY layer. Never. Ever. When you update to the next
64 release, you'll lose all of your work. ALL OF IT.**
65
66#. **Don't be fooled by documentation searching results:**
67 Yocto Project documentation is always being updated. Unfortunately, when you
68 use Google to search for Yocto Project concepts or terms, Google consistently
69 searches and retrieves older versions of Yocto Project manuals. For example,
70 searching for a particular topic using Google could result in a "hit" on a
71 Yocto Project manual that is several releases old. To be sure that you are
72 using the most current Yocto Project documentation, use the drop-down menu at
73 the top of any of its page.
74
75 Many developers look through the :yocto_docs:`All-in-one 'Mega' Manual </singleindex.html>`
76 for a concept or term by doing a search through the whole page. This manual
77 is a concatenation of the core set of Yocto Project manual. Thus, a simple
78 string search using Ctrl-F in this manual produces all the "hits" for a
79 desired term or concept. Once you find the area in which you are
80 interested, you can display the actual manual, if desired. It is also
81 possible to use the search bar in the menu or in the left navigation pane.
82
83#. **Understand the basic concepts of how the build system works: the workflow:**
84 Understanding the Yocto Project workflow is important as it can help you both
85 pinpoint where trouble is occurring and how the build is breaking. The
86 workflow breaks down into the following steps:
87
88 #. Fetch – get the source code
89 #. Extract – unpack the sources
90 #. Patch – apply patches for bug fixes and new capability
91 #. Configure – set up your environment specifications
92 #. Build – compile and link
93 #. Install – copy files to target directories
94 #. Package – bundle files for installation
95
96 During "fetch", there may be an inability to find code. During "extract",
97 there is likely an invalid zip or something similar. In other words, the
98 function of a particular part of the workflow gives you an idea of what might
99 be going wrong.
100
101 .. image:: figures/yp-how-it-works-new-diagram.png
Andrew Geisslerd5838332022-05-27 11:33:10 -0500102 :width: 100%
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500103
104#. **Know that you can generate a dependency graph and learn how to do it:**
105 A dependency graph shows dependencies between recipes, tasks, and targets.
106 You can use the "-g" option with BitBake to generate this graph. When you
107 start a build and the build breaks, you could see packages you have no clue
108 about or have any idea why the build system has included them. The
109 dependency graph can clarify that confusion. You can learn more about
110 dependency graphs and how to generate them in the
111 :ref:`bitbake-user-manual/bitbake-user-manual-intro:generating dependency
112 graphs` section in the BitBake User Manual.
113
114#. **Here's how you decode "magic" folder names in tmp/work:**
115 The build system fetches, unpacks, preprocesses, and builds. If something
116 goes wrong, the build system reports to you directly the path to a folder
117 where the temporary (build/tmp) files and packages reside resulting from the
118 build. For a detailed example of this process, see the :yocto_wiki:`example
119 </Cookbook:Example:Adding_packages_to_your_OS_image>`. Unfortunately this
120 example is on an earlier release of Yocto Project.
121
122 When you perform a build, you can use the "-u" BitBake command-line option to
123 specify a user interface viewer into the dependency graph (e.g. knotty,
124 ncurses, or taskexp) that helps you understand the build dependencies better.
125
126#. **You can build more than just images:**
127 You can build and run a specific task for a specific package (including
128 devshell) or even a single recipe. When developers first start using the
129 Yocto Project, the instructions found in the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600130 :doc:`brief-yoctoprojectqs/index` show how to create an image
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500131 and then run or flash that image. However, you can actually build just a
132 single recipe. Thus, if some dependency or recipe isn't working, you can just
133 say "bitbake foo" where "foo" is the name for a specific recipe. As you
134 become more advanced using the Yocto Project, and if builds are failing, it
135 can be useful to make sure the fetch itself works as desired. Here are some
Andrew Geissler517393d2023-01-13 08:55:19 -0600136 valuable links: :ref:`dev-manual/development-shell:Using a Development
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500137 Shell` for information on how to build and run a specific task using
138 devshell. Also, the :ref:`SDK manual shows how to build out a specific recipe
Andrew Geissler09209ee2020-12-13 08:44:15 -0600139 <sdk-manual/extensible:use \`\`devtool modify\`\` to modify the source of an existing component>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500140
141#. **An ambiguous definition: Package vs Recipe:**
142 A recipe contains instructions the build system uses to create
143 packages. Recipes and Packages are the difference between the front end and
144 the result of the build process.
145
146 As mentioned, the build system takes the recipe and creates packages from the
147 recipe's instructions. The resulting packages are related to the one thing
148 the recipe is building but are different parts (packages) of the build
149 (i.e. the main package, the doc package, the debug symbols package, the
150 separate utilities package, and so forth). The build system splits out the
151 packages so that you don't need to install the packages you don't want or
152 need, which is advantageous because you are building for small devices when
153 developing for embedded and IoT.
154
Andrew Geissler595f6302022-01-24 19:11:47 +0000155#. **You will want to learn about and know what's packaged in the root filesystem.**
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500156
157#. **Create your own image recipe:**
158 There are a number of ways to create your own image recipe. We suggest you
159 create your own image recipe as opposed to appending an existing recipe. It
160 is trivial and easy to write an image recipe. Again, do not try appending to
161 an existing image recipe. Create your own and do it right from the start.
162
163#. **Finally, here is a list of the basic skills you will need as a systems
164 developer. You must be able to:**
165
166 * deal with corporate proxies
167 * add a package to an image
168 * understand the difference between a recipe and package
169 * build a package by itself and why that's useful
170 * find out what packages are created by a recipe
171 * find out what files are in a package
172 * find out what files are in an image
173 * add an ssh server to an image (enable transferring of files to target)
174 * know the anatomy of a recipe
175 * know how to create and use layers
Andrew Geisslerd1e89492021-02-12 15:35:20 -0600176 * find recipes (with the :oe_layerindex:`OpenEmbedded Layer index <>`)
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500177 * understand difference between machine and distro settings
178 * find and use the right BSP (machine) for your hardware
179 * find examples of distro features and know where to set them
180 * understanding the task pipeline and executing individual tasks
181 * understand devtool and how it simplifies your workflow
182 * improve build speeds with shared downloads and shared state cache
183 * generate and understand a dependency graph
Andrew Geisslerd5838332022-05-27 11:33:10 -0500184 * generate and understand BitBake environment
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500185 * build an Extensible SDK for applications development
186
187#. **Depending on what you primary interests are with the Yocto Project, you
188 could consider any of the following reading:**
189
190 * **Look Through the Yocto Project Development Tasks Manual**: This manual
191 contains procedural information grouped to help you get set up, work with
192 layers, customize images, write new recipes, work with libraries, and use
193 QEMU. The information is task-based and spans the breadth of the Yocto
Andrew Geissler09209ee2020-12-13 08:44:15 -0600194 Project. See the :doc:`/dev-manual/index`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500195
196 * **Look Through the Yocto Project Application Development and the Extensible
197 Software Development Kit (eSDK) manual**: This manual describes how to use
198 both the standard SDK and the extensible SDK, which are used primarily for
Andrew Geissler09209ee2020-12-13 08:44:15 -0600199 application development. The :doc:`/sdk-manual/extensible` also provides
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500200 example workflows that use devtool. See the section
Andrew Geissler09209ee2020-12-13 08:44:15 -0600201 :ref:`sdk-manual/extensible:using \`\`devtool\`\` in your sdk workflow`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500202 for more information.
203
204 * **Learn About Kernel Development**: If you want to see how to work with the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600205 kernel and understand Yocto Linux kernels, see the :doc:`/kernel-dev/index`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500206 This manual provides information on how to patch the kernel, modify kernel
207 recipes, and configure the kernel.
208
209 * **Learn About Board Support Packages (BSPs)**: If you want to learn about
Andrew Geissler09209ee2020-12-13 08:44:15 -0600210 BSPs, see the :doc:`/bsp-guide/index`. This manual also provides an
211 example BSP creation workflow. See the :doc:`/bsp-guide/bsp` section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500212
213 * **Learn About Toaster**: Toaster is a web interface to the Yocto Project's
214 OpenEmbedded build system. If you are interested in using this type of
Andrew Geissler09209ee2020-12-13 08:44:15 -0600215 interface to create images, see the :doc:`/toaster-manual/index`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500216
217 * **Have Available the Yocto Project Reference Manual**: Unlike the rest of
218 the Yocto Project manual set, this manual is comprised of material suited
219 for reference rather than procedures. You can get build details, a closer
220 look at how the pieces of the Yocto Project development environment work
221 together, information on various technical details, guidance on migrating
222 to a newer Yocto Project release, reference material on the directory
Andrew Geissler09209ee2020-12-13 08:44:15 -0600223 structure, classes, and tasks. The :doc:`/ref-manual/index` also
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500224 contains a fairly comprehensive glossary of variables used within the Yocto
225 Project.
226
227.. include:: /boilerplate.rst