blob: 4e3b7c325083d462f67c98ef899fe2dff05b4c1d [file] [log] [blame]
Andrew Geisslerf0343792020-11-18 10:42:21 -06001.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Andrew Geisslerc9f78652020-09-18 14:11:35 -05002
3*****************************
4Introducing the Yocto Project
5*****************************
6
7What is the Yocto Project?
8==========================
9
10The Yocto Project is an open source collaboration project that helps
11developers create custom Linux-based systems that are designed for
12embedded products regardless of the product's hardware architecture.
13Yocto Project provides a flexible toolset and a development environment
14that allows embedded device developers across the world to collaborate
15through shared technologies, software stacks, configurations, and best
16practices used to create these tailored Linux images.
17
18Thousands of developers worldwide have discovered that Yocto Project
19provides advantages in both systems and applications development,
20archival and management benefits, and customizations used for speed,
21footprint, and memory utilization. The project is a standard when it
22comes to delivering embedded software stacks. The project allows
23software customizations and build interchange for multiple hardware
24platforms as well as software stacks that can be maintained and scaled.
25
26.. image:: figures/key-dev-elements.png
Andrew Geisslerd5838332022-05-27 11:33:10 -050027 :width: 100%
Andrew Geisslerc9f78652020-09-18 14:11:35 -050028
29For further introductory information on the Yocto Project, you might be
30interested in this
31`article <https://www.embedded.com/electronics-blogs/say-what-/4458600/Why-the-Yocto-Project-for-my-IoT-Project->`__
32by Drew Moseley and in this short introductory
33`video <https://www.youtube.com/watch?v=utZpKM7i5Z4>`__.
34
35The remainder of this section overviews advantages and challenges tied
36to the Yocto Project.
37
Andrew Geisslerc9f78652020-09-18 14:11:35 -050038Features
39--------
40
William A. Kennington IIIac69b482021-06-02 12:28:27 -070041Here are features and advantages of the Yocto Project:
Andrew Geisslerc9f78652020-09-18 14:11:35 -050042
Andrew Geissler3b8a17c2021-04-15 15:55:55 -050043- *Widely Adopted Across the Industry:* Many semiconductor, operating
44 system, software, and service vendors adopt and support the Yocto
45 Project in their products and services. For a look at the Yocto
Andrew Geisslerc9f78652020-09-18 14:11:35 -050046 Project community and the companies involved with the Yocto Project,
47 see the "COMMUNITY" and "ECOSYSTEM" tabs on the
48 :yocto_home:`Yocto Project <>` home page.
49
50- *Architecture Agnostic:* Yocto Project supports Intel, ARM, MIPS,
51 AMD, PPC and other architectures. Most ODMs, OSVs, and chip vendors
52 create and supply BSPs that support their hardware. If you have
53 custom silicon, you can create a BSP that supports that architecture.
54
Andrew Geissler3b8a17c2021-04-15 15:55:55 -050055 Aside from broad architecture support, the Yocto Project fully
56 supports a wide range of devices emulated by the Quick EMUlator
Andrew Geisslerc9f78652020-09-18 14:11:35 -050057 (QEMU).
58
59- *Images and Code Transfer Easily:* Yocto Project output can easily
60 move between architectures without moving to new development
61 environments. Additionally, if you have used the Yocto Project to
62 create an image or application and you find yourself not able to
63 support it, commercial Linux vendors such as Wind River, Mentor
64 Graphics, Timesys, and ENEA could take it and provide ongoing
65 support. These vendors have offerings that are built using the Yocto
66 Project.
67
68- *Flexibility:* Corporations use the Yocto Project many different
69 ways. One example is to create an internal Linux distribution as a
70 code base the corporation can use across multiple product groups.
71 Through customization and layering, a project group can leverage the
72 base Linux distribution to create a distribution that works for their
73 product needs.
74
75- *Ideal for Constrained Embedded and IoT devices:* Unlike a full Linux
76 distribution, you can use the Yocto Project to create exactly what
77 you need for embedded devices. You only add the feature support or
78 packages that you absolutely need for the device. For devices that
79 have display hardware, you can use available system components such
Andrew Geissler3b8a17c2021-04-15 15:55:55 -050080 as X11, Wayland, GTK+, Qt, Clutter, and SDL (among others) to create
81 a rich user experience. For devices that do not have a display or
82 where you want to use alternative UI frameworks, you can choose to
83 not build these components.
Andrew Geisslerc9f78652020-09-18 14:11:35 -050084
85- *Comprehensive Toolchain Capabilities:* Toolchains for supported
86 architectures satisfy most use cases. However, if your hardware
87 supports features that are not part of a standard toolchain, you can
88 easily customize that toolchain through specification of
89 platform-specific tuning parameters. And, should you need to use a
90 third-party toolchain, mechanisms built into the Yocto Project allow
91 for that.
92
93- *Mechanism Rules Over Policy:* Focusing on mechanism rather than
94 policy ensures that you are free to set policies based on the needs
95 of your design instead of adopting decisions enforced by some system
96 software provider.
97
Andrew Geissler3b8a17c2021-04-15 15:55:55 -050098- *Uses a Layer Model:* The Yocto Project :ref:`layer
99 infrastructure <overview-manual/yp-intro:the yocto project layer model>`
100 groups related functionality into separate bundles. You can incrementally
101 add these grouped functionalities to your project as needed. Using layers to
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500102 isolate and group functionality reduces project complexity and
103 redundancy, allows you to easily extend the system, make
104 customizations, and keep functionality organized.
105
106- *Supports Partial Builds:* You can build and rebuild individual
107 packages as needed. Yocto Project accomplishes this through its
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500108 :ref:`overview-manual/concepts:shared state cache` (sstate) scheme.
109 Being able to build and debug components individually eases project
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500110 development.
111
112- *Releases According to a Strict Schedule:* Major releases occur on a
Andrew Geissler09209ee2020-12-13 08:44:15 -0600113 :doc:`six-month cycle </ref-manual/release-process>`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500114 predictably in October and April. The most recent two releases
115 support point releases to address common vulnerabilities and
116 exposures. This predictability is crucial for projects based on the
117 Yocto Project and allows development teams to plan activities.
118
119- *Rich Ecosystem of Individuals and Organizations:* For open source
120 projects, the value of community is very important. Support forums,
121 expertise, and active developers who continue to push the Yocto
122 Project forward are readily available.
123
124- *Binary Reproducibility:* The Yocto Project allows you to be very
125 specific about dependencies and achieves very high percentages of
126 binary reproducibility (e.g. 99.8% for ``core-image-minimal``). When
127 distributions are not specific about which packages are pulled in and
128 in what order to support dependencies, other build systems can
129 arbitrarily include packages.
130
131- *License Manifest:* The Yocto Project provides a :ref:`license
Andrew Geissler09209ee2020-12-13 08:44:15 -0600132 manifest <dev-manual/common-tasks:maintaining open source license compliance during your product's lifecycle>`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500133 for review by people who need to track the use of open source
134 licenses (e.g. legal teams).
135
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500136Challenges
137----------
138
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700139Here are challenges you might encounter when developing using the Yocto Project:
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500140
141- *Steep Learning Curve:* The Yocto Project has a steep learning curve
142 and has many different ways to accomplish similar tasks. It can be
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700143 difficult to choose between such ways.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500144
145- *Understanding What Changes You Need to Make For Your Design Requires
146 Some Research:* Beyond the simple tutorial stage, understanding what
147 changes need to be made for your particular design can require a
148 significant amount of research and investigation. For information
149 that helps you transition from trying out the Yocto Project to using
150 it for your project, see the ":ref:`what-i-wish-id-known:what i wish i'd known about yocto project`" and
151 ":ref:`transitioning-to-a-custom-environment:transitioning to a custom environment for systems development`"
152 documents on the Yocto Project website.
153
Patrick Williams45852732022-04-02 08:58:32 -0500154- *Project Workflow Could Be Confusing:* The :ref:`Yocto Project
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500155 workflow <overview-manual/development-environment:the yocto project development environment>`
156 could be confusing if you are used to traditional desktop and server
157 software development.
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700158 In a desktop development environment, there are mechanisms to easily pull
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500159 and install new packages, which are typically pre-compiled binaries
160 from servers accessible over the Internet. Using the Yocto Project,
161 you must modify your configuration and rebuild to add additional
162 packages.
163
164- *Working in a Cross-Build Environment Can Feel Unfamiliar:* When
165 developing code to run on a target, compilation, execution, and
166 testing done on the actual target can be faster than running a
167 BitBake build on a development host and then deploying binaries to
168 the target for test. While the Yocto Project does support development
169 tools on the target, the additional step of integrating your changes
170 back into the Yocto Project build environment would be required.
171 Yocto Project supports an intermediate approach that involves making
172 changes on the development system within the BitBake environment and
173 then deploying only the updated packages to the target.
174
175 The Yocto Project :term:`OpenEmbedded Build System`
176 produces packages
177 in standard formats (i.e. RPM, DEB, IPK, and TAR). You can deploy
178 these packages into the running system on the target by using
179 utilities on the target such as ``rpm`` or ``ipk``.
180
181- *Initial Build Times Can be Significant:* Long initial build times
182 are unfortunately unavoidable due to the large number of packages
183 initially built from scratch for a fully functioning Linux system.
184 Once that initial build is completed, however, the shared-state
185 (sstate) cache mechanism Yocto Project uses keeps the system from
186 rebuilding packages that have not been "touched" since the last
187 build. The sstate mechanism significantly reduces times for
188 successive builds.
189
190The Yocto Project Layer Model
191=============================
192
193The Yocto Project's "Layer Model" is a development model for embedded
194and IoT Linux creation that distinguishes the Yocto Project from other
195simple build systems. The Layer Model simultaneously supports
196collaboration and customization. Layers are repositories that contain
197related sets of instructions that tell the :term:`OpenEmbedded Build System`
198what to do. You can
199collaborate, share, and reuse layers.
200
201Layers can contain changes to previous instructions or settings at any
202time. This powerful override capability is what allows you to customize
203previously supplied collaborative or community layers to suit your
204product requirements.
205
206You use different layers to logically separate information in your
207build. As an example, you could have BSP, GUI, distro configuration,
208middleware, or application layers. Putting your entire build into one
209layer limits and complicates future customization and reuse. Isolating
210information into layers, on the other hand, helps simplify future
211customizations and reuse. You might find it tempting to keep everything
212in one layer when working on a single project. However, the more modular
213your Metadata, the easier it is to cope with future changes.
214
215.. note::
216
217 - Use Board Support Package (BSP) layers from silicon vendors when
218 possible.
219
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000220 - Familiarize yourself with the
Patrick Williams03907ee2022-05-01 06:28:52 -0500221 :yocto_home:`Yocto Project Compatible Layers </software-overview/layers/>`
222 or the :oe_layerindex:`OpenEmbedded Layer Index <>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500223 The latter contains more layers but they are less universally
224 validated.
225
226 - Layers support the inclusion of technologies, hardware components,
227 and software components. The :ref:`Yocto Project
Andrew Geissler09209ee2020-12-13 08:44:15 -0600228 Compatible <dev-manual/common-tasks:making sure your layer is compatible with yocto project>`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500229 designation provides a minimum level of standardization that
230 contributes to a strong ecosystem. "YP Compatible" is applied to
231 appropriate products and software components such as BSPs, other
232 OE-compatible layers, and related open-source projects, allowing
233 the producer to use Yocto Project badges and branding assets.
234
235To illustrate how layers are used to keep things modular, consider
236machine customizations. These types of customizations typically reside
237in a special layer, rather than a general layer, called a BSP Layer.
238Furthermore, the machine customizations should be isolated from recipes
239and Metadata that support a new GUI environment, for example. This
240situation gives you a couple of layers: one for the machine
241configurations, and one for the GUI environment. It is important to
242understand, however, that the BSP layer can still make machine-specific
243additions to recipes within the GUI environment layer without polluting
244the GUI layer itself with those machine-specific changes. You can
245accomplish this through a recipe that is a BitBake append
246(``.bbappend``) file, which is described later in this section.
247
248.. note::
249
250 For general information on BSP layer structure, see the
Andrew Geissler09036742021-06-25 14:25:14 -0500251 :doc:`/bsp-guide/index`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500252
253The :term:`Source Directory`
254contains both general layers and BSP layers right out of the box. You
255can easily identify layers that ship with a Yocto Project release in the
256Source Directory by their names. Layers typically have names that begin
257with the string ``meta-``.
258
259.. note::
260
261 It is not a requirement that a layer name begin with the prefix
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500262 ``meta-``, but it is a commonly accepted standard in the Yocto Project
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500263 community.
264
Andrew Geissler09209ee2020-12-13 08:44:15 -0600265For example, if you were to examine the :yocto_git:`tree view </poky/tree/>`
266of the ``poky`` repository, you will see several layers: ``meta``,
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500267``meta-skeleton``, ``meta-selftest``, ``meta-poky``, and
268``meta-yocto-bsp``. Each of these repositories represents a distinct
269layer.
270
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500271For procedures on how to create layers, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600272":ref:`dev-manual/common-tasks:understanding and creating layers`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500273section in the Yocto Project Development Tasks Manual.
274
275Components and Tools
276====================
277
278The Yocto Project employs a collection of components and tools used by
279the project itself, by project developers, and by those using the Yocto
280Project. These components and tools are open source projects and
281metadata that are separate from the reference distribution
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500282(:term:`Poky`) and the :term:`OpenEmbedded Build System`. Most of the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500283components and tools are downloaded separately.
284
285This section provides brief overviews of the components and tools
286associated with the Yocto Project.
287
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500288Development Tools
289-----------------
290
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700291Here are tools that help you develop images and applications using
292the Yocto Project:
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500293
294- *CROPS:* `CROPS <https://github.com/crops/poky-container/>`__ is an
295 open source, cross-platform development framework that leverages
296 `Docker Containers <https://www.docker.com/>`__. CROPS provides an
297 easily managed, extensible environment that allows you to build
298 binaries for a variety of architectures on Windows, Linux and Mac OS
299 X hosts.
300
301- *devtool:* This command-line tool is available as part of the
302 extensible SDK (eSDK) and is its cornerstone. You can use ``devtool``
303 to help build, test, and package software within the eSDK. You can
304 use the tool to optionally integrate what you build into an image
305 built by the OpenEmbedded build system.
306
307 The ``devtool`` command employs a number of sub-commands that allow
308 you to add, modify, and upgrade recipes. As with the OpenEmbedded
309 build system, "recipes" represent software packages within
310 ``devtool``. When you use ``devtool add``, a recipe is automatically
311 created. When you use ``devtool modify``, the specified existing
312 recipe is used in order to determine where to get the source code and
313 how to patch it. In both cases, an environment is set up so that when
314 you build the recipe a source tree that is under your control is used
315 in order to allow you to make changes to the source as desired. By
316 default, both new recipes and the source go into a "workspace"
317 directory under the eSDK. The ``devtool upgrade`` command updates an
318 existing recipe so that you can build it for an updated set of source
319 files.
320
321 You can read about the ``devtool`` workflow in the Yocto Project
322 Application Development and Extensible Software Development Kit
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500323 (eSDK) Manual in the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600324 ":ref:`sdk-manual/extensible:using \`\`devtool\`\` in your sdk workflow`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500325 section.
326
327- *Extensible Software Development Kit (eSDK):* The eSDK provides a
328 cross-development toolchain and libraries tailored to the contents of
329 a specific image. The eSDK makes it easy to add new applications and
330 libraries to an image, modify the source for an existing component,
331 test changes on the target hardware, and integrate into the rest of
332 the OpenEmbedded build system. The eSDK gives you a toolchain
333 experience supplemented with the powerful set of ``devtool`` commands
334 tailored for the Yocto Project environment.
335
Andrew Geissler09209ee2020-12-13 08:44:15 -0600336 For information on the eSDK, see the :doc:`/sdk-manual/index` Manual.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500337
338- *Toaster:* Toaster is a web interface to the Yocto Project
339 OpenEmbedded build system. Toaster allows you to configure, run, and
340 view information about builds. For information on Toaster, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600341 :doc:`/toaster-manual/index`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500342
343Production Tools
344----------------
345
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700346Here are tools that help with production related activities using the
347Yocto Project:
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500348
349- *Auto Upgrade Helper:* This utility when used in conjunction with the
350 :term:`OpenEmbedded Build System`
351 (BitBake and
352 OE-Core) automatically generates upgrades for recipes that are based
353 on new versions of the recipes published upstream. See
Andrew Geissler09209ee2020-12-13 08:44:15 -0600354 :ref:`dev-manual/common-tasks:using the auto upgrade helper (auh)`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500355 for how to set it up.
356
357- *Recipe Reporting System:* The Recipe Reporting System tracks recipe
358 versions available for Yocto Project. The main purpose of the system
359 is to help you manage the recipes you maintain and to offer a dynamic
360 overview of the project. The Recipe Reporting System is built on top
Andrew Geisslerd1e89492021-02-12 15:35:20 -0600361 of the :oe_layerindex:`OpenEmbedded Layer Index <>`, which
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500362 is a website that indexes OpenEmbedded-Core layers.
363
364- *Patchwork:* `Patchwork <http://jk.ozlabs.org/projects/patchwork/>`__
365 is a fork of a project originally started by
Andrew Geisslerd1e89492021-02-12 15:35:20 -0600366 `OzLabs <https://ozlabs.org/>`__. The project is a web-based tracking
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500367 system designed to streamline the process of bringing contributions
368 into a project. The Yocto Project uses Patchwork as an organizational
369 tool to handle patches, which number in the thousands for every
370 release.
371
372- *AutoBuilder:* AutoBuilder is a project that automates build tests
373 and quality assurance (QA). By using the public AutoBuilder, anyone
Andrew Geissler595f6302022-01-24 19:11:47 +0000374 can determine the status of the current development branch of Poky.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500375
376 .. note::
377
378 AutoBuilder is based on buildbot.
379
380 A goal of the Yocto Project is to lead the open source industry with
381 a project that automates testing and QA procedures. In doing so, the
382 project encourages a development community that publishes QA and test
383 plans, publicly demonstrates QA and test plans, and encourages
384 development of tools that automate and test and QA procedures for the
385 benefit of the development community.
386
387 You can learn more about the AutoBuilder used by the Yocto Project
Andrew Geissler09209ee2020-12-13 08:44:15 -0600388 Autobuilder :doc:`here </test-manual/understand-autobuilder>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500389
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500390- *Pseudo:* Pseudo is the Yocto Project implementation of
391 `fakeroot <http://man.he.net/man1/fakeroot>`__, which is used to run
392 commands in an environment that seemingly has root privileges.
393
394 During a build, it can be necessary to perform operations that
395 require system administrator privileges. For example, file ownership
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500396 or permissions might need to be defined. Pseudo is a tool that you
397 can either use directly or through the environment variable
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700398 ``LD_PRELOAD``. Either method allows these operations to succeed
399 even without system administrator privileges.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500400
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500401 Thanks to Pseudo, the Yocto Project never needs root privileges to
402 build images for your target system.
403
404 You can read more about Pseudo in the
405 ":ref:`overview-manual/concepts:fakeroot and pseudo`" section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500406
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500407Open-Embedded Build System Components
408-------------------------------------
409
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700410Here are components associated with the :term:`OpenEmbedded Build System`:
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500411
412- *BitBake:* BitBake is a core component of the Yocto Project and is
413 used by the OpenEmbedded build system to build images. While BitBake
414 is key to the build system, BitBake is maintained separately from the
415 Yocto Project.
416
417 BitBake is a generic task execution engine that allows shell and
418 Python tasks to be run efficiently and in parallel while working
419 within complex inter-task dependency constraints. In short, BitBake
420 is a build engine that works through recipes written in a specific
421 format in order to perform sets of tasks.
422
423 You can learn more about BitBake in the :doc:`BitBake User
424 Manual <bitbake:index>`.
425
426- *OpenEmbedded-Core:* OpenEmbedded-Core (OE-Core) is a common layer of
427 metadata (i.e. recipes, classes, and associated files) used by
428 OpenEmbedded-derived systems, which includes the Yocto Project. The
429 Yocto Project and the OpenEmbedded Project both maintain the
430 OpenEmbedded-Core. You can find the OE-Core metadata in the Yocto
Andrew Geissler09209ee2020-12-13 08:44:15 -0600431 Project :yocto_git:`Source Repositories </poky/tree/meta>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500432
433 Historically, the Yocto Project integrated the OE-Core metadata
434 throughout the Yocto Project source repository reference system
435 (Poky). After Yocto Project Version 1.0, the Yocto Project and
436 OpenEmbedded agreed to work together and share a common core set of
437 metadata (OE-Core), which contained much of the functionality
438 previously found in Poky. This collaboration achieved a long-standing
439 OpenEmbedded objective for having a more tightly controlled and
440 quality-assured core. The results also fit well with the Yocto
441 Project objective of achieving a smaller number of fully featured
442 tools as compared to many different ones.
443
444 Sharing a core set of metadata results in Poky as an integration
445 layer on top of OE-Core. You can see that in this
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500446 :ref:`figure <overview-manual/yp-intro:what is the yocto project?>`.
447 The Yocto Project combines various components such as BitBake, OE-Core,
448 script "glue", and documentation for its build system.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500449
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500450Reference Distribution (Poky)
451-----------------------------
452
453Poky is the Yocto Project reference distribution. It contains the
454:term:`OpenEmbedded Build System`
455(BitBake and OE-Core) as well as a set of metadata to get you started
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500456building your own distribution. See the figure in
457":ref:`overview-manual/yp-intro:what is the yocto project?`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500458section for an illustration that shows Poky and its relationship with
459other parts of the Yocto Project.
460
461To use the Yocto Project tools and components, you can download
462(``clone``) Poky and use it to bootstrap your own distribution.
463
464.. note::
465
466 Poky does not contain binary files. It is a working example of how to
467 build your own custom Linux distribution from source.
468
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500469You can read more about Poky in the
470":ref:`overview-manual/yp-intro:reference embedded distribution (poky)`"
471section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500472
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500473Packages for Finished Targets
474-----------------------------
475
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700476Here are components associated with packages for finished targets:
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500477
478- *Matchbox:* Matchbox is an Open Source, base environment for the X
479 Window System running on non-desktop, embedded platforms such as
480 handhelds, set-top boxes, kiosks, and anything else for which screen
481 space, input mechanisms, or system resources are limited.
482
483 Matchbox consists of a number of interchangeable and optional
484 applications that you can tailor to a specific, non-desktop platform
485 to enhance usability in constrained environments.
486
487 You can find the Matchbox source in the Yocto Project
488 :yocto_git:`Source Repositories <>`.
489
490- *Opkg:* Open PacKaGe management (opkg) is a lightweight package
491 management system based on the itsy package (ipkg) management system.
492 Opkg is written in C and resembles Advanced Package Tool (APT) and
493 Debian Package (dpkg) in operation.
494
495 Opkg is intended for use on embedded Linux devices and is used in
Andrew Geisslerd1e89492021-02-12 15:35:20 -0600496 this capacity in the :oe_home:`OpenEmbedded <>` and
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500497 `OpenWrt <https://openwrt.org/>`__ projects, as well as the Yocto
498 Project.
499
500 .. note::
501
502 As best it can, opkg maintains backwards compatibility with ipkg
503 and conforms to a subset of Debian's policy manual regarding
504 control files.
505
506 You can find the opkg source in the Yocto Project
507 :yocto_git:`Source Repositories <>`.
508
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500509Archived Components
510-------------------
511
512The Build Appliance is a virtual machine image that enables you to build
513and boot a custom embedded Linux image with the Yocto Project using a
514non-Linux development system.
515
516Historically, the Build Appliance was the second of three methods by
517which you could use the Yocto Project on a system that was not native to
518Linux.
519
5201. *Hob:* Hob, which is now deprecated and is no longer available since
521 the 2.1 release of the Yocto Project provided a rudimentary,
522 GUI-based interface to the Yocto Project. Toaster has fully replaced
523 Hob.
524
5252. *Build Appliance:* Post Hob, the Build Appliance became available. It
526 was never recommended that you use the Build Appliance as a
527 day-to-day production development environment with the Yocto Project.
528 Build Appliance was useful as a way to try out development in the
529 Yocto Project environment.
530
5313. *CROPS:* The final and best solution available now for developing
532 using the Yocto Project on a system not native to Linux is with
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500533 :ref:`CROPS <overview-manual/yp-intro:development tools>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500534
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500535Development Methods
536===================
537
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500538The Yocto Project development environment usually involves a
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500539:term:`Build Host` and target
540hardware. You use the Build Host to build images and develop
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500541applications, while you use the target hardware to execute deployed
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500542software.
543
544This section provides an introduction to the choices or development
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500545methods you have when setting up your Build Host. Depending on your
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500546particular workflow preference and the type of operating system your
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700547Build Host runs, you have several choices.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500548
549.. note::
550
551 For additional detail about the Yocto Project development
Andrew Geissler09209ee2020-12-13 08:44:15 -0600552 environment, see the ":doc:`/overview-manual/development-environment`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500553 chapter.
554
555- *Native Linux Host:* By far the best option for a Build Host. A
556 system running Linux as its native operating system allows you to
557 develop software by directly using the
558 :term:`BitBake` tool. You can
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500559 accomplish all aspects of development from a regular shell in a
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500560 supported Linux distribution.
561
562 For information on how to set up a Build Host on a system running
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500563 Linux as its native operating system, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600564 ":ref:`dev-manual/start:setting up a native linux host`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500565 section in the Yocto Project Development Tasks Manual.
566
567- *CROss PlatformS (CROPS):* Typically, you use
568 `CROPS <https://github.com/crops/poky-container/>`__, which leverages
569 `Docker Containers <https://www.docker.com/>`__, to set up a Build
570 Host that is not running Linux (e.g. Microsoft Windows or macOS).
571
572 .. note::
573
574 You can, however, use CROPS on a Linux-based system.
575
576 CROPS is an open source, cross-platform development framework that
577 provides an easily managed, extensible environment for building
578 binaries targeted for a variety of architectures on Windows, macOS,
579 or Linux hosts. Once the Build Host is set up using CROPS, you can
580 prepare a shell environment to mimic that of a shell being used on a
581 system natively running Linux.
582
583 For information on how to set up a Build Host with CROPS, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600584 ":ref:`dev-manual/start:setting up to use cross platforms (crops)`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500585 section in the Yocto Project Development Tasks Manual.
586
587- *Windows Subsystem For Linux (WSLv2):* You may use Windows Subsystem
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500588 For Linux v2 to set up a Build Host using Windows 10.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500589
590 .. note::
591
592 The Yocto Project is not compatible with WSLv1, it is compatible
593 but not officially supported nor validated with WSLv2, if you
594 still decide to use WSL please upgrade to WSLv2.
595
596 The Windows Subsystem For Linux allows Windows 10 to run a real Linux
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500597 kernel inside of a lightweight virtual machine (VM).
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500598
599 For information on how to set up a Build Host with WSLv2, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600600 ":ref:`dev-manual/start:setting up to use windows subsystem for linux (wslv2)`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500601 section in the Yocto Project Development Tasks Manual.
602
603- *Toaster:* Regardless of what your Build Host is running, you can use
604 Toaster to develop software using the Yocto Project. Toaster is a web
605 interface to the Yocto Project's :term:`OpenEmbedded Build System`.
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500606 The interface allows you to configure and run your builds. Information
607 about builds is collected and stored in a database. You can use Toaster
608 to configure and start builds on multiple remote build servers.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500609
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500610 For information about and how to use Toaster, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600611 :doc:`/toaster-manual/index`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500612
613Reference Embedded Distribution (Poky)
614======================================
615
616"Poky", which is pronounced *Pock*-ee, is the name of the Yocto
617Project's reference distribution or Reference OS Kit. Poky contains the
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500618:term:`OpenEmbedded Build System` (:term:`BitBake` and
619:term:`OpenEmbedded-Core (OE-Core)`) as well as a set of
620:term:`Metadata` to get you started building your own distro. In other
621words, Poky is a base specification of the functionality needed for a
622typical embedded system as well as the components from the Yocto Project
623that allow you to build a distribution into a usable binary image.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500624
625Poky is a combined repository of BitBake, OpenEmbedded-Core (which is
626found in ``meta``), ``meta-poky``, ``meta-yocto-bsp``, and documentation
627provided all together and known to work well together. You can view
628these items that make up the Poky repository in the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600629:yocto_git:`Source Repositories </poky/tree/>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500630
631.. note::
632
633 If you are interested in all the contents of the
634 poky
Andrew Geissler09209ee2020-12-13 08:44:15 -0600635 Git repository, see the ":ref:`ref-manual/structure:top-level core components`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500636 section in the Yocto Project Reference Manual.
637
638The following figure illustrates what generally comprises Poky:
639
640.. image:: figures/poky-reference-distribution.png
Andrew Geisslerd5838332022-05-27 11:33:10 -0500641 :width: 100%
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500642
643- BitBake is a task executor and scheduler that is the heart of the
644 OpenEmbedded build system.
645
646- ``meta-poky``, which is Poky-specific metadata.
647
648- ``meta-yocto-bsp``, which are Yocto Project-specific Board Support
649 Packages (BSPs).
650
651- OpenEmbedded-Core (OE-Core) metadata, which includes shared
652 configurations, global variable definitions, shared classes,
653 packaging, and recipes. Classes define the encapsulation and
654 inheritance of build logic. Recipes are the logical units of software
655 and images to be built.
656
657- Documentation, which contains the Yocto Project source files used to
658 make the set of user manuals.
659
660.. note::
661
662 While Poky is a "complete" distribution specification and is tested
663 and put through QA, you cannot use it as a product "out of the box"
664 in its current form.
665
666To use the Yocto Project tools, you can use Git to clone (download) the
667Poky repository then use your local copy of the reference distribution
668to bootstrap your own distribution.
669
670.. note::
671
672 Poky does not contain binary files. It is a working example of how to
673 build your own custom Linux distribution from source.
674
675Poky has a regular, well established, six-month release cycle under its
676own version. Major releases occur at the same time major releases (point
677releases) occur for the Yocto Project, which are typically in the Spring
678and Fall. For more information on the Yocto Project release schedule and
Andrew Geissler09209ee2020-12-13 08:44:15 -0600679cadence, see the ":doc:`/ref-manual/release-process`" chapter in the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500680Yocto Project Reference Manual.
681
682Much has been said about Poky being a "default configuration". A default
683configuration provides a starting image footprint. You can use Poky out
684of the box to create an image ranging from a shell-accessible minimal
685image all the way up to a Linux Standard Base-compliant image that uses
686a GNOME Mobile and Embedded (GMAE) based reference user interface called
687Sato.
688
689One of the most powerful properties of Poky is that every aspect of a
690build is controlled by the metadata. You can use metadata to augment
Andrew Geisslerd5838332022-05-27 11:33:10 -0500691these base image types by adding metadata :ref:`layers
692<overview-manual/yp-intro:the yocto project layer model>` that extend
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500693functionality.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500694These layers can provide, for example, an additional software stack for
695an image type, add a board support package (BSP) for additional
696hardware, or even create a new image type.
697
698Metadata is loosely grouped into configuration files or package recipes.
699A recipe is a collection of non-executable metadata used by BitBake to
700set variables or define additional build-time tasks. A recipe contains
701fields such as the recipe description, the recipe version, the license
702of the package and the upstream source repository. A recipe might also
703indicate that the build process uses autotools, make, distutils or any
704other build process, in which case the basic functionality can be
705defined by the classes it inherits from the OE-Core layer's class
706definitions in ``./meta/classes``. Within a recipe you can also define
707additional tasks as well as task prerequisites. Recipe syntax through
Patrick Williams0ca19cc2021-08-16 14:03:13 -0500708BitBake also supports both ``:prepend`` and ``:append`` operators as a
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500709method of extending task functionality. These operators inject code into
710the beginning or end of a task. For information on these BitBake
711operators, see the
712":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:appending and prepending (override style syntax)`"
713section in the BitBake User's Manual.
714
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500715The OpenEmbedded Build System Workflow
716======================================
717
718The :term:`OpenEmbedded Build System` uses a "workflow" to
719accomplish image and SDK generation. The following figure overviews that
720workflow:
721
722.. image:: figures/YP-flow-diagram.png
Andrew Geisslerd5838332022-05-27 11:33:10 -0500723 :width: 100%
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500724
725Following is a brief summary of the "workflow":
726
7271. Developers specify architecture, policies, patches and configuration
728 details.
729
7302. The build system fetches and downloads the source code from the
731 specified location. The build system supports standard methods such
732 as tarballs or source code repositories systems such as Git.
733
7343. Once source code is downloaded, the build system extracts the sources
735 into a local work area where patches are applied and common steps for
736 configuring and compiling the software are run.
737
7384. The build system then installs the software into a temporary staging
739 area where the binary package format you select (DEB, RPM, or IPK) is
740 used to roll up the software.
741
7425. Different QA and sanity checks run throughout entire build process.
743
7446. After the binaries are created, the build system generates a binary
745 package feed that is used to create the final root file image.
746
7477. The build system generates the file system image and a customized
748 Extensible SDK (eSDK) for application development in parallel.
749
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500750For a very detailed look at this workflow, see the
751":ref:`overview-manual/concepts:openembedded build system concepts`" section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500752
753Some Basic Terms
754================
755
756It helps to understand some basic fundamental terms when learning the
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700757Yocto Project. Although there is a list of terms in the ":doc:`Yocto Project
Andrew Geissler09209ee2020-12-13 08:44:15 -0600758Terms </ref-manual/terms>`" section of the Yocto Project
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500759Reference Manual, this section provides the definitions of some terms
760helpful for getting started:
761
762- *Configuration Files:* Files that hold global definitions of
763 variables, user-defined variables, and hardware configuration
764 information. These files tell the :term:`OpenEmbedded Build System`
765 what to build and
766 what to put into the image to support a particular platform.
767
768- *Extensible Software Development Kit (eSDK):* A custom SDK for
769 application developers. This eSDK allows developers to incorporate
770 their library and programming changes back into the image to make
771 their code available to other application developers. For information
Andrew Geissler09209ee2020-12-13 08:44:15 -0600772 on the eSDK, see the :doc:`/sdk-manual/index` manual.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500773
774- *Layer:* A collection of related recipes. Layers allow you to
775 consolidate related metadata to customize your build. Layers also
776 isolate information used when building for multiple architectures.
777 Layers are hierarchical in their ability to override previous
778 specifications. You can include any number of available layers from
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500779 the Yocto Project and customize the build by adding your own layers
780 after them. You can search the Layer Index for layers used within
781 Yocto Project.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500782
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500783 For more detailed information on layers, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600784 ":ref:`dev-manual/common-tasks:understanding and creating layers`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500785 section in the Yocto Project Development Tasks Manual. For a
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500786 discussion specifically on BSP Layers, see the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500787 ":ref:`bsp-guide/bsp:bsp layers`" section in the Yocto
788 Project Board Support Packages (BSP) Developer's Guide.
789
790- *Metadata:* A key element of the Yocto Project is the Metadata that
791 is used to construct a Linux distribution and is contained in the
792 files that the OpenEmbedded build system parses when building an
793 image. In general, Metadata includes recipes, configuration files,
794 and other information that refers to the build instructions
795 themselves, as well as the data used to control what things get built
796 and the effects of the build. Metadata also includes commands and
797 data used to indicate what versions of software are used, from where
798 they are obtained, and changes or additions to the software itself
799 (patches or auxiliary files) that are used to fix bugs or customize
800 the software for use in a particular situation. OpenEmbedded-Core is
801 an important set of validated metadata.
802
803- *OpenEmbedded Build System:* The terms "BitBake" and "build system"
804 are sometimes used for the OpenEmbedded Build System.
805
806 BitBake is a task scheduler and execution engine that parses
807 instructions (i.e. recipes) and configuration data. After a parsing
808 phase, BitBake creates a dependency tree to order the compilation,
809 schedules the compilation of the included code, and finally executes
810 the building of the specified custom Linux image (distribution).
811 BitBake is similar to the ``make`` tool.
812
813 During a build process, the build system tracks dependencies and
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500814 performs a native or cross-compilation of each package. As a first
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500815 step in a cross-build setup, the framework attempts to create a
816 cross-compiler toolchain (i.e. Extensible SDK) suited for the target
817 platform.
818
819- *OpenEmbedded-Core (OE-Core):* OE-Core is metadata comprised of
820 foundation recipes, classes, and associated files that are meant to
821 be common among many different OpenEmbedded-derived systems,
822 including the Yocto Project. OE-Core is a curated subset of an
823 original repository developed by the OpenEmbedded community that has
824 been pared down into a smaller, core set of continuously validated
825 recipes. The result is a tightly controlled and quality-assured core
826 set of recipes.
827
828 You can see the Metadata in the ``meta`` directory of the Yocto
Andrew Geissler09209ee2020-12-13 08:44:15 -0600829 Project :yocto_git:`Source Repositories <>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500830
831- *Packages:* In the context of the Yocto Project, this term refers to
832 a recipe's packaged output produced by BitBake (i.e. a "baked
833 recipe"). A package is generally the compiled binaries produced from
834 the recipe's sources. You "bake" something by running it through
835 BitBake.
836
837 It is worth noting that the term "package" can, in general, have
838 subtle meanings. For example, the packages referred to in the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600839 ":ref:`ref-manual/system-requirements:required packages for the build host`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500840 section in the Yocto Project Reference Manual are compiled binaries
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500841 that, when installed, add functionality to your host Linux
842 distribution.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500843
844 Another point worth noting is that historically within the Yocto
Andrew Geissler615f2f12022-07-15 14:00:58 -0500845 Project, recipes were referred to as packages --- thus, the existence
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500846 of several BitBake variables that are seemingly mis-named, (e.g.
847 :term:`PR`,
848 :term:`PV`, and
849 :term:`PE`).
850
851- *Poky:* Poky is a reference embedded distribution and a reference
852 test configuration. Poky provides the following:
853
854 - A base-level functional distro used to illustrate how to customize
855 a distribution.
856
857 - A means by which to test the Yocto Project components (i.e. Poky
858 is used to validate the Yocto Project).
859
860 - A vehicle through which you can download the Yocto Project.
861
862 Poky is not a product level distro. Rather, it is a good starting
863 point for customization.
864
865 .. note::
866
867 Poky is an integration layer on top of OE-Core.
868
869- *Recipe:* The most common form of metadata. A recipe contains a list
870 of settings and tasks (i.e. instructions) for building packages that
871 are then used to build the binary image. A recipe describes where you
872 get source code and which patches to apply. Recipes describe
873 dependencies for libraries or for other recipes as well as
874 configuration and compilation options. Related recipes are
875 consolidated into a layer.