blob: 7aee9db04f8cc7d17d8fede67205e5e6c071288d [file] [log] [blame]
Andrew Geisslerf0343792020-11-18 10:42:21 -06001.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Andrew Geisslerc9f78652020-09-18 14:11:35 -05002
3*****************************
4Introducing the Yocto Project
5*****************************
6
7What is the Yocto Project?
8==========================
9
10The Yocto Project is an open source collaboration project that helps
11developers create custom Linux-based systems that are designed for
12embedded products regardless of the product's hardware architecture.
13Yocto Project provides a flexible toolset and a development environment
14that allows embedded device developers across the world to collaborate
15through shared technologies, software stacks, configurations, and best
16practices used to create these tailored Linux images.
17
18Thousands of developers worldwide have discovered that Yocto Project
19provides advantages in both systems and applications development,
20archival and management benefits, and customizations used for speed,
21footprint, and memory utilization. The project is a standard when it
22comes to delivering embedded software stacks. The project allows
23software customizations and build interchange for multiple hardware
24platforms as well as software stacks that can be maintained and scaled.
25
26.. image:: figures/key-dev-elements.png
27 :align: center
28
29For further introductory information on the Yocto Project, you might be
30interested in this
31`article <https://www.embedded.com/electronics-blogs/say-what-/4458600/Why-the-Yocto-Project-for-my-IoT-Project->`__
32by Drew Moseley and in this short introductory
33`video <https://www.youtube.com/watch?v=utZpKM7i5Z4>`__.
34
35The remainder of this section overviews advantages and challenges tied
36to the Yocto Project.
37
Andrew Geisslerc9f78652020-09-18 14:11:35 -050038Features
39--------
40
William A. Kennington IIIac69b482021-06-02 12:28:27 -070041Here are features and advantages of the Yocto Project:
Andrew Geisslerc9f78652020-09-18 14:11:35 -050042
Andrew Geissler3b8a17c2021-04-15 15:55:55 -050043- *Widely Adopted Across the Industry:* Many semiconductor, operating
44 system, software, and service vendors adopt and support the Yocto
45 Project in their products and services. For a look at the Yocto
Andrew Geisslerc9f78652020-09-18 14:11:35 -050046 Project community and the companies involved with the Yocto Project,
47 see the "COMMUNITY" and "ECOSYSTEM" tabs on the
48 :yocto_home:`Yocto Project <>` home page.
49
50- *Architecture Agnostic:* Yocto Project supports Intel, ARM, MIPS,
51 AMD, PPC and other architectures. Most ODMs, OSVs, and chip vendors
52 create and supply BSPs that support their hardware. If you have
53 custom silicon, you can create a BSP that supports that architecture.
54
Andrew Geissler3b8a17c2021-04-15 15:55:55 -050055 Aside from broad architecture support, the Yocto Project fully
56 supports a wide range of devices emulated by the Quick EMUlator
Andrew Geisslerc9f78652020-09-18 14:11:35 -050057 (QEMU).
58
59- *Images and Code Transfer Easily:* Yocto Project output can easily
60 move between architectures without moving to new development
61 environments. Additionally, if you have used the Yocto Project to
62 create an image or application and you find yourself not able to
63 support it, commercial Linux vendors such as Wind River, Mentor
64 Graphics, Timesys, and ENEA could take it and provide ongoing
65 support. These vendors have offerings that are built using the Yocto
66 Project.
67
68- *Flexibility:* Corporations use the Yocto Project many different
69 ways. One example is to create an internal Linux distribution as a
70 code base the corporation can use across multiple product groups.
71 Through customization and layering, a project group can leverage the
72 base Linux distribution to create a distribution that works for their
73 product needs.
74
75- *Ideal for Constrained Embedded and IoT devices:* Unlike a full Linux
76 distribution, you can use the Yocto Project to create exactly what
77 you need for embedded devices. You only add the feature support or
78 packages that you absolutely need for the device. For devices that
79 have display hardware, you can use available system components such
Andrew Geissler3b8a17c2021-04-15 15:55:55 -050080 as X11, Wayland, GTK+, Qt, Clutter, and SDL (among others) to create
81 a rich user experience. For devices that do not have a display or
82 where you want to use alternative UI frameworks, you can choose to
83 not build these components.
Andrew Geisslerc9f78652020-09-18 14:11:35 -050084
85- *Comprehensive Toolchain Capabilities:* Toolchains for supported
86 architectures satisfy most use cases. However, if your hardware
87 supports features that are not part of a standard toolchain, you can
88 easily customize that toolchain through specification of
89 platform-specific tuning parameters. And, should you need to use a
90 third-party toolchain, mechanisms built into the Yocto Project allow
91 for that.
92
93- *Mechanism Rules Over Policy:* Focusing on mechanism rather than
94 policy ensures that you are free to set policies based on the needs
95 of your design instead of adopting decisions enforced by some system
96 software provider.
97
Andrew Geissler3b8a17c2021-04-15 15:55:55 -050098- *Uses a Layer Model:* The Yocto Project :ref:`layer
99 infrastructure <overview-manual/yp-intro:the yocto project layer model>`
100 groups related functionality into separate bundles. You can incrementally
101 add these grouped functionalities to your project as needed. Using layers to
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500102 isolate and group functionality reduces project complexity and
103 redundancy, allows you to easily extend the system, make
104 customizations, and keep functionality organized.
105
106- *Supports Partial Builds:* You can build and rebuild individual
107 packages as needed. Yocto Project accomplishes this through its
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500108 :ref:`overview-manual/concepts:shared state cache` (sstate) scheme.
109 Being able to build and debug components individually eases project
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500110 development.
111
112- *Releases According to a Strict Schedule:* Major releases occur on a
Andrew Geissler09209ee2020-12-13 08:44:15 -0600113 :doc:`six-month cycle </ref-manual/release-process>`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500114 predictably in October and April. The most recent two releases
115 support point releases to address common vulnerabilities and
116 exposures. This predictability is crucial for projects based on the
117 Yocto Project and allows development teams to plan activities.
118
119- *Rich Ecosystem of Individuals and Organizations:* For open source
120 projects, the value of community is very important. Support forums,
121 expertise, and active developers who continue to push the Yocto
122 Project forward are readily available.
123
124- *Binary Reproducibility:* The Yocto Project allows you to be very
125 specific about dependencies and achieves very high percentages of
126 binary reproducibility (e.g. 99.8% for ``core-image-minimal``). When
127 distributions are not specific about which packages are pulled in and
128 in what order to support dependencies, other build systems can
129 arbitrarily include packages.
130
131- *License Manifest:* The Yocto Project provides a :ref:`license
Andrew Geissler09209ee2020-12-13 08:44:15 -0600132 manifest <dev-manual/common-tasks:maintaining open source license compliance during your product's lifecycle>`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500133 for review by people who need to track the use of open source
134 licenses (e.g. legal teams).
135
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500136Challenges
137----------
138
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700139Here are challenges you might encounter when developing using the Yocto Project:
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500140
141- *Steep Learning Curve:* The Yocto Project has a steep learning curve
142 and has many different ways to accomplish similar tasks. It can be
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700143 difficult to choose between such ways.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500144
145- *Understanding What Changes You Need to Make For Your Design Requires
146 Some Research:* Beyond the simple tutorial stage, understanding what
147 changes need to be made for your particular design can require a
148 significant amount of research and investigation. For information
149 that helps you transition from trying out the Yocto Project to using
150 it for your project, see the ":ref:`what-i-wish-id-known:what i wish i'd known about yocto project`" and
151 ":ref:`transitioning-to-a-custom-environment:transitioning to a custom environment for systems development`"
152 documents on the Yocto Project website.
153
154- *Project Workflow Could Be Confusing:* The `Yocto Project
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500155 workflow <overview-manual/development-environment:the yocto project development environment>`
156 could be confusing if you are used to traditional desktop and server
157 software development.
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700158 In a desktop development environment, there are mechanisms to easily pull
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500159 and install new packages, which are typically pre-compiled binaries
160 from servers accessible over the Internet. Using the Yocto Project,
161 you must modify your configuration and rebuild to add additional
162 packages.
163
164- *Working in a Cross-Build Environment Can Feel Unfamiliar:* When
165 developing code to run on a target, compilation, execution, and
166 testing done on the actual target can be faster than running a
167 BitBake build on a development host and then deploying binaries to
168 the target for test. While the Yocto Project does support development
169 tools on the target, the additional step of integrating your changes
170 back into the Yocto Project build environment would be required.
171 Yocto Project supports an intermediate approach that involves making
172 changes on the development system within the BitBake environment and
173 then deploying only the updated packages to the target.
174
175 The Yocto Project :term:`OpenEmbedded Build System`
176 produces packages
177 in standard formats (i.e. RPM, DEB, IPK, and TAR). You can deploy
178 these packages into the running system on the target by using
179 utilities on the target such as ``rpm`` or ``ipk``.
180
181- *Initial Build Times Can be Significant:* Long initial build times
182 are unfortunately unavoidable due to the large number of packages
183 initially built from scratch for a fully functioning Linux system.
184 Once that initial build is completed, however, the shared-state
185 (sstate) cache mechanism Yocto Project uses keeps the system from
186 rebuilding packages that have not been "touched" since the last
187 build. The sstate mechanism significantly reduces times for
188 successive builds.
189
190The Yocto Project Layer Model
191=============================
192
193The Yocto Project's "Layer Model" is a development model for embedded
194and IoT Linux creation that distinguishes the Yocto Project from other
195simple build systems. The Layer Model simultaneously supports
196collaboration and customization. Layers are repositories that contain
197related sets of instructions that tell the :term:`OpenEmbedded Build System`
198what to do. You can
199collaborate, share, and reuse layers.
200
201Layers can contain changes to previous instructions or settings at any
202time. This powerful override capability is what allows you to customize
203previously supplied collaborative or community layers to suit your
204product requirements.
205
206You use different layers to logically separate information in your
207build. As an example, you could have BSP, GUI, distro configuration,
208middleware, or application layers. Putting your entire build into one
209layer limits and complicates future customization and reuse. Isolating
210information into layers, on the other hand, helps simplify future
211customizations and reuse. You might find it tempting to keep everything
212in one layer when working on a single project. However, the more modular
213your Metadata, the easier it is to cope with future changes.
214
215.. note::
216
217 - Use Board Support Package (BSP) layers from silicon vendors when
218 possible.
219
220 - Familiarize yourself with the `Yocto Project curated layer
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500221 index <https://www.yoctoproject.org/software-overview/layers/>`__
Andrew Geisslerd1e89492021-02-12 15:35:20 -0600222 or the :oe_layerindex:`OpenEmbedded layer index <>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500223 The latter contains more layers but they are less universally
224 validated.
225
226 - Layers support the inclusion of technologies, hardware components,
227 and software components. The :ref:`Yocto Project
Andrew Geissler09209ee2020-12-13 08:44:15 -0600228 Compatible <dev-manual/common-tasks:making sure your layer is compatible with yocto project>`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500229 designation provides a minimum level of standardization that
230 contributes to a strong ecosystem. "YP Compatible" is applied to
231 appropriate products and software components such as BSPs, other
232 OE-compatible layers, and related open-source projects, allowing
233 the producer to use Yocto Project badges and branding assets.
234
235To illustrate how layers are used to keep things modular, consider
236machine customizations. These types of customizations typically reside
237in a special layer, rather than a general layer, called a BSP Layer.
238Furthermore, the machine customizations should be isolated from recipes
239and Metadata that support a new GUI environment, for example. This
240situation gives you a couple of layers: one for the machine
241configurations, and one for the GUI environment. It is important to
242understand, however, that the BSP layer can still make machine-specific
243additions to recipes within the GUI environment layer without polluting
244the GUI layer itself with those machine-specific changes. You can
245accomplish this through a recipe that is a BitBake append
246(``.bbappend``) file, which is described later in this section.
247
248.. note::
249
250 For general information on BSP layer structure, see the
Andrew Geissler09036742021-06-25 14:25:14 -0500251 :doc:`/bsp-guide/index`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500252
253The :term:`Source Directory`
254contains both general layers and BSP layers right out of the box. You
255can easily identify layers that ship with a Yocto Project release in the
256Source Directory by their names. Layers typically have names that begin
257with the string ``meta-``.
258
259.. note::
260
261 It is not a requirement that a layer name begin with the prefix
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500262 ``meta-``, but it is a commonly accepted standard in the Yocto Project
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500263 community.
264
Andrew Geissler09209ee2020-12-13 08:44:15 -0600265For example, if you were to examine the :yocto_git:`tree view </poky/tree/>`
266of the ``poky`` repository, you will see several layers: ``meta``,
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500267``meta-skeleton``, ``meta-selftest``, ``meta-poky``, and
268``meta-yocto-bsp``. Each of these repositories represents a distinct
269layer.
270
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500271For procedures on how to create layers, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600272":ref:`dev-manual/common-tasks:understanding and creating layers`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500273section in the Yocto Project Development Tasks Manual.
274
275Components and Tools
276====================
277
278The Yocto Project employs a collection of components and tools used by
279the project itself, by project developers, and by those using the Yocto
280Project. These components and tools are open source projects and
281metadata that are separate from the reference distribution
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500282(:term:`Poky`) and the :term:`OpenEmbedded Build System`. Most of the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500283components and tools are downloaded separately.
284
285This section provides brief overviews of the components and tools
286associated with the Yocto Project.
287
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500288Development Tools
289-----------------
290
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700291Here are tools that help you develop images and applications using
292the Yocto Project:
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500293
294- *CROPS:* `CROPS <https://github.com/crops/poky-container/>`__ is an
295 open source, cross-platform development framework that leverages
296 `Docker Containers <https://www.docker.com/>`__. CROPS provides an
297 easily managed, extensible environment that allows you to build
298 binaries for a variety of architectures on Windows, Linux and Mac OS
299 X hosts.
300
301- *devtool:* This command-line tool is available as part of the
302 extensible SDK (eSDK) and is its cornerstone. You can use ``devtool``
303 to help build, test, and package software within the eSDK. You can
304 use the tool to optionally integrate what you build into an image
305 built by the OpenEmbedded build system.
306
307 The ``devtool`` command employs a number of sub-commands that allow
308 you to add, modify, and upgrade recipes. As with the OpenEmbedded
309 build system, "recipes" represent software packages within
310 ``devtool``. When you use ``devtool add``, a recipe is automatically
311 created. When you use ``devtool modify``, the specified existing
312 recipe is used in order to determine where to get the source code and
313 how to patch it. In both cases, an environment is set up so that when
314 you build the recipe a source tree that is under your control is used
315 in order to allow you to make changes to the source as desired. By
316 default, both new recipes and the source go into a "workspace"
317 directory under the eSDK. The ``devtool upgrade`` command updates an
318 existing recipe so that you can build it for an updated set of source
319 files.
320
321 You can read about the ``devtool`` workflow in the Yocto Project
322 Application Development and Extensible Software Development Kit
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500323 (eSDK) Manual in the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600324 ":ref:`sdk-manual/extensible:using \`\`devtool\`\` in your sdk workflow`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500325 section.
326
327- *Extensible Software Development Kit (eSDK):* The eSDK provides a
328 cross-development toolchain and libraries tailored to the contents of
329 a specific image. The eSDK makes it easy to add new applications and
330 libraries to an image, modify the source for an existing component,
331 test changes on the target hardware, and integrate into the rest of
332 the OpenEmbedded build system. The eSDK gives you a toolchain
333 experience supplemented with the powerful set of ``devtool`` commands
334 tailored for the Yocto Project environment.
335
Andrew Geissler09209ee2020-12-13 08:44:15 -0600336 For information on the eSDK, see the :doc:`/sdk-manual/index` Manual.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500337
338- *Toaster:* Toaster is a web interface to the Yocto Project
339 OpenEmbedded build system. Toaster allows you to configure, run, and
340 view information about builds. For information on Toaster, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600341 :doc:`/toaster-manual/index`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500342
343Production Tools
344----------------
345
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700346Here are tools that help with production related activities using the
347Yocto Project:
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500348
349- *Auto Upgrade Helper:* This utility when used in conjunction with the
350 :term:`OpenEmbedded Build System`
351 (BitBake and
352 OE-Core) automatically generates upgrades for recipes that are based
353 on new versions of the recipes published upstream. See
Andrew Geissler09209ee2020-12-13 08:44:15 -0600354 :ref:`dev-manual/common-tasks:using the auto upgrade helper (auh)`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500355 for how to set it up.
356
357- *Recipe Reporting System:* The Recipe Reporting System tracks recipe
358 versions available for Yocto Project. The main purpose of the system
359 is to help you manage the recipes you maintain and to offer a dynamic
360 overview of the project. The Recipe Reporting System is built on top
Andrew Geisslerd1e89492021-02-12 15:35:20 -0600361 of the :oe_layerindex:`OpenEmbedded Layer Index <>`, which
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500362 is a website that indexes OpenEmbedded-Core layers.
363
364- *Patchwork:* `Patchwork <http://jk.ozlabs.org/projects/patchwork/>`__
365 is a fork of a project originally started by
Andrew Geisslerd1e89492021-02-12 15:35:20 -0600366 `OzLabs <https://ozlabs.org/>`__. The project is a web-based tracking
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500367 system designed to streamline the process of bringing contributions
368 into a project. The Yocto Project uses Patchwork as an organizational
369 tool to handle patches, which number in the thousands for every
370 release.
371
372- *AutoBuilder:* AutoBuilder is a project that automates build tests
373 and quality assurance (QA). By using the public AutoBuilder, anyone
374 can determine the status of the current "master" branch of Poky.
375
376 .. note::
377
378 AutoBuilder is based on buildbot.
379
380 A goal of the Yocto Project is to lead the open source industry with
381 a project that automates testing and QA procedures. In doing so, the
382 project encourages a development community that publishes QA and test
383 plans, publicly demonstrates QA and test plans, and encourages
384 development of tools that automate and test and QA procedures for the
385 benefit of the development community.
386
387 You can learn more about the AutoBuilder used by the Yocto Project
Andrew Geissler09209ee2020-12-13 08:44:15 -0600388 Autobuilder :doc:`here </test-manual/understand-autobuilder>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500389
390- *Cross-Prelink:* Prelinking is the process of pre-computing the load
391 addresses and link tables generated by the dynamic linker as compared
392 to doing this at runtime. Doing this ahead of time results in
393 performance improvements when the application is launched and reduced
394 memory usage for libraries shared by many applications.
395
396 Historically, cross-prelink is a variant of prelink, which was
397 conceived by `Jakub
Andrew Geisslerd1e89492021-02-12 15:35:20 -0600398 Jelínek <https://people.redhat.com/jakub/prelink.pdf>`__ a number of
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500399 years ago. Both prelink and cross-prelink are maintained in the same
400 repository albeit on separate branches. By providing an emulated
401 runtime dynamic linker (i.e. ``glibc``-derived ``ld.so`` emulation),
402 the cross-prelink project extends the prelink software's ability to
403 prelink a sysroot environment. Additionally, the cross-prelink
404 software enables the ability to work in sysroot style environments.
405
406 The dynamic linker determines standard load address calculations
407 based on a variety of factors such as mapping addresses, library
408 usage, and library function conflicts. The prelink tool uses this
409 information, from the dynamic linker, to determine unique load
410 addresses for executable and linkable format (ELF) binaries that are
411 shared libraries and dynamically linked. The prelink tool modifies
412 these ELF binaries with the pre-computed information. The result is
413 faster loading and often lower memory consumption because more of the
414 library code can be re-used from shared Copy-On-Write (COW) pages.
415
416 The original upstream prelink project only supports running prelink
417 on the end target device due to the reliance on the target device's
418 dynamic linker. This restriction causes issues when developing a
419 cross-compiled system. The cross-prelink adds a synthesized dynamic
420 loader that runs on the host, thus permitting cross-prelinking
421 without ever having to run on a read-write target filesystem.
422
423- *Pseudo:* Pseudo is the Yocto Project implementation of
424 `fakeroot <http://man.he.net/man1/fakeroot>`__, which is used to run
425 commands in an environment that seemingly has root privileges.
426
427 During a build, it can be necessary to perform operations that
428 require system administrator privileges. For example, file ownership
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500429 or permissions might need to be defined. Pseudo is a tool that you
430 can either use directly or through the environment variable
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700431 ``LD_PRELOAD``. Either method allows these operations to succeed
432 even without system administrator privileges.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500433
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500434 Thanks to Pseudo, the Yocto Project never needs root privileges to
435 build images for your target system.
436
437 You can read more about Pseudo in the
438 ":ref:`overview-manual/concepts:fakeroot and pseudo`" section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500439
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500440Open-Embedded Build System Components
441-------------------------------------
442
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700443Here are components associated with the :term:`OpenEmbedded Build System`:
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500444
445- *BitBake:* BitBake is a core component of the Yocto Project and is
446 used by the OpenEmbedded build system to build images. While BitBake
447 is key to the build system, BitBake is maintained separately from the
448 Yocto Project.
449
450 BitBake is a generic task execution engine that allows shell and
451 Python tasks to be run efficiently and in parallel while working
452 within complex inter-task dependency constraints. In short, BitBake
453 is a build engine that works through recipes written in a specific
454 format in order to perform sets of tasks.
455
456 You can learn more about BitBake in the :doc:`BitBake User
457 Manual <bitbake:index>`.
458
459- *OpenEmbedded-Core:* OpenEmbedded-Core (OE-Core) is a common layer of
460 metadata (i.e. recipes, classes, and associated files) used by
461 OpenEmbedded-derived systems, which includes the Yocto Project. The
462 Yocto Project and the OpenEmbedded Project both maintain the
463 OpenEmbedded-Core. You can find the OE-Core metadata in the Yocto
Andrew Geissler09209ee2020-12-13 08:44:15 -0600464 Project :yocto_git:`Source Repositories </poky/tree/meta>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500465
466 Historically, the Yocto Project integrated the OE-Core metadata
467 throughout the Yocto Project source repository reference system
468 (Poky). After Yocto Project Version 1.0, the Yocto Project and
469 OpenEmbedded agreed to work together and share a common core set of
470 metadata (OE-Core), which contained much of the functionality
471 previously found in Poky. This collaboration achieved a long-standing
472 OpenEmbedded objective for having a more tightly controlled and
473 quality-assured core. The results also fit well with the Yocto
474 Project objective of achieving a smaller number of fully featured
475 tools as compared to many different ones.
476
477 Sharing a core set of metadata results in Poky as an integration
478 layer on top of OE-Core. You can see that in this
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500479 :ref:`figure <overview-manual/yp-intro:what is the yocto project?>`.
480 The Yocto Project combines various components such as BitBake, OE-Core,
481 script "glue", and documentation for its build system.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500482
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500483Reference Distribution (Poky)
484-----------------------------
485
486Poky is the Yocto Project reference distribution. It contains the
487:term:`OpenEmbedded Build System`
488(BitBake and OE-Core) as well as a set of metadata to get you started
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500489building your own distribution. See the figure in
490":ref:`overview-manual/yp-intro:what is the yocto project?`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500491section for an illustration that shows Poky and its relationship with
492other parts of the Yocto Project.
493
494To use the Yocto Project tools and components, you can download
495(``clone``) Poky and use it to bootstrap your own distribution.
496
497.. note::
498
499 Poky does not contain binary files. It is a working example of how to
500 build your own custom Linux distribution from source.
501
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500502You can read more about Poky in the
503":ref:`overview-manual/yp-intro:reference embedded distribution (poky)`"
504section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500505
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500506Packages for Finished Targets
507-----------------------------
508
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700509Here are components associated with packages for finished targets:
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500510
511- *Matchbox:* Matchbox is an Open Source, base environment for the X
512 Window System running on non-desktop, embedded platforms such as
513 handhelds, set-top boxes, kiosks, and anything else for which screen
514 space, input mechanisms, or system resources are limited.
515
516 Matchbox consists of a number of interchangeable and optional
517 applications that you can tailor to a specific, non-desktop platform
518 to enhance usability in constrained environments.
519
520 You can find the Matchbox source in the Yocto Project
521 :yocto_git:`Source Repositories <>`.
522
523- *Opkg:* Open PacKaGe management (opkg) is a lightweight package
524 management system based on the itsy package (ipkg) management system.
525 Opkg is written in C and resembles Advanced Package Tool (APT) and
526 Debian Package (dpkg) in operation.
527
528 Opkg is intended for use on embedded Linux devices and is used in
Andrew Geisslerd1e89492021-02-12 15:35:20 -0600529 this capacity in the :oe_home:`OpenEmbedded <>` and
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500530 `OpenWrt <https://openwrt.org/>`__ projects, as well as the Yocto
531 Project.
532
533 .. note::
534
535 As best it can, opkg maintains backwards compatibility with ipkg
536 and conforms to a subset of Debian's policy manual regarding
537 control files.
538
539 You can find the opkg source in the Yocto Project
540 :yocto_git:`Source Repositories <>`.
541
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500542Archived Components
543-------------------
544
545The Build Appliance is a virtual machine image that enables you to build
546and boot a custom embedded Linux image with the Yocto Project using a
547non-Linux development system.
548
549Historically, the Build Appliance was the second of three methods by
550which you could use the Yocto Project on a system that was not native to
551Linux.
552
5531. *Hob:* Hob, which is now deprecated and is no longer available since
554 the 2.1 release of the Yocto Project provided a rudimentary,
555 GUI-based interface to the Yocto Project. Toaster has fully replaced
556 Hob.
557
5582. *Build Appliance:* Post Hob, the Build Appliance became available. It
559 was never recommended that you use the Build Appliance as a
560 day-to-day production development environment with the Yocto Project.
561 Build Appliance was useful as a way to try out development in the
562 Yocto Project environment.
563
5643. *CROPS:* The final and best solution available now for developing
565 using the Yocto Project on a system not native to Linux is with
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500566 :ref:`CROPS <overview-manual/yp-intro:development tools>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500567
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500568Development Methods
569===================
570
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500571The Yocto Project development environment usually involves a
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500572:term:`Build Host` and target
573hardware. You use the Build Host to build images and develop
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500574applications, while you use the target hardware to execute deployed
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500575software.
576
577This section provides an introduction to the choices or development
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500578methods you have when setting up your Build Host. Depending on your
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500579particular workflow preference and the type of operating system your
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700580Build Host runs, you have several choices.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500581
582.. note::
583
584 For additional detail about the Yocto Project development
Andrew Geissler09209ee2020-12-13 08:44:15 -0600585 environment, see the ":doc:`/overview-manual/development-environment`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500586 chapter.
587
588- *Native Linux Host:* By far the best option for a Build Host. A
589 system running Linux as its native operating system allows you to
590 develop software by directly using the
591 :term:`BitBake` tool. You can
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500592 accomplish all aspects of development from a regular shell in a
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500593 supported Linux distribution.
594
595 For information on how to set up a Build Host on a system running
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500596 Linux as its native operating system, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600597 ":ref:`dev-manual/start:setting up a native linux host`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500598 section in the Yocto Project Development Tasks Manual.
599
600- *CROss PlatformS (CROPS):* Typically, you use
601 `CROPS <https://github.com/crops/poky-container/>`__, which leverages
602 `Docker Containers <https://www.docker.com/>`__, to set up a Build
603 Host that is not running Linux (e.g. Microsoft Windows or macOS).
604
605 .. note::
606
607 You can, however, use CROPS on a Linux-based system.
608
609 CROPS is an open source, cross-platform development framework that
610 provides an easily managed, extensible environment for building
611 binaries targeted for a variety of architectures on Windows, macOS,
612 or Linux hosts. Once the Build Host is set up using CROPS, you can
613 prepare a shell environment to mimic that of a shell being used on a
614 system natively running Linux.
615
616 For information on how to set up a Build Host with CROPS, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600617 ":ref:`dev-manual/start:setting up to use cross platforms (crops)`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500618 section in the Yocto Project Development Tasks Manual.
619
620- *Windows Subsystem For Linux (WSLv2):* You may use Windows Subsystem
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500621 For Linux v2 to set up a Build Host using Windows 10.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500622
623 .. note::
624
625 The Yocto Project is not compatible with WSLv1, it is compatible
626 but not officially supported nor validated with WSLv2, if you
627 still decide to use WSL please upgrade to WSLv2.
628
629 The Windows Subsystem For Linux allows Windows 10 to run a real Linux
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500630 kernel inside of a lightweight virtual machine (VM).
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500631
632 For information on how to set up a Build Host with WSLv2, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600633 ":ref:`dev-manual/start:setting up to use windows subsystem for linux (wslv2)`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500634 section in the Yocto Project Development Tasks Manual.
635
636- *Toaster:* Regardless of what your Build Host is running, you can use
637 Toaster to develop software using the Yocto Project. Toaster is a web
638 interface to the Yocto Project's :term:`OpenEmbedded Build System`.
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500639 The interface allows you to configure and run your builds. Information
640 about builds is collected and stored in a database. You can use Toaster
641 to configure and start builds on multiple remote build servers.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500642
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500643 For information about and how to use Toaster, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600644 :doc:`/toaster-manual/index`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500645
646Reference Embedded Distribution (Poky)
647======================================
648
649"Poky", which is pronounced *Pock*-ee, is the name of the Yocto
650Project's reference distribution or Reference OS Kit. Poky contains the
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500651:term:`OpenEmbedded Build System` (:term:`BitBake` and
652:term:`OpenEmbedded-Core (OE-Core)`) as well as a set of
653:term:`Metadata` to get you started building your own distro. In other
654words, Poky is a base specification of the functionality needed for a
655typical embedded system as well as the components from the Yocto Project
656that allow you to build a distribution into a usable binary image.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500657
658Poky is a combined repository of BitBake, OpenEmbedded-Core (which is
659found in ``meta``), ``meta-poky``, ``meta-yocto-bsp``, and documentation
660provided all together and known to work well together. You can view
661these items that make up the Poky repository in the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600662:yocto_git:`Source Repositories </poky/tree/>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500663
664.. note::
665
666 If you are interested in all the contents of the
667 poky
Andrew Geissler09209ee2020-12-13 08:44:15 -0600668 Git repository, see the ":ref:`ref-manual/structure:top-level core components`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500669 section in the Yocto Project Reference Manual.
670
671The following figure illustrates what generally comprises Poky:
672
673.. image:: figures/poky-reference-distribution.png
674 :align: center
675
676- BitBake is a task executor and scheduler that is the heart of the
677 OpenEmbedded build system.
678
679- ``meta-poky``, which is Poky-specific metadata.
680
681- ``meta-yocto-bsp``, which are Yocto Project-specific Board Support
682 Packages (BSPs).
683
684- OpenEmbedded-Core (OE-Core) metadata, which includes shared
685 configurations, global variable definitions, shared classes,
686 packaging, and recipes. Classes define the encapsulation and
687 inheritance of build logic. Recipes are the logical units of software
688 and images to be built.
689
690- Documentation, which contains the Yocto Project source files used to
691 make the set of user manuals.
692
693.. note::
694
695 While Poky is a "complete" distribution specification and is tested
696 and put through QA, you cannot use it as a product "out of the box"
697 in its current form.
698
699To use the Yocto Project tools, you can use Git to clone (download) the
700Poky repository then use your local copy of the reference distribution
701to bootstrap your own distribution.
702
703.. note::
704
705 Poky does not contain binary files. It is a working example of how to
706 build your own custom Linux distribution from source.
707
708Poky has a regular, well established, six-month release cycle under its
709own version. Major releases occur at the same time major releases (point
710releases) occur for the Yocto Project, which are typically in the Spring
711and Fall. For more information on the Yocto Project release schedule and
Andrew Geissler09209ee2020-12-13 08:44:15 -0600712cadence, see the ":doc:`/ref-manual/release-process`" chapter in the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500713Yocto Project Reference Manual.
714
715Much has been said about Poky being a "default configuration". A default
716configuration provides a starting image footprint. You can use Poky out
717of the box to create an image ranging from a shell-accessible minimal
718image all the way up to a Linux Standard Base-compliant image that uses
719a GNOME Mobile and Embedded (GMAE) based reference user interface called
720Sato.
721
722One of the most powerful properties of Poky is that every aspect of a
723build is controlled by the metadata. You can use metadata to augment
724these base image types by adding metadata
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500725`layers <overview-manual/yp-intro:the yocto project layer model>` that extend
726functionality.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500727These layers can provide, for example, an additional software stack for
728an image type, add a board support package (BSP) for additional
729hardware, or even create a new image type.
730
731Metadata is loosely grouped into configuration files or package recipes.
732A recipe is a collection of non-executable metadata used by BitBake to
733set variables or define additional build-time tasks. A recipe contains
734fields such as the recipe description, the recipe version, the license
735of the package and the upstream source repository. A recipe might also
736indicate that the build process uses autotools, make, distutils or any
737other build process, in which case the basic functionality can be
738defined by the classes it inherits from the OE-Core layer's class
739definitions in ``./meta/classes``. Within a recipe you can also define
740additional tasks as well as task prerequisites. Recipe syntax through
Patrick Williams0ca19cc2021-08-16 14:03:13 -0500741BitBake also supports both ``:prepend`` and ``:append`` operators as a
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500742method of extending task functionality. These operators inject code into
743the beginning or end of a task. For information on these BitBake
744operators, see the
745":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:appending and prepending (override style syntax)`"
746section in the BitBake User's Manual.
747
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500748The OpenEmbedded Build System Workflow
749======================================
750
751The :term:`OpenEmbedded Build System` uses a "workflow" to
752accomplish image and SDK generation. The following figure overviews that
753workflow:
754
755.. image:: figures/YP-flow-diagram.png
756 :align: center
757
758Following is a brief summary of the "workflow":
759
7601. Developers specify architecture, policies, patches and configuration
761 details.
762
7632. The build system fetches and downloads the source code from the
764 specified location. The build system supports standard methods such
765 as tarballs or source code repositories systems such as Git.
766
7673. Once source code is downloaded, the build system extracts the sources
768 into a local work area where patches are applied and common steps for
769 configuring and compiling the software are run.
770
7714. The build system then installs the software into a temporary staging
772 area where the binary package format you select (DEB, RPM, or IPK) is
773 used to roll up the software.
774
7755. Different QA and sanity checks run throughout entire build process.
776
7776. After the binaries are created, the build system generates a binary
778 package feed that is used to create the final root file image.
779
7807. The build system generates the file system image and a customized
781 Extensible SDK (eSDK) for application development in parallel.
782
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500783For a very detailed look at this workflow, see the
784":ref:`overview-manual/concepts:openembedded build system concepts`" section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500785
786Some Basic Terms
787================
788
789It helps to understand some basic fundamental terms when learning the
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700790Yocto Project. Although there is a list of terms in the ":doc:`Yocto Project
Andrew Geissler09209ee2020-12-13 08:44:15 -0600791Terms </ref-manual/terms>`" section of the Yocto Project
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500792Reference Manual, this section provides the definitions of some terms
793helpful for getting started:
794
795- *Configuration Files:* Files that hold global definitions of
796 variables, user-defined variables, and hardware configuration
797 information. These files tell the :term:`OpenEmbedded Build System`
798 what to build and
799 what to put into the image to support a particular platform.
800
801- *Extensible Software Development Kit (eSDK):* A custom SDK for
802 application developers. This eSDK allows developers to incorporate
803 their library and programming changes back into the image to make
804 their code available to other application developers. For information
Andrew Geissler09209ee2020-12-13 08:44:15 -0600805 on the eSDK, see the :doc:`/sdk-manual/index` manual.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500806
807- *Layer:* A collection of related recipes. Layers allow you to
808 consolidate related metadata to customize your build. Layers also
809 isolate information used when building for multiple architectures.
810 Layers are hierarchical in their ability to override previous
811 specifications. You can include any number of available layers from
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500812 the Yocto Project and customize the build by adding your own layers
813 after them. You can search the Layer Index for layers used within
814 Yocto Project.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500815
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500816 For more detailed information on layers, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600817 ":ref:`dev-manual/common-tasks:understanding and creating layers`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500818 section in the Yocto Project Development Tasks Manual. For a
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500819 discussion specifically on BSP Layers, see the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500820 ":ref:`bsp-guide/bsp:bsp layers`" section in the Yocto
821 Project Board Support Packages (BSP) Developer's Guide.
822
823- *Metadata:* A key element of the Yocto Project is the Metadata that
824 is used to construct a Linux distribution and is contained in the
825 files that the OpenEmbedded build system parses when building an
826 image. In general, Metadata includes recipes, configuration files,
827 and other information that refers to the build instructions
828 themselves, as well as the data used to control what things get built
829 and the effects of the build. Metadata also includes commands and
830 data used to indicate what versions of software are used, from where
831 they are obtained, and changes or additions to the software itself
832 (patches or auxiliary files) that are used to fix bugs or customize
833 the software for use in a particular situation. OpenEmbedded-Core is
834 an important set of validated metadata.
835
836- *OpenEmbedded Build System:* The terms "BitBake" and "build system"
837 are sometimes used for the OpenEmbedded Build System.
838
839 BitBake is a task scheduler and execution engine that parses
840 instructions (i.e. recipes) and configuration data. After a parsing
841 phase, BitBake creates a dependency tree to order the compilation,
842 schedules the compilation of the included code, and finally executes
843 the building of the specified custom Linux image (distribution).
844 BitBake is similar to the ``make`` tool.
845
846 During a build process, the build system tracks dependencies and
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500847 performs a native or cross-compilation of each package. As a first
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500848 step in a cross-build setup, the framework attempts to create a
849 cross-compiler toolchain (i.e. Extensible SDK) suited for the target
850 platform.
851
852- *OpenEmbedded-Core (OE-Core):* OE-Core is metadata comprised of
853 foundation recipes, classes, and associated files that are meant to
854 be common among many different OpenEmbedded-derived systems,
855 including the Yocto Project. OE-Core is a curated subset of an
856 original repository developed by the OpenEmbedded community that has
857 been pared down into a smaller, core set of continuously validated
858 recipes. The result is a tightly controlled and quality-assured core
859 set of recipes.
860
861 You can see the Metadata in the ``meta`` directory of the Yocto
Andrew Geissler09209ee2020-12-13 08:44:15 -0600862 Project :yocto_git:`Source Repositories <>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500863
864- *Packages:* In the context of the Yocto Project, this term refers to
865 a recipe's packaged output produced by BitBake (i.e. a "baked
866 recipe"). A package is generally the compiled binaries produced from
867 the recipe's sources. You "bake" something by running it through
868 BitBake.
869
870 It is worth noting that the term "package" can, in general, have
871 subtle meanings. For example, the packages referred to in the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600872 ":ref:`ref-manual/system-requirements:required packages for the build host`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500873 section in the Yocto Project Reference Manual are compiled binaries
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500874 that, when installed, add functionality to your host Linux
875 distribution.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500876
877 Another point worth noting is that historically within the Yocto
878 Project, recipes were referred to as packages - thus, the existence
879 of several BitBake variables that are seemingly mis-named, (e.g.
880 :term:`PR`,
881 :term:`PV`, and
882 :term:`PE`).
883
884- *Poky:* Poky is a reference embedded distribution and a reference
885 test configuration. Poky provides the following:
886
887 - A base-level functional distro used to illustrate how to customize
888 a distribution.
889
890 - A means by which to test the Yocto Project components (i.e. Poky
891 is used to validate the Yocto Project).
892
893 - A vehicle through which you can download the Yocto Project.
894
895 Poky is not a product level distro. Rather, it is a good starting
896 point for customization.
897
898 .. note::
899
900 Poky is an integration layer on top of OE-Core.
901
902- *Recipe:* The most common form of metadata. A recipe contains a list
903 of settings and tasks (i.e. instructions) for building packages that
904 are then used to build the binary image. A recipe describes where you
905 get source code and which patches to apply. Recipes describe
906 dependencies for libraries or for other recipes as well as
907 configuration and compilation options. Related recipes are
908 consolidated into a layer.