blob: 5cdab7ca4afbf52e346fd4c259a6bf26c22a2576 [file] [log] [blame]
Andrew Geissler4873add2020-11-02 18:44:49 -06001.. SPDX-License-Identifier: CC-BY-2.0-UK
Andrew Geisslerc9f78652020-09-18 14:11:35 -05002
3*****************************
4Introducing the Yocto Project
5*****************************
6
7What is the Yocto Project?
8==========================
9
10The Yocto Project is an open source collaboration project that helps
11developers create custom Linux-based systems that are designed for
12embedded products regardless of the product's hardware architecture.
13Yocto Project provides a flexible toolset and a development environment
14that allows embedded device developers across the world to collaborate
15through shared technologies, software stacks, configurations, and best
16practices used to create these tailored Linux images.
17
18Thousands of developers worldwide have discovered that Yocto Project
19provides advantages in both systems and applications development,
20archival and management benefits, and customizations used for speed,
21footprint, and memory utilization. The project is a standard when it
22comes to delivering embedded software stacks. The project allows
23software customizations and build interchange for multiple hardware
24platforms as well as software stacks that can be maintained and scaled.
25
26.. image:: figures/key-dev-elements.png
27 :align: center
28
29For further introductory information on the Yocto Project, you might be
30interested in this
31`article <https://www.embedded.com/electronics-blogs/say-what-/4458600/Why-the-Yocto-Project-for-my-IoT-Project->`__
32by Drew Moseley and in this short introductory
33`video <https://www.youtube.com/watch?v=utZpKM7i5Z4>`__.
34
35The remainder of this section overviews advantages and challenges tied
36to the Yocto Project.
37
38.. _gs-features:
39
40Features
41--------
42
43The following list describes features and advantages of the Yocto
44Project:
45
46- *Widely Adopted Across the Industry:* Semiconductor, operating
47 system, software, and service vendors exist whose products and
48 services adopt and support the Yocto Project. For a look at the Yocto
49 Project community and the companies involved with the Yocto Project,
50 see the "COMMUNITY" and "ECOSYSTEM" tabs on the
51 :yocto_home:`Yocto Project <>` home page.
52
53- *Architecture Agnostic:* Yocto Project supports Intel, ARM, MIPS,
54 AMD, PPC and other architectures. Most ODMs, OSVs, and chip vendors
55 create and supply BSPs that support their hardware. If you have
56 custom silicon, you can create a BSP that supports that architecture.
57
58 Aside from lots of architecture support, the Yocto Project fully
59 supports a wide range of device emulation through the Quick EMUlator
60 (QEMU).
61
62- *Images and Code Transfer Easily:* Yocto Project output can easily
63 move between architectures without moving to new development
64 environments. Additionally, if you have used the Yocto Project to
65 create an image or application and you find yourself not able to
66 support it, commercial Linux vendors such as Wind River, Mentor
67 Graphics, Timesys, and ENEA could take it and provide ongoing
68 support. These vendors have offerings that are built using the Yocto
69 Project.
70
71- *Flexibility:* Corporations use the Yocto Project many different
72 ways. One example is to create an internal Linux distribution as a
73 code base the corporation can use across multiple product groups.
74 Through customization and layering, a project group can leverage the
75 base Linux distribution to create a distribution that works for their
76 product needs.
77
78- *Ideal for Constrained Embedded and IoT devices:* Unlike a full Linux
79 distribution, you can use the Yocto Project to create exactly what
80 you need for embedded devices. You only add the feature support or
81 packages that you absolutely need for the device. For devices that
82 have display hardware, you can use available system components such
83 as X11, GTK+, Qt, Clutter, and SDL (among others) to create a rich
84 user experience. For devices that do not have a display or where you
85 want to use alternative UI frameworks, you can choose to not install
86 these components.
87
88- *Comprehensive Toolchain Capabilities:* Toolchains for supported
89 architectures satisfy most use cases. However, if your hardware
90 supports features that are not part of a standard toolchain, you can
91 easily customize that toolchain through specification of
92 platform-specific tuning parameters. And, should you need to use a
93 third-party toolchain, mechanisms built into the Yocto Project allow
94 for that.
95
96- *Mechanism Rules Over Policy:* Focusing on mechanism rather than
97 policy ensures that you are free to set policies based on the needs
98 of your design instead of adopting decisions enforced by some system
99 software provider.
100
101- *Uses a Layer Model:* The Yocto Project `layer
102 infrastructure <#the-yocto-project-layer-model>`__ groups related
103 functionality into separate bundles. You can incrementally add these
104 grouped functionalities to your project as needed. Using layers to
105 isolate and group functionality reduces project complexity and
106 redundancy, allows you to easily extend the system, make
107 customizations, and keep functionality organized.
108
109- *Supports Partial Builds:* You can build and rebuild individual
110 packages as needed. Yocto Project accomplishes this through its
111 `shared-state cache <#shared-state-cache>`__ (sstate) scheme. Being
112 able to build and debug components individually eases project
113 development.
114
115- *Releases According to a Strict Schedule:* Major releases occur on a
116 :doc:`six-month cycle <../ref-manual/ref-release-process>`
117 predictably in October and April. The most recent two releases
118 support point releases to address common vulnerabilities and
119 exposures. This predictability is crucial for projects based on the
120 Yocto Project and allows development teams to plan activities.
121
122- *Rich Ecosystem of Individuals and Organizations:* For open source
123 projects, the value of community is very important. Support forums,
124 expertise, and active developers who continue to push the Yocto
125 Project forward are readily available.
126
127- *Binary Reproducibility:* The Yocto Project allows you to be very
128 specific about dependencies and achieves very high percentages of
129 binary reproducibility (e.g. 99.8% for ``core-image-minimal``). When
130 distributions are not specific about which packages are pulled in and
131 in what order to support dependencies, other build systems can
132 arbitrarily include packages.
133
134- *License Manifest:* The Yocto Project provides a :ref:`license
135 manifest <dev-manual/dev-manual-common-tasks:maintaining open source license compliance during your product's lifecycle>`
136 for review by people who need to track the use of open source
137 licenses (e.g. legal teams).
138
139.. _gs-challenges:
140
141Challenges
142----------
143
144The following list presents challenges you might encounter when
145developing using the Yocto Project:
146
147- *Steep Learning Curve:* The Yocto Project has a steep learning curve
148 and has many different ways to accomplish similar tasks. It can be
149 difficult to choose how to proceed when varying methods exist by
150 which to accomplish a given task.
151
152- *Understanding What Changes You Need to Make For Your Design Requires
153 Some Research:* Beyond the simple tutorial stage, understanding what
154 changes need to be made for your particular design can require a
155 significant amount of research and investigation. For information
156 that helps you transition from trying out the Yocto Project to using
157 it for your project, see the ":ref:`what-i-wish-id-known:what i wish i'd known about yocto project`" and
158 ":ref:`transitioning-to-a-custom-environment:transitioning to a custom environment for systems development`"
159 documents on the Yocto Project website.
160
161- *Project Workflow Could Be Confusing:* The `Yocto Project
162 workflow <#overview-development-environment>`__ could be confusing if
163 you are used to traditional desktop and server software development.
164 In a desktop development environment, mechanisms exist to easily pull
165 and install new packages, which are typically pre-compiled binaries
166 from servers accessible over the Internet. Using the Yocto Project,
167 you must modify your configuration and rebuild to add additional
168 packages.
169
170- *Working in a Cross-Build Environment Can Feel Unfamiliar:* When
171 developing code to run on a target, compilation, execution, and
172 testing done on the actual target can be faster than running a
173 BitBake build on a development host and then deploying binaries to
174 the target for test. While the Yocto Project does support development
175 tools on the target, the additional step of integrating your changes
176 back into the Yocto Project build environment would be required.
177 Yocto Project supports an intermediate approach that involves making
178 changes on the development system within the BitBake environment and
179 then deploying only the updated packages to the target.
180
181 The Yocto Project :term:`OpenEmbedded Build System`
182 produces packages
183 in standard formats (i.e. RPM, DEB, IPK, and TAR). You can deploy
184 these packages into the running system on the target by using
185 utilities on the target such as ``rpm`` or ``ipk``.
186
187- *Initial Build Times Can be Significant:* Long initial build times
188 are unfortunately unavoidable due to the large number of packages
189 initially built from scratch for a fully functioning Linux system.
190 Once that initial build is completed, however, the shared-state
191 (sstate) cache mechanism Yocto Project uses keeps the system from
192 rebuilding packages that have not been "touched" since the last
193 build. The sstate mechanism significantly reduces times for
194 successive builds.
195
196The Yocto Project Layer Model
197=============================
198
199The Yocto Project's "Layer Model" is a development model for embedded
200and IoT Linux creation that distinguishes the Yocto Project from other
201simple build systems. The Layer Model simultaneously supports
202collaboration and customization. Layers are repositories that contain
203related sets of instructions that tell the :term:`OpenEmbedded Build System`
204what to do. You can
205collaborate, share, and reuse layers.
206
207Layers can contain changes to previous instructions or settings at any
208time. This powerful override capability is what allows you to customize
209previously supplied collaborative or community layers to suit your
210product requirements.
211
212You use different layers to logically separate information in your
213build. As an example, you could have BSP, GUI, distro configuration,
214middleware, or application layers. Putting your entire build into one
215layer limits and complicates future customization and reuse. Isolating
216information into layers, on the other hand, helps simplify future
217customizations and reuse. You might find it tempting to keep everything
218in one layer when working on a single project. However, the more modular
219your Metadata, the easier it is to cope with future changes.
220
221.. note::
222
223 - Use Board Support Package (BSP) layers from silicon vendors when
224 possible.
225
226 - Familiarize yourself with the `Yocto Project curated layer
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500227 index <https://www.yoctoproject.org/software-overview/layers/>`__
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500228 or the `OpenEmbedded layer
229 index <http://layers.openembedded.org/layerindex/branch/master/layers/>`__.
230 The latter contains more layers but they are less universally
231 validated.
232
233 - Layers support the inclusion of technologies, hardware components,
234 and software components. The :ref:`Yocto Project
235 Compatible <dev-manual/dev-manual-common-tasks:making sure your layer is compatible with yocto project>`
236 designation provides a minimum level of standardization that
237 contributes to a strong ecosystem. "YP Compatible" is applied to
238 appropriate products and software components such as BSPs, other
239 OE-compatible layers, and related open-source projects, allowing
240 the producer to use Yocto Project badges and branding assets.
241
242To illustrate how layers are used to keep things modular, consider
243machine customizations. These types of customizations typically reside
244in a special layer, rather than a general layer, called a BSP Layer.
245Furthermore, the machine customizations should be isolated from recipes
246and Metadata that support a new GUI environment, for example. This
247situation gives you a couple of layers: one for the machine
248configurations, and one for the GUI environment. It is important to
249understand, however, that the BSP layer can still make machine-specific
250additions to recipes within the GUI environment layer without polluting
251the GUI layer itself with those machine-specific changes. You can
252accomplish this through a recipe that is a BitBake append
253(``.bbappend``) file, which is described later in this section.
254
255.. note::
256
257 For general information on BSP layer structure, see the
258 :doc:`../bsp-guide/bsp-guide`
259 .
260
261The :term:`Source Directory`
262contains both general layers and BSP layers right out of the box. You
263can easily identify layers that ship with a Yocto Project release in the
264Source Directory by their names. Layers typically have names that begin
265with the string ``meta-``.
266
267.. note::
268
269 It is not a requirement that a layer name begin with the prefix
270 meta-
271 , but it is a commonly accepted standard in the Yocto Project
272 community.
273
274For example, if you were to examine the `tree
275view <https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/>`__ of the
276``poky`` repository, you will see several layers: ``meta``,
277``meta-skeleton``, ``meta-selftest``, ``meta-poky``, and
278``meta-yocto-bsp``. Each of these repositories represents a distinct
279layer.
280
281For procedures on how to create layers, see the
282":ref:`dev-manual/dev-manual-common-tasks:understanding and creating layers`"
283section in the Yocto Project Development Tasks Manual.
284
285Components and Tools
286====================
287
288The Yocto Project employs a collection of components and tools used by
289the project itself, by project developers, and by those using the Yocto
290Project. These components and tools are open source projects and
291metadata that are separate from the reference distribution
292(:term:`Poky`) and the
293:term:`OpenEmbedded Build System`. Most of the
294components and tools are downloaded separately.
295
296This section provides brief overviews of the components and tools
297associated with the Yocto Project.
298
299.. _gs-development-tools:
300
301Development Tools
302-----------------
303
304The following list consists of tools that help you develop images and
305applications using the Yocto Project:
306
307- *CROPS:* `CROPS <https://github.com/crops/poky-container/>`__ is an
308 open source, cross-platform development framework that leverages
309 `Docker Containers <https://www.docker.com/>`__. CROPS provides an
310 easily managed, extensible environment that allows you to build
311 binaries for a variety of architectures on Windows, Linux and Mac OS
312 X hosts.
313
314- *devtool:* This command-line tool is available as part of the
315 extensible SDK (eSDK) and is its cornerstone. You can use ``devtool``
316 to help build, test, and package software within the eSDK. You can
317 use the tool to optionally integrate what you build into an image
318 built by the OpenEmbedded build system.
319
320 The ``devtool`` command employs a number of sub-commands that allow
321 you to add, modify, and upgrade recipes. As with the OpenEmbedded
322 build system, "recipes" represent software packages within
323 ``devtool``. When you use ``devtool add``, a recipe is automatically
324 created. When you use ``devtool modify``, the specified existing
325 recipe is used in order to determine where to get the source code and
326 how to patch it. In both cases, an environment is set up so that when
327 you build the recipe a source tree that is under your control is used
328 in order to allow you to make changes to the source as desired. By
329 default, both new recipes and the source go into a "workspace"
330 directory under the eSDK. The ``devtool upgrade`` command updates an
331 existing recipe so that you can build it for an updated set of source
332 files.
333
334 You can read about the ``devtool`` workflow in the Yocto Project
335 Application Development and Extensible Software Development Kit
336 (eSDK) Manual in the
337 ":ref:`sdk-manual/sdk-extensible:using \`\`devtool\`\` in your sdk workflow`"
338 section.
339
340- *Extensible Software Development Kit (eSDK):* The eSDK provides a
341 cross-development toolchain and libraries tailored to the contents of
342 a specific image. The eSDK makes it easy to add new applications and
343 libraries to an image, modify the source for an existing component,
344 test changes on the target hardware, and integrate into the rest of
345 the OpenEmbedded build system. The eSDK gives you a toolchain
346 experience supplemented with the powerful set of ``devtool`` commands
347 tailored for the Yocto Project environment.
348
349 For information on the eSDK, see the :doc:`../sdk-manual/sdk-manual` Manual.
350
351- *Toaster:* Toaster is a web interface to the Yocto Project
352 OpenEmbedded build system. Toaster allows you to configure, run, and
353 view information about builds. For information on Toaster, see the
354 :doc:`../toaster-manual/toaster-manual`.
355
356.. _gs-production-tools:
357
358Production Tools
359----------------
360
361The following list consists of tools that help production related
362activities using the Yocto Project:
363
364- *Auto Upgrade Helper:* This utility when used in conjunction with the
365 :term:`OpenEmbedded Build System`
366 (BitBake and
367 OE-Core) automatically generates upgrades for recipes that are based
368 on new versions of the recipes published upstream. See
369 :ref:`dev-manual/dev-manual-common-tasks:using the auto upgrade helper (auh)`
370 for how to set it up.
371
372- *Recipe Reporting System:* The Recipe Reporting System tracks recipe
373 versions available for Yocto Project. The main purpose of the system
374 is to help you manage the recipes you maintain and to offer a dynamic
375 overview of the project. The Recipe Reporting System is built on top
376 of the `OpenEmbedded Layer
377 Index <http://layers.openembedded.org/layerindex/layers/>`__, which
378 is a website that indexes OpenEmbedded-Core layers.
379
380- *Patchwork:* `Patchwork <http://jk.ozlabs.org/projects/patchwork/>`__
381 is a fork of a project originally started by
382 `OzLabs <http://ozlabs.org/>`__. The project is a web-based tracking
383 system designed to streamline the process of bringing contributions
384 into a project. The Yocto Project uses Patchwork as an organizational
385 tool to handle patches, which number in the thousands for every
386 release.
387
388- *AutoBuilder:* AutoBuilder is a project that automates build tests
389 and quality assurance (QA). By using the public AutoBuilder, anyone
390 can determine the status of the current "master" branch of Poky.
391
392 .. note::
393
394 AutoBuilder is based on buildbot.
395
396 A goal of the Yocto Project is to lead the open source industry with
397 a project that automates testing and QA procedures. In doing so, the
398 project encourages a development community that publishes QA and test
399 plans, publicly demonstrates QA and test plans, and encourages
400 development of tools that automate and test and QA procedures for the
401 benefit of the development community.
402
403 You can learn more about the AutoBuilder used by the Yocto Project
404 Autobuilder :doc:`here <../test-manual/test-manual-understand-autobuilder>`.
405
406- *Cross-Prelink:* Prelinking is the process of pre-computing the load
407 addresses and link tables generated by the dynamic linker as compared
408 to doing this at runtime. Doing this ahead of time results in
409 performance improvements when the application is launched and reduced
410 memory usage for libraries shared by many applications.
411
412 Historically, cross-prelink is a variant of prelink, which was
413 conceived by `Jakub
414 Jelínek <http://people.redhat.com/jakub/prelink.pdf>`__ a number of
415 years ago. Both prelink and cross-prelink are maintained in the same
416 repository albeit on separate branches. By providing an emulated
417 runtime dynamic linker (i.e. ``glibc``-derived ``ld.so`` emulation),
418 the cross-prelink project extends the prelink software's ability to
419 prelink a sysroot environment. Additionally, the cross-prelink
420 software enables the ability to work in sysroot style environments.
421
422 The dynamic linker determines standard load address calculations
423 based on a variety of factors such as mapping addresses, library
424 usage, and library function conflicts. The prelink tool uses this
425 information, from the dynamic linker, to determine unique load
426 addresses for executable and linkable format (ELF) binaries that are
427 shared libraries and dynamically linked. The prelink tool modifies
428 these ELF binaries with the pre-computed information. The result is
429 faster loading and often lower memory consumption because more of the
430 library code can be re-used from shared Copy-On-Write (COW) pages.
431
432 The original upstream prelink project only supports running prelink
433 on the end target device due to the reliance on the target device's
434 dynamic linker. This restriction causes issues when developing a
435 cross-compiled system. The cross-prelink adds a synthesized dynamic
436 loader that runs on the host, thus permitting cross-prelinking
437 without ever having to run on a read-write target filesystem.
438
439- *Pseudo:* Pseudo is the Yocto Project implementation of
440 `fakeroot <http://man.he.net/man1/fakeroot>`__, which is used to run
441 commands in an environment that seemingly has root privileges.
442
443 During a build, it can be necessary to perform operations that
444 require system administrator privileges. For example, file ownership
445 or permissions might need definition. Pseudo is a tool that you can
446 either use directly or through the environment variable
447 ``LD_PRELOAD``. Either method allows these operations to succeed as
448 if system administrator privileges exist even when they do not.
449
450 You can read more about Pseudo in the "`Fakeroot and
451 Pseudo <#fakeroot-and-pseudo>`__" section.
452
453.. _gs-openembedded-build-system:
454
455Open-Embedded Build System Components
456-------------------------------------
457
458The following list consists of components associated with the
459:term:`OpenEmbedded Build System`:
460
461- *BitBake:* BitBake is a core component of the Yocto Project and is
462 used by the OpenEmbedded build system to build images. While BitBake
463 is key to the build system, BitBake is maintained separately from the
464 Yocto Project.
465
466 BitBake is a generic task execution engine that allows shell and
467 Python tasks to be run efficiently and in parallel while working
468 within complex inter-task dependency constraints. In short, BitBake
469 is a build engine that works through recipes written in a specific
470 format in order to perform sets of tasks.
471
472 You can learn more about BitBake in the :doc:`BitBake User
473 Manual <bitbake:index>`.
474
475- *OpenEmbedded-Core:* OpenEmbedded-Core (OE-Core) is a common layer of
476 metadata (i.e. recipes, classes, and associated files) used by
477 OpenEmbedded-derived systems, which includes the Yocto Project. The
478 Yocto Project and the OpenEmbedded Project both maintain the
479 OpenEmbedded-Core. You can find the OE-Core metadata in the Yocto
480 Project :yocto_git:`Source Repositories </cgit/cgit.cgi/poky/tree/meta>`.
481
482 Historically, the Yocto Project integrated the OE-Core metadata
483 throughout the Yocto Project source repository reference system
484 (Poky). After Yocto Project Version 1.0, the Yocto Project and
485 OpenEmbedded agreed to work together and share a common core set of
486 metadata (OE-Core), which contained much of the functionality
487 previously found in Poky. This collaboration achieved a long-standing
488 OpenEmbedded objective for having a more tightly controlled and
489 quality-assured core. The results also fit well with the Yocto
490 Project objective of achieving a smaller number of fully featured
491 tools as compared to many different ones.
492
493 Sharing a core set of metadata results in Poky as an integration
494 layer on top of OE-Core. You can see that in this
495 `figure <#yp-key-dev-elements>`__. The Yocto Project combines various
496 components such as BitBake, OE-Core, script "glue", and documentation
497 for its build system.
498
499.. _gs-reference-distribution-poky:
500
501Reference Distribution (Poky)
502-----------------------------
503
504Poky is the Yocto Project reference distribution. It contains the
505:term:`OpenEmbedded Build System`
506(BitBake and OE-Core) as well as a set of metadata to get you started
507building your own distribution. See the
508`figure <#what-is-the-yocto-project>`__ in "What is the Yocto Project?"
509section for an illustration that shows Poky and its relationship with
510other parts of the Yocto Project.
511
512To use the Yocto Project tools and components, you can download
513(``clone``) Poky and use it to bootstrap your own distribution.
514
515.. note::
516
517 Poky does not contain binary files. It is a working example of how to
518 build your own custom Linux distribution from source.
519
520You can read more about Poky in the "`Reference Embedded Distribution
521(Poky) <#reference-embedded-distribution>`__" section.
522
523.. _gs-packages-for-finished-targets:
524
525Packages for Finished Targets
526-----------------------------
527
528The following lists components associated with packages for finished
529targets:
530
531- *Matchbox:* Matchbox is an Open Source, base environment for the X
532 Window System running on non-desktop, embedded platforms such as
533 handhelds, set-top boxes, kiosks, and anything else for which screen
534 space, input mechanisms, or system resources are limited.
535
536 Matchbox consists of a number of interchangeable and optional
537 applications that you can tailor to a specific, non-desktop platform
538 to enhance usability in constrained environments.
539
540 You can find the Matchbox source in the Yocto Project
541 :yocto_git:`Source Repositories <>`.
542
543- *Opkg:* Open PacKaGe management (opkg) is a lightweight package
544 management system based on the itsy package (ipkg) management system.
545 Opkg is written in C and resembles Advanced Package Tool (APT) and
546 Debian Package (dpkg) in operation.
547
548 Opkg is intended for use on embedded Linux devices and is used in
549 this capacity in the
550 `OpenEmbedded <http://www.openembedded.org/wiki/Main_Page>`__ and
551 `OpenWrt <https://openwrt.org/>`__ projects, as well as the Yocto
552 Project.
553
554 .. note::
555
556 As best it can, opkg maintains backwards compatibility with ipkg
557 and conforms to a subset of Debian's policy manual regarding
558 control files.
559
560 You can find the opkg source in the Yocto Project
561 :yocto_git:`Source Repositories <>`.
562
563.. _gs-archived-components:
564
565Archived Components
566-------------------
567
568The Build Appliance is a virtual machine image that enables you to build
569and boot a custom embedded Linux image with the Yocto Project using a
570non-Linux development system.
571
572Historically, the Build Appliance was the second of three methods by
573which you could use the Yocto Project on a system that was not native to
574Linux.
575
5761. *Hob:* Hob, which is now deprecated and is no longer available since
577 the 2.1 release of the Yocto Project provided a rudimentary,
578 GUI-based interface to the Yocto Project. Toaster has fully replaced
579 Hob.
580
5812. *Build Appliance:* Post Hob, the Build Appliance became available. It
582 was never recommended that you use the Build Appliance as a
583 day-to-day production development environment with the Yocto Project.
584 Build Appliance was useful as a way to try out development in the
585 Yocto Project environment.
586
5873. *CROPS:* The final and best solution available now for developing
588 using the Yocto Project on a system not native to Linux is with
589 `CROPS <#gs-crops-overview>`__.
590
591.. _gs-development-methods:
592
593Development Methods
594===================
595
596The Yocto Project development environment usually involves a
597:term:`Build Host` and target
598hardware. You use the Build Host to build images and develop
599applications, while you use the target hardware to test deployed
600software.
601
602This section provides an introduction to the choices or development
603methods you have when setting up your Build Host. Depending on the your
604particular workflow preference and the type of operating system your
605Build Host runs, several choices exist that allow you to use the Yocto
606Project.
607
608.. note::
609
610 For additional detail about the Yocto Project development
611 environment, see the ":doc:`overview-manual-development-environment`"
612 chapter.
613
614- *Native Linux Host:* By far the best option for a Build Host. A
615 system running Linux as its native operating system allows you to
616 develop software by directly using the
617 :term:`BitBake` tool. You can
618 accomplish all aspects of development from a familiar shell of a
619 supported Linux distribution.
620
621 For information on how to set up a Build Host on a system running
622 Linux as its native operating system, see the
623 ":ref:`dev-manual/dev-manual-start:setting up a native linux host`"
624 section in the Yocto Project Development Tasks Manual.
625
626- *CROss PlatformS (CROPS):* Typically, you use
627 `CROPS <https://github.com/crops/poky-container/>`__, which leverages
628 `Docker Containers <https://www.docker.com/>`__, to set up a Build
629 Host that is not running Linux (e.g. Microsoft Windows or macOS).
630
631 .. note::
632
633 You can, however, use CROPS on a Linux-based system.
634
635 CROPS is an open source, cross-platform development framework that
636 provides an easily managed, extensible environment for building
637 binaries targeted for a variety of architectures on Windows, macOS,
638 or Linux hosts. Once the Build Host is set up using CROPS, you can
639 prepare a shell environment to mimic that of a shell being used on a
640 system natively running Linux.
641
642 For information on how to set up a Build Host with CROPS, see the
643 ":ref:`dev-manual/dev-manual-start:setting up to use cross platforms (crops)`"
644 section in the Yocto Project Development Tasks Manual.
645
646- *Windows Subsystem For Linux (WSLv2):* You may use Windows Subsystem
647 For Linux v2 to set up a build host using Windows 10.
648
649 .. note::
650
651 The Yocto Project is not compatible with WSLv1, it is compatible
652 but not officially supported nor validated with WSLv2, if you
653 still decide to use WSL please upgrade to WSLv2.
654
655 The Windows Subsystem For Linux allows Windows 10 to run a real Linux
656 kernel inside of a lightweight utility virtual machine (VM) using
657 virtualization technology.
658
659 For information on how to set up a Build Host with WSLv2, see the
660 ":ref:`dev-manual/dev-manual-start:setting up to use windows subsystem for linux (wslv2)`"
661 section in the Yocto Project Development Tasks Manual.
662
663- *Toaster:* Regardless of what your Build Host is running, you can use
664 Toaster to develop software using the Yocto Project. Toaster is a web
665 interface to the Yocto Project's :term:`OpenEmbedded Build System`.
666 The interface
667 enables you to configure and run your builds. Information about
668 builds is collected and stored in a database. You can use Toaster to
669 configure and start builds on multiple remote build servers.
670
671 For information about and how to use Toaster, see the
672 :doc:`../toaster-manual/toaster-manual`.
673
674.. _reference-embedded-distribution:
675
676Reference Embedded Distribution (Poky)
677======================================
678
679"Poky", which is pronounced *Pock*-ee, is the name of the Yocto
680Project's reference distribution or Reference OS Kit. Poky contains the
681:term:`OpenEmbedded Build System`
682(:term:`BitBake` and
683:term:`OpenEmbedded-Core (OE-Core)`) as well as a set
684of :term:`Metadata` to get you started
685building your own distro. In other words, Poky is a base specification
686of the functionality needed for a typical embedded system as well as the
687components from the Yocto Project that allow you to build a distribution
688into a usable binary image.
689
690Poky is a combined repository of BitBake, OpenEmbedded-Core (which is
691found in ``meta``), ``meta-poky``, ``meta-yocto-bsp``, and documentation
692provided all together and known to work well together. You can view
693these items that make up the Poky repository in the
694:yocto_git:`Source Repositories </cgit/cgit.cgi/poky/tree/>`.
695
696.. note::
697
698 If you are interested in all the contents of the
699 poky
700 Git repository, see the ":ref:`ref-manual/ref-structure:top-level core components`"
701 section in the Yocto Project Reference Manual.
702
703The following figure illustrates what generally comprises Poky:
704
705.. image:: figures/poky-reference-distribution.png
706 :align: center
707
708- BitBake is a task executor and scheduler that is the heart of the
709 OpenEmbedded build system.
710
711- ``meta-poky``, which is Poky-specific metadata.
712
713- ``meta-yocto-bsp``, which are Yocto Project-specific Board Support
714 Packages (BSPs).
715
716- OpenEmbedded-Core (OE-Core) metadata, which includes shared
717 configurations, global variable definitions, shared classes,
718 packaging, and recipes. Classes define the encapsulation and
719 inheritance of build logic. Recipes are the logical units of software
720 and images to be built.
721
722- Documentation, which contains the Yocto Project source files used to
723 make the set of user manuals.
724
725.. note::
726
727 While Poky is a "complete" distribution specification and is tested
728 and put through QA, you cannot use it as a product "out of the box"
729 in its current form.
730
731To use the Yocto Project tools, you can use Git to clone (download) the
732Poky repository then use your local copy of the reference distribution
733to bootstrap your own distribution.
734
735.. note::
736
737 Poky does not contain binary files. It is a working example of how to
738 build your own custom Linux distribution from source.
739
740Poky has a regular, well established, six-month release cycle under its
741own version. Major releases occur at the same time major releases (point
742releases) occur for the Yocto Project, which are typically in the Spring
743and Fall. For more information on the Yocto Project release schedule and
744cadence, see the ":doc:`../ref-manual/ref-release-process`" chapter in the
745Yocto Project Reference Manual.
746
747Much has been said about Poky being a "default configuration". A default
748configuration provides a starting image footprint. You can use Poky out
749of the box to create an image ranging from a shell-accessible minimal
750image all the way up to a Linux Standard Base-compliant image that uses
751a GNOME Mobile and Embedded (GMAE) based reference user interface called
752Sato.
753
754One of the most powerful properties of Poky is that every aspect of a
755build is controlled by the metadata. You can use metadata to augment
756these base image types by adding metadata
757`layers <#the-yocto-project-layer-model>`__ that extend functionality.
758These layers can provide, for example, an additional software stack for
759an image type, add a board support package (BSP) for additional
760hardware, or even create a new image type.
761
762Metadata is loosely grouped into configuration files or package recipes.
763A recipe is a collection of non-executable metadata used by BitBake to
764set variables or define additional build-time tasks. A recipe contains
765fields such as the recipe description, the recipe version, the license
766of the package and the upstream source repository. A recipe might also
767indicate that the build process uses autotools, make, distutils or any
768other build process, in which case the basic functionality can be
769defined by the classes it inherits from the OE-Core layer's class
770definitions in ``./meta/classes``. Within a recipe you can also define
771additional tasks as well as task prerequisites. Recipe syntax through
772BitBake also supports both ``_prepend`` and ``_append`` operators as a
773method of extending task functionality. These operators inject code into
774the beginning or end of a task. For information on these BitBake
775operators, see the
776":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:appending and prepending (override style syntax)`"
777section in the BitBake User's Manual.
778
779.. _openembedded-build-system-workflow:
780
781The OpenEmbedded Build System Workflow
782======================================
783
784The :term:`OpenEmbedded Build System` uses a "workflow" to
785accomplish image and SDK generation. The following figure overviews that
786workflow:
787
788.. image:: figures/YP-flow-diagram.png
789 :align: center
790
791Following is a brief summary of the "workflow":
792
7931. Developers specify architecture, policies, patches and configuration
794 details.
795
7962. The build system fetches and downloads the source code from the
797 specified location. The build system supports standard methods such
798 as tarballs or source code repositories systems such as Git.
799
8003. Once source code is downloaded, the build system extracts the sources
801 into a local work area where patches are applied and common steps for
802 configuring and compiling the software are run.
803
8044. The build system then installs the software into a temporary staging
805 area where the binary package format you select (DEB, RPM, or IPK) is
806 used to roll up the software.
807
8085. Different QA and sanity checks run throughout entire build process.
809
8106. After the binaries are created, the build system generates a binary
811 package feed that is used to create the final root file image.
812
8137. The build system generates the file system image and a customized
814 Extensible SDK (eSDK) for application development in parallel.
815
816For a very detailed look at this workflow, see the "`OpenEmbedded Build
817System Concepts <#openembedded-build-system-build-concepts>`__" section.
818
819Some Basic Terms
820================
821
822It helps to understand some basic fundamental terms when learning the
823Yocto Project. Although a list of terms exists in the ":doc:`Yocto Project
824Terms <../ref-manual/ref-terms>`" section of the Yocto Project
825Reference Manual, this section provides the definitions of some terms
826helpful for getting started:
827
828- *Configuration Files:* Files that hold global definitions of
829 variables, user-defined variables, and hardware configuration
830 information. These files tell the :term:`OpenEmbedded Build System`
831 what to build and
832 what to put into the image to support a particular platform.
833
834- *Extensible Software Development Kit (eSDK):* A custom SDK for
835 application developers. This eSDK allows developers to incorporate
836 their library and programming changes back into the image to make
837 their code available to other application developers. For information
838 on the eSDK, see the :doc:`../sdk-manual/sdk-manual` manual.
839
840- *Layer:* A collection of related recipes. Layers allow you to
841 consolidate related metadata to customize your build. Layers also
842 isolate information used when building for multiple architectures.
843 Layers are hierarchical in their ability to override previous
844 specifications. You can include any number of available layers from
845 the Yocto Project and customize the build by adding your layers after
846 them. You can search the Layer Index for layers used within Yocto
847 Project.
848
849 For more detailed information on layers, see the
850 ":ref:`dev-manual/dev-manual-common-tasks:understanding and creating layers`"
851 section in the Yocto Project Development Tasks Manual. For a
852 discussion specifically on BSP Layers, see the
853 ":ref:`bsp-guide/bsp:bsp layers`" section in the Yocto
854 Project Board Support Packages (BSP) Developer's Guide.
855
856- *Metadata:* A key element of the Yocto Project is the Metadata that
857 is used to construct a Linux distribution and is contained in the
858 files that the OpenEmbedded build system parses when building an
859 image. In general, Metadata includes recipes, configuration files,
860 and other information that refers to the build instructions
861 themselves, as well as the data used to control what things get built
862 and the effects of the build. Metadata also includes commands and
863 data used to indicate what versions of software are used, from where
864 they are obtained, and changes or additions to the software itself
865 (patches or auxiliary files) that are used to fix bugs or customize
866 the software for use in a particular situation. OpenEmbedded-Core is
867 an important set of validated metadata.
868
869- *OpenEmbedded Build System:* The terms "BitBake" and "build system"
870 are sometimes used for the OpenEmbedded Build System.
871
872 BitBake is a task scheduler and execution engine that parses
873 instructions (i.e. recipes) and configuration data. After a parsing
874 phase, BitBake creates a dependency tree to order the compilation,
875 schedules the compilation of the included code, and finally executes
876 the building of the specified custom Linux image (distribution).
877 BitBake is similar to the ``make`` tool.
878
879 During a build process, the build system tracks dependencies and
880 performs a native or cross-compilation of the package. As a first
881 step in a cross-build setup, the framework attempts to create a
882 cross-compiler toolchain (i.e. Extensible SDK) suited for the target
883 platform.
884
885- *OpenEmbedded-Core (OE-Core):* OE-Core is metadata comprised of
886 foundation recipes, classes, and associated files that are meant to
887 be common among many different OpenEmbedded-derived systems,
888 including the Yocto Project. OE-Core is a curated subset of an
889 original repository developed by the OpenEmbedded community that has
890 been pared down into a smaller, core set of continuously validated
891 recipes. The result is a tightly controlled and quality-assured core
892 set of recipes.
893
894 You can see the Metadata in the ``meta`` directory of the Yocto
895 Project `Source
896 Repositories <http://git.yoctoproject.org/cgit/cgit.cgi>`__.
897
898- *Packages:* In the context of the Yocto Project, this term refers to
899 a recipe's packaged output produced by BitBake (i.e. a "baked
900 recipe"). A package is generally the compiled binaries produced from
901 the recipe's sources. You "bake" something by running it through
902 BitBake.
903
904 It is worth noting that the term "package" can, in general, have
905 subtle meanings. For example, the packages referred to in the
906 ":ref:`ref-manual/ref-system-requirements:required packages for the build host`"
907 section in the Yocto Project Reference Manual are compiled binaries
908 that, when installed, add functionality to your Linux distribution.
909
910 Another point worth noting is that historically within the Yocto
911 Project, recipes were referred to as packages - thus, the existence
912 of several BitBake variables that are seemingly mis-named, (e.g.
913 :term:`PR`,
914 :term:`PV`, and
915 :term:`PE`).
916
917- *Poky:* Poky is a reference embedded distribution and a reference
918 test configuration. Poky provides the following:
919
920 - A base-level functional distro used to illustrate how to customize
921 a distribution.
922
923 - A means by which to test the Yocto Project components (i.e. Poky
924 is used to validate the Yocto Project).
925
926 - A vehicle through which you can download the Yocto Project.
927
928 Poky is not a product level distro. Rather, it is a good starting
929 point for customization.
930
931 .. note::
932
933 Poky is an integration layer on top of OE-Core.
934
935- *Recipe:* The most common form of metadata. A recipe contains a list
936 of settings and tasks (i.e. instructions) for building packages that
937 are then used to build the binary image. A recipe describes where you
938 get source code and which patches to apply. Recipes describe
939 dependencies for libraries or for other recipes as well as
940 configuration and compilation options. Related recipes are
941 consolidated into a layer.