blob: 66a88c9521a6431a55b0219acbe1631de3dcf2eb [file] [log] [blame]
Andrew Geisslerf0343792020-11-18 10:42:21 -06001.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Andrew Geisslerc9f78652020-09-18 14:11:35 -05002
3*****************************
4Introducing the Yocto Project
5*****************************
6
7What is the Yocto Project?
8==========================
9
10The Yocto Project is an open source collaboration project that helps
11developers create custom Linux-based systems that are designed for
12embedded products regardless of the product's hardware architecture.
13Yocto Project provides a flexible toolset and a development environment
14that allows embedded device developers across the world to collaborate
15through shared technologies, software stacks, configurations, and best
16practices used to create these tailored Linux images.
17
18Thousands of developers worldwide have discovered that Yocto Project
19provides advantages in both systems and applications development,
20archival and management benefits, and customizations used for speed,
21footprint, and memory utilization. The project is a standard when it
22comes to delivering embedded software stacks. The project allows
23software customizations and build interchange for multiple hardware
24platforms as well as software stacks that can be maintained and scaled.
25
26.. image:: figures/key-dev-elements.png
27 :align: center
28
29For further introductory information on the Yocto Project, you might be
30interested in this
31`article <https://www.embedded.com/electronics-blogs/say-what-/4458600/Why-the-Yocto-Project-for-my-IoT-Project->`__
32by Drew Moseley and in this short introductory
33`video <https://www.youtube.com/watch?v=utZpKM7i5Z4>`__.
34
35The remainder of this section overviews advantages and challenges tied
36to the Yocto Project.
37
Andrew Geisslerc9f78652020-09-18 14:11:35 -050038Features
39--------
40
41The following list describes features and advantages of the Yocto
42Project:
43
44- *Widely Adopted Across the Industry:* Semiconductor, operating
45 system, software, and service vendors exist whose products and
46 services adopt and support the Yocto Project. For a look at the Yocto
47 Project community and the companies involved with the Yocto Project,
48 see the "COMMUNITY" and "ECOSYSTEM" tabs on the
49 :yocto_home:`Yocto Project <>` home page.
50
51- *Architecture Agnostic:* Yocto Project supports Intel, ARM, MIPS,
52 AMD, PPC and other architectures. Most ODMs, OSVs, and chip vendors
53 create and supply BSPs that support their hardware. If you have
54 custom silicon, you can create a BSP that supports that architecture.
55
56 Aside from lots of architecture support, the Yocto Project fully
57 supports a wide range of device emulation through the Quick EMUlator
58 (QEMU).
59
60- *Images and Code Transfer Easily:* Yocto Project output can easily
61 move between architectures without moving to new development
62 environments. Additionally, if you have used the Yocto Project to
63 create an image or application and you find yourself not able to
64 support it, commercial Linux vendors such as Wind River, Mentor
65 Graphics, Timesys, and ENEA could take it and provide ongoing
66 support. These vendors have offerings that are built using the Yocto
67 Project.
68
69- *Flexibility:* Corporations use the Yocto Project many different
70 ways. One example is to create an internal Linux distribution as a
71 code base the corporation can use across multiple product groups.
72 Through customization and layering, a project group can leverage the
73 base Linux distribution to create a distribution that works for their
74 product needs.
75
76- *Ideal for Constrained Embedded and IoT devices:* Unlike a full Linux
77 distribution, you can use the Yocto Project to create exactly what
78 you need for embedded devices. You only add the feature support or
79 packages that you absolutely need for the device. For devices that
80 have display hardware, you can use available system components such
81 as X11, GTK+, Qt, Clutter, and SDL (among others) to create a rich
82 user experience. For devices that do not have a display or where you
83 want to use alternative UI frameworks, you can choose to not install
84 these components.
85
86- *Comprehensive Toolchain Capabilities:* Toolchains for supported
87 architectures satisfy most use cases. However, if your hardware
88 supports features that are not part of a standard toolchain, you can
89 easily customize that toolchain through specification of
90 platform-specific tuning parameters. And, should you need to use a
91 third-party toolchain, mechanisms built into the Yocto Project allow
92 for that.
93
94- *Mechanism Rules Over Policy:* Focusing on mechanism rather than
95 policy ensures that you are free to set policies based on the needs
96 of your design instead of adopting decisions enforced by some system
97 software provider.
98
99- *Uses a Layer Model:* The Yocto Project `layer
100 infrastructure <#the-yocto-project-layer-model>`__ groups related
101 functionality into separate bundles. You can incrementally add these
102 grouped functionalities to your project as needed. Using layers to
103 isolate and group functionality reduces project complexity and
104 redundancy, allows you to easily extend the system, make
105 customizations, and keep functionality organized.
106
107- *Supports Partial Builds:* You can build and rebuild individual
108 packages as needed. Yocto Project accomplishes this through its
109 `shared-state cache <#shared-state-cache>`__ (sstate) scheme. Being
110 able to build and debug components individually eases project
111 development.
112
113- *Releases According to a Strict Schedule:* Major releases occur on a
Andrew Geissler09209ee2020-12-13 08:44:15 -0600114 :doc:`six-month cycle </ref-manual/release-process>`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500115 predictably in October and April. The most recent two releases
116 support point releases to address common vulnerabilities and
117 exposures. This predictability is crucial for projects based on the
118 Yocto Project and allows development teams to plan activities.
119
120- *Rich Ecosystem of Individuals and Organizations:* For open source
121 projects, the value of community is very important. Support forums,
122 expertise, and active developers who continue to push the Yocto
123 Project forward are readily available.
124
125- *Binary Reproducibility:* The Yocto Project allows you to be very
126 specific about dependencies and achieves very high percentages of
127 binary reproducibility (e.g. 99.8% for ``core-image-minimal``). When
128 distributions are not specific about which packages are pulled in and
129 in what order to support dependencies, other build systems can
130 arbitrarily include packages.
131
132- *License Manifest:* The Yocto Project provides a :ref:`license
Andrew Geissler09209ee2020-12-13 08:44:15 -0600133 manifest <dev-manual/common-tasks:maintaining open source license compliance during your product's lifecycle>`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500134 for review by people who need to track the use of open source
135 licenses (e.g. legal teams).
136
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500137Challenges
138----------
139
140The following list presents challenges you might encounter when
141developing using the Yocto Project:
142
143- *Steep Learning Curve:* The Yocto Project has a steep learning curve
144 and has many different ways to accomplish similar tasks. It can be
145 difficult to choose how to proceed when varying methods exist by
146 which to accomplish a given task.
147
148- *Understanding What Changes You Need to Make For Your Design Requires
149 Some Research:* Beyond the simple tutorial stage, understanding what
150 changes need to be made for your particular design can require a
151 significant amount of research and investigation. For information
152 that helps you transition from trying out the Yocto Project to using
153 it for your project, see the ":ref:`what-i-wish-id-known:what i wish i'd known about yocto project`" and
154 ":ref:`transitioning-to-a-custom-environment:transitioning to a custom environment for systems development`"
155 documents on the Yocto Project website.
156
157- *Project Workflow Could Be Confusing:* The `Yocto Project
158 workflow <#overview-development-environment>`__ could be confusing if
159 you are used to traditional desktop and server software development.
160 In a desktop development environment, mechanisms exist to easily pull
161 and install new packages, which are typically pre-compiled binaries
162 from servers accessible over the Internet. Using the Yocto Project,
163 you must modify your configuration and rebuild to add additional
164 packages.
165
166- *Working in a Cross-Build Environment Can Feel Unfamiliar:* When
167 developing code to run on a target, compilation, execution, and
168 testing done on the actual target can be faster than running a
169 BitBake build on a development host and then deploying binaries to
170 the target for test. While the Yocto Project does support development
171 tools on the target, the additional step of integrating your changes
172 back into the Yocto Project build environment would be required.
173 Yocto Project supports an intermediate approach that involves making
174 changes on the development system within the BitBake environment and
175 then deploying only the updated packages to the target.
176
177 The Yocto Project :term:`OpenEmbedded Build System`
178 produces packages
179 in standard formats (i.e. RPM, DEB, IPK, and TAR). You can deploy
180 these packages into the running system on the target by using
181 utilities on the target such as ``rpm`` or ``ipk``.
182
183- *Initial Build Times Can be Significant:* Long initial build times
184 are unfortunately unavoidable due to the large number of packages
185 initially built from scratch for a fully functioning Linux system.
186 Once that initial build is completed, however, the shared-state
187 (sstate) cache mechanism Yocto Project uses keeps the system from
188 rebuilding packages that have not been "touched" since the last
189 build. The sstate mechanism significantly reduces times for
190 successive builds.
191
192The Yocto Project Layer Model
193=============================
194
195The Yocto Project's "Layer Model" is a development model for embedded
196and IoT Linux creation that distinguishes the Yocto Project from other
197simple build systems. The Layer Model simultaneously supports
198collaboration and customization. Layers are repositories that contain
199related sets of instructions that tell the :term:`OpenEmbedded Build System`
200what to do. You can
201collaborate, share, and reuse layers.
202
203Layers can contain changes to previous instructions or settings at any
204time. This powerful override capability is what allows you to customize
205previously supplied collaborative or community layers to suit your
206product requirements.
207
208You use different layers to logically separate information in your
209build. As an example, you could have BSP, GUI, distro configuration,
210middleware, or application layers. Putting your entire build into one
211layer limits and complicates future customization and reuse. Isolating
212information into layers, on the other hand, helps simplify future
213customizations and reuse. You might find it tempting to keep everything
214in one layer when working on a single project. However, the more modular
215your Metadata, the easier it is to cope with future changes.
216
217.. note::
218
219 - Use Board Support Package (BSP) layers from silicon vendors when
220 possible.
221
222 - Familiarize yourself with the `Yocto Project curated layer
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500223 index <https://www.yoctoproject.org/software-overview/layers/>`__
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500224 or the `OpenEmbedded layer
225 index <http://layers.openembedded.org/layerindex/branch/master/layers/>`__.
226 The latter contains more layers but they are less universally
227 validated.
228
229 - Layers support the inclusion of technologies, hardware components,
230 and software components. The :ref:`Yocto Project
Andrew Geissler09209ee2020-12-13 08:44:15 -0600231 Compatible <dev-manual/common-tasks:making sure your layer is compatible with yocto project>`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500232 designation provides a minimum level of standardization that
233 contributes to a strong ecosystem. "YP Compatible" is applied to
234 appropriate products and software components such as BSPs, other
235 OE-compatible layers, and related open-source projects, allowing
236 the producer to use Yocto Project badges and branding assets.
237
238To illustrate how layers are used to keep things modular, consider
239machine customizations. These types of customizations typically reside
240in a special layer, rather than a general layer, called a BSP Layer.
241Furthermore, the machine customizations should be isolated from recipes
242and Metadata that support a new GUI environment, for example. This
243situation gives you a couple of layers: one for the machine
244configurations, and one for the GUI environment. It is important to
245understand, however, that the BSP layer can still make machine-specific
246additions to recipes within the GUI environment layer without polluting
247the GUI layer itself with those machine-specific changes. You can
248accomplish this through a recipe that is a BitBake append
249(``.bbappend``) file, which is described later in this section.
250
251.. note::
252
253 For general information on BSP layer structure, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600254 :doc:`/bsp-guide/index`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500255 .
256
257The :term:`Source Directory`
258contains both general layers and BSP layers right out of the box. You
259can easily identify layers that ship with a Yocto Project release in the
260Source Directory by their names. Layers typically have names that begin
261with the string ``meta-``.
262
263.. note::
264
265 It is not a requirement that a layer name begin with the prefix
266 meta-
267 , but it is a commonly accepted standard in the Yocto Project
268 community.
269
Andrew Geissler09209ee2020-12-13 08:44:15 -0600270For example, if you were to examine the :yocto_git:`tree view </poky/tree/>`
271of the ``poky`` repository, you will see several layers: ``meta``,
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500272``meta-skeleton``, ``meta-selftest``, ``meta-poky``, and
273``meta-yocto-bsp``. Each of these repositories represents a distinct
274layer.
275
276For procedures on how to create layers, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600277":ref:`dev-manual/common-tasks:understanding and creating layers`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500278section in the Yocto Project Development Tasks Manual.
279
280Components and Tools
281====================
282
283The Yocto Project employs a collection of components and tools used by
284the project itself, by project developers, and by those using the Yocto
285Project. These components and tools are open source projects and
286metadata that are separate from the reference distribution
287(:term:`Poky`) and the
288:term:`OpenEmbedded Build System`. Most of the
289components and tools are downloaded separately.
290
291This section provides brief overviews of the components and tools
292associated with the Yocto Project.
293
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500294Development Tools
295-----------------
296
297The following list consists of tools that help you develop images and
298applications using the Yocto Project:
299
300- *CROPS:* `CROPS <https://github.com/crops/poky-container/>`__ is an
301 open source, cross-platform development framework that leverages
302 `Docker Containers <https://www.docker.com/>`__. CROPS provides an
303 easily managed, extensible environment that allows you to build
304 binaries for a variety of architectures on Windows, Linux and Mac OS
305 X hosts.
306
307- *devtool:* This command-line tool is available as part of the
308 extensible SDK (eSDK) and is its cornerstone. You can use ``devtool``
309 to help build, test, and package software within the eSDK. You can
310 use the tool to optionally integrate what you build into an image
311 built by the OpenEmbedded build system.
312
313 The ``devtool`` command employs a number of sub-commands that allow
314 you to add, modify, and upgrade recipes. As with the OpenEmbedded
315 build system, "recipes" represent software packages within
316 ``devtool``. When you use ``devtool add``, a recipe is automatically
317 created. When you use ``devtool modify``, the specified existing
318 recipe is used in order to determine where to get the source code and
319 how to patch it. In both cases, an environment is set up so that when
320 you build the recipe a source tree that is under your control is used
321 in order to allow you to make changes to the source as desired. By
322 default, both new recipes and the source go into a "workspace"
323 directory under the eSDK. The ``devtool upgrade`` command updates an
324 existing recipe so that you can build it for an updated set of source
325 files.
326
327 You can read about the ``devtool`` workflow in the Yocto Project
328 Application Development and Extensible Software Development Kit
329 (eSDK) Manual in the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600330 ":ref:`sdk-manual/extensible:using \`\`devtool\`\` in your sdk workflow`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500331 section.
332
333- *Extensible Software Development Kit (eSDK):* The eSDK provides a
334 cross-development toolchain and libraries tailored to the contents of
335 a specific image. The eSDK makes it easy to add new applications and
336 libraries to an image, modify the source for an existing component,
337 test changes on the target hardware, and integrate into the rest of
338 the OpenEmbedded build system. The eSDK gives you a toolchain
339 experience supplemented with the powerful set of ``devtool`` commands
340 tailored for the Yocto Project environment.
341
Andrew Geissler09209ee2020-12-13 08:44:15 -0600342 For information on the eSDK, see the :doc:`/sdk-manual/index` Manual.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500343
344- *Toaster:* Toaster is a web interface to the Yocto Project
345 OpenEmbedded build system. Toaster allows you to configure, run, and
346 view information about builds. For information on Toaster, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600347 :doc:`/toaster-manual/index`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500348
349Production Tools
350----------------
351
352The following list consists of tools that help production related
353activities using the Yocto Project:
354
355- *Auto Upgrade Helper:* This utility when used in conjunction with the
356 :term:`OpenEmbedded Build System`
357 (BitBake and
358 OE-Core) automatically generates upgrades for recipes that are based
359 on new versions of the recipes published upstream. See
Andrew Geissler09209ee2020-12-13 08:44:15 -0600360 :ref:`dev-manual/common-tasks:using the auto upgrade helper (auh)`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500361 for how to set it up.
362
363- *Recipe Reporting System:* The Recipe Reporting System tracks recipe
364 versions available for Yocto Project. The main purpose of the system
365 is to help you manage the recipes you maintain and to offer a dynamic
366 overview of the project. The Recipe Reporting System is built on top
367 of the `OpenEmbedded Layer
368 Index <http://layers.openembedded.org/layerindex/layers/>`__, which
369 is a website that indexes OpenEmbedded-Core layers.
370
371- *Patchwork:* `Patchwork <http://jk.ozlabs.org/projects/patchwork/>`__
372 is a fork of a project originally started by
373 `OzLabs <http://ozlabs.org/>`__. The project is a web-based tracking
374 system designed to streamline the process of bringing contributions
375 into a project. The Yocto Project uses Patchwork as an organizational
376 tool to handle patches, which number in the thousands for every
377 release.
378
379- *AutoBuilder:* AutoBuilder is a project that automates build tests
380 and quality assurance (QA). By using the public AutoBuilder, anyone
381 can determine the status of the current "master" branch of Poky.
382
383 .. note::
384
385 AutoBuilder is based on buildbot.
386
387 A goal of the Yocto Project is to lead the open source industry with
388 a project that automates testing and QA procedures. In doing so, the
389 project encourages a development community that publishes QA and test
390 plans, publicly demonstrates QA and test plans, and encourages
391 development of tools that automate and test and QA procedures for the
392 benefit of the development community.
393
394 You can learn more about the AutoBuilder used by the Yocto Project
Andrew Geissler09209ee2020-12-13 08:44:15 -0600395 Autobuilder :doc:`here </test-manual/understand-autobuilder>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500396
397- *Cross-Prelink:* Prelinking is the process of pre-computing the load
398 addresses and link tables generated by the dynamic linker as compared
399 to doing this at runtime. Doing this ahead of time results in
400 performance improvements when the application is launched and reduced
401 memory usage for libraries shared by many applications.
402
403 Historically, cross-prelink is a variant of prelink, which was
404 conceived by `Jakub
405 Jelínek <http://people.redhat.com/jakub/prelink.pdf>`__ a number of
406 years ago. Both prelink and cross-prelink are maintained in the same
407 repository albeit on separate branches. By providing an emulated
408 runtime dynamic linker (i.e. ``glibc``-derived ``ld.so`` emulation),
409 the cross-prelink project extends the prelink software's ability to
410 prelink a sysroot environment. Additionally, the cross-prelink
411 software enables the ability to work in sysroot style environments.
412
413 The dynamic linker determines standard load address calculations
414 based on a variety of factors such as mapping addresses, library
415 usage, and library function conflicts. The prelink tool uses this
416 information, from the dynamic linker, to determine unique load
417 addresses for executable and linkable format (ELF) binaries that are
418 shared libraries and dynamically linked. The prelink tool modifies
419 these ELF binaries with the pre-computed information. The result is
420 faster loading and often lower memory consumption because more of the
421 library code can be re-used from shared Copy-On-Write (COW) pages.
422
423 The original upstream prelink project only supports running prelink
424 on the end target device due to the reliance on the target device's
425 dynamic linker. This restriction causes issues when developing a
426 cross-compiled system. The cross-prelink adds a synthesized dynamic
427 loader that runs on the host, thus permitting cross-prelinking
428 without ever having to run on a read-write target filesystem.
429
430- *Pseudo:* Pseudo is the Yocto Project implementation of
431 `fakeroot <http://man.he.net/man1/fakeroot>`__, which is used to run
432 commands in an environment that seemingly has root privileges.
433
434 During a build, it can be necessary to perform operations that
435 require system administrator privileges. For example, file ownership
436 or permissions might need definition. Pseudo is a tool that you can
437 either use directly or through the environment variable
438 ``LD_PRELOAD``. Either method allows these operations to succeed as
439 if system administrator privileges exist even when they do not.
440
441 You can read more about Pseudo in the "`Fakeroot and
442 Pseudo <#fakeroot-and-pseudo>`__" section.
443
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500444Open-Embedded Build System Components
445-------------------------------------
446
447The following list consists of components associated with the
448:term:`OpenEmbedded Build System`:
449
450- *BitBake:* BitBake is a core component of the Yocto Project and is
451 used by the OpenEmbedded build system to build images. While BitBake
452 is key to the build system, BitBake is maintained separately from the
453 Yocto Project.
454
455 BitBake is a generic task execution engine that allows shell and
456 Python tasks to be run efficiently and in parallel while working
457 within complex inter-task dependency constraints. In short, BitBake
458 is a build engine that works through recipes written in a specific
459 format in order to perform sets of tasks.
460
461 You can learn more about BitBake in the :doc:`BitBake User
462 Manual <bitbake:index>`.
463
464- *OpenEmbedded-Core:* OpenEmbedded-Core (OE-Core) is a common layer of
465 metadata (i.e. recipes, classes, and associated files) used by
466 OpenEmbedded-derived systems, which includes the Yocto Project. The
467 Yocto Project and the OpenEmbedded Project both maintain the
468 OpenEmbedded-Core. You can find the OE-Core metadata in the Yocto
Andrew Geissler09209ee2020-12-13 08:44:15 -0600469 Project :yocto_git:`Source Repositories </poky/tree/meta>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500470
471 Historically, the Yocto Project integrated the OE-Core metadata
472 throughout the Yocto Project source repository reference system
473 (Poky). After Yocto Project Version 1.0, the Yocto Project and
474 OpenEmbedded agreed to work together and share a common core set of
475 metadata (OE-Core), which contained much of the functionality
476 previously found in Poky. This collaboration achieved a long-standing
477 OpenEmbedded objective for having a more tightly controlled and
478 quality-assured core. The results also fit well with the Yocto
479 Project objective of achieving a smaller number of fully featured
480 tools as compared to many different ones.
481
482 Sharing a core set of metadata results in Poky as an integration
483 layer on top of OE-Core. You can see that in this
484 `figure <#yp-key-dev-elements>`__. The Yocto Project combines various
485 components such as BitBake, OE-Core, script "glue", and documentation
486 for its build system.
487
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500488Reference Distribution (Poky)
489-----------------------------
490
491Poky is the Yocto Project reference distribution. It contains the
492:term:`OpenEmbedded Build System`
493(BitBake and OE-Core) as well as a set of metadata to get you started
494building your own distribution. See the
495`figure <#what-is-the-yocto-project>`__ in "What is the Yocto Project?"
496section for an illustration that shows Poky and its relationship with
497other parts of the Yocto Project.
498
499To use the Yocto Project tools and components, you can download
500(``clone``) Poky and use it to bootstrap your own distribution.
501
502.. note::
503
504 Poky does not contain binary files. It is a working example of how to
505 build your own custom Linux distribution from source.
506
507You can read more about Poky in the "`Reference Embedded Distribution
508(Poky) <#reference-embedded-distribution>`__" section.
509
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500510Packages for Finished Targets
511-----------------------------
512
513The following lists components associated with packages for finished
514targets:
515
516- *Matchbox:* Matchbox is an Open Source, base environment for the X
517 Window System running on non-desktop, embedded platforms such as
518 handhelds, set-top boxes, kiosks, and anything else for which screen
519 space, input mechanisms, or system resources are limited.
520
521 Matchbox consists of a number of interchangeable and optional
522 applications that you can tailor to a specific, non-desktop platform
523 to enhance usability in constrained environments.
524
525 You can find the Matchbox source in the Yocto Project
526 :yocto_git:`Source Repositories <>`.
527
528- *Opkg:* Open PacKaGe management (opkg) is a lightweight package
529 management system based on the itsy package (ipkg) management system.
530 Opkg is written in C and resembles Advanced Package Tool (APT) and
531 Debian Package (dpkg) in operation.
532
533 Opkg is intended for use on embedded Linux devices and is used in
534 this capacity in the
535 `OpenEmbedded <http://www.openembedded.org/wiki/Main_Page>`__ and
536 `OpenWrt <https://openwrt.org/>`__ projects, as well as the Yocto
537 Project.
538
539 .. note::
540
541 As best it can, opkg maintains backwards compatibility with ipkg
542 and conforms to a subset of Debian's policy manual regarding
543 control files.
544
545 You can find the opkg source in the Yocto Project
546 :yocto_git:`Source Repositories <>`.
547
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500548Archived Components
549-------------------
550
551The Build Appliance is a virtual machine image that enables you to build
552and boot a custom embedded Linux image with the Yocto Project using a
553non-Linux development system.
554
555Historically, the Build Appliance was the second of three methods by
556which you could use the Yocto Project on a system that was not native to
557Linux.
558
5591. *Hob:* Hob, which is now deprecated and is no longer available since
560 the 2.1 release of the Yocto Project provided a rudimentary,
561 GUI-based interface to the Yocto Project. Toaster has fully replaced
562 Hob.
563
5642. *Build Appliance:* Post Hob, the Build Appliance became available. It
565 was never recommended that you use the Build Appliance as a
566 day-to-day production development environment with the Yocto Project.
567 Build Appliance was useful as a way to try out development in the
568 Yocto Project environment.
569
5703. *CROPS:* The final and best solution available now for developing
571 using the Yocto Project on a system not native to Linux is with
572 `CROPS <#gs-crops-overview>`__.
573
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500574Development Methods
575===================
576
577The Yocto Project development environment usually involves a
578:term:`Build Host` and target
579hardware. You use the Build Host to build images and develop
580applications, while you use the target hardware to test deployed
581software.
582
583This section provides an introduction to the choices or development
584methods you have when setting up your Build Host. Depending on the your
585particular workflow preference and the type of operating system your
586Build Host runs, several choices exist that allow you to use the Yocto
587Project.
588
589.. note::
590
591 For additional detail about the Yocto Project development
Andrew Geissler09209ee2020-12-13 08:44:15 -0600592 environment, see the ":doc:`/overview-manual/development-environment`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500593 chapter.
594
595- *Native Linux Host:* By far the best option for a Build Host. A
596 system running Linux as its native operating system allows you to
597 develop software by directly using the
598 :term:`BitBake` tool. You can
599 accomplish all aspects of development from a familiar shell of a
600 supported Linux distribution.
601
602 For information on how to set up a Build Host on a system running
603 Linux as its native operating system, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600604 ":ref:`dev-manual/start:setting up a native linux host`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500605 section in the Yocto Project Development Tasks Manual.
606
607- *CROss PlatformS (CROPS):* Typically, you use
608 `CROPS <https://github.com/crops/poky-container/>`__, which leverages
609 `Docker Containers <https://www.docker.com/>`__, to set up a Build
610 Host that is not running Linux (e.g. Microsoft Windows or macOS).
611
612 .. note::
613
614 You can, however, use CROPS on a Linux-based system.
615
616 CROPS is an open source, cross-platform development framework that
617 provides an easily managed, extensible environment for building
618 binaries targeted for a variety of architectures on Windows, macOS,
619 or Linux hosts. Once the Build Host is set up using CROPS, you can
620 prepare a shell environment to mimic that of a shell being used on a
621 system natively running Linux.
622
623 For information on how to set up a Build Host with CROPS, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600624 ":ref:`dev-manual/start:setting up to use cross platforms (crops)`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500625 section in the Yocto Project Development Tasks Manual.
626
627- *Windows Subsystem For Linux (WSLv2):* You may use Windows Subsystem
628 For Linux v2 to set up a build host using Windows 10.
629
630 .. note::
631
632 The Yocto Project is not compatible with WSLv1, it is compatible
633 but not officially supported nor validated with WSLv2, if you
634 still decide to use WSL please upgrade to WSLv2.
635
636 The Windows Subsystem For Linux allows Windows 10 to run a real Linux
637 kernel inside of a lightweight utility virtual machine (VM) using
638 virtualization technology.
639
640 For information on how to set up a Build Host with WSLv2, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600641 ":ref:`dev-manual/start:setting up to use windows subsystem for linux (wslv2)`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500642 section in the Yocto Project Development Tasks Manual.
643
644- *Toaster:* Regardless of what your Build Host is running, you can use
645 Toaster to develop software using the Yocto Project. Toaster is a web
646 interface to the Yocto Project's :term:`OpenEmbedded Build System`.
647 The interface
648 enables you to configure and run your builds. Information about
649 builds is collected and stored in a database. You can use Toaster to
650 configure and start builds on multiple remote build servers.
651
652 For information about and how to use Toaster, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600653 :doc:`/toaster-manual/index`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500654
655Reference Embedded Distribution (Poky)
656======================================
657
658"Poky", which is pronounced *Pock*-ee, is the name of the Yocto
659Project's reference distribution or Reference OS Kit. Poky contains the
660:term:`OpenEmbedded Build System`
661(:term:`BitBake` and
662:term:`OpenEmbedded-Core (OE-Core)`) as well as a set
663of :term:`Metadata` to get you started
664building your own distro. In other words, Poky is a base specification
665of the functionality needed for a typical embedded system as well as the
666components from the Yocto Project that allow you to build a distribution
667into a usable binary image.
668
669Poky is a combined repository of BitBake, OpenEmbedded-Core (which is
670found in ``meta``), ``meta-poky``, ``meta-yocto-bsp``, and documentation
671provided all together and known to work well together. You can view
672these items that make up the Poky repository in the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600673:yocto_git:`Source Repositories </poky/tree/>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500674
675.. note::
676
677 If you are interested in all the contents of the
678 poky
Andrew Geissler09209ee2020-12-13 08:44:15 -0600679 Git repository, see the ":ref:`ref-manual/structure:top-level core components`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500680 section in the Yocto Project Reference Manual.
681
682The following figure illustrates what generally comprises Poky:
683
684.. image:: figures/poky-reference-distribution.png
685 :align: center
686
687- BitBake is a task executor and scheduler that is the heart of the
688 OpenEmbedded build system.
689
690- ``meta-poky``, which is Poky-specific metadata.
691
692- ``meta-yocto-bsp``, which are Yocto Project-specific Board Support
693 Packages (BSPs).
694
695- OpenEmbedded-Core (OE-Core) metadata, which includes shared
696 configurations, global variable definitions, shared classes,
697 packaging, and recipes. Classes define the encapsulation and
698 inheritance of build logic. Recipes are the logical units of software
699 and images to be built.
700
701- Documentation, which contains the Yocto Project source files used to
702 make the set of user manuals.
703
704.. note::
705
706 While Poky is a "complete" distribution specification and is tested
707 and put through QA, you cannot use it as a product "out of the box"
708 in its current form.
709
710To use the Yocto Project tools, you can use Git to clone (download) the
711Poky repository then use your local copy of the reference distribution
712to bootstrap your own distribution.
713
714.. note::
715
716 Poky does not contain binary files. It is a working example of how to
717 build your own custom Linux distribution from source.
718
719Poky has a regular, well established, six-month release cycle under its
720own version. Major releases occur at the same time major releases (point
721releases) occur for the Yocto Project, which are typically in the Spring
722and Fall. For more information on the Yocto Project release schedule and
Andrew Geissler09209ee2020-12-13 08:44:15 -0600723cadence, see the ":doc:`/ref-manual/release-process`" chapter in the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500724Yocto Project Reference Manual.
725
726Much has been said about Poky being a "default configuration". A default
727configuration provides a starting image footprint. You can use Poky out
728of the box to create an image ranging from a shell-accessible minimal
729image all the way up to a Linux Standard Base-compliant image that uses
730a GNOME Mobile and Embedded (GMAE) based reference user interface called
731Sato.
732
733One of the most powerful properties of Poky is that every aspect of a
734build is controlled by the metadata. You can use metadata to augment
735these base image types by adding metadata
736`layers <#the-yocto-project-layer-model>`__ that extend functionality.
737These layers can provide, for example, an additional software stack for
738an image type, add a board support package (BSP) for additional
739hardware, or even create a new image type.
740
741Metadata is loosely grouped into configuration files or package recipes.
742A recipe is a collection of non-executable metadata used by BitBake to
743set variables or define additional build-time tasks. A recipe contains
744fields such as the recipe description, the recipe version, the license
745of the package and the upstream source repository. A recipe might also
746indicate that the build process uses autotools, make, distutils or any
747other build process, in which case the basic functionality can be
748defined by the classes it inherits from the OE-Core layer's class
749definitions in ``./meta/classes``. Within a recipe you can also define
750additional tasks as well as task prerequisites. Recipe syntax through
751BitBake also supports both ``_prepend`` and ``_append`` operators as a
752method of extending task functionality. These operators inject code into
753the beginning or end of a task. For information on these BitBake
754operators, see the
755":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:appending and prepending (override style syntax)`"
756section in the BitBake User's Manual.
757
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500758The OpenEmbedded Build System Workflow
759======================================
760
761The :term:`OpenEmbedded Build System` uses a "workflow" to
762accomplish image and SDK generation. The following figure overviews that
763workflow:
764
765.. image:: figures/YP-flow-diagram.png
766 :align: center
767
768Following is a brief summary of the "workflow":
769
7701. Developers specify architecture, policies, patches and configuration
771 details.
772
7732. The build system fetches and downloads the source code from the
774 specified location. The build system supports standard methods such
775 as tarballs or source code repositories systems such as Git.
776
7773. Once source code is downloaded, the build system extracts the sources
778 into a local work area where patches are applied and common steps for
779 configuring and compiling the software are run.
780
7814. The build system then installs the software into a temporary staging
782 area where the binary package format you select (DEB, RPM, or IPK) is
783 used to roll up the software.
784
7855. Different QA and sanity checks run throughout entire build process.
786
7876. After the binaries are created, the build system generates a binary
788 package feed that is used to create the final root file image.
789
7907. The build system generates the file system image and a customized
791 Extensible SDK (eSDK) for application development in parallel.
792
793For a very detailed look at this workflow, see the "`OpenEmbedded Build
794System Concepts <#openembedded-build-system-build-concepts>`__" section.
795
796Some Basic Terms
797================
798
799It helps to understand some basic fundamental terms when learning the
800Yocto Project. Although a list of terms exists in the ":doc:`Yocto Project
Andrew Geissler09209ee2020-12-13 08:44:15 -0600801Terms </ref-manual/terms>`" section of the Yocto Project
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500802Reference Manual, this section provides the definitions of some terms
803helpful for getting started:
804
805- *Configuration Files:* Files that hold global definitions of
806 variables, user-defined variables, and hardware configuration
807 information. These files tell the :term:`OpenEmbedded Build System`
808 what to build and
809 what to put into the image to support a particular platform.
810
811- *Extensible Software Development Kit (eSDK):* A custom SDK for
812 application developers. This eSDK allows developers to incorporate
813 their library and programming changes back into the image to make
814 their code available to other application developers. For information
Andrew Geissler09209ee2020-12-13 08:44:15 -0600815 on the eSDK, see the :doc:`/sdk-manual/index` manual.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500816
817- *Layer:* A collection of related recipes. Layers allow you to
818 consolidate related metadata to customize your build. Layers also
819 isolate information used when building for multiple architectures.
820 Layers are hierarchical in their ability to override previous
821 specifications. You can include any number of available layers from
822 the Yocto Project and customize the build by adding your layers after
823 them. You can search the Layer Index for layers used within Yocto
824 Project.
825
826 For more detailed information on layers, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600827 ":ref:`dev-manual/common-tasks:understanding and creating layers`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500828 section in the Yocto Project Development Tasks Manual. For a
829 discussion specifically on BSP Layers, see the
830 ":ref:`bsp-guide/bsp:bsp layers`" section in the Yocto
831 Project Board Support Packages (BSP) Developer's Guide.
832
833- *Metadata:* A key element of the Yocto Project is the Metadata that
834 is used to construct a Linux distribution and is contained in the
835 files that the OpenEmbedded build system parses when building an
836 image. In general, Metadata includes recipes, configuration files,
837 and other information that refers to the build instructions
838 themselves, as well as the data used to control what things get built
839 and the effects of the build. Metadata also includes commands and
840 data used to indicate what versions of software are used, from where
841 they are obtained, and changes or additions to the software itself
842 (patches or auxiliary files) that are used to fix bugs or customize
843 the software for use in a particular situation. OpenEmbedded-Core is
844 an important set of validated metadata.
845
846- *OpenEmbedded Build System:* The terms "BitBake" and "build system"
847 are sometimes used for the OpenEmbedded Build System.
848
849 BitBake is a task scheduler and execution engine that parses
850 instructions (i.e. recipes) and configuration data. After a parsing
851 phase, BitBake creates a dependency tree to order the compilation,
852 schedules the compilation of the included code, and finally executes
853 the building of the specified custom Linux image (distribution).
854 BitBake is similar to the ``make`` tool.
855
856 During a build process, the build system tracks dependencies and
857 performs a native or cross-compilation of the package. As a first
858 step in a cross-build setup, the framework attempts to create a
859 cross-compiler toolchain (i.e. Extensible SDK) suited for the target
860 platform.
861
862- *OpenEmbedded-Core (OE-Core):* OE-Core is metadata comprised of
863 foundation recipes, classes, and associated files that are meant to
864 be common among many different OpenEmbedded-derived systems,
865 including the Yocto Project. OE-Core is a curated subset of an
866 original repository developed by the OpenEmbedded community that has
867 been pared down into a smaller, core set of continuously validated
868 recipes. The result is a tightly controlled and quality-assured core
869 set of recipes.
870
871 You can see the Metadata in the ``meta`` directory of the Yocto
Andrew Geissler09209ee2020-12-13 08:44:15 -0600872 Project :yocto_git:`Source Repositories <>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500873
874- *Packages:* In the context of the Yocto Project, this term refers to
875 a recipe's packaged output produced by BitBake (i.e. a "baked
876 recipe"). A package is generally the compiled binaries produced from
877 the recipe's sources. You "bake" something by running it through
878 BitBake.
879
880 It is worth noting that the term "package" can, in general, have
881 subtle meanings. For example, the packages referred to in the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600882 ":ref:`ref-manual/system-requirements:required packages for the build host`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500883 section in the Yocto Project Reference Manual are compiled binaries
884 that, when installed, add functionality to your Linux distribution.
885
886 Another point worth noting is that historically within the Yocto
887 Project, recipes were referred to as packages - thus, the existence
888 of several BitBake variables that are seemingly mis-named, (e.g.
889 :term:`PR`,
890 :term:`PV`, and
891 :term:`PE`).
892
893- *Poky:* Poky is a reference embedded distribution and a reference
894 test configuration. Poky provides the following:
895
896 - A base-level functional distro used to illustrate how to customize
897 a distribution.
898
899 - A means by which to test the Yocto Project components (i.e. Poky
900 is used to validate the Yocto Project).
901
902 - A vehicle through which you can download the Yocto Project.
903
904 Poky is not a product level distro. Rather, it is a good starting
905 point for customization.
906
907 .. note::
908
909 Poky is an integration layer on top of OE-Core.
910
911- *Recipe:* The most common form of metadata. A recipe contains a list
912 of settings and tasks (i.e. instructions) for building packages that
913 are then used to build the binary image. A recipe describes where you
914 get source code and which patches to apply. Recipes describe
915 dependencies for libraries or for other recipes as well as
916 configuration and compilation options. Related recipes are
917 consolidated into a layer.