blob: a3957934930e564964bef19ff360324d8b1b0098 [file] [log] [blame]
Andrew Geissler517393d2023-01-13 08:55:19 -06001.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
2
3Building
4********
5
6This section describes various build procedures, such as the steps
7needed for a simple build, building a target for multiple configurations,
8generating an image for more than one machine, and so forth.
9
10Building a Simple Image
11=======================
12
13In the development environment, you need to build an image whenever you
14change hardware support, add or change system libraries, or add or
15change services that have dependencies. There are several methods that allow
16you to build an image within the Yocto Project. This section presents
17the basic steps you need to build a simple image using BitBake from a
18build host running Linux.
19
20.. note::
21
22 - For information on how to build an image using
23 :term:`Toaster`, see the
24 :doc:`/toaster-manual/index`.
25
26 - For information on how to use ``devtool`` to build images, see the
27 ":ref:`sdk-manual/extensible:using \`\`devtool\`\` in your sdk workflow`"
28 section in the Yocto Project Application Development and the
29 Extensible Software Development Kit (eSDK) manual.
30
31 - For a quick example on how to build an image using the
32 OpenEmbedded build system, see the
33 :doc:`/brief-yoctoprojectqs/index` document.
34
35The build process creates an entire Linux distribution from source and
36places it in your :term:`Build Directory` under ``tmp/deploy/images``. For
37detailed information on the build process using BitBake, see the
38":ref:`overview-manual/concepts:images`" section in the Yocto Project Overview
39and Concepts Manual.
40
41The following figure and list overviews the build process:
42
43.. image:: figures/bitbake-build-flow.png
44 :width: 100%
45
46#. *Set up Your Host Development System to Support Development Using the
47 Yocto Project*: See the ":doc:`start`" section for options on how to get a
48 build host ready to use the Yocto Project.
49
50#. *Initialize the Build Environment:* Initialize the build environment
51 by sourcing the build environment script (i.e.
52 :ref:`structure-core-script`)::
53
54 $ source oe-init-build-env [build_dir]
55
56 When you use the initialization script, the OpenEmbedded build system
57 uses ``build`` as the default :term:`Build Directory` in your current work
58 directory. You can use a `build_dir` argument with the script to
59 specify a different :term:`Build Directory`.
60
61 .. note::
62
63 A common practice is to use a different :term:`Build Directory` for
64 different targets; for example, ``~/build/x86`` for a ``qemux86``
65 target, and ``~/build/arm`` for a ``qemuarm`` target. In any
66 event, it's typically cleaner to locate the :term:`Build Directory`
67 somewhere outside of your source directory.
68
69#. *Make Sure Your* ``local.conf`` *File is Correct*: Ensure the
70 ``conf/local.conf`` configuration file, which is found in the
71 :term:`Build Directory`, is set up how you want it. This file defines many
72 aspects of the build environment including the target machine architecture
73 through the :term:`MACHINE` variable, the packaging format used during
74 the build (:term:`PACKAGE_CLASSES`), and a centralized tarball download
75 directory through the :term:`DL_DIR` variable.
76
77#. *Build the Image:* Build the image using the ``bitbake`` command::
78
79 $ bitbake target
80
81 .. note::
82
83 For information on BitBake, see the :doc:`bitbake:index`.
84
85 The target is the name of the recipe you want to build. Common
86 targets are the images in ``meta/recipes-core/images``,
87 ``meta/recipes-sato/images``, and so forth all found in the
88 :term:`Source Directory`. Alternatively, the target
89 can be the name of a recipe for a specific piece of software such as
90 BusyBox. For more details about the images the OpenEmbedded build
91 system supports, see the
92 ":ref:`ref-manual/images:Images`" chapter in the Yocto
93 Project Reference Manual.
94
95 As an example, the following command builds the
96 ``core-image-minimal`` image::
97
98 $ bitbake core-image-minimal
99
100 Once an
101 image has been built, it often needs to be installed. The images and
102 kernels built by the OpenEmbedded build system are placed in the
103 :term:`Build Directory` in ``tmp/deploy/images``. For information on how to
104 run pre-built images such as ``qemux86`` and ``qemuarm``, see the
105 :doc:`/sdk-manual/index` manual. For
106 information about how to install these images, see the documentation
107 for your particular board or machine.
108
109Building Images for Multiple Targets Using Multiple Configurations
110==================================================================
111
112You can use a single ``bitbake`` command to build multiple images or
113packages for different targets where each image or package requires a
114different configuration (multiple configuration builds). The builds, in
115this scenario, are sometimes referred to as "multiconfigs", and this
116section uses that term throughout.
117
118This section describes how to set up for multiple configuration builds
119and how to account for cross-build dependencies between the
120multiconfigs.
121
122Setting Up and Running a Multiple Configuration Build
123-----------------------------------------------------
124
125To accomplish a multiple configuration build, you must define each
126target's configuration separately using a parallel configuration file in
127the :term:`Build Directory` or configuration directory within a layer, and you
128must follow a required file hierarchy. Additionally, you must enable the
129multiple configuration builds in your ``local.conf`` file.
130
131Follow these steps to set up and execute multiple configuration builds:
132
133- *Create Separate Configuration Files*: You need to create a single
134 configuration file for each build target (each multiconfig).
135 The configuration definitions are implementation dependent but often
136 each configuration file will define the machine and the
137 temporary directory BitBake uses for the build. Whether the same
138 temporary directory (:term:`TMPDIR`) can be shared will depend on what is
139 similar and what is different between the configurations. Multiple MACHINE
140 targets can share the same (:term:`TMPDIR`) as long as the rest of the
141 configuration is the same, multiple :term:`DISTRO` settings would need separate
142 (:term:`TMPDIR`) directories.
143
144 For example, consider a scenario with two different multiconfigs for the same
145 :term:`MACHINE`: "qemux86" built
146 for two distributions such as "poky" and "poky-lsb". In this case,
147 you would need to use the different :term:`TMPDIR`.
148
149 Here is an example showing the minimal statements needed in a
150 configuration file for a "qemux86" target whose temporary build
151 directory is ``tmpmultix86``::
152
153 MACHINE = "qemux86"
154 TMPDIR = "${TOPDIR}/tmpmultix86"
155
156 The location for these multiconfig configuration files is specific.
157 They must reside in the current :term:`Build Directory` in a sub-directory of
158 ``conf`` named ``multiconfig`` or within a layer's ``conf`` directory
159 under a directory named ``multiconfig``. Following is an example that defines
160 two configuration files for the "x86" and "arm" multiconfigs:
161
162 .. image:: figures/multiconfig_files.png
163 :align: center
164 :width: 50%
165
166 The usual :term:`BBPATH` search path is used to locate multiconfig files in
167 a similar way to other conf files.
168
169- *Add the BitBake Multi-configuration Variable to the Local
170 Configuration File*: Use the
171 :term:`BBMULTICONFIG`
172 variable in your ``conf/local.conf`` configuration file to specify
173 each multiconfig. Continuing with the example from the previous
174 figure, the :term:`BBMULTICONFIG` variable needs to enable two
175 multiconfigs: "x86" and "arm" by specifying each configuration file::
176
177 BBMULTICONFIG = "x86 arm"
178
179 .. note::
180
181 A "default" configuration already exists by definition. This
182 configuration is named: "" (i.e. empty string) and is defined by
183 the variables coming from your ``local.conf``
184 file. Consequently, the previous example actually adds two
185 additional configurations to your build: "arm" and "x86" along
186 with "".
187
188- *Launch BitBake*: Use the following BitBake command form to launch
189 the multiple configuration build::
190
191 $ bitbake [mc:multiconfigname:]target [[[mc:multiconfigname:]target] ... ]
192
193 For the example in this section, the following command applies::
194
195 $ bitbake mc:x86:core-image-minimal mc:arm:core-image-sato mc::core-image-base
196
197 The previous BitBake command builds a ``core-image-minimal`` image
198 that is configured through the ``x86.conf`` configuration file, a
199 ``core-image-sato`` image that is configured through the ``arm.conf``
200 configuration file and a ``core-image-base`` that is configured
201 through your ``local.conf`` configuration file.
202
203.. note::
204
205 Support for multiple configuration builds in the Yocto Project &DISTRO;
206 (&DISTRO_NAME;) Release does not include Shared State (sstate)
207 optimizations. Consequently, if a build uses the same object twice
208 in, for example, two different :term:`TMPDIR`
209 directories, the build either loads from an existing sstate cache for
210 that build at the start or builds the object fresh.
211
212Enabling Multiple Configuration Build Dependencies
213--------------------------------------------------
214
215Sometimes dependencies can exist between targets (multiconfigs) in a
216multiple configuration build. For example, suppose that in order to
217build a ``core-image-sato`` image for an "x86" multiconfig, the root
218filesystem of an "arm" multiconfig must exist. This dependency is
219essentially that the
220:ref:`ref-tasks-image` task in the
221``core-image-sato`` recipe depends on the completion of the
222:ref:`ref-tasks-rootfs` task of the
223``core-image-minimal`` recipe.
224
225To enable dependencies in a multiple configuration build, you must
226declare the dependencies in the recipe using the following statement
227form::
228
229 task_or_package[mcdepends] = "mc:from_multiconfig:to_multiconfig:recipe_name:task_on_which_to_depend"
230
231To better show how to use this statement, consider the example scenario
232from the first paragraph of this section. The following statement needs
233to be added to the recipe that builds the ``core-image-sato`` image::
234
235 do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs"
236
237In this example, the `from_multiconfig` is "x86". The `to_multiconfig` is "arm". The
238task on which the :ref:`ref-tasks-image` task in the recipe depends is the
239:ref:`ref-tasks-rootfs` task from the ``core-image-minimal`` recipe associated
240with the "arm" multiconfig.
241
242Once you set up this dependency, you can build the "x86" multiconfig
243using a BitBake command as follows::
244
245 $ bitbake mc:x86:core-image-sato
246
247This command executes all the tasks needed to create the
248``core-image-sato`` image for the "x86" multiconfig. Because of the
249dependency, BitBake also executes through the :ref:`ref-tasks-rootfs` task for the
250"arm" multiconfig build.
251
252Having a recipe depend on the root filesystem of another build might not
253seem that useful. Consider this change to the statement in the
254``core-image-sato`` recipe::
255
256 do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_image"
257
258In this case, BitBake must
259create the ``core-image-minimal`` image for the "arm" build since the
260"x86" build depends on it.
261
262Because "x86" and "arm" are enabled for multiple configuration builds
263and have separate configuration files, BitBake places the artifacts for
264each build in the respective temporary build directories (i.e.
265:term:`TMPDIR`).
266
267Building an Initial RAM Filesystem (Initramfs) Image
268====================================================
269
270An initial RAM filesystem (:term:`Initramfs`) image provides a temporary root
271filesystem used for early system initialization, typically providing tools and
272loading modules needed to locate and mount the final root filesystem.
273
274Follow these steps to create an :term:`Initramfs` image:
275
Andrew Geissler5082cc72023-09-11 08:41:39 -0400276#. *Create the Initramfs Image Recipe:* You can reference the
Andrew Geissler517393d2023-01-13 08:55:19 -0600277 ``core-image-minimal-initramfs.bb`` recipe found in the
278 ``meta/recipes-core`` directory of the :term:`Source Directory`
279 as an example from which to work.
280
Andrew Geissler5082cc72023-09-11 08:41:39 -0400281#. *Decide if You Need to Bundle the Initramfs Image Into the Kernel
Andrew Geissler517393d2023-01-13 08:55:19 -0600282 Image:* If you want the :term:`Initramfs` image that is built to be bundled
283 in with the kernel image, set the :term:`INITRAMFS_IMAGE_BUNDLE`
284 variable to ``"1"`` in your ``local.conf`` configuration file and set the
285 :term:`INITRAMFS_IMAGE` variable in the recipe that builds the kernel image.
286
287 Setting the :term:`INITRAMFS_IMAGE_BUNDLE` flag causes the :term:`Initramfs`
288 image to be unpacked into the ``${B}/usr/`` directory. The unpacked
289 :term:`Initramfs` image is then passed to the kernel's ``Makefile`` using the
290 :term:`CONFIG_INITRAMFS_SOURCE` variable, allowing the :term:`Initramfs`
291 image to be built into the kernel normally.
292
293#. *Optionally Add Items to the Initramfs Image Through the Initramfs
294 Image Recipe:* If you add items to the :term:`Initramfs` image by way of its
295 recipe, you should use :term:`PACKAGE_INSTALL` rather than
296 :term:`IMAGE_INSTALL`. :term:`PACKAGE_INSTALL` gives more direct control of
297 what is added to the image as compared to the defaults you might not
298 necessarily want that are set by the :ref:`ref-classes-image`
299 or :ref:`ref-classes-core-image` classes.
300
301#. *Build the Kernel Image and the Initramfs Image:* Build your kernel
302 image using BitBake. Because the :term:`Initramfs` image recipe is a
303 dependency of the kernel image, the :term:`Initramfs` image is built as well
304 and bundled with the kernel image if you used the
305 :term:`INITRAMFS_IMAGE_BUNDLE` variable described earlier.
306
307Bundling an Initramfs Image From a Separate Multiconfig
308-------------------------------------------------------
309
310There may be a case where we want to build an :term:`Initramfs` image which does not
311inherit the same distro policy as our main image, for example, we may want
312our main image to use ``TCLIBC="glibc"``, but to use ``TCLIBC="musl"`` in our :term:`Initramfs`
313image to keep a smaller footprint. However, by performing the steps mentioned
314above the :term:`Initramfs` image will inherit ``TCLIBC="glibc"`` without allowing us
315to override it.
316
317To achieve this, you need to perform some additional steps:
318
319#. *Create a multiconfig for your Initramfs image:* You can perform the steps
320 on ":ref:`dev-manual/building:building images for multiple targets using multiple configurations`" to create a separate multiconfig.
321 For the sake of simplicity let's assume such multiconfig is called: ``initramfscfg.conf`` and
322 contains the variables::
323
324 TMPDIR="${TOPDIR}/tmp-initramfscfg"
325 TCLIBC="musl"
326
327#. *Set additional Initramfs variables on your main configuration:*
328 Additionally, on your main configuration (``local.conf``) you need to set the
329 variables::
330
331 INITRAMFS_MULTICONFIG = "initramfscfg"
332 INITRAMFS_DEPLOY_DIR_IMAGE = "${TOPDIR}/tmp-initramfscfg/deploy/images/${MACHINE}"
333
334 The variables :term:`INITRAMFS_MULTICONFIG` and :term:`INITRAMFS_DEPLOY_DIR_IMAGE`
335 are used to create a multiconfig dependency from the kernel to the :term:`INITRAMFS_IMAGE`
336 to be built coming from the ``initramfscfg`` multiconfig, and to let the
337 buildsystem know where the :term:`INITRAMFS_IMAGE` will be located.
338
339 Building a system with such configuration will build the kernel using the
340 main configuration but the :ref:`ref-tasks-bundle_initramfs` task will grab the
341 selected :term:`INITRAMFS_IMAGE` from :term:`INITRAMFS_DEPLOY_DIR_IMAGE`
342 instead, resulting in a musl based :term:`Initramfs` image bundled in the kernel
343 but a glibc based main image.
344
345 The same is applicable to avoid inheriting :term:`DISTRO_FEATURES` on :term:`INITRAMFS_IMAGE`
346 or to build a different :term:`DISTRO` for it such as ``poky-tiny``.
347
348
349Building a Tiny System
350======================
351
352Very small distributions have some significant advantages such as
353requiring less on-die or in-package memory (cheaper), better performance
354through efficient cache usage, lower power requirements due to less
355memory, faster boot times, and reduced development overhead. Some
356real-world examples where a very small distribution gives you distinct
357advantages are digital cameras, medical devices, and small headless
358systems.
359
360This section presents information that shows you how you can trim your
361distribution to even smaller sizes than the ``poky-tiny`` distribution,
362which is around 5 Mbytes, that can be built out-of-the-box using the
363Yocto Project.
364
365Tiny System Overview
366--------------------
367
368The following list presents the overall steps you need to consider and
369perform to create distributions with smaller root filesystems, achieve
370faster boot times, maintain your critical functionality, and avoid
371initial RAM disks:
372
373- :ref:`Determine your goals and guiding principles
374 <dev-manual/building:goals and guiding principles>`
375
376- :ref:`dev-manual/building:understand what contributes to your image size`
377
378- :ref:`Reduce the size of the root filesystem
379 <dev-manual/building:trim the root filesystem>`
380
381- :ref:`Reduce the size of the kernel <dev-manual/building:trim the kernel>`
382
383- :ref:`dev-manual/building:remove package management requirements`
384
385- :ref:`dev-manual/building:look for other ways to minimize size`
386
387- :ref:`dev-manual/building:iterate on the process`
388
389Goals and Guiding Principles
390----------------------------
391
392Before you can reach your destination, you need to know where you are
393going. Here is an example list that you can use as a guide when creating
394very small distributions:
395
396- Determine how much space you need (e.g. a kernel that is 1 Mbyte or
397 less and a root filesystem that is 3 Mbytes or less).
398
399- Find the areas that are currently taking 90% of the space and
400 concentrate on reducing those areas.
401
402- Do not create any difficult "hacks" to achieve your goals.
403
404- Leverage the device-specific options.
405
406- Work in a separate layer so that you keep changes isolated. For
407 information on how to create layers, see the
408 ":ref:`dev-manual/layers:understanding and creating layers`" section.
409
410Understand What Contributes to Your Image Size
411----------------------------------------------
412
413It is easiest to have something to start with when creating your own
414distribution. You can use the Yocto Project out-of-the-box to create the
415``poky-tiny`` distribution. Ultimately, you will want to make changes in
416your own distribution that are likely modeled after ``poky-tiny``.
417
418.. note::
419
420 To use ``poky-tiny`` in your build, set the :term:`DISTRO` variable in your
421 ``local.conf`` file to "poky-tiny" as described in the
422 ":ref:`dev-manual/custom-distribution:creating your own distribution`"
423 section.
424
425Understanding some memory concepts will help you reduce the system size.
426Memory consists of static, dynamic, and temporary memory. Static memory
427is the TEXT (code), DATA (initialized data in the code), and BSS
428(uninitialized data) sections. Dynamic memory represents memory that is
429allocated at runtime: stacks, hash tables, and so forth. Temporary
430memory is recovered after the boot process. This memory consists of
431memory used for decompressing the kernel and for the ``__init__``
432functions.
433
434To help you see where you currently are with kernel and root filesystem
435sizes, you can use two tools found in the :term:`Source Directory`
436in the
437``scripts/tiny/`` directory:
438
439- ``ksize.py``: Reports component sizes for the kernel build objects.
440
441- ``dirsize.py``: Reports component sizes for the root filesystem.
442
443This next tool and command help you organize configuration fragments and
444view file dependencies in a human-readable form:
445
446- ``merge_config.sh``: Helps you manage configuration files and
447 fragments within the kernel. With this tool, you can merge individual
448 configuration fragments together. The tool allows you to make
449 overrides and warns you of any missing configuration options. The
450 tool is ideal for allowing you to iterate on configurations, create
451 minimal configurations, and create configuration files for different
452 machines without having to duplicate your process.
453
454 The ``merge_config.sh`` script is part of the Linux Yocto kernel Git
455 repositories (i.e. ``linux-yocto-3.14``, ``linux-yocto-3.10``,
456 ``linux-yocto-3.8``, and so forth) in the ``scripts/kconfig``
457 directory.
458
459 For more information on configuration fragments, see the
460 ":ref:`kernel-dev/common:creating configuration fragments`"
461 section in the Yocto Project Linux Kernel Development Manual.
462
463- ``bitbake -u taskexp -g bitbake_target``: Using the BitBake command
464 with these options brings up a Dependency Explorer from which you can
465 view file dependencies. Understanding these dependencies allows you
466 to make informed decisions when cutting out various pieces of the
467 kernel and root filesystem.
468
469Trim the Root Filesystem
470------------------------
471
472The root filesystem is made up of packages for booting, libraries, and
473applications. To change things, you can configure how the packaging
474happens, which changes the way you build them. You can also modify the
475filesystem itself or select a different filesystem.
476
477First, find out what is hogging your root filesystem by running the
478``dirsize.py`` script from your root directory::
479
480 $ cd root-directory-of-image
481 $ dirsize.py 100000 > dirsize-100k.log
482 $ cat dirsize-100k.log
483
484You can apply a filter to the script to ignore files
485under a certain size. The previous example filters out any files below
486100 Kbytes. The sizes reported by the tool are uncompressed, and thus
487will be smaller by a relatively constant factor in a compressed root
488filesystem. When you examine your log file, you can focus on areas of
489the root filesystem that take up large amounts of memory.
490
491You need to be sure that what you eliminate does not cripple the
492functionality you need. One way to see how packages relate to each other
493is by using the Dependency Explorer UI with the BitBake command::
494
495 $ cd image-directory
496 $ bitbake -u taskexp -g image
497
498Use the interface to
499select potential packages you wish to eliminate and see their dependency
500relationships.
501
502When deciding how to reduce the size, get rid of packages that result in
503minimal impact on the feature set. For example, you might not need a VGA
504display. Or, you might be able to get by with ``devtmpfs`` and ``mdev``
505instead of ``udev``.
506
507Use your ``local.conf`` file to make changes. For example, to eliminate
508``udev`` and ``glib``, set the following in the local configuration
509file::
510
511 VIRTUAL-RUNTIME_dev_manager = ""
512
513Finally, you should consider exactly the type of root filesystem you
514need to meet your needs while also reducing its size. For example,
515consider ``cramfs``, ``squashfs``, ``ubifs``, ``ext2``, or an
516:term:`Initramfs` using ``initramfs``. Be aware that ``ext3`` requires a 1
517Mbyte journal. If you are okay with running read-only, you do not need
518this journal.
519
520.. note::
521
522 After each round of elimination, you need to rebuild your system and
523 then use the tools to see the effects of your reductions.
524
525Trim the Kernel
526---------------
527
528The kernel is built by including policies for hardware-independent
529aspects. What subsystems do you enable? For what architecture are you
530building? Which drivers do you build by default?
531
532.. note::
533
534 You can modify the kernel source if you want to help with boot time.
535
536Run the ``ksize.py`` script from the top-level Linux build directory to
537get an idea of what is making up the kernel::
538
539 $ cd top-level-linux-build-directory
540 $ ksize.py > ksize.log
541 $ cat ksize.log
542
543When you examine the log, you will see how much space is taken up with
544the built-in ``.o`` files for drivers, networking, core kernel files,
545filesystem, sound, and so forth. The sizes reported by the tool are
546uncompressed, and thus will be smaller by a relatively constant factor
547in a compressed kernel image. Look to reduce the areas that are large
548and taking up around the "90% rule."
549
550To examine, or drill down, into any particular area, use the ``-d``
551option with the script::
552
553 $ ksize.py -d > ksize.log
554
555Using this option
556breaks out the individual file information for each area of the kernel
557(e.g. drivers, networking, and so forth).
558
559Use your log file to see what you can eliminate from the kernel based on
560features you can let go. For example, if you are not going to need
561sound, you do not need any drivers that support sound.
562
563After figuring out what to eliminate, you need to reconfigure the kernel
564to reflect those changes during the next build. You could run
565``menuconfig`` and make all your changes at once. However, that makes it
566difficult to see the effects of your individual eliminations and also
567makes it difficult to replicate the changes for perhaps another target
568device. A better method is to start with no configurations using
569``allnoconfig``, create configuration fragments for individual changes,
570and then manage the fragments into a single configuration file using
571``merge_config.sh``. The tool makes it easy for you to iterate using the
572configuration change and build cycle.
573
574Each time you make configuration changes, you need to rebuild the kernel
575and check to see what impact your changes had on the overall size.
576
577Remove Package Management Requirements
578--------------------------------------
579
580Packaging requirements add size to the image. One way to reduce the size
581of the image is to remove all the packaging requirements from the image.
582This reduction includes both removing the package manager and its unique
583dependencies as well as removing the package management data itself.
584
585To eliminate all the packaging requirements for an image, be sure that
586"package-management" is not part of your
587:term:`IMAGE_FEATURES`
588statement for the image. When you remove this feature, you are removing
589the package manager as well as its dependencies from the root
590filesystem.
591
592Look for Other Ways to Minimize Size
593------------------------------------
594
595Depending on your particular circumstances, other areas that you can
596trim likely exist. The key to finding these areas is through tools and
597methods described here combined with experimentation and iteration. Here
598are a couple of areas to experiment with:
599
600- ``glibc``: In general, follow this process:
601
602 #. Remove ``glibc`` features from
603 :term:`DISTRO_FEATURES`
604 that you think you do not need.
605
606 #. Build your distribution.
607
608 #. If the build fails due to missing symbols in a package, determine
609 if you can reconfigure the package to not need those features. For
610 example, change the configuration to not support wide character
611 support as is done for ``ncurses``. Or, if support for those
612 characters is needed, determine what ``glibc`` features provide
613 the support and restore the configuration.
614
615 4. Rebuild and repeat the process.
616
617- ``busybox``: For BusyBox, use a process similar as described for
618 ``glibc``. A difference is you will need to boot the resulting system
619 to see if you are able to do everything you expect from the running
620 system. You need to be sure to integrate configuration fragments into
621 Busybox because BusyBox handles its own core features and then allows
622 you to add configuration fragments on top.
623
624Iterate on the Process
625----------------------
626
627If you have not reached your goals on system size, you need to iterate
628on the process. The process is the same. Use the tools and see just what
629is taking up 90% of the root filesystem and the kernel. Decide what you
630can eliminate without limiting your device beyond what you need.
631
632Depending on your system, a good place to look might be Busybox, which
633provides a stripped down version of Unix tools in a single, executable
634file. You might be able to drop virtual terminal services or perhaps
635ipv6.
636
637Building Images for More than One Machine
638=========================================
639
640A common scenario developers face is creating images for several
641different machines that use the same software environment. In this
642situation, it is tempting to set the tunings and optimization flags for
643each build specifically for the targeted hardware (i.e. "maxing out" the
644tunings). Doing so can considerably add to build times and package feed
645maintenance collectively for the machines. For example, selecting tunes
646that are extremely specific to a CPU core used in a system might enable
647some micro optimizations in GCC for that particular system but would
648otherwise not gain you much of a performance difference across the other
649systems as compared to using a more general tuning across all the builds
650(e.g. setting :term:`DEFAULTTUNE`
651specifically for each machine's build). Rather than "max out" each
652build's tunings, you can take steps that cause the OpenEmbedded build
653system to reuse software across the various machines where it makes
654sense.
655
656If build speed and package feed maintenance are considerations, you
657should consider the points in this section that can help you optimize
658your tunings to best consider build times and package feed maintenance.
659
660- *Share the :term:`Build Directory`:* If at all possible, share the
661 :term:`TMPDIR` across builds. The Yocto Project supports switching between
662 different :term:`MACHINE` values in the same :term:`TMPDIR`. This practice
663 is well supported and regularly used by developers when building for
664 multiple machines. When you use the same :term:`TMPDIR` for multiple
665 machine builds, the OpenEmbedded build system can reuse the existing native
666 and often cross-recipes for multiple machines. Thus, build time decreases.
667
668 .. note::
669
670 If :term:`DISTRO` settings change or fundamental configuration settings
671 such as the filesystem layout, you need to work with a clean :term:`TMPDIR`.
672 Sharing :term:`TMPDIR` under these circumstances might work but since it is
673 not guaranteed, you should use a clean :term:`TMPDIR`.
674
675- *Enable the Appropriate Package Architecture:* By default, the
676 OpenEmbedded build system enables three levels of package
677 architectures: "all", "tune" or "package", and "machine". Any given
678 recipe usually selects one of these package architectures (types) for
679 its output. Depending for what a given recipe creates packages,
680 making sure you enable the appropriate package architecture can
681 directly impact the build time.
682
683 A recipe that just generates scripts can enable "all" architecture
684 because there are no binaries to build. To specifically enable "all"
685 architecture, be sure your recipe inherits the
686 :ref:`ref-classes-allarch` class.
687 This class is useful for "all" architectures because it configures
688 many variables so packages can be used across multiple architectures.
689
690 If your recipe needs to generate packages that are machine-specific
691 or when one of the build or runtime dependencies is already
692 machine-architecture dependent, which makes your recipe also
693 machine-architecture dependent, make sure your recipe enables the
694 "machine" package architecture through the
695 :term:`MACHINE_ARCH`
696 variable::
697
698 PACKAGE_ARCH = "${MACHINE_ARCH}"
699
700 When you do not
701 specifically enable a package architecture through the
702 :term:`PACKAGE_ARCH`, The
703 OpenEmbedded build system defaults to the
704 :term:`TUNE_PKGARCH` setting::
705
706 PACKAGE_ARCH = "${TUNE_PKGARCH}"
707
708- *Choose a Generic Tuning File if Possible:* Some tunes are more
709 generic and can run on multiple targets (e.g. an ``armv5`` set of
710 packages could run on ``armv6`` and ``armv7`` processors in most
711 cases). Similarly, ``i486`` binaries could work on ``i586`` and
712 higher processors. You should realize, however, that advances on
713 newer processor versions would not be used.
714
715 If you select the same tune for several different machines, the
716 OpenEmbedded build system reuses software previously built, thus
717 speeding up the overall build time. Realize that even though a new
718 sysroot for each machine is generated, the software is not recompiled
719 and only one package feed exists.
720
721- *Manage Granular Level Packaging:* Sometimes there are cases where
722 injecting another level of package architecture beyond the three
723 higher levels noted earlier can be useful. For example, consider how
724 NXP (formerly Freescale) allows for the easy reuse of binary packages
725 in their layer
726 :yocto_git:`meta-freescale </meta-freescale/>`.
727 In this example, the
728 :yocto_git:`fsl-dynamic-packagearch </meta-freescale/tree/classes/fsl-dynamic-packagearch.bbclass>`
729 class shares GPU packages for i.MX53 boards because all boards share
730 the AMD GPU. The i.MX6-based boards can do the same because all
731 boards share the Vivante GPU. This class inspects the BitBake
732 datastore to identify if the package provides or depends on one of
733 the sub-architecture values. If so, the class sets the
734 :term:`PACKAGE_ARCH` value
735 based on the ``MACHINE_SUBARCH`` value. If the package does not
736 provide or depend on one of the sub-architecture values but it
737 matches a value in the machine-specific filter, it sets
738 :term:`MACHINE_ARCH`. This
739 behavior reduces the number of packages built and saves build time by
740 reusing binaries.
741
742- *Use Tools to Debug Issues:* Sometimes you can run into situations
743 where software is being rebuilt when you think it should not be. For
744 example, the OpenEmbedded build system might not be using shared
745 state between machines when you think it should be. These types of
746 situations are usually due to references to machine-specific
747 variables such as :term:`MACHINE`,
748 :term:`SERIAL_CONSOLES`,
749 :term:`XSERVER`,
750 :term:`MACHINE_FEATURES`,
751 and so forth in code that is supposed to only be tune-specific or
752 when the recipe depends
753 (:term:`DEPENDS`,
754 :term:`RDEPENDS`,
755 :term:`RRECOMMENDS`,
756 :term:`RSUGGESTS`, and so forth)
757 on some other recipe that already has
758 :term:`PACKAGE_ARCH` defined
759 as "${MACHINE_ARCH}".
760
761 .. note::
762
763 Patches to fix any issues identified are most welcome as these
764 issues occasionally do occur.
765
766 For such cases, you can use some tools to help you sort out the
767 situation:
768
769 - ``state-diff-machines.sh``*:* You can find this tool in the
770 ``scripts`` directory of the Source Repositories. See the comments
771 in the script for information on how to use the tool.
772
773 - *BitBake's "-S printdiff" Option:* Using this option causes
774 BitBake to try to establish the closest signature match it can
775 (e.g. in the shared state cache) and then run ``bitbake-diffsigs``
776 over the matches to determine the stamps and delta where these two
777 stamp trees diverge.
778
779Building Software from an External Source
780=========================================
781
782By default, the OpenEmbedded build system uses the :term:`Build Directory`
783when building source code. The build process involves fetching the source
784files, unpacking them, and then patching them if necessary before the build
785takes place.
786
787There are situations where you might want to build software from source
788files that are external to and thus outside of the OpenEmbedded build
789system. For example, suppose you have a project that includes a new BSP
790with a heavily customized kernel. And, you want to minimize exposing the
791build system to the development team so that they can focus on their
792project and maintain everyone's workflow as much as possible. In this
793case, you want a kernel source directory on the development machine
794where the development occurs. You want the recipe's
795:term:`SRC_URI` variable to point to
796the external directory and use it as is, not copy it.
797
798To build from software that comes from an external source, all you need to do
799is inherit the :ref:`ref-classes-externalsrc` class and then set
800the :term:`EXTERNALSRC` variable to point to your external source code. Here
801are the statements to put in your ``local.conf`` file::
802
803 INHERIT += "externalsrc"
804 EXTERNALSRC:pn-myrecipe = "path-to-your-source-tree"
805
806This next example shows how to accomplish the same thing by setting
807:term:`EXTERNALSRC` in the recipe itself or in the recipe's append file::
808
809 EXTERNALSRC = "path"
810 EXTERNALSRC_BUILD = "path"
811
812.. note::
813
814 In order for these settings to take effect, you must globally or
815 locally inherit the :ref:`ref-classes-externalsrc` class.
816
817By default, :ref:`ref-classes-externalsrc` builds the source code in a
818directory separate from the external source directory as specified by
819:term:`EXTERNALSRC`. If you need
820to have the source built in the same directory in which it resides, or
821some other nominated directory, you can set
822:term:`EXTERNALSRC_BUILD`
823to point to that directory::
824
825 EXTERNALSRC_BUILD:pn-myrecipe = "path-to-your-source-tree"
826
827Replicating a Build Offline
828===========================
829
830It can be useful to take a "snapshot" of upstream sources used in a
831build and then use that "snapshot" later to replicate the build offline.
832To do so, you need to first prepare and populate your downloads
833directory your "snapshot" of files. Once your downloads directory is
834ready, you can use it at any time and from any machine to replicate your
835build.
836
837Follow these steps to populate your Downloads directory:
838
839#. *Create a Clean Downloads Directory:* Start with an empty downloads
840 directory (:term:`DL_DIR`). You
841 start with an empty downloads directory by either removing the files
842 in the existing directory or by setting :term:`DL_DIR` to point to either
843 an empty location or one that does not yet exist.
844
845#. *Generate Tarballs of the Source Git Repositories:* Edit your
846 ``local.conf`` configuration file as follows::
847
848 DL_DIR = "/home/your-download-dir/"
849 BB_GENERATE_MIRROR_TARBALLS = "1"
850
851 During
852 the fetch process in the next step, BitBake gathers the source files
853 and creates tarballs in the directory pointed to by :term:`DL_DIR`. See
854 the
855 :term:`BB_GENERATE_MIRROR_TARBALLS`
856 variable for more information.
857
858#. *Populate Your Downloads Directory Without Building:* Use BitBake to
859 fetch your sources but inhibit the build::
860
861 $ bitbake target --runonly=fetch
862
863 The downloads directory (i.e. ``${DL_DIR}``) now has
864 a "snapshot" of the source files in the form of tarballs, which can
865 be used for the build.
866
867#. *Optionally Remove Any Git or other SCM Subdirectories From the
868 Downloads Directory:* If you want, you can clean up your downloads
869 directory by removing any Git or other Source Control Management
870 (SCM) subdirectories such as ``${DL_DIR}/git2/*``. The tarballs
871 already contain these subdirectories.
872
873Once your downloads directory has everything it needs regarding source
874files, you can create your "own-mirror" and build your target.
875Understand that you can use the files to build the target offline from
876any machine and at any time.
877
878Follow these steps to build your target using the files in the downloads
879directory:
880
881#. *Using Local Files Only:* Inside your ``local.conf`` file, add the
882 :term:`SOURCE_MIRROR_URL` variable, inherit the
883 :ref:`ref-classes-own-mirrors` class, and use the
884 :term:`BB_NO_NETWORK` variable to your ``local.conf``::
885
886 SOURCE_MIRROR_URL ?= "file:///home/your-download-dir/"
887 INHERIT += "own-mirrors"
888 BB_NO_NETWORK = "1"
889
890 The :term:`SOURCE_MIRROR_URL` and :ref:`ref-classes-own-mirrors`
891 class set up the system to use the downloads directory as your "own
892 mirror". Using the :term:`BB_NO_NETWORK` variable makes sure that
893 BitBake's fetching process in step 3 stays local, which means files
894 from your "own-mirror" are used.
895
896#. *Start With a Clean Build:* You can start with a clean build by
897 removing the ``${``\ :term:`TMPDIR`\ ``}`` directory or using a new
898 :term:`Build Directory`.
899
900#. *Build Your Target:* Use BitBake to build your target::
901
902 $ bitbake target
903
904 The build completes using the known local "snapshot" of source
905 files from your mirror. The resulting tarballs for your "snapshot" of
906 source files are in the downloads directory.
907
908 .. note::
909
910 The offline build does not work if recipes attempt to find the
911 latest version of software by setting
912 :term:`SRCREV` to
913 ``${``\ :term:`AUTOREV`\ ``}``::
914
915 SRCREV = "${AUTOREV}"
916
917 When a recipe sets :term:`SRCREV` to
918 ``${``\ :term:`AUTOREV`\ ``}``, the build system accesses the network in an
919 attempt to determine the latest version of software from the SCM.
920 Typically, recipes that use :term:`AUTOREV` are custom or modified
921 recipes. Recipes that reside in public repositories usually do not
922 use :term:`AUTOREV`.
923
924 If you do have recipes that use :term:`AUTOREV`, you can take steps to
925 still use the recipes in an offline build. Do the following:
926
927 #. Use a configuration generated by enabling :ref:`build
928 history <dev-manual/build-quality:maintaining build output quality>`.
929
930 #. Use the ``buildhistory-collect-srcrevs`` command to collect the
931 stored :term:`SRCREV` values from the build's history. For more
932 information on collecting these values, see the
933 ":ref:`dev-manual/build-quality:build history package information`"
934 section.
935
936 #. Once you have the correct source revisions, you can modify
937 those recipes to set :term:`SRCREV` to specific versions of the
938 software.
939