blob: 09b71698111b30ea42e7153a3fc8e8338f932cac [file] [log] [blame]
Andrew Geisslerf0343792020-11-18 10:42:21 -06001.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Andrew Geisslerc9f78652020-09-18 14:11:35 -05002
3************************
4Using the Extensible SDK
5************************
6
7This chapter describes the extensible SDK and how to install it.
8Information covers the pieces of the SDK, how to install it, and
9presents a look at using the ``devtool`` functionality. The extensible
10SDK makes it easy to add new applications and libraries to an image,
11modify the source for an existing component, test changes on the target
12hardware, and ease integration into the rest of the
13:term:`OpenEmbedded Build System`.
14
15.. note::
16
17 For a side-by-side comparison of main features supported for an
Andrew Geissler09036742021-06-25 14:25:14 -050018 extensible SDK as compared to a standard SDK, see the
19 :ref:`sdk-manual/intro:introduction` section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -050020
21In addition to the functionality available through ``devtool``, you can
22alternatively make use of the toolchain directly, for example from
Andrew Geissler3b8a17c2021-04-15 15:55:55 -050023Makefile and Autotools. See the
24":ref:`sdk-manual/working-projects:using the sdk toolchain directly`" chapter
25for more information.
Andrew Geisslerc9f78652020-09-18 14:11:35 -050026
Andrew Geisslerc9f78652020-09-18 14:11:35 -050027Why use the Extensible SDK and What is in It?
28=============================================
29
30The extensible SDK provides a cross-development toolchain and libraries
31tailored to the contents of a specific image. You would use the
32Extensible SDK if you want a toolchain experience supplemented with the
33powerful set of ``devtool`` commands tailored for the Yocto Project
34environment.
35
36The installed extensible SDK consists of several files and directories.
37Basically, it contains an SDK environment setup script, some
38configuration files, an internal build system, and the ``devtool``
39functionality.
40
Andrew Geisslerc9f78652020-09-18 14:11:35 -050041Installing the Extensible SDK
42=============================
43
Patrick Williams92b42cb2022-09-03 06:53:57 -050044Two ways to install the Extensible SDK
45--------------------------------------
46
47Extensible SDK can be installed in two different ways, and both have
48their own pros and cons:
49
501. *Setting up the Extensible SDK environment directly in a Yocto build*. This
51avoids having to produce, test, distribute and maintain separate SDK installer
52archives, which can get very large. There is only one environment for the regular
53Yocto build and the SDK and less code paths where things can go not according to plan.
54It's easier to update the SDK: it simply means updating the Yocto layers with
55git fetch or layer management tooling. The SDK extensibility is better than in the
56second option: just run ``bitbake`` again to add more things to the sysroot, or add layers
57if even more things are required.
58
592. *Setting up the Extensible SDK from a standalone installer*. This has the benefit of
60having a single, self-contained archive that includes all the needed binary artifacts.
61So nothing needs to be rebuilt, and there is no need to provide a well-functioning
62binary artefact cache over the network for developers with underpowered laptops.
63
64Setting up the Extensible SDK environment directly in a Yocto build
65-------------------------------------------------------------------
66
671. Set up all the needed layers and a Yocto build directory, e.g. a regular Yocto
68 build where ``bitbake`` can be executed.
69
702. Run:
71 $ bitbake meta-ide-support
72 $ bitbake -c populate_sysroot gtk+3
73 (or any other target or native item that the application developer would need)
74 $ bitbake populate-sysroots
75
76
77Setting up the Extensible SDK from a standalone installer
78---------------------------------------------------------
79
Andrew Geisslerc9f78652020-09-18 14:11:35 -050080The first thing you need to do is install the SDK on your :term:`Build
81Host` by running the ``*.sh`` installation script.
82
83You can download a tarball installer, which includes the pre-built
84toolchain, the ``runqemu`` script, the internal build system,
85``devtool``, and support files from the appropriate
Andrew Geissler09209ee2020-12-13 08:44:15 -060086:yocto_dl:`toolchain </releases/yocto/yocto-&DISTRO;/toolchain/>` directory within the Index of
Andrew Geisslerc9f78652020-09-18 14:11:35 -050087Releases. Toolchains are available for several 32-bit and 64-bit
88architectures with the ``x86_64`` directories, respectively. The
89toolchains the Yocto Project provides are based off the
90``core-image-sato`` and ``core-image-minimal`` images and contain
91libraries appropriate for developing against that image.
92
93The names of the tarball installer scripts are such that a string
94representing the host system appears first in the filename and then is
95immediately followed by a string representing the target architecture.
96An extensible SDK has the string "-ext" as part of the name. Following
Andrew Geisslerc926e172021-05-07 16:11:35 -050097is the general form::
Andrew Geisslerc9f78652020-09-18 14:11:35 -050098
99 poky-glibc-host_system-image_type-arch-toolchain-ext-release_version.sh
100
101 Where:
102 host_system is a string representing your development system:
103
104 i686 or x86_64.
105
106 image_type is the image for which the SDK was built:
107
108 core-image-sato or core-image-minimal
109
110 arch is a string representing the tuned target architecture:
111
112 aarch64, armv5e, core2-64, i586, mips32r2, mips64, ppc7400, or cortexa8hf-neon
113
114 release_version is a string representing the release number of the Yocto Project:
115
Andrew Geissler09209ee2020-12-13 08:44:15 -0600116 &DISTRO;, &DISTRO;+snapshot
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500117
118For example, the following SDK installer is for a 64-bit
119development host system and a i586-tuned target architecture based off
Andrew Geisslerc926e172021-05-07 16:11:35 -0500120the SDK for ``core-image-sato`` and using the current &DISTRO; snapshot::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500121
Andrew Geisslerd1e89492021-02-12 15:35:20 -0600122 poky-glibc-x86_64-core-image-sato-i586-toolchain-ext-&DISTRO;.sh
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500123
124.. note::
125
126 As an alternative to downloading an SDK, you can build the SDK
Andrew Geissler09036742021-06-25 14:25:14 -0500127 installer. For information on building the installer, see the
128 :ref:`sdk-manual/appendix-obtain:building an sdk installer`
129 section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500130
131The SDK and toolchains are self-contained and by default are installed
132into the ``poky_sdk`` folder in your home directory. You can choose to
133install the extensible SDK in any location when you run the installer.
134However, because files need to be written under that directory during
135the normal course of operation, the location you choose for installation
136must be writable for whichever users need to use the SDK.
137
138The following command shows how to run the installer given a toolchain
139tarball for a 64-bit x86 development host system and a 64-bit x86 target
140architecture. The example assumes the SDK installer is located in
141``~/Downloads/`` and has execution rights.
142
143.. note::
144
145 If you do not have write permissions for the directory into which you
146 are installing the SDK, the installer notifies you and exits. For
147 that case, set up the proper permissions in the directory and run the
148 installer again.
149
150::
151
152 $ ./Downloads/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.5.sh
153 Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.5
154 ==========================================================================
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500155 Enter target directory for SDK (default: poky_sdk):
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500156 You are about to install the SDK to "/home/scottrif/poky_sdk". Proceed [Y/n]? Y
157 Extracting SDK..............done
158 Setting it up...
159 Extracting buildtools...
160 Preparing build system...
161 Parsing recipes: 100% |##################################################################| Time: 0:00:52
162 Initialising tasks: 100% |###############################################################| Time: 0:00:00
163 Checking sstate mirror object availability: 100% |#######################################| Time: 0:00:00
164 Loading cache: 100% |####################################################################| Time: 0:00:00
165 Initialising tasks: 100% |###############################################################| Time: 0:00:00
166 done
167 SDK has been successfully set up and is ready to be used.
168 Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
169 $ . /home/scottrif/poky_sdk/environment-setup-core2-64-poky-linux
170
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500171Running the Extensible SDK Environment Setup Script
172===================================================
173
174Once you have the SDK installed, you must run the SDK environment setup
Patrick Williams92b42cb2022-09-03 06:53:57 -0500175script before you can actually use the SDK.
176
177When using a SDK directly in a Yocto build, you will find the script in
178``tmp/deploy/images/qemux86-64/`` in your build directory.
179
180When using a standalone SDK installer, this setup script resides in
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500181the directory you chose when you installed the SDK, which is either the
182default ``poky_sdk`` directory or the directory you chose during
183installation.
184
185Before running the script, be sure it is the one that matches the
186architecture for which you are developing. Environment setup scripts
187begin with the string "``environment-setup``" and include as part of
188their name the tuned target architecture. As an example, the following
189commands set the working directory to where the SDK was installed and
190then source the environment setup script. In this example, the setup
Andrew Geisslerc926e172021-05-07 16:11:35 -0500191script is for an IA-based target machine using i586 tuning::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500192
193 $ cd /home/scottrif/poky_sdk
194 $ source environment-setup-core2-64-poky-linux
195 SDK environment now set up; additionally you may now run devtool to perform development tasks.
196 Run devtool --help for further details.
197
Patrick Williams92b42cb2022-09-03 06:53:57 -0500198When using the environment script directly in a Yocto build, it can
199be run similarly:
200
201 $ source tmp/deploy/images/qemux86-64/environment-setup-core2-64-poky-linux
202
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500203Running the setup script defines many environment variables needed in
204order to use the SDK (e.g. ``PATH``,
205:term:`CC`,
206:term:`LD`, and so forth). If you want to
207see all the environment variables the script exports, examine the
208installation file itself.
209
210Using ``devtool`` in Your SDK Workflow
211======================================
212
213The cornerstone of the extensible SDK is a command-line tool called
214``devtool``. This tool provides a number of features that help you
215build, test and package software within the extensible SDK, and
216optionally integrate it into an image built by the OpenEmbedded build
217system.
218
219.. note::
220
221 The use of
222 devtool
223 is not limited to the extensible SDK. You can use
224 devtool
225 to help you easily develop any project whose build output must be
226 part of an image built using the build system.
227
228The ``devtool`` command line is organized similarly to
Andrew Geissler09209ee2020-12-13 08:44:15 -0600229:ref:`overview-manual/development-environment:git` in that it has a number of
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500230sub-commands for each function. You can run ``devtool --help`` to see
231all the commands.
232
233.. note::
234
235 See the "
236 devtool
237  Quick Reference
238 " in the Yocto Project Reference Manual for a
239 devtool
240 quick reference.
241
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700242Three ``devtool`` subcommands provide entry-points into
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500243development:
244
245- *devtool add*: Assists in adding new software to be built.
246
247- *devtool modify*: Sets up an environment to enable you to modify
248 the source of an existing component.
249
250- *devtool upgrade*: Updates an existing recipe so that you can
251 build it for an updated set of source files.
252
253As with the build system, "recipes" represent software packages within
254``devtool``. When you use ``devtool add``, a recipe is automatically
255created. When you use ``devtool modify``, the specified existing recipe
256is used in order to determine where to get the source code and how to
257patch it. In both cases, an environment is set up so that when you build
258the recipe a source tree that is under your control is used in order to
259allow you to make changes to the source as desired. By default, new
260recipes and the source go into a "workspace" directory under the SDK.
261
262The remainder of this section presents the ``devtool add``,
263``devtool modify``, and ``devtool upgrade`` workflows.
264
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500265Use ``devtool add`` to Add an Application
266-----------------------------------------
267
268The ``devtool add`` command generates a new recipe based on existing
269source code. This command takes advantage of the
270:ref:`devtool-the-workspace-layer-structure`
271layer that many ``devtool`` commands use. The command is flexible enough
272to allow you to extract source code into both the workspace or a
273separate local Git repository and to use existing code that does not
274need to be extracted.
275
276Depending on your particular scenario, the arguments and options you use
277with ``devtool add`` form different combinations. The following diagram
278shows common development flows you would use with the ``devtool add``
279command:
280
281.. image:: figures/sdk-devtool-add-flow.png
Andrew Geisslerd5838332022-05-27 11:33:10 -0500282 :width: 100%
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500283
2841. *Generating the New Recipe*: The top part of the flow shows three
285 scenarios by which you could use ``devtool add`` to generate a recipe
286 based on existing source code.
287
288 In a shared development environment, it is typical for other
289 developers to be responsible for various areas of source code. As a
290 developer, you are probably interested in using that source code as
291 part of your development within the Yocto Project. All you need is
292 access to the code, a recipe, and a controlled area in which to do
293 your work.
294
295 Within the diagram, three possible scenarios feed into the
296 ``devtool add`` workflow:
297
298 - *Left*: The left scenario in the figure represents a common
299 situation where the source code does not exist locally and needs
300 to be extracted. In this situation, the source code is extracted
Andrew Geissler615f2f12022-07-15 14:00:58 -0500301 to the default workspace --- you do not want the files in some
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500302 specific location outside of the workspace. Thus, everything you
Andrew Geisslerc926e172021-05-07 16:11:35 -0500303 need will be located in the workspace::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500304
305 $ devtool add recipe fetchuri
306
307 With this command, ``devtool`` extracts the upstream
308 source files into a local Git repository within the ``sources``
309 folder. The command then creates a recipe named recipe and a
310 corresponding append file in the workspace. If you do not provide
311 recipe, the command makes an attempt to determine the recipe name.
312
313 - *Middle*: The middle scenario in the figure also represents a
314 situation where the source code does not exist locally. In this
315 case, the code is again upstream and needs to be extracted to some
Andrew Geissler615f2f12022-07-15 14:00:58 -0500316 local area --- this time outside of the default workspace.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500317
318 .. note::
319
320 If required,
321 devtool
322 always creates a Git repository locally during the extraction.
323
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700324 Furthermore, the first positional argument ``srctree`` in this case
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500325 identifies where the ``devtool add`` command will locate the
326 extracted code outside of the workspace. You need to specify an
Andrew Geisslerc926e172021-05-07 16:11:35 -0500327 empty directory::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500328
329 $ devtool add recipe srctree fetchuri
330
331 In summary,
332 the source code is pulled from fetchuri and extracted into the
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700333 location defined by ``srctree`` as a local Git repository.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500334
335 Within workspace, ``devtool`` creates a recipe named recipe along
336 with an associated append file.
337
338 - *Right*: The right scenario in the figure represents a situation
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700339 where the ``srctree`` has been previously prepared outside of the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500340 ``devtool`` workspace.
341
342 The following command provides a new recipe name and identifies
Andrew Geisslerc926e172021-05-07 16:11:35 -0500343 the existing source tree location::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500344
345 $ devtool add recipe srctree
346
347 The command examines the source code and creates a recipe named
348 recipe for the code and places the recipe into the workspace.
349
350 Because the extracted source code already exists, ``devtool`` does
Andrew Geissler615f2f12022-07-15 14:00:58 -0500351 not try to relocate the source code into the workspace --- only the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500352 new recipe is placed in the workspace.
353
354 Aside from a recipe folder, the command also creates an associated
355 append folder and places an initial ``*.bbappend`` file within.
356
3572. *Edit the Recipe*: You can use ``devtool edit-recipe`` to open up the
358 editor as defined by the ``$EDITOR`` environment variable and modify
Andrew Geisslerc926e172021-05-07 16:11:35 -0500359 the file::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500360
361 $ devtool edit-recipe recipe
362
363 From within the editor, you
Andrew Geissler595f6302022-01-24 19:11:47 +0000364 can make modifications to the recipe that take effect when you build
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500365 it later.
366
3673. *Build the Recipe or Rebuild the Image*: The next step you take
368 depends on what you are going to do with the new code.
369
370 If you need to eventually move the build output to the target
371 hardware, use the following ``devtool`` command:
372 :;
373
374 $ devtool build recipe
375
376 On the other hand, if you want an image to contain the recipe's
377 packages from the workspace for immediate deployment onto a device
378 (e.g. for testing purposes), you can use the ``devtool build-image``
Andrew Geisslerc926e172021-05-07 16:11:35 -0500379 command::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500380
381 $ devtool build-image image
382
3834. *Deploy the Build Output*: When you use the ``devtool build`` command
384 to build out your recipe, you probably want to see if the resulting
385 build output works as expected on the target hardware.
386
387 .. note::
388
389 This step assumes you have a previously built image that is
390 already either running in QEMU or is running on actual hardware.
391 Also, it is assumed that for deployment of the image to the
392 target, SSH is installed in the image and, if the image is running
393 on real hardware, you have network access to and from your
394 development machine.
395
396 You can deploy your build output to that target hardware by using the
397 ``devtool deploy-target`` command: $ devtool deploy-target recipe
398 target The target is a live target machine running as an SSH server.
399
400 You can, of course, also deploy the image you build to actual
401 hardware by using the ``devtool build-image`` command. However,
402 ``devtool`` does not provide a specific command that allows you to
403 deploy the image to actual hardware.
404
4055. *Finish Your Work With the Recipe*: The ``devtool finish`` command
406 creates any patches corresponding to commits in the local Git
407 repository, moves the new recipe to a more permanent layer, and then
408 resets the recipe so that the recipe is built normally rather than
409 from the workspace.
410 ::
411
412 $ devtool finish recipe layer
413
414 .. note::
415
416 Any changes you want to turn into patches must be committed to the
417 Git repository in the source tree.
418
419 As mentioned, the ``devtool finish`` command moves the final recipe
420 to its permanent layer.
421
422 As a final process of the ``devtool finish`` command, the state of
423 the standard layers and the upstream source is restored so that you
424 can build the recipe from those areas rather than the workspace.
425
426 .. note::
427
428 You can use the
429 devtool reset
430 command to put things back should you decide you do not want to
431 proceed with your work. If you do use this command, realize that
432 the source tree is preserved.
433
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500434Use ``devtool modify`` to Modify the Source of an Existing Component
435--------------------------------------------------------------------
436
437The ``devtool modify`` command prepares the way to work on existing code
438that already has a local recipe in place that is used to build the
439software. The command is flexible enough to allow you to extract code
440from an upstream source, specify the existing recipe, and keep track of
441and gather any patch files from other developers that are associated
442with the code.
443
444Depending on your particular scenario, the arguments and options you use
445with ``devtool modify`` form different combinations. The following
446diagram shows common development flows for the ``devtool modify``
447command:
448
449.. image:: figures/sdk-devtool-modify-flow.png
Andrew Geisslerd5838332022-05-27 11:33:10 -0500450 :width: 100%
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500451
4521. *Preparing to Modify the Code*: The top part of the flow shows three
453 scenarios by which you could use ``devtool modify`` to prepare to
454 work on source files. Each scenario assumes the following:
455
456 - The recipe exists locally in a layer external to the ``devtool``
457 workspace.
458
459 - The source files exist either upstream in an un-extracted state or
460 locally in a previously extracted state.
461
462 The typical situation is where another developer has created a layer
463 for use with the Yocto Project and their recipe already resides in
464 that layer. Furthermore, their source code is readily available
465 either upstream or locally.
466
467 - *Left*: The left scenario in the figure represents a common
468 situation where the source code does not exist locally and it
469 needs to be extracted from an upstream source. In this situation,
470 the source is extracted into the default ``devtool`` workspace
471 location. The recipe, in this scenario, is in its own layer
472 outside the workspace (i.e. ``meta-``\ layername).
473
474 The following command identifies the recipe and, by default,
Andrew Geisslerc926e172021-05-07 16:11:35 -0500475 extracts the source files::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500476
477 $ devtool modify recipe
478
479 Once
480 ``devtool``\ locates the recipe, ``devtool`` uses the recipe's
481 :term:`SRC_URI` statements to
482 locate the source code and any local patch files from other
483 developers.
484
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700485 With this scenario, there is no ``srctree`` argument. Consequently, the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500486 default behavior of the ``devtool modify`` command is to extract
Andrew Geissler09036742021-06-25 14:25:14 -0500487 the source files pointed to by the :term:`SRC_URI` statements into a
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500488 local Git structure. Furthermore, the location for the extracted
489 source is the default area within the ``devtool`` workspace. The
490 result is that the command sets up both the source code and an
491 append file within the workspace while the recipe remains in its
492 original location.
493
494 Additionally, if you have any non-patch local files (i.e. files
Andrew Geissler09036742021-06-25 14:25:14 -0500495 referred to with ``file://`` entries in :term:`SRC_URI` statement
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500496 excluding ``*.patch/`` or ``*.diff``), these files are copied to
497 an ``oe-local-files`` folder under the newly created source tree.
498 Copying the files here gives you a convenient area from which you
499 can modify the files. Any changes or additions you make to those
500 files are incorporated into the build the next time you build the
501 software just as are other changes you might have made to the
502 source.
503
504 - *Middle*: The middle scenario in the figure represents a situation
505 where the source code also does not exist locally. In this case,
506 the code is again upstream and needs to be extracted to some local
507 area as a Git repository. The recipe, in this scenario, is again
508 local and in its own layer outside the workspace.
509
510 The following command tells ``devtool`` the recipe with which to
511 work and, in this case, identifies a local area for the extracted
512 source files that exists outside of the default ``devtool``
Andrew Geisslerc926e172021-05-07 16:11:35 -0500513 workspace::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500514
515 $ devtool modify recipe srctree
516
517 .. note::
518
519 You cannot provide a URL for
520 srctree
521 using the
522 devtool
523 command.
524
Andrew Geissler09036742021-06-25 14:25:14 -0500525 As with all extractions, the command uses the recipe's :term:`SRC_URI`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500526 statements to locate the source files and any associated patch
527 files. Non-patch files are copied to an ``oe-local-files`` folder
528 under the newly created source tree.
529
530 Once the files are located, the command by default extracts them
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700531 into ``srctree``.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500532
533 Within workspace, ``devtool`` creates an append file for the
534 recipe. The recipe remains in its original location but the source
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700535 files are extracted to the location you provide with ``srctree``.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500536
537 - *Right*: The right scenario in the figure represents a situation
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700538 where the source tree (``srctree``) already exists locally as a
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500539 previously extracted Git structure outside of the ``devtool``
540 workspace. In this example, the recipe also exists elsewhere
541 locally in its own layer.
542
543 The following command tells ``devtool`` the recipe with which to
544 work, uses the "-n" option to indicate source does not need to be
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700545 extracted, and uses ``srctree`` to point to the previously extracted
Andrew Geisslerc926e172021-05-07 16:11:35 -0500546 source files::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500547
548 $ devtool modify -n recipe srctree
549
550 If an ``oe-local-files`` subdirectory happens to exist and it
551 contains non-patch files, the files are used. However, if the
552 subdirectory does not exist and you run the ``devtool finish``
553 command, any non-patch files that might exist next to the recipe
554 are removed because it appears to ``devtool`` that you have
555 deleted those files.
556
557 Once the ``devtool modify`` command finishes, it creates only an
558 append file for the recipe in the ``devtool`` workspace. The
559 recipe and the source code remain in their original locations.
560
5612. *Edit the Source*: Once you have used the ``devtool modify`` command,
562 you are free to make changes to the source files. You can use any
563 editor you like to make and save your source code modifications.
564
5653. *Build the Recipe or Rebuild the Image*: The next step you take
566 depends on what you are going to do with the new code.
567
568 If you need to eventually move the build output to the target
Andrew Geisslerc926e172021-05-07 16:11:35 -0500569 hardware, use the following ``devtool`` command::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500570
571 $ devtool build recipe
572
573 On the other hand, if you want an image to contain the recipe's
574 packages from the workspace for immediate deployment onto a device
575 (e.g. for testing purposes), you can use the ``devtool build-image``
576 command: $ devtool build-image image
577
5784. *Deploy the Build Output*: When you use the ``devtool build`` command
579 to build out your recipe, you probably want to see if the resulting
580 build output works as expected on target hardware.
581
582 .. note::
583
584 This step assumes you have a previously built image that is
585 already either running in QEMU or running on actual hardware.
586 Also, it is assumed that for deployment of the image to the
587 target, SSH is installed in the image and if the image is running
588 on real hardware that you have network access to and from your
589 development machine.
590
591 You can deploy your build output to that target hardware by using the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500592 ``devtool deploy-target`` command::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500593
594 $ devtool deploy-target recipe target
595
596 The target is a live target machine running as an SSH server.
597
598 You can, of course, use other methods to deploy the image you built
599 using the ``devtool build-image`` command to actual hardware.
600 ``devtool`` does not provide a specific command to deploy the image
601 to actual hardware.
602
6035. *Finish Your Work With the Recipe*: The ``devtool finish`` command
604 creates any patches corresponding to commits in the local Git
605 repository, updates the recipe to point to them (or creates a
606 ``.bbappend`` file to do so, depending on the specified destination
607 layer), and then resets the recipe so that the recipe is built
608 normally rather than from the workspace.
609 ::
610
611 $ devtool finish recipe layer
612
613 .. note::
614
615 Any changes you want to turn into patches must be staged and
616 committed within the local Git repository before you use the
617 devtool finish
618 command.
619
620 Because there is no need to move the recipe, ``devtool finish``
621 either updates the original recipe in the original layer or the
622 command creates a ``.bbappend`` file in a different layer as provided
623 by layer. Any work you did in the ``oe-local-files`` directory is
624 preserved in the original files next to the recipe during the
625 ``devtool finish`` command.
626
627 As a final process of the ``devtool finish`` command, the state of
628 the standard layers and the upstream source is restored so that you
629 can build the recipe from those areas rather than from the workspace.
630
631 .. note::
632
633 You can use the
634 devtool reset
635 command to put things back should you decide you do not want to
636 proceed with your work. If you do use this command, realize that
637 the source tree is preserved.
638
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500639Use ``devtool upgrade`` to Create a Version of the Recipe that Supports a Newer Version of the Software
640-------------------------------------------------------------------------------------------------------
641
642The ``devtool upgrade`` command upgrades an existing recipe to that of a
643more up-to-date version found upstream. Throughout the life of software,
644recipes continually undergo version upgrades by their upstream
645publishers. You can use the ``devtool upgrade`` workflow to make sure
646your recipes you are using for builds are up-to-date with their upstream
647counterparts.
648
649.. note::
650
651 Several methods exist by which you can upgrade recipes -
Andrew Geissler09036742021-06-25 14:25:14 -0500652 ``devtool upgrade``
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500653 happens to be one. You can read about all the methods by which you
Andrew Geissler09036742021-06-25 14:25:14 -0500654 can upgrade recipes in the
655 :ref:`dev-manual/common-tasks:upgrading recipes` section
656 of the Yocto Project Development Tasks Manual.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500657
658The ``devtool upgrade`` command is flexible enough to allow you to
659specify source code revision and versioning schemes, extract code into
660or out of the ``devtool``
661:ref:`devtool-the-workspace-layer-structure`,
662and work with any source file forms that the
Patrick Williams213cb262021-08-07 19:21:33 -0500663:ref:`bitbake:bitbake-user-manual/bitbake-user-manual-fetching:fetchers` support.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500664
665The following diagram shows the common development flow used with the
666``devtool upgrade`` command:
667
668.. image:: figures/sdk-devtool-upgrade-flow.png
Andrew Geisslerd5838332022-05-27 11:33:10 -0500669 :width: 100%
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500670
6711. *Initiate the Upgrade*: The top part of the flow shows the typical
672 scenario by which you use the ``devtool upgrade`` command. The
673 following conditions exist:
674
675 - The recipe exists in a local layer external to the ``devtool``
676 workspace.
677
678 - The source files for the new release exist in the same location
679 pointed to by :term:`SRC_URI`
680 in the recipe (e.g. a tarball with the new version number in the
681 name, or as a different revision in the upstream Git repository).
682
683 A common situation is where third-party software has undergone a
684 revision so that it has been upgraded. The recipe you have access to
685 is likely in your own layer. Thus, you need to upgrade the recipe to
Andrew Geisslerc926e172021-05-07 16:11:35 -0500686 use the newer version of the software::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500687
688 $ devtool upgrade -V version recipe
689
690 By default, the ``devtool upgrade`` command extracts source
691 code into the ``sources`` directory in the
692 :ref:`devtool-the-workspace-layer-structure`.
693 If you want the code extracted to any other location, you need to
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700694 provide the ``srctree`` positional argument with the command as follows::
695
696 $ devtool upgrade -V version recipe srctree
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500697
698 .. note::
699
700 In this example, the "-V" option specifies the new version. If you
701 don't use "-V", the command upgrades the recipe to the latest
702 version.
703
Andrew Geissler09036742021-06-25 14:25:14 -0500704 If the source files pointed to by the :term:`SRC_URI` statement in the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500705 recipe are in a Git repository, you must provide the "-S" option and
706 specify a revision for the software.
707
Andrew Geissler09036742021-06-25 14:25:14 -0500708 Once ``devtool`` locates the recipe, it uses the :term:`SRC_URI` variable
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500709 to locate the source code and any local patch files from other
710 developers. The result is that the command sets up the source code,
711 the new version of the recipe, and an append file all within the
712 workspace.
713
714 Additionally, if you have any non-patch local files (i.e. files
Andrew Geissler09036742021-06-25 14:25:14 -0500715 referred to with ``file://`` entries in :term:`SRC_URI` statement
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500716 excluding ``*.patch/`` or ``*.diff``), these files are copied to an
717 ``oe-local-files`` folder under the newly created source tree.
718 Copying the files here gives you a convenient area from which you can
719 modify the files. Any changes or additions you make to those files
720 are incorporated into the build the next time you build the software
721 just as are other changes you might have made to the source.
722
William A. Kennington IIIac69b482021-06-02 12:28:27 -07007232. *Resolve any Conflicts created by the Upgrade*: Conflicts could happen
724 after upgrading the software to a new version. Conflicts occur
Andrew Geissler09036742021-06-25 14:25:14 -0500725 if your recipe specifies some patch files in :term:`SRC_URI` that
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500726 conflict with changes made in the new version of the software. For
727 such cases, you need to resolve the conflicts by editing the source
728 and following the normal ``git rebase`` conflict resolution process.
729
730 Before moving onto the next step, be sure to resolve any such
731 conflicts created through use of a newer or different version of the
732 software.
733
7343. *Build the Recipe or Rebuild the Image*: The next step you take
735 depends on what you are going to do with the new code.
736
737 If you need to eventually move the build output to the target
Andrew Geisslerc926e172021-05-07 16:11:35 -0500738 hardware, use the following ``devtool`` command::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500739
740 $ devtool build recipe
741
742 On the other hand, if you want an image to contain the recipe's
743 packages from the workspace for immediate deployment onto a device
744 (e.g. for testing purposes), you can use the ``devtool build-image``
Andrew Geisslerc926e172021-05-07 16:11:35 -0500745 command::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500746
747 $ devtool build-image image
748
7494. *Deploy the Build Output*: When you use the ``devtool build`` command
750 or ``bitbake`` to build your recipe, you probably want to see if the
751 resulting build output works as expected on target hardware.
752
753 .. note::
754
755 This step assumes you have a previously built image that is
756 already either running in QEMU or running on actual hardware.
757 Also, it is assumed that for deployment of the image to the
758 target, SSH is installed in the image and if the image is running
759 on real hardware that you have network access to and from your
760 development machine.
761
762 You can deploy your build output to that target hardware by using the
763 ``devtool deploy-target`` command: $ devtool deploy-target recipe
764 target The target is a live target machine running as an SSH server.
765
766 You can, of course, also deploy the image you build using the
767 ``devtool build-image`` command to actual hardware. However,
768 ``devtool`` does not provide a specific command that allows you to do
769 this.
770
7715. *Finish Your Work With the Recipe*: The ``devtool finish`` command
772 creates any patches corresponding to commits in the local Git
773 repository, moves the new recipe to a more permanent layer, and then
774 resets the recipe so that the recipe is built normally rather than
775 from the workspace.
776
777 Any work you did in the ``oe-local-files`` directory is preserved in
778 the original files next to the recipe during the ``devtool finish``
779 command.
780
781 If you specify a destination layer that is the same as the original
782 source, then the old version of the recipe and associated files are
783 removed prior to adding the new version.
784 ::
785
786 $ devtool finish recipe layer
787
788 .. note::
789
790 Any changes you want to turn into patches must be committed to the
791 Git repository in the source tree.
792
793 As a final process of the ``devtool finish`` command, the state of
794 the standard layers and the upstream source is restored so that you
795 can build the recipe from those areas rather than the workspace.
796
797 .. note::
798
799 You can use the
800 devtool reset
801 command to put things back should you decide you do not want to
802 proceed with your work. If you do use this command, realize that
803 the source tree is preserved.
804
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500805A Closer Look at ``devtool add``
806================================
807
808The ``devtool add`` command automatically creates a recipe based on the
809source tree you provide with the command. Currently, the command has
810support for the following:
811
812- Autotools (``autoconf`` and ``automake``)
813
814- CMake
815
816- Scons
817
818- ``qmake``
819
820- Plain ``Makefile``
821
822- Out-of-tree kernel module
823
824- Binary package (i.e. "-b" option)
825
826- Node.js module
827
828- Python modules that use ``setuptools`` or ``distutils``
829
830Apart from binary packages, the determination of how a source tree
831should be treated is automatic based on the files present within that
832source tree. For example, if a ``CMakeLists.txt`` file is found, then
833the source tree is assumed to be using CMake and is treated accordingly.
834
835.. note::
836
837 In most cases, you need to edit the automatically generated recipe in
838 order to make it build properly. Typically, you would go through
839 several edit and build cycles until the recipe successfully builds.
840 Once the recipe builds, you could use possible further iterations to
841 test the recipe on the target device.
842
843The remainder of this section covers specifics regarding how parts of
844the recipe are generated.
845
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500846Name and Version
847----------------
848
849If you do not specify a name and version on the command line,
850``devtool add`` uses various metadata within the source tree in an
851attempt to determine the name and version of the software being built.
852Based on what the tool determines, ``devtool`` sets the name of the
853created recipe file accordingly.
854
855If ``devtool`` cannot determine the name and version, the command prints
856an error. For such cases, you must re-run the command and provide the
857name and version, just the name, or just the version as part of the
858command line.
859
860Sometimes the name or version determined from the source tree might be
Andrew Geisslerc926e172021-05-07 16:11:35 -0500861incorrect. For such a case, you must reset the recipe::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500862
863 $ devtool reset -n recipename
864
865After running the ``devtool reset`` command, you need to
866run ``devtool add`` again and provide the name or the version.
867
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500868Dependency Detection and Mapping
869--------------------------------
870
871The ``devtool add`` command attempts to detect build-time dependencies
872and map them to other recipes in the system. During this mapping, the
873command fills in the names of those recipes as part of the
874:term:`DEPENDS` variable within the
875recipe. If a dependency cannot be mapped, ``devtool`` places a comment
876in the recipe indicating such. The inability to map a dependency can
877result from naming not being recognized or because the dependency simply
878is not available. For cases where the dependency is not available, you
879must use the ``devtool add`` command to add an additional recipe that
880satisfies the dependency. Once you add that recipe, you need to update
Andrew Geissler09036742021-06-25 14:25:14 -0500881the :term:`DEPENDS` variable in the original recipe to include the new
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500882recipe.
883
884If you need to add runtime dependencies, you can do so by adding the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500885following to your recipe::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500886
Patrick Williams0ca19cc2021-08-16 14:03:13 -0500887 RDEPENDS:${PN} += "dependency1 dependency2 ..."
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500888
889.. note::
890
891 The
892 devtool add
893 command often cannot distinguish between mandatory and optional
894 dependencies. Consequently, some of the detected dependencies might
895 in fact be optional. When in doubt, consult the documentation or the
896 configure script for the software the recipe is building for further
897 details. In some cases, you might find you can substitute the
898 dependency with an option that disables the associated functionality
899 passed to the configure script.
900
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500901License Detection
902-----------------
903
904The ``devtool add`` command attempts to determine if the software you
905are adding is able to be distributed under a common, open-source
906license. If so, the command sets the
907:term:`LICENSE` value accordingly.
908You should double-check the value added by the command against the
909documentation or source files for the software you are building and, if
Andrew Geissler09036742021-06-25 14:25:14 -0500910necessary, update that :term:`LICENSE` value.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500911
912The ``devtool add`` command also sets the
913:term:`LIC_FILES_CHKSUM`
914value to point to all files that appear to be license-related. Realize
915that license statements often appear in comments at the top of source
916files or within the documentation. In such cases, the command does not
917recognize those license statements. Consequently, you might need to
Andrew Geissler09036742021-06-25 14:25:14 -0500918amend the :term:`LIC_FILES_CHKSUM` variable to point to one or more of those
919comments if present. Setting :term:`LIC_FILES_CHKSUM` is particularly
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500920important for third-party software. The mechanism attempts to ensure
921correct licensing should you upgrade the recipe to a newer upstream
922version in future. Any change in licensing is detected and you receive
923an error prompting you to check the license text again.
924
925If the ``devtool add`` command cannot determine licensing information,
Andrew Geissler09036742021-06-25 14:25:14 -0500926``devtool`` sets the :term:`LICENSE` value to "CLOSED" and leaves the
927:term:`LIC_FILES_CHKSUM` value unset. This behavior allows you to continue
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500928with development even though the settings are unlikely to be correct in
929all cases. You should check the documentation or source files for the
930software you are building to determine the actual license.
931
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500932Adding Makefile-Only Software
933-----------------------------
934
935The use of Make by itself is very common in both proprietary and
936open-source software. Unfortunately, Makefiles are often not written
937with cross-compilation in mind. Thus, ``devtool add`` often cannot do
938very much to ensure that these Makefiles build correctly. It is very
939common, for example, to explicitly call ``gcc`` instead of using the
940:term:`CC` variable. Usually, in a
941cross-compilation environment, ``gcc`` is the compiler for the build
942host and the cross-compiler is named something similar to
943``arm-poky-linux-gnueabi-gcc`` and might require arguments (e.g. to
944point to the associated sysroot for the target machine).
945
946When writing a recipe for Makefile-only software, keep the following in
947mind:
948
949- You probably need to patch the Makefile to use variables instead of
950 hardcoding tools within the toolchain such as ``gcc`` and ``g++``.
951
952- The environment in which Make runs is set up with various standard
Andrew Geissler09036742021-06-25 14:25:14 -0500953 variables for compilation (e.g. :term:`CC`, :term:`CXX`, and so forth) in a
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500954 similar manner to the environment set up by the SDK's environment
955 setup script. One easy way to see these variables is to run the
956 ``devtool build`` command on the recipe and then look in
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700957 ``oe-logs/run.do_compile``. Towards the top of this file, there is
958 a list of environment variables that are set. You can take
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500959 advantage of these variables within the Makefile.
960
961- If the Makefile sets a default for a variable using "=", that default
962 overrides the value set in the environment, which is usually not
963 desirable. For this case, you can either patch the Makefile so it
964 sets the default using the "?=" operator, or you can alternatively
965 force the value on the ``make`` command line. To force the value on
966 the command line, add the variable setting to
967 :term:`EXTRA_OEMAKE` or
968 :term:`PACKAGECONFIG_CONFARGS`
Andrew Geissler09036742021-06-25 14:25:14 -0500969 within the recipe. Here is an example using :term:`EXTRA_OEMAKE`::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500970
971 EXTRA_OEMAKE += "'CC=${CC}' 'CXX=${CXX}'"
972
973 In the above example,
974 single quotes are used around the variable settings as the values are
975 likely to contain spaces because required default options are passed
976 to the compiler.
977
978- Hardcoding paths inside Makefiles is often problematic in a
979 cross-compilation environment. This is particularly true because
980 those hardcoded paths often point to locations on the build host and
981 thus will either be read-only or will introduce contamination into
982 the cross-compilation because they are specific to the build host
983 rather than the target. Patching the Makefile to use prefix variables
984 or other path variables is usually the way to handle this situation.
985
986- Sometimes a Makefile runs target-specific commands such as
987 ``ldconfig``. For such cases, you might be able to apply patches that
988 remove these commands from the Makefile.
989
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500990Adding Native Tools
991-------------------
992
993Often, you need to build additional tools that run on the :term:`Build
994Host` as opposed to
995the target. You should indicate this requirement by using one of the
996following methods when you run ``devtool add``:
997
998- Specify the name of the recipe such that it ends with "-native".
999 Specifying the name like this produces a recipe that only builds for
1000 the build host.
1001
William A. Kennington IIIac69b482021-06-02 12:28:27 -07001002- Specify the "--also-native" option with the ``devtool add``
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001003 command. Specifying this option creates a recipe file that still
1004 builds for the target but also creates a variant with a "-native"
1005 suffix that builds for the build host.
1006
1007.. note::
1008
1009 If you need to add a tool that is shipped as part of a source tree
1010 that builds code for the target, you can typically accomplish this by
1011 building the native and target parts separately rather than within
1012 the same compilation process. Realize though that with the
William A. Kennington IIIac69b482021-06-02 12:28:27 -07001013 "--also-native" option, you can add the tool using just one
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001014 recipe file.
1015
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001016Adding Node.js Modules
1017----------------------
1018
1019You can use the ``devtool add`` command two different ways to add
1020Node.js modules: 1) Through ``npm`` and, 2) from a repository or local
1021source.
1022
Andrew Geisslerc926e172021-05-07 16:11:35 -05001023Use the following form to add Node.js modules through ``npm``::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001024
1025 $ devtool add "npm://registry.npmjs.org;name=forever;version=0.15.1"
1026
1027The name and
1028version parameters are mandatory. Lockdown and shrinkwrap files are
1029generated and pointed to by the recipe in order to freeze the version
1030that is fetched for the dependencies according to the first time. This
1031also saves checksums that are verified on future fetches. Together,
1032these behaviors ensure the reproducibility and integrity of the build.
1033
1034.. note::
1035
1036 - You must use quotes around the URL. The ``devtool add`` does not
1037 require the quotes, but the shell considers ";" as a splitter
1038 between multiple commands. Thus, without the quotes,
1039 ``devtool add`` does not receive the other parts, which results in
1040 several "command not found" errors.
1041
1042 - In order to support adding Node.js modules, a ``nodejs`` recipe
1043 must be part of your SDK.
1044
1045As mentioned earlier, you can also add Node.js modules directly from a
1046repository or local source tree. To add modules this way, use
Andrew Geisslerc926e172021-05-07 16:11:35 -05001047``devtool add`` in the following form::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001048
1049 $ devtool add https://github.com/diversario/node-ssdp
1050
1051In this example, ``devtool``
1052fetches the specified Git repository, detects the code as Node.js code,
1053fetches dependencies using ``npm``, and sets
1054:term:`SRC_URI` accordingly.
1055
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001056Working With Recipes
1057====================
1058
1059When building a recipe using the ``devtool build`` command, the typical
1060build progresses as follows:
1061
10621. Fetch the source
1063
10642. Unpack the source
1065
10663. Configure the source
1067
10684. Compile the source
1069
10705. Install the build output
1071
10726. Package the installed output
1073
1074For recipes in the workspace, fetching and unpacking is disabled as the
1075source tree has already been prepared and is persistent. Each of these
1076build steps is defined as a function (task), usually with a "do\_" prefix
1077(e.g. :ref:`ref-tasks-fetch`,
1078:ref:`ref-tasks-unpack`, and so
1079forth). These functions are typically shell scripts but can instead be
1080written in Python.
1081
1082If you look at the contents of a recipe, you will see that the recipe
1083does not include complete instructions for building the software.
1084Instead, common functionality is encapsulated in classes inherited with
1085the ``inherit`` directive. This technique leaves the recipe to describe
William A. Kennington IIIac69b482021-06-02 12:28:27 -07001086just the things that are specific to the software being built. There is
1087a :ref:`base <ref-classes-base>` class that
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001088is implicitly inherited by all recipes and provides the functionality
1089that most recipes typically need.
1090
1091The remainder of this section presents information useful when working
1092with recipes.
1093
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001094Finding Logs and Work Files
1095---------------------------
1096
1097After the first run of the ``devtool build`` command, recipes that were
1098previously created using the ``devtool add`` command or whose sources
1099were modified using the ``devtool modify`` command contain symbolic
1100links created within the source tree:
1101
1102- ``oe-logs``: This link points to the directory in which log files and
1103 run scripts for each build step are created.
1104
1105- ``oe-workdir``: This link points to the temporary work area for the
1106 recipe. The following locations under ``oe-workdir`` are particularly
1107 useful:
1108
1109 - ``image/``: Contains all of the files installed during the
1110 :ref:`ref-tasks-install` stage.
1111 Within a recipe, this directory is referred to by the expression
1112 ``${``\ :term:`D`\ ``}``.
1113
1114 - ``sysroot-destdir/``: Contains a subset of files installed within
Patrick Williams2194f502022-10-16 14:26:09 -05001115 :ref:`ref-tasks-install` that have been put into the shared sysroot. For
Andrew Geissler3b8a17c2021-04-15 15:55:55 -05001116 more information, see the
1117 ":ref:`dev-manual/common-tasks:sharing files between recipes`" section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001118
1119 - ``packages-split/``: Contains subdirectories for each package
1120 produced by the recipe. For more information, see the
Andrew Geissler3b8a17c2021-04-15 15:55:55 -05001121 ":ref:`sdk-manual/extensible:packaging`" section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001122
1123You can use these links to get more information on what is happening at
1124each build step.
1125
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001126Setting Configure Arguments
1127---------------------------
1128
1129If the software your recipe is building uses GNU autoconf, then a fixed
1130set of arguments is passed to it to enable cross-compilation plus any
1131extras specified by
1132:term:`EXTRA_OECONF` or
1133:term:`PACKAGECONFIG_CONFARGS`
1134set within the recipe. If you wish to pass additional options, add them
Andrew Geissler09036742021-06-25 14:25:14 -05001135to :term:`EXTRA_OECONF` or :term:`PACKAGECONFIG_CONFARGS`. Other supported build
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001136tools have similar variables (e.g.
1137:term:`EXTRA_OECMAKE` for
1138CMake, :term:`EXTRA_OESCONS`
1139for Scons, and so forth). If you need to pass anything on the ``make``
Andrew Geissler09036742021-06-25 14:25:14 -05001140command line, you can use :term:`EXTRA_OEMAKE` or the
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001141:term:`PACKAGECONFIG_CONFARGS`
1142variables to do so.
1143
1144You can use the ``devtool configure-help`` command to help you set the
1145arguments listed in the previous paragraph. The command determines the
1146exact options being passed, and shows them to you along with any custom
Andrew Geissler09036742021-06-25 14:25:14 -05001147arguments specified through :term:`EXTRA_OECONF` or
1148:term:`PACKAGECONFIG_CONFARGS`. If applicable, the command also shows you
William A. Kennington IIIac69b482021-06-02 12:28:27 -07001149the output of the configure script's "--help" option as a
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001150reference.
1151
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001152Sharing Files Between Recipes
1153-----------------------------
1154
1155Recipes often need to use files provided by other recipes on the
1156:term:`Build Host`. For example,
1157an application linking to a common library needs access to the library
1158itself and its associated headers. The way this access is accomplished
William A. Kennington IIIac69b482021-06-02 12:28:27 -07001159within the extensible SDK is through the sysroot. There is one sysroot per
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001160"machine" for which the SDK is being built. In practical terms, this
William A. Kennington IIIac69b482021-06-02 12:28:27 -07001161means there is a sysroot for the target machine, and a sysroot for
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001162the build host.
1163
1164Recipes should never write files directly into the sysroot. Instead,
1165files should be installed into standard locations during the
1166:ref:`ref-tasks-install` task within
1167the ``${``\ :term:`D`\ ``}`` directory. A
1168subset of these files automatically goes into the sysroot. The reason
1169for this limitation is that almost all files that go into the sysroot
1170are cataloged in manifests in order to ensure they can be removed later
1171when a recipe is modified or removed. Thus, the sysroot is able to
1172remain free from stale files.
1173
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001174Packaging
1175---------
1176
1177Packaging is not always particularly relevant within the extensible SDK.
1178However, if you examine how build output gets into the final image on
1179the target device, it is important to understand packaging because the
1180contents of the image are expressed in terms of packages and not
1181recipes.
1182
1183During the :ref:`ref-tasks-package`
1184task, files installed during the
1185:ref:`ref-tasks-install` task are
1186split into one main package, which is almost always named the same as
1187the recipe, and into several other packages. This separation exists
1188because not all of those installed files are useful in every image. For
1189example, you probably do not need any of the documentation installed in
1190a production image. Consequently, for each recipe the documentation
1191files are separated into a ``-doc`` package. Recipes that package
1192software containing optional modules or plugins might undergo additional
1193package splitting as well.
1194
1195After building a recipe, you can see where files have gone by looking in
1196the ``oe-workdir/packages-split`` directory, which contains a
1197subdirectory for each package. Apart from some advanced cases, the
1198:term:`PACKAGES` and
1199:term:`FILES` variables controls
Andrew Geissler09036742021-06-25 14:25:14 -05001200splitting. The :term:`PACKAGES` variable lists all of the packages to be
1201produced, while the :term:`FILES` variable specifies which files to include
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001202in each package by using an override to specify the package. For
Patrick Williams0ca19cc2021-08-16 14:03:13 -05001203example, ``FILES:${PN}`` specifies the files to go into the main package
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001204(i.e. the main package has the same name as the recipe and
1205``${``\ :term:`PN`\ ``}`` evaluates to the
Andrew Geissler09036742021-06-25 14:25:14 -05001206recipe name). The order of the :term:`PACKAGES` value is significant. For
1207each installed file, the first package whose :term:`FILES` value matches the
1208file is the package into which the file goes. Both the :term:`PACKAGES` and
1209:term:`FILES` variables have default values. Consequently, you might find
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001210you do not even need to set these variables in your recipe unless the
1211software the recipe is building installs files into non-standard
1212locations.
1213
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001214Restoring the Target Device to its Original State
1215=================================================
1216
1217If you use the ``devtool deploy-target`` command to write a recipe's
1218build output to the target, and you are working on an existing component
1219of the system, then you might find yourself in a situation where you
1220need to restore the original files that existed prior to running the
1221``devtool deploy-target`` command. Because the ``devtool deploy-target``
1222command backs up any files it overwrites, you can use the
1223``devtool undeploy-target`` command to restore those files and remove
Andrew Geisslerc926e172021-05-07 16:11:35 -05001224any other files the recipe deployed. Consider the following example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001225
1226 $ devtool undeploy-target lighttpd root@192.168.7.2
1227
1228If you have deployed
1229multiple applications, you can remove them all using the "-a" option
Andrew Geisslerc926e172021-05-07 16:11:35 -05001230thus restoring the target device to its original state::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001231
1232 $ devtool undeploy-target -a root@192.168.7.2
1233
1234Information about files deployed to
1235the target as well as any backed up files are stored on the target
1236itself. This storage, of course, requires some additional space on the
1237target machine.
1238
1239.. note::
1240
1241 The
1242 devtool deploy-target
1243 and
1244 devtool undeploy-target
1245 commands do not currently interact with any package management system
1246 on the target device (e.g. RPM or OPKG). Consequently, you should not
1247 intermingle
1248 devtool deploy-target
1249 and package manager operations on the target device. Doing so could
1250 result in a conflicting set of files.
1251
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001252Installing Additional Items Into the Extensible SDK
1253===================================================
1254
1255Out of the box the extensible SDK typically only comes with a small
1256number of tools and libraries. A minimal SDK starts mostly empty and is
1257populated on-demand. Sometimes you must explicitly install extra items
1258into the SDK. If you need these extra items, you can first search for
1259the items using the ``devtool search`` command. For example, suppose you
1260need to link to libGL but you are not sure which recipe provides libGL.
Andrew Geisslerc926e172021-05-07 16:11:35 -05001261You can use the following command to find out::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001262
1263 $ devtool search libGL mesa
Patrick Williams92b42cb2022-09-03 06:53:57 -05001264 A free implementation of the OpenGL API
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001265
Patrick Williams92b42cb2022-09-03 06:53:57 -05001266Once you know the recipe
1267(i.e. ``mesa`` in this example), you can install it.
1268
1269When using the extensible SDK directly in a Yocto build
1270-------------------------------------------------------
1271
1272In this scenario, the Yocto build tooling, e.g. ``bitbake``
1273is directly accessible to build additional items, and it
1274can simply be executed directly:
1275
1276 $ bitbake mesa
1277 $ bitbake populate-sysroots
1278
1279When using a standalone installer for the Extensible SDK
1280--------------------------------------------------------
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001281
1282 $ devtool sdk-install mesa
1283
1284By default, the ``devtool sdk-install`` command assumes
1285the item is available in pre-built form from your SDK provider. If the
1286item is not available and it is acceptable to build the item from
Andrew Geisslerc926e172021-05-07 16:11:35 -05001287source, you can add the "-s" option as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001288
1289 $ devtool sdk-install -s mesa
1290
1291It is important to remember that building the item from source
1292takes significantly longer than installing the pre-built artifact. Also,
William A. Kennington IIIac69b482021-06-02 12:28:27 -07001293if there is no recipe for the item you want to add to the SDK, you must
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001294instead add the item using the ``devtool add`` command.
1295
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001296Applying Updates to an Installed Extensible SDK
1297===============================================
1298
1299If you are working with an installed extensible SDK that gets
1300occasionally updated (e.g. a third-party SDK), then you will need to
1301manually "pull down" the updates into the installed SDK.
1302
Andrew Geisslerc926e172021-05-07 16:11:35 -05001303To update your installed SDK, use ``devtool`` as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001304
1305 $ devtool sdk-update
1306
1307The previous command assumes your SDK provider has set the
Andrew Geissler3b8a17c2021-04-15 15:55:55 -05001308default update URL for you through the :term:`SDK_UPDATE_URL`
1309variable as described in the
1310":ref:`sdk-manual/appendix-customizing:Providing Updates to the Extensible SDK After Installation`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001311section. If the SDK provider has not set that default URL, you need to
1312specify it yourself in the command as follows: $ devtool sdk-update
1313path_to_update_directory
1314
1315.. note::
1316
1317 The URL needs to point specifically to a published SDK and not to an
1318 SDK installer that you would download and install.
1319
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001320Creating a Derivative SDK With Additional Components
1321====================================================
1322
1323You might need to produce an SDK that contains your own custom
1324libraries. A good example would be if you were a vendor with customers
1325that use your SDK to build their own platform-specific software and
1326those customers need an SDK that has custom libraries. In such a case,
1327you can produce a derivative SDK based on the currently installed SDK
1328fairly easily by following these steps:
1329
13301. If necessary, install an extensible SDK that you want to use as a
1331 base for your derivative SDK.
1332
13332. Source the environment script for the SDK.
1334
13353. Add the extra libraries or other components you want by using the
1336 ``devtool add`` command.
1337
13384. Run the ``devtool build-sdk`` command.
1339
1340The previous steps take the recipes added to the workspace and construct
1341a new SDK installer that contains those recipes and the resulting binary
1342artifacts. The recipes go into their own separate layer in the
1343constructed derivative SDK, which leaves the workspace clean and ready
1344for users to add their own recipes.