blob: 8ef44e39acc0ce9e4287cc3a46ef8f300d5f7d02 [file] [log] [blame]
Andrew Geisslerf0343792020-11-18 10:42:21 -06001.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Andrew Geisslerc9f78652020-09-18 14:11:35 -05002
3************************
4Using the Extensible SDK
5************************
6
7This chapter describes the extensible SDK and how to install it.
8Information covers the pieces of the SDK, how to install it, and
9presents a look at using the ``devtool`` functionality. The extensible
10SDK makes it easy to add new applications and libraries to an image,
11modify the source for an existing component, test changes on the target
12hardware, and ease integration into the rest of the
13:term:`OpenEmbedded Build System`.
14
15.. note::
16
17 For a side-by-side comparison of main features supported for an
Andrew Geissler09036742021-06-25 14:25:14 -050018 extensible SDK as compared to a standard SDK, see the
19 :ref:`sdk-manual/intro:introduction` section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -050020
21In addition to the functionality available through ``devtool``, you can
22alternatively make use of the toolchain directly, for example from
Andrew Geissler3b8a17c2021-04-15 15:55:55 -050023Makefile and Autotools. See the
24":ref:`sdk-manual/working-projects:using the sdk toolchain directly`" chapter
25for more information.
Andrew Geisslerc9f78652020-09-18 14:11:35 -050026
Andrew Geisslerc9f78652020-09-18 14:11:35 -050027Why use the Extensible SDK and What is in It?
28=============================================
29
30The extensible SDK provides a cross-development toolchain and libraries
31tailored to the contents of a specific image. You would use the
32Extensible SDK if you want a toolchain experience supplemented with the
33powerful set of ``devtool`` commands tailored for the Yocto Project
34environment.
35
36The installed extensible SDK consists of several files and directories.
37Basically, it contains an SDK environment setup script, some
38configuration files, an internal build system, and the ``devtool``
39functionality.
40
Andrew Geisslerc9f78652020-09-18 14:11:35 -050041Installing the Extensible SDK
42=============================
43
44The first thing you need to do is install the SDK on your :term:`Build
45Host` by running the ``*.sh`` installation script.
46
47You can download a tarball installer, which includes the pre-built
48toolchain, the ``runqemu`` script, the internal build system,
49``devtool``, and support files from the appropriate
Andrew Geissler09209ee2020-12-13 08:44:15 -060050:yocto_dl:`toolchain </releases/yocto/yocto-&DISTRO;/toolchain/>` directory within the Index of
Andrew Geisslerc9f78652020-09-18 14:11:35 -050051Releases. Toolchains are available for several 32-bit and 64-bit
52architectures with the ``x86_64`` directories, respectively. The
53toolchains the Yocto Project provides are based off the
54``core-image-sato`` and ``core-image-minimal`` images and contain
55libraries appropriate for developing against that image.
56
57The names of the tarball installer scripts are such that a string
58representing the host system appears first in the filename and then is
59immediately followed by a string representing the target architecture.
60An extensible SDK has the string "-ext" as part of the name. Following
Andrew Geisslerc926e172021-05-07 16:11:35 -050061is the general form::
Andrew Geisslerc9f78652020-09-18 14:11:35 -050062
63 poky-glibc-host_system-image_type-arch-toolchain-ext-release_version.sh
64
65 Where:
66 host_system is a string representing your development system:
67
68 i686 or x86_64.
69
70 image_type is the image for which the SDK was built:
71
72 core-image-sato or core-image-minimal
73
74 arch is a string representing the tuned target architecture:
75
76 aarch64, armv5e, core2-64, i586, mips32r2, mips64, ppc7400, or cortexa8hf-neon
77
78 release_version is a string representing the release number of the Yocto Project:
79
Andrew Geissler09209ee2020-12-13 08:44:15 -060080 &DISTRO;, &DISTRO;+snapshot
Andrew Geisslerc9f78652020-09-18 14:11:35 -050081
82For example, the following SDK installer is for a 64-bit
83development host system and a i586-tuned target architecture based off
Andrew Geisslerc926e172021-05-07 16:11:35 -050084the SDK for ``core-image-sato`` and using the current &DISTRO; snapshot::
Andrew Geisslerc9f78652020-09-18 14:11:35 -050085
Andrew Geisslerd1e89492021-02-12 15:35:20 -060086 poky-glibc-x86_64-core-image-sato-i586-toolchain-ext-&DISTRO;.sh
Andrew Geisslerc9f78652020-09-18 14:11:35 -050087
88.. note::
89
90 As an alternative to downloading an SDK, you can build the SDK
Andrew Geissler09036742021-06-25 14:25:14 -050091 installer. For information on building the installer, see the
92 :ref:`sdk-manual/appendix-obtain:building an sdk installer`
93 section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -050094
95The SDK and toolchains are self-contained and by default are installed
96into the ``poky_sdk`` folder in your home directory. You can choose to
97install the extensible SDK in any location when you run the installer.
98However, because files need to be written under that directory during
99the normal course of operation, the location you choose for installation
100must be writable for whichever users need to use the SDK.
101
102The following command shows how to run the installer given a toolchain
103tarball for a 64-bit x86 development host system and a 64-bit x86 target
104architecture. The example assumes the SDK installer is located in
105``~/Downloads/`` and has execution rights.
106
107.. note::
108
109 If you do not have write permissions for the directory into which you
110 are installing the SDK, the installer notifies you and exits. For
111 that case, set up the proper permissions in the directory and run the
112 installer again.
113
114::
115
116 $ ./Downloads/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.5.sh
117 Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.5
118 ==========================================================================
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500119 Enter target directory for SDK (default: poky_sdk):
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500120 You are about to install the SDK to "/home/scottrif/poky_sdk". Proceed [Y/n]? Y
121 Extracting SDK..............done
122 Setting it up...
123 Extracting buildtools...
124 Preparing build system...
125 Parsing recipes: 100% |##################################################################| Time: 0:00:52
126 Initialising tasks: 100% |###############################################################| Time: 0:00:00
127 Checking sstate mirror object availability: 100% |#######################################| Time: 0:00:00
128 Loading cache: 100% |####################################################################| Time: 0:00:00
129 Initialising tasks: 100% |###############################################################| Time: 0:00:00
130 done
131 SDK has been successfully set up and is ready to be used.
132 Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
133 $ . /home/scottrif/poky_sdk/environment-setup-core2-64-poky-linux
134
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500135Running the Extensible SDK Environment Setup Script
136===================================================
137
138Once you have the SDK installed, you must run the SDK environment setup
139script before you can actually use the SDK. This setup script resides in
140the directory you chose when you installed the SDK, which is either the
141default ``poky_sdk`` directory or the directory you chose during
142installation.
143
144Before running the script, be sure it is the one that matches the
145architecture for which you are developing. Environment setup scripts
146begin with the string "``environment-setup``" and include as part of
147their name the tuned target architecture. As an example, the following
148commands set the working directory to where the SDK was installed and
149then source the environment setup script. In this example, the setup
Andrew Geisslerc926e172021-05-07 16:11:35 -0500150script is for an IA-based target machine using i586 tuning::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500151
152 $ cd /home/scottrif/poky_sdk
153 $ source environment-setup-core2-64-poky-linux
154 SDK environment now set up; additionally you may now run devtool to perform development tasks.
155 Run devtool --help for further details.
156
157Running the setup script defines many environment variables needed in
158order to use the SDK (e.g. ``PATH``,
159:term:`CC`,
160:term:`LD`, and so forth). If you want to
161see all the environment variables the script exports, examine the
162installation file itself.
163
164Using ``devtool`` in Your SDK Workflow
165======================================
166
167The cornerstone of the extensible SDK is a command-line tool called
168``devtool``. This tool provides a number of features that help you
169build, test and package software within the extensible SDK, and
170optionally integrate it into an image built by the OpenEmbedded build
171system.
172
173.. note::
174
175 The use of
176 devtool
177 is not limited to the extensible SDK. You can use
178 devtool
179 to help you easily develop any project whose build output must be
180 part of an image built using the build system.
181
182The ``devtool`` command line is organized similarly to
Andrew Geissler09209ee2020-12-13 08:44:15 -0600183:ref:`overview-manual/development-environment:git` in that it has a number of
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500184sub-commands for each function. You can run ``devtool --help`` to see
185all the commands.
186
187.. note::
188
189 See the "
190 devtool
191  Quick Reference
192 " in the Yocto Project Reference Manual for a
193 devtool
194 quick reference.
195
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700196Three ``devtool`` subcommands provide entry-points into
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500197development:
198
199- *devtool add*: Assists in adding new software to be built.
200
201- *devtool modify*: Sets up an environment to enable you to modify
202 the source of an existing component.
203
204- *devtool upgrade*: Updates an existing recipe so that you can
205 build it for an updated set of source files.
206
207As with the build system, "recipes" represent software packages within
208``devtool``. When you use ``devtool add``, a recipe is automatically
209created. When you use ``devtool modify``, the specified existing recipe
210is used in order to determine where to get the source code and how to
211patch it. In both cases, an environment is set up so that when you build
212the recipe a source tree that is under your control is used in order to
213allow you to make changes to the source as desired. By default, new
214recipes and the source go into a "workspace" directory under the SDK.
215
216The remainder of this section presents the ``devtool add``,
217``devtool modify``, and ``devtool upgrade`` workflows.
218
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500219Use ``devtool add`` to Add an Application
220-----------------------------------------
221
222The ``devtool add`` command generates a new recipe based on existing
223source code. This command takes advantage of the
224:ref:`devtool-the-workspace-layer-structure`
225layer that many ``devtool`` commands use. The command is flexible enough
226to allow you to extract source code into both the workspace or a
227separate local Git repository and to use existing code that does not
228need to be extracted.
229
230Depending on your particular scenario, the arguments and options you use
231with ``devtool add`` form different combinations. The following diagram
232shows common development flows you would use with the ``devtool add``
233command:
234
235.. image:: figures/sdk-devtool-add-flow.png
236 :align: center
237
2381. *Generating the New Recipe*: The top part of the flow shows three
239 scenarios by which you could use ``devtool add`` to generate a recipe
240 based on existing source code.
241
242 In a shared development environment, it is typical for other
243 developers to be responsible for various areas of source code. As a
244 developer, you are probably interested in using that source code as
245 part of your development within the Yocto Project. All you need is
246 access to the code, a recipe, and a controlled area in which to do
247 your work.
248
249 Within the diagram, three possible scenarios feed into the
250 ``devtool add`` workflow:
251
252 - *Left*: The left scenario in the figure represents a common
253 situation where the source code does not exist locally and needs
254 to be extracted. In this situation, the source code is extracted
255 to the default workspace - you do not want the files in some
256 specific location outside of the workspace. Thus, everything you
Andrew Geisslerc926e172021-05-07 16:11:35 -0500257 need will be located in the workspace::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500258
259 $ devtool add recipe fetchuri
260
261 With this command, ``devtool`` extracts the upstream
262 source files into a local Git repository within the ``sources``
263 folder. The command then creates a recipe named recipe and a
264 corresponding append file in the workspace. If you do not provide
265 recipe, the command makes an attempt to determine the recipe name.
266
267 - *Middle*: The middle scenario in the figure also represents a
268 situation where the source code does not exist locally. In this
269 case, the code is again upstream and needs to be extracted to some
270 local area - this time outside of the default workspace.
271
272 .. note::
273
274 If required,
275 devtool
276 always creates a Git repository locally during the extraction.
277
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700278 Furthermore, the first positional argument ``srctree`` in this case
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500279 identifies where the ``devtool add`` command will locate the
280 extracted code outside of the workspace. You need to specify an
Andrew Geisslerc926e172021-05-07 16:11:35 -0500281 empty directory::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500282
283 $ devtool add recipe srctree fetchuri
284
285 In summary,
286 the source code is pulled from fetchuri and extracted into the
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700287 location defined by ``srctree`` as a local Git repository.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500288
289 Within workspace, ``devtool`` creates a recipe named recipe along
290 with an associated append file.
291
292 - *Right*: The right scenario in the figure represents a situation
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700293 where the ``srctree`` has been previously prepared outside of the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500294 ``devtool`` workspace.
295
296 The following command provides a new recipe name and identifies
Andrew Geisslerc926e172021-05-07 16:11:35 -0500297 the existing source tree location::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500298
299 $ devtool add recipe srctree
300
301 The command examines the source code and creates a recipe named
302 recipe for the code and places the recipe into the workspace.
303
304 Because the extracted source code already exists, ``devtool`` does
305 not try to relocate the source code into the workspace - only the
306 new recipe is placed in the workspace.
307
308 Aside from a recipe folder, the command also creates an associated
309 append folder and places an initial ``*.bbappend`` file within.
310
3112. *Edit the Recipe*: You can use ``devtool edit-recipe`` to open up the
312 editor as defined by the ``$EDITOR`` environment variable and modify
Andrew Geisslerc926e172021-05-07 16:11:35 -0500313 the file::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500314
315 $ devtool edit-recipe recipe
316
317 From within the editor, you
318 can make modifications to the recipe that take affect when you build
319 it later.
320
3213. *Build the Recipe or Rebuild the Image*: The next step you take
322 depends on what you are going to do with the new code.
323
324 If you need to eventually move the build output to the target
325 hardware, use the following ``devtool`` command:
326 :;
327
328 $ devtool build recipe
329
330 On the other hand, if you want an image to contain the recipe's
331 packages from the workspace for immediate deployment onto a device
332 (e.g. for testing purposes), you can use the ``devtool build-image``
Andrew Geisslerc926e172021-05-07 16:11:35 -0500333 command::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500334
335 $ devtool build-image image
336
3374. *Deploy the Build Output*: When you use the ``devtool build`` command
338 to build out your recipe, you probably want to see if the resulting
339 build output works as expected on the target hardware.
340
341 .. note::
342
343 This step assumes you have a previously built image that is
344 already either running in QEMU or is running on actual hardware.
345 Also, it is assumed that for deployment of the image to the
346 target, SSH is installed in the image and, if the image is running
347 on real hardware, you have network access to and from your
348 development machine.
349
350 You can deploy your build output to that target hardware by using the
351 ``devtool deploy-target`` command: $ devtool deploy-target recipe
352 target The target is a live target machine running as an SSH server.
353
354 You can, of course, also deploy the image you build to actual
355 hardware by using the ``devtool build-image`` command. However,
356 ``devtool`` does not provide a specific command that allows you to
357 deploy the image to actual hardware.
358
3595. *Finish Your Work With the Recipe*: The ``devtool finish`` command
360 creates any patches corresponding to commits in the local Git
361 repository, moves the new recipe to a more permanent layer, and then
362 resets the recipe so that the recipe is built normally rather than
363 from the workspace.
364 ::
365
366 $ devtool finish recipe layer
367
368 .. note::
369
370 Any changes you want to turn into patches must be committed to the
371 Git repository in the source tree.
372
373 As mentioned, the ``devtool finish`` command moves the final recipe
374 to its permanent layer.
375
376 As a final process of the ``devtool finish`` command, the state of
377 the standard layers and the upstream source is restored so that you
378 can build the recipe from those areas rather than the workspace.
379
380 .. note::
381
382 You can use the
383 devtool reset
384 command to put things back should you decide you do not want to
385 proceed with your work. If you do use this command, realize that
386 the source tree is preserved.
387
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500388Use ``devtool modify`` to Modify the Source of an Existing Component
389--------------------------------------------------------------------
390
391The ``devtool modify`` command prepares the way to work on existing code
392that already has a local recipe in place that is used to build the
393software. The command is flexible enough to allow you to extract code
394from an upstream source, specify the existing recipe, and keep track of
395and gather any patch files from other developers that are associated
396with the code.
397
398Depending on your particular scenario, the arguments and options you use
399with ``devtool modify`` form different combinations. The following
400diagram shows common development flows for the ``devtool modify``
401command:
402
403.. image:: figures/sdk-devtool-modify-flow.png
404 :align: center
405
4061. *Preparing to Modify the Code*: The top part of the flow shows three
407 scenarios by which you could use ``devtool modify`` to prepare to
408 work on source files. Each scenario assumes the following:
409
410 - The recipe exists locally in a layer external to the ``devtool``
411 workspace.
412
413 - The source files exist either upstream in an un-extracted state or
414 locally in a previously extracted state.
415
416 The typical situation is where another developer has created a layer
417 for use with the Yocto Project and their recipe already resides in
418 that layer. Furthermore, their source code is readily available
419 either upstream or locally.
420
421 - *Left*: The left scenario in the figure represents a common
422 situation where the source code does not exist locally and it
423 needs to be extracted from an upstream source. In this situation,
424 the source is extracted into the default ``devtool`` workspace
425 location. The recipe, in this scenario, is in its own layer
426 outside the workspace (i.e. ``meta-``\ layername).
427
428 The following command identifies the recipe and, by default,
Andrew Geisslerc926e172021-05-07 16:11:35 -0500429 extracts the source files::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500430
431 $ devtool modify recipe
432
433 Once
434 ``devtool``\ locates the recipe, ``devtool`` uses the recipe's
435 :term:`SRC_URI` statements to
436 locate the source code and any local patch files from other
437 developers.
438
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700439 With this scenario, there is no ``srctree`` argument. Consequently, the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500440 default behavior of the ``devtool modify`` command is to extract
Andrew Geissler09036742021-06-25 14:25:14 -0500441 the source files pointed to by the :term:`SRC_URI` statements into a
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500442 local Git structure. Furthermore, the location for the extracted
443 source is the default area within the ``devtool`` workspace. The
444 result is that the command sets up both the source code and an
445 append file within the workspace while the recipe remains in its
446 original location.
447
448 Additionally, if you have any non-patch local files (i.e. files
Andrew Geissler09036742021-06-25 14:25:14 -0500449 referred to with ``file://`` entries in :term:`SRC_URI` statement
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500450 excluding ``*.patch/`` or ``*.diff``), these files are copied to
451 an ``oe-local-files`` folder under the newly created source tree.
452 Copying the files here gives you a convenient area from which you
453 can modify the files. Any changes or additions you make to those
454 files are incorporated into the build the next time you build the
455 software just as are other changes you might have made to the
456 source.
457
458 - *Middle*: The middle scenario in the figure represents a situation
459 where the source code also does not exist locally. In this case,
460 the code is again upstream and needs to be extracted to some local
461 area as a Git repository. The recipe, in this scenario, is again
462 local and in its own layer outside the workspace.
463
464 The following command tells ``devtool`` the recipe with which to
465 work and, in this case, identifies a local area for the extracted
466 source files that exists outside of the default ``devtool``
Andrew Geisslerc926e172021-05-07 16:11:35 -0500467 workspace::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500468
469 $ devtool modify recipe srctree
470
471 .. note::
472
473 You cannot provide a URL for
474 srctree
475 using the
476 devtool
477 command.
478
Andrew Geissler09036742021-06-25 14:25:14 -0500479 As with all extractions, the command uses the recipe's :term:`SRC_URI`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500480 statements to locate the source files and any associated patch
481 files. Non-patch files are copied to an ``oe-local-files`` folder
482 under the newly created source tree.
483
484 Once the files are located, the command by default extracts them
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700485 into ``srctree``.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500486
487 Within workspace, ``devtool`` creates an append file for the
488 recipe. The recipe remains in its original location but the source
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700489 files are extracted to the location you provide with ``srctree``.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500490
491 - *Right*: The right scenario in the figure represents a situation
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700492 where the source tree (``srctree``) already exists locally as a
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500493 previously extracted Git structure outside of the ``devtool``
494 workspace. In this example, the recipe also exists elsewhere
495 locally in its own layer.
496
497 The following command tells ``devtool`` the recipe with which to
498 work, uses the "-n" option to indicate source does not need to be
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700499 extracted, and uses ``srctree`` to point to the previously extracted
Andrew Geisslerc926e172021-05-07 16:11:35 -0500500 source files::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500501
502 $ devtool modify -n recipe srctree
503
504 If an ``oe-local-files`` subdirectory happens to exist and it
505 contains non-patch files, the files are used. However, if the
506 subdirectory does not exist and you run the ``devtool finish``
507 command, any non-patch files that might exist next to the recipe
508 are removed because it appears to ``devtool`` that you have
509 deleted those files.
510
511 Once the ``devtool modify`` command finishes, it creates only an
512 append file for the recipe in the ``devtool`` workspace. The
513 recipe and the source code remain in their original locations.
514
5152. *Edit the Source*: Once you have used the ``devtool modify`` command,
516 you are free to make changes to the source files. You can use any
517 editor you like to make and save your source code modifications.
518
5193. *Build the Recipe or Rebuild the Image*: The next step you take
520 depends on what you are going to do with the new code.
521
522 If you need to eventually move the build output to the target
Andrew Geisslerc926e172021-05-07 16:11:35 -0500523 hardware, use the following ``devtool`` command::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500524
525 $ devtool build recipe
526
527 On the other hand, if you want an image to contain the recipe's
528 packages from the workspace for immediate deployment onto a device
529 (e.g. for testing purposes), you can use the ``devtool build-image``
530 command: $ devtool build-image image
531
5324. *Deploy the Build Output*: When you use the ``devtool build`` command
533 to build out your recipe, you probably want to see if the resulting
534 build output works as expected on target hardware.
535
536 .. note::
537
538 This step assumes you have a previously built image that is
539 already either running in QEMU or running on actual hardware.
540 Also, it is assumed that for deployment of the image to the
541 target, SSH is installed in the image and if the image is running
542 on real hardware that you have network access to and from your
543 development machine.
544
545 You can deploy your build output to that target hardware by using the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500546 ``devtool deploy-target`` command::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500547
548 $ devtool deploy-target recipe target
549
550 The target is a live target machine running as an SSH server.
551
552 You can, of course, use other methods to deploy the image you built
553 using the ``devtool build-image`` command to actual hardware.
554 ``devtool`` does not provide a specific command to deploy the image
555 to actual hardware.
556
5575. *Finish Your Work With the Recipe*: The ``devtool finish`` command
558 creates any patches corresponding to commits in the local Git
559 repository, updates the recipe to point to them (or creates a
560 ``.bbappend`` file to do so, depending on the specified destination
561 layer), and then resets the recipe so that the recipe is built
562 normally rather than from the workspace.
563 ::
564
565 $ devtool finish recipe layer
566
567 .. note::
568
569 Any changes you want to turn into patches must be staged and
570 committed within the local Git repository before you use the
571 devtool finish
572 command.
573
574 Because there is no need to move the recipe, ``devtool finish``
575 either updates the original recipe in the original layer or the
576 command creates a ``.bbappend`` file in a different layer as provided
577 by layer. Any work you did in the ``oe-local-files`` directory is
578 preserved in the original files next to the recipe during the
579 ``devtool finish`` command.
580
581 As a final process of the ``devtool finish`` command, the state of
582 the standard layers and the upstream source is restored so that you
583 can build the recipe from those areas rather than from the workspace.
584
585 .. note::
586
587 You can use the
588 devtool reset
589 command to put things back should you decide you do not want to
590 proceed with your work. If you do use this command, realize that
591 the source tree is preserved.
592
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500593Use ``devtool upgrade`` to Create a Version of the Recipe that Supports a Newer Version of the Software
594-------------------------------------------------------------------------------------------------------
595
596The ``devtool upgrade`` command upgrades an existing recipe to that of a
597more up-to-date version found upstream. Throughout the life of software,
598recipes continually undergo version upgrades by their upstream
599publishers. You can use the ``devtool upgrade`` workflow to make sure
600your recipes you are using for builds are up-to-date with their upstream
601counterparts.
602
603.. note::
604
605 Several methods exist by which you can upgrade recipes -
Andrew Geissler09036742021-06-25 14:25:14 -0500606 ``devtool upgrade``
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500607 happens to be one. You can read about all the methods by which you
Andrew Geissler09036742021-06-25 14:25:14 -0500608 can upgrade recipes in the
609 :ref:`dev-manual/common-tasks:upgrading recipes` section
610 of the Yocto Project Development Tasks Manual.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500611
612The ``devtool upgrade`` command is flexible enough to allow you to
613specify source code revision and versioning schemes, extract code into
614or out of the ``devtool``
615:ref:`devtool-the-workspace-layer-structure`,
616and work with any source file forms that the
Patrick Williams213cb262021-08-07 19:21:33 -0500617:ref:`bitbake:bitbake-user-manual/bitbake-user-manual-fetching:fetchers` support.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500618
619The following diagram shows the common development flow used with the
620``devtool upgrade`` command:
621
622.. image:: figures/sdk-devtool-upgrade-flow.png
623 :align: center
624
6251. *Initiate the Upgrade*: The top part of the flow shows the typical
626 scenario by which you use the ``devtool upgrade`` command. The
627 following conditions exist:
628
629 - The recipe exists in a local layer external to the ``devtool``
630 workspace.
631
632 - The source files for the new release exist in the same location
633 pointed to by :term:`SRC_URI`
634 in the recipe (e.g. a tarball with the new version number in the
635 name, or as a different revision in the upstream Git repository).
636
637 A common situation is where third-party software has undergone a
638 revision so that it has been upgraded. The recipe you have access to
639 is likely in your own layer. Thus, you need to upgrade the recipe to
Andrew Geisslerc926e172021-05-07 16:11:35 -0500640 use the newer version of the software::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500641
642 $ devtool upgrade -V version recipe
643
644 By default, the ``devtool upgrade`` command extracts source
645 code into the ``sources`` directory in the
646 :ref:`devtool-the-workspace-layer-structure`.
647 If you want the code extracted to any other location, you need to
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700648 provide the ``srctree`` positional argument with the command as follows::
649
650 $ devtool upgrade -V version recipe srctree
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500651
652 .. note::
653
654 In this example, the "-V" option specifies the new version. If you
655 don't use "-V", the command upgrades the recipe to the latest
656 version.
657
Andrew Geissler09036742021-06-25 14:25:14 -0500658 If the source files pointed to by the :term:`SRC_URI` statement in the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500659 recipe are in a Git repository, you must provide the "-S" option and
660 specify a revision for the software.
661
Andrew Geissler09036742021-06-25 14:25:14 -0500662 Once ``devtool`` locates the recipe, it uses the :term:`SRC_URI` variable
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500663 to locate the source code and any local patch files from other
664 developers. The result is that the command sets up the source code,
665 the new version of the recipe, and an append file all within the
666 workspace.
667
668 Additionally, if you have any non-patch local files (i.e. files
Andrew Geissler09036742021-06-25 14:25:14 -0500669 referred to with ``file://`` entries in :term:`SRC_URI` statement
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500670 excluding ``*.patch/`` or ``*.diff``), these files are copied to an
671 ``oe-local-files`` folder under the newly created source tree.
672 Copying the files here gives you a convenient area from which you can
673 modify the files. Any changes or additions you make to those files
674 are incorporated into the build the next time you build the software
675 just as are other changes you might have made to the source.
676
William A. Kennington IIIac69b482021-06-02 12:28:27 -07006772. *Resolve any Conflicts created by the Upgrade*: Conflicts could happen
678 after upgrading the software to a new version. Conflicts occur
Andrew Geissler09036742021-06-25 14:25:14 -0500679 if your recipe specifies some patch files in :term:`SRC_URI` that
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500680 conflict with changes made in the new version of the software. For
681 such cases, you need to resolve the conflicts by editing the source
682 and following the normal ``git rebase`` conflict resolution process.
683
684 Before moving onto the next step, be sure to resolve any such
685 conflicts created through use of a newer or different version of the
686 software.
687
6883. *Build the Recipe or Rebuild the Image*: The next step you take
689 depends on what you are going to do with the new code.
690
691 If you need to eventually move the build output to the target
Andrew Geisslerc926e172021-05-07 16:11:35 -0500692 hardware, use the following ``devtool`` command::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500693
694 $ devtool build recipe
695
696 On the other hand, if you want an image to contain the recipe's
697 packages from the workspace for immediate deployment onto a device
698 (e.g. for testing purposes), you can use the ``devtool build-image``
Andrew Geisslerc926e172021-05-07 16:11:35 -0500699 command::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500700
701 $ devtool build-image image
702
7034. *Deploy the Build Output*: When you use the ``devtool build`` command
704 or ``bitbake`` to build your recipe, you probably want to see if the
705 resulting build output works as expected on target hardware.
706
707 .. note::
708
709 This step assumes you have a previously built image that is
710 already either running in QEMU or running on actual hardware.
711 Also, it is assumed that for deployment of the image to the
712 target, SSH is installed in the image and if the image is running
713 on real hardware that you have network access to and from your
714 development machine.
715
716 You can deploy your build output to that target hardware by using the
717 ``devtool deploy-target`` command: $ devtool deploy-target recipe
718 target The target is a live target machine running as an SSH server.
719
720 You can, of course, also deploy the image you build using the
721 ``devtool build-image`` command to actual hardware. However,
722 ``devtool`` does not provide a specific command that allows you to do
723 this.
724
7255. *Finish Your Work With the Recipe*: The ``devtool finish`` command
726 creates any patches corresponding to commits in the local Git
727 repository, moves the new recipe to a more permanent layer, and then
728 resets the recipe so that the recipe is built normally rather than
729 from the workspace.
730
731 Any work you did in the ``oe-local-files`` directory is preserved in
732 the original files next to the recipe during the ``devtool finish``
733 command.
734
735 If you specify a destination layer that is the same as the original
736 source, then the old version of the recipe and associated files are
737 removed prior to adding the new version.
738 ::
739
740 $ devtool finish recipe layer
741
742 .. note::
743
744 Any changes you want to turn into patches must be committed to the
745 Git repository in the source tree.
746
747 As a final process of the ``devtool finish`` command, the state of
748 the standard layers and the upstream source is restored so that you
749 can build the recipe from those areas rather than the workspace.
750
751 .. note::
752
753 You can use the
754 devtool reset
755 command to put things back should you decide you do not want to
756 proceed with your work. If you do use this command, realize that
757 the source tree is preserved.
758
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500759A Closer Look at ``devtool add``
760================================
761
762The ``devtool add`` command automatically creates a recipe based on the
763source tree you provide with the command. Currently, the command has
764support for the following:
765
766- Autotools (``autoconf`` and ``automake``)
767
768- CMake
769
770- Scons
771
772- ``qmake``
773
774- Plain ``Makefile``
775
776- Out-of-tree kernel module
777
778- Binary package (i.e. "-b" option)
779
780- Node.js module
781
782- Python modules that use ``setuptools`` or ``distutils``
783
784Apart from binary packages, the determination of how a source tree
785should be treated is automatic based on the files present within that
786source tree. For example, if a ``CMakeLists.txt`` file is found, then
787the source tree is assumed to be using CMake and is treated accordingly.
788
789.. note::
790
791 In most cases, you need to edit the automatically generated recipe in
792 order to make it build properly. Typically, you would go through
793 several edit and build cycles until the recipe successfully builds.
794 Once the recipe builds, you could use possible further iterations to
795 test the recipe on the target device.
796
797The remainder of this section covers specifics regarding how parts of
798the recipe are generated.
799
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500800Name and Version
801----------------
802
803If you do not specify a name and version on the command line,
804``devtool add`` uses various metadata within the source tree in an
805attempt to determine the name and version of the software being built.
806Based on what the tool determines, ``devtool`` sets the name of the
807created recipe file accordingly.
808
809If ``devtool`` cannot determine the name and version, the command prints
810an error. For such cases, you must re-run the command and provide the
811name and version, just the name, or just the version as part of the
812command line.
813
814Sometimes the name or version determined from the source tree might be
Andrew Geisslerc926e172021-05-07 16:11:35 -0500815incorrect. For such a case, you must reset the recipe::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500816
817 $ devtool reset -n recipename
818
819After running the ``devtool reset`` command, you need to
820run ``devtool add`` again and provide the name or the version.
821
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500822Dependency Detection and Mapping
823--------------------------------
824
825The ``devtool add`` command attempts to detect build-time dependencies
826and map them to other recipes in the system. During this mapping, the
827command fills in the names of those recipes as part of the
828:term:`DEPENDS` variable within the
829recipe. If a dependency cannot be mapped, ``devtool`` places a comment
830in the recipe indicating such. The inability to map a dependency can
831result from naming not being recognized or because the dependency simply
832is not available. For cases where the dependency is not available, you
833must use the ``devtool add`` command to add an additional recipe that
834satisfies the dependency. Once you add that recipe, you need to update
Andrew Geissler09036742021-06-25 14:25:14 -0500835the :term:`DEPENDS` variable in the original recipe to include the new
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500836recipe.
837
838If you need to add runtime dependencies, you can do so by adding the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500839following to your recipe::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500840
Patrick Williams0ca19cc2021-08-16 14:03:13 -0500841 RDEPENDS:${PN} += "dependency1 dependency2 ..."
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500842
843.. note::
844
845 The
846 devtool add
847 command often cannot distinguish between mandatory and optional
848 dependencies. Consequently, some of the detected dependencies might
849 in fact be optional. When in doubt, consult the documentation or the
850 configure script for the software the recipe is building for further
851 details. In some cases, you might find you can substitute the
852 dependency with an option that disables the associated functionality
853 passed to the configure script.
854
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500855License Detection
856-----------------
857
858The ``devtool add`` command attempts to determine if the software you
859are adding is able to be distributed under a common, open-source
860license. If so, the command sets the
861:term:`LICENSE` value accordingly.
862You should double-check the value added by the command against the
863documentation or source files for the software you are building and, if
Andrew Geissler09036742021-06-25 14:25:14 -0500864necessary, update that :term:`LICENSE` value.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500865
866The ``devtool add`` command also sets the
867:term:`LIC_FILES_CHKSUM`
868value to point to all files that appear to be license-related. Realize
869that license statements often appear in comments at the top of source
870files or within the documentation. In such cases, the command does not
871recognize those license statements. Consequently, you might need to
Andrew Geissler09036742021-06-25 14:25:14 -0500872amend the :term:`LIC_FILES_CHKSUM` variable to point to one or more of those
873comments if present. Setting :term:`LIC_FILES_CHKSUM` is particularly
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500874important for third-party software. The mechanism attempts to ensure
875correct licensing should you upgrade the recipe to a newer upstream
876version in future. Any change in licensing is detected and you receive
877an error prompting you to check the license text again.
878
879If the ``devtool add`` command cannot determine licensing information,
Andrew Geissler09036742021-06-25 14:25:14 -0500880``devtool`` sets the :term:`LICENSE` value to "CLOSED" and leaves the
881:term:`LIC_FILES_CHKSUM` value unset. This behavior allows you to continue
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500882with development even though the settings are unlikely to be correct in
883all cases. You should check the documentation or source files for the
884software you are building to determine the actual license.
885
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500886Adding Makefile-Only Software
887-----------------------------
888
889The use of Make by itself is very common in both proprietary and
890open-source software. Unfortunately, Makefiles are often not written
891with cross-compilation in mind. Thus, ``devtool add`` often cannot do
892very much to ensure that these Makefiles build correctly. It is very
893common, for example, to explicitly call ``gcc`` instead of using the
894:term:`CC` variable. Usually, in a
895cross-compilation environment, ``gcc`` is the compiler for the build
896host and the cross-compiler is named something similar to
897``arm-poky-linux-gnueabi-gcc`` and might require arguments (e.g. to
898point to the associated sysroot for the target machine).
899
900When writing a recipe for Makefile-only software, keep the following in
901mind:
902
903- You probably need to patch the Makefile to use variables instead of
904 hardcoding tools within the toolchain such as ``gcc`` and ``g++``.
905
906- The environment in which Make runs is set up with various standard
Andrew Geissler09036742021-06-25 14:25:14 -0500907 variables for compilation (e.g. :term:`CC`, :term:`CXX`, and so forth) in a
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500908 similar manner to the environment set up by the SDK's environment
909 setup script. One easy way to see these variables is to run the
910 ``devtool build`` command on the recipe and then look in
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700911 ``oe-logs/run.do_compile``. Towards the top of this file, there is
912 a list of environment variables that are set. You can take
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500913 advantage of these variables within the Makefile.
914
915- If the Makefile sets a default for a variable using "=", that default
916 overrides the value set in the environment, which is usually not
917 desirable. For this case, you can either patch the Makefile so it
918 sets the default using the "?=" operator, or you can alternatively
919 force the value on the ``make`` command line. To force the value on
920 the command line, add the variable setting to
921 :term:`EXTRA_OEMAKE` or
922 :term:`PACKAGECONFIG_CONFARGS`
Andrew Geissler09036742021-06-25 14:25:14 -0500923 within the recipe. Here is an example using :term:`EXTRA_OEMAKE`::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500924
925 EXTRA_OEMAKE += "'CC=${CC}' 'CXX=${CXX}'"
926
927 In the above example,
928 single quotes are used around the variable settings as the values are
929 likely to contain spaces because required default options are passed
930 to the compiler.
931
932- Hardcoding paths inside Makefiles is often problematic in a
933 cross-compilation environment. This is particularly true because
934 those hardcoded paths often point to locations on the build host and
935 thus will either be read-only or will introduce contamination into
936 the cross-compilation because they are specific to the build host
937 rather than the target. Patching the Makefile to use prefix variables
938 or other path variables is usually the way to handle this situation.
939
940- Sometimes a Makefile runs target-specific commands such as
941 ``ldconfig``. For such cases, you might be able to apply patches that
942 remove these commands from the Makefile.
943
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500944Adding Native Tools
945-------------------
946
947Often, you need to build additional tools that run on the :term:`Build
948Host` as opposed to
949the target. You should indicate this requirement by using one of the
950following methods when you run ``devtool add``:
951
952- Specify the name of the recipe such that it ends with "-native".
953 Specifying the name like this produces a recipe that only builds for
954 the build host.
955
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700956- Specify the "--also-native" option with the ``devtool add``
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500957 command. Specifying this option creates a recipe file that still
958 builds for the target but also creates a variant with a "-native"
959 suffix that builds for the build host.
960
961.. note::
962
963 If you need to add a tool that is shipped as part of a source tree
964 that builds code for the target, you can typically accomplish this by
965 building the native and target parts separately rather than within
966 the same compilation process. Realize though that with the
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700967 "--also-native" option, you can add the tool using just one
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500968 recipe file.
969
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500970Adding Node.js Modules
971----------------------
972
973You can use the ``devtool add`` command two different ways to add
974Node.js modules: 1) Through ``npm`` and, 2) from a repository or local
975source.
976
Andrew Geisslerc926e172021-05-07 16:11:35 -0500977Use the following form to add Node.js modules through ``npm``::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500978
979 $ devtool add "npm://registry.npmjs.org;name=forever;version=0.15.1"
980
981The name and
982version parameters are mandatory. Lockdown and shrinkwrap files are
983generated and pointed to by the recipe in order to freeze the version
984that is fetched for the dependencies according to the first time. This
985also saves checksums that are verified on future fetches. Together,
986these behaviors ensure the reproducibility and integrity of the build.
987
988.. note::
989
990 - You must use quotes around the URL. The ``devtool add`` does not
991 require the quotes, but the shell considers ";" as a splitter
992 between multiple commands. Thus, without the quotes,
993 ``devtool add`` does not receive the other parts, which results in
994 several "command not found" errors.
995
996 - In order to support adding Node.js modules, a ``nodejs`` recipe
997 must be part of your SDK.
998
999As mentioned earlier, you can also add Node.js modules directly from a
1000repository or local source tree. To add modules this way, use
Andrew Geisslerc926e172021-05-07 16:11:35 -05001001``devtool add`` in the following form::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001002
1003 $ devtool add https://github.com/diversario/node-ssdp
1004
1005In this example, ``devtool``
1006fetches the specified Git repository, detects the code as Node.js code,
1007fetches dependencies using ``npm``, and sets
1008:term:`SRC_URI` accordingly.
1009
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001010Working With Recipes
1011====================
1012
1013When building a recipe using the ``devtool build`` command, the typical
1014build progresses as follows:
1015
10161. Fetch the source
1017
10182. Unpack the source
1019
10203. Configure the source
1021
10224. Compile the source
1023
10245. Install the build output
1025
10266. Package the installed output
1027
1028For recipes in the workspace, fetching and unpacking is disabled as the
1029source tree has already been prepared and is persistent. Each of these
1030build steps is defined as a function (task), usually with a "do\_" prefix
1031(e.g. :ref:`ref-tasks-fetch`,
1032:ref:`ref-tasks-unpack`, and so
1033forth). These functions are typically shell scripts but can instead be
1034written in Python.
1035
1036If you look at the contents of a recipe, you will see that the recipe
1037does not include complete instructions for building the software.
1038Instead, common functionality is encapsulated in classes inherited with
1039the ``inherit`` directive. This technique leaves the recipe to describe
William A. Kennington IIIac69b482021-06-02 12:28:27 -07001040just the things that are specific to the software being built. There is
1041a :ref:`base <ref-classes-base>` class that
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001042is implicitly inherited by all recipes and provides the functionality
1043that most recipes typically need.
1044
1045The remainder of this section presents information useful when working
1046with recipes.
1047
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001048Finding Logs and Work Files
1049---------------------------
1050
1051After the first run of the ``devtool build`` command, recipes that were
1052previously created using the ``devtool add`` command or whose sources
1053were modified using the ``devtool modify`` command contain symbolic
1054links created within the source tree:
1055
1056- ``oe-logs``: This link points to the directory in which log files and
1057 run scripts for each build step are created.
1058
1059- ``oe-workdir``: This link points to the temporary work area for the
1060 recipe. The following locations under ``oe-workdir`` are particularly
1061 useful:
1062
1063 - ``image/``: Contains all of the files installed during the
1064 :ref:`ref-tasks-install` stage.
1065 Within a recipe, this directory is referred to by the expression
1066 ``${``\ :term:`D`\ ``}``.
1067
1068 - ``sysroot-destdir/``: Contains a subset of files installed within
1069 ``do_install`` that have been put into the shared sysroot. For
Andrew Geissler3b8a17c2021-04-15 15:55:55 -05001070 more information, see the
1071 ":ref:`dev-manual/common-tasks:sharing files between recipes`" section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001072
1073 - ``packages-split/``: Contains subdirectories for each package
1074 produced by the recipe. For more information, see the
Andrew Geissler3b8a17c2021-04-15 15:55:55 -05001075 ":ref:`sdk-manual/extensible:packaging`" section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001076
1077You can use these links to get more information on what is happening at
1078each build step.
1079
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001080Setting Configure Arguments
1081---------------------------
1082
1083If the software your recipe is building uses GNU autoconf, then a fixed
1084set of arguments is passed to it to enable cross-compilation plus any
1085extras specified by
1086:term:`EXTRA_OECONF` or
1087:term:`PACKAGECONFIG_CONFARGS`
1088set within the recipe. If you wish to pass additional options, add them
Andrew Geissler09036742021-06-25 14:25:14 -05001089to :term:`EXTRA_OECONF` or :term:`PACKAGECONFIG_CONFARGS`. Other supported build
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001090tools have similar variables (e.g.
1091:term:`EXTRA_OECMAKE` for
1092CMake, :term:`EXTRA_OESCONS`
1093for Scons, and so forth). If you need to pass anything on the ``make``
Andrew Geissler09036742021-06-25 14:25:14 -05001094command line, you can use :term:`EXTRA_OEMAKE` or the
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001095:term:`PACKAGECONFIG_CONFARGS`
1096variables to do so.
1097
1098You can use the ``devtool configure-help`` command to help you set the
1099arguments listed in the previous paragraph. The command determines the
1100exact options being passed, and shows them to you along with any custom
Andrew Geissler09036742021-06-25 14:25:14 -05001101arguments specified through :term:`EXTRA_OECONF` or
1102:term:`PACKAGECONFIG_CONFARGS`. If applicable, the command also shows you
William A. Kennington IIIac69b482021-06-02 12:28:27 -07001103the output of the configure script's "--help" option as a
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001104reference.
1105
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001106Sharing Files Between Recipes
1107-----------------------------
1108
1109Recipes often need to use files provided by other recipes on the
1110:term:`Build Host`. For example,
1111an application linking to a common library needs access to the library
1112itself and its associated headers. The way this access is accomplished
William A. Kennington IIIac69b482021-06-02 12:28:27 -07001113within the extensible SDK is through the sysroot. There is one sysroot per
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001114"machine" for which the SDK is being built. In practical terms, this
William A. Kennington IIIac69b482021-06-02 12:28:27 -07001115means there is a sysroot for the target machine, and a sysroot for
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001116the build host.
1117
1118Recipes should never write files directly into the sysroot. Instead,
1119files should be installed into standard locations during the
1120:ref:`ref-tasks-install` task within
1121the ``${``\ :term:`D`\ ``}`` directory. A
1122subset of these files automatically goes into the sysroot. The reason
1123for this limitation is that almost all files that go into the sysroot
1124are cataloged in manifests in order to ensure they can be removed later
1125when a recipe is modified or removed. Thus, the sysroot is able to
1126remain free from stale files.
1127
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001128Packaging
1129---------
1130
1131Packaging is not always particularly relevant within the extensible SDK.
1132However, if you examine how build output gets into the final image on
1133the target device, it is important to understand packaging because the
1134contents of the image are expressed in terms of packages and not
1135recipes.
1136
1137During the :ref:`ref-tasks-package`
1138task, files installed during the
1139:ref:`ref-tasks-install` task are
1140split into one main package, which is almost always named the same as
1141the recipe, and into several other packages. This separation exists
1142because not all of those installed files are useful in every image. For
1143example, you probably do not need any of the documentation installed in
1144a production image. Consequently, for each recipe the documentation
1145files are separated into a ``-doc`` package. Recipes that package
1146software containing optional modules or plugins might undergo additional
1147package splitting as well.
1148
1149After building a recipe, you can see where files have gone by looking in
1150the ``oe-workdir/packages-split`` directory, which contains a
1151subdirectory for each package. Apart from some advanced cases, the
1152:term:`PACKAGES` and
1153:term:`FILES` variables controls
Andrew Geissler09036742021-06-25 14:25:14 -05001154splitting. The :term:`PACKAGES` variable lists all of the packages to be
1155produced, while the :term:`FILES` variable specifies which files to include
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001156in each package by using an override to specify the package. For
Patrick Williams0ca19cc2021-08-16 14:03:13 -05001157example, ``FILES:${PN}`` specifies the files to go into the main package
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001158(i.e. the main package has the same name as the recipe and
1159``${``\ :term:`PN`\ ``}`` evaluates to the
Andrew Geissler09036742021-06-25 14:25:14 -05001160recipe name). The order of the :term:`PACKAGES` value is significant. For
1161each installed file, the first package whose :term:`FILES` value matches the
1162file is the package into which the file goes. Both the :term:`PACKAGES` and
1163:term:`FILES` variables have default values. Consequently, you might find
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001164you do not even need to set these variables in your recipe unless the
1165software the recipe is building installs files into non-standard
1166locations.
1167
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001168Restoring the Target Device to its Original State
1169=================================================
1170
1171If you use the ``devtool deploy-target`` command to write a recipe's
1172build output to the target, and you are working on an existing component
1173of the system, then you might find yourself in a situation where you
1174need to restore the original files that existed prior to running the
1175``devtool deploy-target`` command. Because the ``devtool deploy-target``
1176command backs up any files it overwrites, you can use the
1177``devtool undeploy-target`` command to restore those files and remove
Andrew Geisslerc926e172021-05-07 16:11:35 -05001178any other files the recipe deployed. Consider the following example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001179
1180 $ devtool undeploy-target lighttpd root@192.168.7.2
1181
1182If you have deployed
1183multiple applications, you can remove them all using the "-a" option
Andrew Geisslerc926e172021-05-07 16:11:35 -05001184thus restoring the target device to its original state::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001185
1186 $ devtool undeploy-target -a root@192.168.7.2
1187
1188Information about files deployed to
1189the target as well as any backed up files are stored on the target
1190itself. This storage, of course, requires some additional space on the
1191target machine.
1192
1193.. note::
1194
1195 The
1196 devtool deploy-target
1197 and
1198 devtool undeploy-target
1199 commands do not currently interact with any package management system
1200 on the target device (e.g. RPM or OPKG). Consequently, you should not
1201 intermingle
1202 devtool deploy-target
1203 and package manager operations on the target device. Doing so could
1204 result in a conflicting set of files.
1205
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001206Installing Additional Items Into the Extensible SDK
1207===================================================
1208
1209Out of the box the extensible SDK typically only comes with a small
1210number of tools and libraries. A minimal SDK starts mostly empty and is
1211populated on-demand. Sometimes you must explicitly install extra items
1212into the SDK. If you need these extra items, you can first search for
1213the items using the ``devtool search`` command. For example, suppose you
1214need to link to libGL but you are not sure which recipe provides libGL.
Andrew Geisslerc926e172021-05-07 16:11:35 -05001215You can use the following command to find out::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001216
1217 $ devtool search libGL mesa
1218
1219A free implementation of the OpenGL API Once you know the recipe
Andrew Geisslerc926e172021-05-07 16:11:35 -05001220(i.e. ``mesa`` in this example), you can install it::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001221
1222 $ devtool sdk-install mesa
1223
1224By default, the ``devtool sdk-install`` command assumes
1225the item is available in pre-built form from your SDK provider. If the
1226item is not available and it is acceptable to build the item from
Andrew Geisslerc926e172021-05-07 16:11:35 -05001227source, you can add the "-s" option as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001228
1229 $ devtool sdk-install -s mesa
1230
1231It is important to remember that building the item from source
1232takes significantly longer than installing the pre-built artifact. Also,
William A. Kennington IIIac69b482021-06-02 12:28:27 -07001233if there is no recipe for the item you want to add to the SDK, you must
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001234instead add the item using the ``devtool add`` command.
1235
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001236Applying Updates to an Installed Extensible SDK
1237===============================================
1238
1239If you are working with an installed extensible SDK that gets
1240occasionally updated (e.g. a third-party SDK), then you will need to
1241manually "pull down" the updates into the installed SDK.
1242
Andrew Geisslerc926e172021-05-07 16:11:35 -05001243To update your installed SDK, use ``devtool`` as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001244
1245 $ devtool sdk-update
1246
1247The previous command assumes your SDK provider has set the
Andrew Geissler3b8a17c2021-04-15 15:55:55 -05001248default update URL for you through the :term:`SDK_UPDATE_URL`
1249variable as described in the
1250":ref:`sdk-manual/appendix-customizing:Providing Updates to the Extensible SDK After Installation`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001251section. If the SDK provider has not set that default URL, you need to
1252specify it yourself in the command as follows: $ devtool sdk-update
1253path_to_update_directory
1254
1255.. note::
1256
1257 The URL needs to point specifically to a published SDK and not to an
1258 SDK installer that you would download and install.
1259
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001260Creating a Derivative SDK With Additional Components
1261====================================================
1262
1263You might need to produce an SDK that contains your own custom
1264libraries. A good example would be if you were a vendor with customers
1265that use your SDK to build their own platform-specific software and
1266those customers need an SDK that has custom libraries. In such a case,
1267you can produce a derivative SDK based on the currently installed SDK
1268fairly easily by following these steps:
1269
12701. If necessary, install an extensible SDK that you want to use as a
1271 base for your derivative SDK.
1272
12732. Source the environment script for the SDK.
1274
12753. Add the extra libraries or other components you want by using the
1276 ``devtool add`` command.
1277
12784. Run the ``devtool build-sdk`` command.
1279
1280The previous steps take the recipes added to the workspace and construct
1281a new SDK installer that contains those recipes and the resulting binary
1282artifacts. The recipes go into their own separate layer in the
1283constructed derivative SDK, which leaves the workspace clean and ready
1284for users to add their own recipes.