blob: c94213d6ca577a3507ca39b67fdb5f844144efa8 [file] [log] [blame]
Andrew Geisslerf0343792020-11-18 10:42:21 -06001.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Andrew Geisslerc9f78652020-09-18 14:11:35 -05002
3************************
4Using the Extensible SDK
5************************
6
7This chapter describes the extensible SDK and how to install it.
8Information covers the pieces of the SDK, how to install it, and
9presents a look at using the ``devtool`` functionality. The extensible
10SDK makes it easy to add new applications and libraries to an image,
11modify the source for an existing component, test changes on the target
12hardware, and ease integration into the rest of the
13:term:`OpenEmbedded Build System`.
14
15.. note::
16
17 For a side-by-side comparison of main features supported for an
18 extensible SDK as compared to a standard SDK, see the "
19 Introduction
20 " section.
21
22In addition to the functionality available through ``devtool``, you can
23alternatively make use of the toolchain directly, for example from
24Makefile and Autotools. See the "`Using the SDK Toolchain
25Directly <#sdk-working-projects>`__" chapter for more information.
26
Andrew Geisslerc9f78652020-09-18 14:11:35 -050027Why use the Extensible SDK and What is in It?
28=============================================
29
30The extensible SDK provides a cross-development toolchain and libraries
31tailored to the contents of a specific image. You would use the
32Extensible SDK if you want a toolchain experience supplemented with the
33powerful set of ``devtool`` commands tailored for the Yocto Project
34environment.
35
36The installed extensible SDK consists of several files and directories.
37Basically, it contains an SDK environment setup script, some
38configuration files, an internal build system, and the ``devtool``
39functionality.
40
Andrew Geisslerc9f78652020-09-18 14:11:35 -050041Installing the Extensible SDK
42=============================
43
44The first thing you need to do is install the SDK on your :term:`Build
45Host` by running the ``*.sh`` installation script.
46
47You can download a tarball installer, which includes the pre-built
48toolchain, the ``runqemu`` script, the internal build system,
49``devtool``, and support files from the appropriate
Andrew Geissler09209ee2020-12-13 08:44:15 -060050:yocto_dl:`toolchain </releases/yocto/yocto-&DISTRO;/toolchain/>` directory within the Index of
Andrew Geisslerc9f78652020-09-18 14:11:35 -050051Releases. Toolchains are available for several 32-bit and 64-bit
52architectures with the ``x86_64`` directories, respectively. The
53toolchains the Yocto Project provides are based off the
54``core-image-sato`` and ``core-image-minimal`` images and contain
55libraries appropriate for developing against that image.
56
57The names of the tarball installer scripts are such that a string
58representing the host system appears first in the filename and then is
59immediately followed by a string representing the target architecture.
60An extensible SDK has the string "-ext" as part of the name. Following
61is the general form:
62::
63
64 poky-glibc-host_system-image_type-arch-toolchain-ext-release_version.sh
65
66 Where:
67 host_system is a string representing your development system:
68
69 i686 or x86_64.
70
71 image_type is the image for which the SDK was built:
72
73 core-image-sato or core-image-minimal
74
75 arch is a string representing the tuned target architecture:
76
77 aarch64, armv5e, core2-64, i586, mips32r2, mips64, ppc7400, or cortexa8hf-neon
78
79 release_version is a string representing the release number of the Yocto Project:
80
Andrew Geissler09209ee2020-12-13 08:44:15 -060081 &DISTRO;, &DISTRO;+snapshot
Andrew Geisslerc9f78652020-09-18 14:11:35 -050082
83For example, the following SDK installer is for a 64-bit
84development host system and a i586-tuned target architecture based off
85the SDK for ``core-image-sato`` and using the current DISTRO snapshot:
86::
87
88 poky-glibc-x86_64-core-image-sato-i586-toolchain-ext-DISTRO.sh
89
90.. note::
91
92 As an alternative to downloading an SDK, you can build the SDK
93 installer. For information on building the installer, see the "
94 Building an SDK Installer
95 " section.
96
97The SDK and toolchains are self-contained and by default are installed
98into the ``poky_sdk`` folder in your home directory. You can choose to
99install the extensible SDK in any location when you run the installer.
100However, because files need to be written under that directory during
101the normal course of operation, the location you choose for installation
102must be writable for whichever users need to use the SDK.
103
104The following command shows how to run the installer given a toolchain
105tarball for a 64-bit x86 development host system and a 64-bit x86 target
106architecture. The example assumes the SDK installer is located in
107``~/Downloads/`` and has execution rights.
108
109.. note::
110
111 If you do not have write permissions for the directory into which you
112 are installing the SDK, the installer notifies you and exits. For
113 that case, set up the proper permissions in the directory and run the
114 installer again.
115
116::
117
118 $ ./Downloads/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.5.sh
119 Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.5
120 ==========================================================================
121 Enter target directory for SDK (default: ~/poky_sdk):
122 You are about to install the SDK to "/home/scottrif/poky_sdk". Proceed [Y/n]? Y
123 Extracting SDK..............done
124 Setting it up...
125 Extracting buildtools...
126 Preparing build system...
127 Parsing recipes: 100% |##################################################################| Time: 0:00:52
128 Initialising tasks: 100% |###############################################################| Time: 0:00:00
129 Checking sstate mirror object availability: 100% |#######################################| Time: 0:00:00
130 Loading cache: 100% |####################################################################| Time: 0:00:00
131 Initialising tasks: 100% |###############################################################| Time: 0:00:00
132 done
133 SDK has been successfully set up and is ready to be used.
134 Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
135 $ . /home/scottrif/poky_sdk/environment-setup-core2-64-poky-linux
136
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500137Running the Extensible SDK Environment Setup Script
138===================================================
139
140Once you have the SDK installed, you must run the SDK environment setup
141script before you can actually use the SDK. This setup script resides in
142the directory you chose when you installed the SDK, which is either the
143default ``poky_sdk`` directory or the directory you chose during
144installation.
145
146Before running the script, be sure it is the one that matches the
147architecture for which you are developing. Environment setup scripts
148begin with the string "``environment-setup``" and include as part of
149their name the tuned target architecture. As an example, the following
150commands set the working directory to where the SDK was installed and
151then source the environment setup script. In this example, the setup
152script is for an IA-based target machine using i586 tuning:
153::
154
155 $ cd /home/scottrif/poky_sdk
156 $ source environment-setup-core2-64-poky-linux
157 SDK environment now set up; additionally you may now run devtool to perform development tasks.
158 Run devtool --help for further details.
159
160Running the setup script defines many environment variables needed in
161order to use the SDK (e.g. ``PATH``,
162:term:`CC`,
163:term:`LD`, and so forth). If you want to
164see all the environment variables the script exports, examine the
165installation file itself.
166
167Using ``devtool`` in Your SDK Workflow
168======================================
169
170The cornerstone of the extensible SDK is a command-line tool called
171``devtool``. This tool provides a number of features that help you
172build, test and package software within the extensible SDK, and
173optionally integrate it into an image built by the OpenEmbedded build
174system.
175
176.. note::
177
178 The use of
179 devtool
180 is not limited to the extensible SDK. You can use
181 devtool
182 to help you easily develop any project whose build output must be
183 part of an image built using the build system.
184
185The ``devtool`` command line is organized similarly to
Andrew Geissler09209ee2020-12-13 08:44:15 -0600186:ref:`overview-manual/development-environment:git` in that it has a number of
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500187sub-commands for each function. You can run ``devtool --help`` to see
188all the commands.
189
190.. note::
191
192 See the "
193 devtool
194  Quick Reference
195 " in the Yocto Project Reference Manual for a
196 devtool
197 quick reference.
198
199Three ``devtool`` subcommands exist that provide entry-points into
200development:
201
202- *devtool add*: Assists in adding new software to be built.
203
204- *devtool modify*: Sets up an environment to enable you to modify
205 the source of an existing component.
206
207- *devtool upgrade*: Updates an existing recipe so that you can
208 build it for an updated set of source files.
209
210As with the build system, "recipes" represent software packages within
211``devtool``. When you use ``devtool add``, a recipe is automatically
212created. When you use ``devtool modify``, the specified existing recipe
213is used in order to determine where to get the source code and how to
214patch it. In both cases, an environment is set up so that when you build
215the recipe a source tree that is under your control is used in order to
216allow you to make changes to the source as desired. By default, new
217recipes and the source go into a "workspace" directory under the SDK.
218
219The remainder of this section presents the ``devtool add``,
220``devtool modify``, and ``devtool upgrade`` workflows.
221
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500222Use ``devtool add`` to Add an Application
223-----------------------------------------
224
225The ``devtool add`` command generates a new recipe based on existing
226source code. This command takes advantage of the
227:ref:`devtool-the-workspace-layer-structure`
228layer that many ``devtool`` commands use. The command is flexible enough
229to allow you to extract source code into both the workspace or a
230separate local Git repository and to use existing code that does not
231need to be extracted.
232
233Depending on your particular scenario, the arguments and options you use
234with ``devtool add`` form different combinations. The following diagram
235shows common development flows you would use with the ``devtool add``
236command:
237
238.. image:: figures/sdk-devtool-add-flow.png
239 :align: center
240
2411. *Generating the New Recipe*: The top part of the flow shows three
242 scenarios by which you could use ``devtool add`` to generate a recipe
243 based on existing source code.
244
245 In a shared development environment, it is typical for other
246 developers to be responsible for various areas of source code. As a
247 developer, you are probably interested in using that source code as
248 part of your development within the Yocto Project. All you need is
249 access to the code, a recipe, and a controlled area in which to do
250 your work.
251
252 Within the diagram, three possible scenarios feed into the
253 ``devtool add`` workflow:
254
255 - *Left*: The left scenario in the figure represents a common
256 situation where the source code does not exist locally and needs
257 to be extracted. In this situation, the source code is extracted
258 to the default workspace - you do not want the files in some
259 specific location outside of the workspace. Thus, everything you
260 need will be located in the workspace:
261 ::
262
263 $ devtool add recipe fetchuri
264
265 With this command, ``devtool`` extracts the upstream
266 source files into a local Git repository within the ``sources``
267 folder. The command then creates a recipe named recipe and a
268 corresponding append file in the workspace. If you do not provide
269 recipe, the command makes an attempt to determine the recipe name.
270
271 - *Middle*: The middle scenario in the figure also represents a
272 situation where the source code does not exist locally. In this
273 case, the code is again upstream and needs to be extracted to some
274 local area - this time outside of the default workspace.
275
276 .. note::
277
278 If required,
279 devtool
280 always creates a Git repository locally during the extraction.
281
282 Furthermore, the first positional argument srctree in this case
283 identifies where the ``devtool add`` command will locate the
284 extracted code outside of the workspace. You need to specify an
285 empty directory:
286 ::
287
288 $ devtool add recipe srctree fetchuri
289
290 In summary,
291 the source code is pulled from fetchuri and extracted into the
292 location defined by srctree as a local Git repository.
293
294 Within workspace, ``devtool`` creates a recipe named recipe along
295 with an associated append file.
296
297 - *Right*: The right scenario in the figure represents a situation
298 where the srctree has been previously prepared outside of the
299 ``devtool`` workspace.
300
301 The following command provides a new recipe name and identifies
302 the existing source tree location:
303 ::
304
305 $ devtool add recipe srctree
306
307 The command examines the source code and creates a recipe named
308 recipe for the code and places the recipe into the workspace.
309
310 Because the extracted source code already exists, ``devtool`` does
311 not try to relocate the source code into the workspace - only the
312 new recipe is placed in the workspace.
313
314 Aside from a recipe folder, the command also creates an associated
315 append folder and places an initial ``*.bbappend`` file within.
316
3172. *Edit the Recipe*: You can use ``devtool edit-recipe`` to open up the
318 editor as defined by the ``$EDITOR`` environment variable and modify
319 the file:
320 ::
321
322 $ devtool edit-recipe recipe
323
324 From within the editor, you
325 can make modifications to the recipe that take affect when you build
326 it later.
327
3283. *Build the Recipe or Rebuild the Image*: The next step you take
329 depends on what you are going to do with the new code.
330
331 If you need to eventually move the build output to the target
332 hardware, use the following ``devtool`` command:
333 :;
334
335 $ devtool build recipe
336
337 On the other hand, if you want an image to contain the recipe's
338 packages from the workspace for immediate deployment onto a device
339 (e.g. for testing purposes), you can use the ``devtool build-image``
340 command:
341 ::
342
343 $ devtool build-image image
344
3454. *Deploy the Build Output*: When you use the ``devtool build`` command
346 to build out your recipe, you probably want to see if the resulting
347 build output works as expected on the target hardware.
348
349 .. note::
350
351 This step assumes you have a previously built image that is
352 already either running in QEMU or is running on actual hardware.
353 Also, it is assumed that for deployment of the image to the
354 target, SSH is installed in the image and, if the image is running
355 on real hardware, you have network access to and from your
356 development machine.
357
358 You can deploy your build output to that target hardware by using the
359 ``devtool deploy-target`` command: $ devtool deploy-target recipe
360 target The target is a live target machine running as an SSH server.
361
362 You can, of course, also deploy the image you build to actual
363 hardware by using the ``devtool build-image`` command. However,
364 ``devtool`` does not provide a specific command that allows you to
365 deploy the image to actual hardware.
366
3675. *Finish Your Work With the Recipe*: The ``devtool finish`` command
368 creates any patches corresponding to commits in the local Git
369 repository, moves the new recipe to a more permanent layer, and then
370 resets the recipe so that the recipe is built normally rather than
371 from the workspace.
372 ::
373
374 $ devtool finish recipe layer
375
376 .. note::
377
378 Any changes you want to turn into patches must be committed to the
379 Git repository in the source tree.
380
381 As mentioned, the ``devtool finish`` command moves the final recipe
382 to its permanent layer.
383
384 As a final process of the ``devtool finish`` command, the state of
385 the standard layers and the upstream source is restored so that you
386 can build the recipe from those areas rather than the workspace.
387
388 .. note::
389
390 You can use the
391 devtool reset
392 command to put things back should you decide you do not want to
393 proceed with your work. If you do use this command, realize that
394 the source tree is preserved.
395
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500396Use ``devtool modify`` to Modify the Source of an Existing Component
397--------------------------------------------------------------------
398
399The ``devtool modify`` command prepares the way to work on existing code
400that already has a local recipe in place that is used to build the
401software. The command is flexible enough to allow you to extract code
402from an upstream source, specify the existing recipe, and keep track of
403and gather any patch files from other developers that are associated
404with the code.
405
406Depending on your particular scenario, the arguments and options you use
407with ``devtool modify`` form different combinations. The following
408diagram shows common development flows for the ``devtool modify``
409command:
410
411.. image:: figures/sdk-devtool-modify-flow.png
412 :align: center
413
4141. *Preparing to Modify the Code*: The top part of the flow shows three
415 scenarios by which you could use ``devtool modify`` to prepare to
416 work on source files. Each scenario assumes the following:
417
418 - The recipe exists locally in a layer external to the ``devtool``
419 workspace.
420
421 - The source files exist either upstream in an un-extracted state or
422 locally in a previously extracted state.
423
424 The typical situation is where another developer has created a layer
425 for use with the Yocto Project and their recipe already resides in
426 that layer. Furthermore, their source code is readily available
427 either upstream or locally.
428
429 - *Left*: The left scenario in the figure represents a common
430 situation where the source code does not exist locally and it
431 needs to be extracted from an upstream source. In this situation,
432 the source is extracted into the default ``devtool`` workspace
433 location. The recipe, in this scenario, is in its own layer
434 outside the workspace (i.e. ``meta-``\ layername).
435
436 The following command identifies the recipe and, by default,
437 extracts the source files:
438 ::
439
440 $ devtool modify recipe
441
442 Once
443 ``devtool``\ locates the recipe, ``devtool`` uses the recipe's
444 :term:`SRC_URI` statements to
445 locate the source code and any local patch files from other
446 developers.
447
448 With this scenario, no srctree argument exists. Consequently, the
449 default behavior of the ``devtool modify`` command is to extract
450 the source files pointed to by the ``SRC_URI`` statements into a
451 local Git structure. Furthermore, the location for the extracted
452 source is the default area within the ``devtool`` workspace. The
453 result is that the command sets up both the source code and an
454 append file within the workspace while the recipe remains in its
455 original location.
456
457 Additionally, if you have any non-patch local files (i.e. files
458 referred to with ``file://`` entries in ``SRC_URI`` statement
459 excluding ``*.patch/`` or ``*.diff``), these files are copied to
460 an ``oe-local-files`` folder under the newly created source tree.
461 Copying the files here gives you a convenient area from which you
462 can modify the files. Any changes or additions you make to those
463 files are incorporated into the build the next time you build the
464 software just as are other changes you might have made to the
465 source.
466
467 - *Middle*: The middle scenario in the figure represents a situation
468 where the source code also does not exist locally. In this case,
469 the code is again upstream and needs to be extracted to some local
470 area as a Git repository. The recipe, in this scenario, is again
471 local and in its own layer outside the workspace.
472
473 The following command tells ``devtool`` the recipe with which to
474 work and, in this case, identifies a local area for the extracted
475 source files that exists outside of the default ``devtool``
476 workspace:
477 ::
478
479 $ devtool modify recipe srctree
480
481 .. note::
482
483 You cannot provide a URL for
484 srctree
485 using the
486 devtool
487 command.
488
489 As with all extractions, the command uses the recipe's ``SRC_URI``
490 statements to locate the source files and any associated patch
491 files. Non-patch files are copied to an ``oe-local-files`` folder
492 under the newly created source tree.
493
494 Once the files are located, the command by default extracts them
495 into srctree.
496
497 Within workspace, ``devtool`` creates an append file for the
498 recipe. The recipe remains in its original location but the source
499 files are extracted to the location you provide with srctree.
500
501 - *Right*: The right scenario in the figure represents a situation
502 where the source tree (srctree) already exists locally as a
503 previously extracted Git structure outside of the ``devtool``
504 workspace. In this example, the recipe also exists elsewhere
505 locally in its own layer.
506
507 The following command tells ``devtool`` the recipe with which to
508 work, uses the "-n" option to indicate source does not need to be
509 extracted, and uses srctree to point to the previously extracted
510 source files:
511 ::
512
513 $ devtool modify -n recipe srctree
514
515 If an ``oe-local-files`` subdirectory happens to exist and it
516 contains non-patch files, the files are used. However, if the
517 subdirectory does not exist and you run the ``devtool finish``
518 command, any non-patch files that might exist next to the recipe
519 are removed because it appears to ``devtool`` that you have
520 deleted those files.
521
522 Once the ``devtool modify`` command finishes, it creates only an
523 append file for the recipe in the ``devtool`` workspace. The
524 recipe and the source code remain in their original locations.
525
5262. *Edit the Source*: Once you have used the ``devtool modify`` command,
527 you are free to make changes to the source files. You can use any
528 editor you like to make and save your source code modifications.
529
5303. *Build the Recipe or Rebuild the Image*: The next step you take
531 depends on what you are going to do with the new code.
532
533 If you need to eventually move the build output to the target
534 hardware, use the following ``devtool`` command:
535 ::
536
537 $ devtool build recipe
538
539 On the other hand, if you want an image to contain the recipe's
540 packages from the workspace for immediate deployment onto a device
541 (e.g. for testing purposes), you can use the ``devtool build-image``
542 command: $ devtool build-image image
543
5444. *Deploy the Build Output*: When you use the ``devtool build`` command
545 to build out your recipe, you probably want to see if the resulting
546 build output works as expected on target hardware.
547
548 .. note::
549
550 This step assumes you have a previously built image that is
551 already either running in QEMU or running on actual hardware.
552 Also, it is assumed that for deployment of the image to the
553 target, SSH is installed in the image and if the image is running
554 on real hardware that you have network access to and from your
555 development machine.
556
557 You can deploy your build output to that target hardware by using the
558 ``devtool deploy-target`` command:
559 ::
560
561 $ devtool deploy-target recipe target
562
563 The target is a live target machine running as an SSH server.
564
565 You can, of course, use other methods to deploy the image you built
566 using the ``devtool build-image`` command to actual hardware.
567 ``devtool`` does not provide a specific command to deploy the image
568 to actual hardware.
569
5705. *Finish Your Work With the Recipe*: The ``devtool finish`` command
571 creates any patches corresponding to commits in the local Git
572 repository, updates the recipe to point to them (or creates a
573 ``.bbappend`` file to do so, depending on the specified destination
574 layer), and then resets the recipe so that the recipe is built
575 normally rather than from the workspace.
576 ::
577
578 $ devtool finish recipe layer
579
580 .. note::
581
582 Any changes you want to turn into patches must be staged and
583 committed within the local Git repository before you use the
584 devtool finish
585 command.
586
587 Because there is no need to move the recipe, ``devtool finish``
588 either updates the original recipe in the original layer or the
589 command creates a ``.bbappend`` file in a different layer as provided
590 by layer. Any work you did in the ``oe-local-files`` directory is
591 preserved in the original files next to the recipe during the
592 ``devtool finish`` command.
593
594 As a final process of the ``devtool finish`` command, the state of
595 the standard layers and the upstream source is restored so that you
596 can build the recipe from those areas rather than from the workspace.
597
598 .. note::
599
600 You can use the
601 devtool reset
602 command to put things back should you decide you do not want to
603 proceed with your work. If you do use this command, realize that
604 the source tree is preserved.
605
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500606Use ``devtool upgrade`` to Create a Version of the Recipe that Supports a Newer Version of the Software
607-------------------------------------------------------------------------------------------------------
608
609The ``devtool upgrade`` command upgrades an existing recipe to that of a
610more up-to-date version found upstream. Throughout the life of software,
611recipes continually undergo version upgrades by their upstream
612publishers. You can use the ``devtool upgrade`` workflow to make sure
613your recipes you are using for builds are up-to-date with their upstream
614counterparts.
615
616.. note::
617
618 Several methods exist by which you can upgrade recipes -
619 devtool upgrade
620 happens to be one. You can read about all the methods by which you
621 can upgrade recipes in the "
622 Upgrading Recipes
623 " section of the Yocto Project Development Tasks Manual.
624
625The ``devtool upgrade`` command is flexible enough to allow you to
626specify source code revision and versioning schemes, extract code into
627or out of the ``devtool``
628:ref:`devtool-the-workspace-layer-structure`,
629and work with any source file forms that the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600630:ref:`fetchers <bitbake:bitbake-user-manual/bitbake-user-manual-fetching:fetchers>` support.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500631
632The following diagram shows the common development flow used with the
633``devtool upgrade`` command:
634
635.. image:: figures/sdk-devtool-upgrade-flow.png
636 :align: center
637
6381. *Initiate the Upgrade*: The top part of the flow shows the typical
639 scenario by which you use the ``devtool upgrade`` command. The
640 following conditions exist:
641
642 - The recipe exists in a local layer external to the ``devtool``
643 workspace.
644
645 - The source files for the new release exist in the same location
646 pointed to by :term:`SRC_URI`
647 in the recipe (e.g. a tarball with the new version number in the
648 name, or as a different revision in the upstream Git repository).
649
650 A common situation is where third-party software has undergone a
651 revision so that it has been upgraded. The recipe you have access to
652 is likely in your own layer. Thus, you need to upgrade the recipe to
653 use the newer version of the software:
654 ::
655
656 $ devtool upgrade -V version recipe
657
658 By default, the ``devtool upgrade`` command extracts source
659 code into the ``sources`` directory in the
660 :ref:`devtool-the-workspace-layer-structure`.
661 If you want the code extracted to any other location, you need to
662 provide the srctree positional argument with the command as follows:
663 $ devtool upgrade -V version recipe srctree
664
665 .. note::
666
667 In this example, the "-V" option specifies the new version. If you
668 don't use "-V", the command upgrades the recipe to the latest
669 version.
670
671 If the source files pointed to by the ``SRC_URI`` statement in the
672 recipe are in a Git repository, you must provide the "-S" option and
673 specify a revision for the software.
674
675 Once ``devtool`` locates the recipe, it uses the ``SRC_URI`` variable
676 to locate the source code and any local patch files from other
677 developers. The result is that the command sets up the source code,
678 the new version of the recipe, and an append file all within the
679 workspace.
680
681 Additionally, if you have any non-patch local files (i.e. files
682 referred to with ``file://`` entries in ``SRC_URI`` statement
683 excluding ``*.patch/`` or ``*.diff``), these files are copied to an
684 ``oe-local-files`` folder under the newly created source tree.
685 Copying the files here gives you a convenient area from which you can
686 modify the files. Any changes or additions you make to those files
687 are incorporated into the build the next time you build the software
688 just as are other changes you might have made to the source.
689
6902. *Resolve any Conflicts created by the Upgrade*: Conflicts could exist
691 due to the software being upgraded to a new version. Conflicts occur
692 if your recipe specifies some patch files in ``SRC_URI`` that
693 conflict with changes made in the new version of the software. For
694 such cases, you need to resolve the conflicts by editing the source
695 and following the normal ``git rebase`` conflict resolution process.
696
697 Before moving onto the next step, be sure to resolve any such
698 conflicts created through use of a newer or different version of the
699 software.
700
7013. *Build the Recipe or Rebuild the Image*: The next step you take
702 depends on what you are going to do with the new code.
703
704 If you need to eventually move the build output to the target
705 hardware, use the following ``devtool`` command:
706 ::
707
708 $ devtool build recipe
709
710 On the other hand, if you want an image to contain the recipe's
711 packages from the workspace for immediate deployment onto a device
712 (e.g. for testing purposes), you can use the ``devtool build-image``
713 command:
714 ::
715
716 $ devtool build-image image
717
7184. *Deploy the Build Output*: When you use the ``devtool build`` command
719 or ``bitbake`` to build your recipe, you probably want to see if the
720 resulting build output works as expected on target hardware.
721
722 .. note::
723
724 This step assumes you have a previously built image that is
725 already either running in QEMU or running on actual hardware.
726 Also, it is assumed that for deployment of the image to the
727 target, SSH is installed in the image and if the image is running
728 on real hardware that you have network access to and from your
729 development machine.
730
731 You can deploy your build output to that target hardware by using the
732 ``devtool deploy-target`` command: $ devtool deploy-target recipe
733 target The target is a live target machine running as an SSH server.
734
735 You can, of course, also deploy the image you build using the
736 ``devtool build-image`` command to actual hardware. However,
737 ``devtool`` does not provide a specific command that allows you to do
738 this.
739
7405. *Finish Your Work With the Recipe*: The ``devtool finish`` command
741 creates any patches corresponding to commits in the local Git
742 repository, moves the new recipe to a more permanent layer, and then
743 resets the recipe so that the recipe is built normally rather than
744 from the workspace.
745
746 Any work you did in the ``oe-local-files`` directory is preserved in
747 the original files next to the recipe during the ``devtool finish``
748 command.
749
750 If you specify a destination layer that is the same as the original
751 source, then the old version of the recipe and associated files are
752 removed prior to adding the new version.
753 ::
754
755 $ devtool finish recipe layer
756
757 .. note::
758
759 Any changes you want to turn into patches must be committed to the
760 Git repository in the source tree.
761
762 As a final process of the ``devtool finish`` command, the state of
763 the standard layers and the upstream source is restored so that you
764 can build the recipe from those areas rather than the workspace.
765
766 .. note::
767
768 You can use the
769 devtool reset
770 command to put things back should you decide you do not want to
771 proceed with your work. If you do use this command, realize that
772 the source tree is preserved.
773
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500774A Closer Look at ``devtool add``
775================================
776
777The ``devtool add`` command automatically creates a recipe based on the
778source tree you provide with the command. Currently, the command has
779support for the following:
780
781- Autotools (``autoconf`` and ``automake``)
782
783- CMake
784
785- Scons
786
787- ``qmake``
788
789- Plain ``Makefile``
790
791- Out-of-tree kernel module
792
793- Binary package (i.e. "-b" option)
794
795- Node.js module
796
797- Python modules that use ``setuptools`` or ``distutils``
798
799Apart from binary packages, the determination of how a source tree
800should be treated is automatic based on the files present within that
801source tree. For example, if a ``CMakeLists.txt`` file is found, then
802the source tree is assumed to be using CMake and is treated accordingly.
803
804.. note::
805
806 In most cases, you need to edit the automatically generated recipe in
807 order to make it build properly. Typically, you would go through
808 several edit and build cycles until the recipe successfully builds.
809 Once the recipe builds, you could use possible further iterations to
810 test the recipe on the target device.
811
812The remainder of this section covers specifics regarding how parts of
813the recipe are generated.
814
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500815Name and Version
816----------------
817
818If you do not specify a name and version on the command line,
819``devtool add`` uses various metadata within the source tree in an
820attempt to determine the name and version of the software being built.
821Based on what the tool determines, ``devtool`` sets the name of the
822created recipe file accordingly.
823
824If ``devtool`` cannot determine the name and version, the command prints
825an error. For such cases, you must re-run the command and provide the
826name and version, just the name, or just the version as part of the
827command line.
828
829Sometimes the name or version determined from the source tree might be
830incorrect. For such a case, you must reset the recipe:
831::
832
833 $ devtool reset -n recipename
834
835After running the ``devtool reset`` command, you need to
836run ``devtool add`` again and provide the name or the version.
837
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500838Dependency Detection and Mapping
839--------------------------------
840
841The ``devtool add`` command attempts to detect build-time dependencies
842and map them to other recipes in the system. During this mapping, the
843command fills in the names of those recipes as part of the
844:term:`DEPENDS` variable within the
845recipe. If a dependency cannot be mapped, ``devtool`` places a comment
846in the recipe indicating such. The inability to map a dependency can
847result from naming not being recognized or because the dependency simply
848is not available. For cases where the dependency is not available, you
849must use the ``devtool add`` command to add an additional recipe that
850satisfies the dependency. Once you add that recipe, you need to update
851the ``DEPENDS`` variable in the original recipe to include the new
852recipe.
853
854If you need to add runtime dependencies, you can do so by adding the
855following to your recipe:
856::
857
858 RDEPENDS_${PN} += "dependency1 dependency2 ..."
859
860.. note::
861
862 The
863 devtool add
864 command often cannot distinguish between mandatory and optional
865 dependencies. Consequently, some of the detected dependencies might
866 in fact be optional. When in doubt, consult the documentation or the
867 configure script for the software the recipe is building for further
868 details. In some cases, you might find you can substitute the
869 dependency with an option that disables the associated functionality
870 passed to the configure script.
871
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500872License Detection
873-----------------
874
875The ``devtool add`` command attempts to determine if the software you
876are adding is able to be distributed under a common, open-source
877license. If so, the command sets the
878:term:`LICENSE` value accordingly.
879You should double-check the value added by the command against the
880documentation or source files for the software you are building and, if
881necessary, update that ``LICENSE`` value.
882
883The ``devtool add`` command also sets the
884:term:`LIC_FILES_CHKSUM`
885value to point to all files that appear to be license-related. Realize
886that license statements often appear in comments at the top of source
887files or within the documentation. In such cases, the command does not
888recognize those license statements. Consequently, you might need to
889amend the ``LIC_FILES_CHKSUM`` variable to point to one or more of those
890comments if present. Setting ``LIC_FILES_CHKSUM`` is particularly
891important for third-party software. The mechanism attempts to ensure
892correct licensing should you upgrade the recipe to a newer upstream
893version in future. Any change in licensing is detected and you receive
894an error prompting you to check the license text again.
895
896If the ``devtool add`` command cannot determine licensing information,
897``devtool`` sets the ``LICENSE`` value to "CLOSED" and leaves the
898``LIC_FILES_CHKSUM`` value unset. This behavior allows you to continue
899with development even though the settings are unlikely to be correct in
900all cases. You should check the documentation or source files for the
901software you are building to determine the actual license.
902
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500903Adding Makefile-Only Software
904-----------------------------
905
906The use of Make by itself is very common in both proprietary and
907open-source software. Unfortunately, Makefiles are often not written
908with cross-compilation in mind. Thus, ``devtool add`` often cannot do
909very much to ensure that these Makefiles build correctly. It is very
910common, for example, to explicitly call ``gcc`` instead of using the
911:term:`CC` variable. Usually, in a
912cross-compilation environment, ``gcc`` is the compiler for the build
913host and the cross-compiler is named something similar to
914``arm-poky-linux-gnueabi-gcc`` and might require arguments (e.g. to
915point to the associated sysroot for the target machine).
916
917When writing a recipe for Makefile-only software, keep the following in
918mind:
919
920- You probably need to patch the Makefile to use variables instead of
921 hardcoding tools within the toolchain such as ``gcc`` and ``g++``.
922
923- The environment in which Make runs is set up with various standard
924 variables for compilation (e.g. ``CC``, ``CXX``, and so forth) in a
925 similar manner to the environment set up by the SDK's environment
926 setup script. One easy way to see these variables is to run the
927 ``devtool build`` command on the recipe and then look in
928 ``oe-logs/run.do_compile``. Towards the top of this file, a list of
929 environment variables exists that are being set. You can take
930 advantage of these variables within the Makefile.
931
932- If the Makefile sets a default for a variable using "=", that default
933 overrides the value set in the environment, which is usually not
934 desirable. For this case, you can either patch the Makefile so it
935 sets the default using the "?=" operator, or you can alternatively
936 force the value on the ``make`` command line. To force the value on
937 the command line, add the variable setting to
938 :term:`EXTRA_OEMAKE` or
939 :term:`PACKAGECONFIG_CONFARGS`
940 within the recipe. Here is an example using ``EXTRA_OEMAKE``:
941 ::
942
943 EXTRA_OEMAKE += "'CC=${CC}' 'CXX=${CXX}'"
944
945 In the above example,
946 single quotes are used around the variable settings as the values are
947 likely to contain spaces because required default options are passed
948 to the compiler.
949
950- Hardcoding paths inside Makefiles is often problematic in a
951 cross-compilation environment. This is particularly true because
952 those hardcoded paths often point to locations on the build host and
953 thus will either be read-only or will introduce contamination into
954 the cross-compilation because they are specific to the build host
955 rather than the target. Patching the Makefile to use prefix variables
956 or other path variables is usually the way to handle this situation.
957
958- Sometimes a Makefile runs target-specific commands such as
959 ``ldconfig``. For such cases, you might be able to apply patches that
960 remove these commands from the Makefile.
961
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500962Adding Native Tools
963-------------------
964
965Often, you need to build additional tools that run on the :term:`Build
966Host` as opposed to
967the target. You should indicate this requirement by using one of the
968following methods when you run ``devtool add``:
969
970- Specify the name of the recipe such that it ends with "-native".
971 Specifying the name like this produces a recipe that only builds for
972 the build host.
973
974- Specify the "DASHDASHalso-native" option with the ``devtool add``
975 command. Specifying this option creates a recipe file that still
976 builds for the target but also creates a variant with a "-native"
977 suffix that builds for the build host.
978
979.. note::
980
981 If you need to add a tool that is shipped as part of a source tree
982 that builds code for the target, you can typically accomplish this by
983 building the native and target parts separately rather than within
984 the same compilation process. Realize though that with the
985 "DASHDASHalso-native" option, you can add the tool using just one
986 recipe file.
987
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500988Adding Node.js Modules
989----------------------
990
991You can use the ``devtool add`` command two different ways to add
992Node.js modules: 1) Through ``npm`` and, 2) from a repository or local
993source.
994
995Use the following form to add Node.js modules through ``npm``:
996::
997
998 $ devtool add "npm://registry.npmjs.org;name=forever;version=0.15.1"
999
1000The name and
1001version parameters are mandatory. Lockdown and shrinkwrap files are
1002generated and pointed to by the recipe in order to freeze the version
1003that is fetched for the dependencies according to the first time. This
1004also saves checksums that are verified on future fetches. Together,
1005these behaviors ensure the reproducibility and integrity of the build.
1006
1007.. note::
1008
1009 - You must use quotes around the URL. The ``devtool add`` does not
1010 require the quotes, but the shell considers ";" as a splitter
1011 between multiple commands. Thus, without the quotes,
1012 ``devtool add`` does not receive the other parts, which results in
1013 several "command not found" errors.
1014
1015 - In order to support adding Node.js modules, a ``nodejs`` recipe
1016 must be part of your SDK.
1017
1018As mentioned earlier, you can also add Node.js modules directly from a
1019repository or local source tree. To add modules this way, use
1020``devtool add`` in the following form:
1021::
1022
1023 $ devtool add https://github.com/diversario/node-ssdp
1024
1025In this example, ``devtool``
1026fetches the specified Git repository, detects the code as Node.js code,
1027fetches dependencies using ``npm``, and sets
1028:term:`SRC_URI` accordingly.
1029
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001030Working With Recipes
1031====================
1032
1033When building a recipe using the ``devtool build`` command, the typical
1034build progresses as follows:
1035
10361. Fetch the source
1037
10382. Unpack the source
1039
10403. Configure the source
1041
10424. Compile the source
1043
10445. Install the build output
1045
10466. Package the installed output
1047
1048For recipes in the workspace, fetching and unpacking is disabled as the
1049source tree has already been prepared and is persistent. Each of these
1050build steps is defined as a function (task), usually with a "do\_" prefix
1051(e.g. :ref:`ref-tasks-fetch`,
1052:ref:`ref-tasks-unpack`, and so
1053forth). These functions are typically shell scripts but can instead be
1054written in Python.
1055
1056If you look at the contents of a recipe, you will see that the recipe
1057does not include complete instructions for building the software.
1058Instead, common functionality is encapsulated in classes inherited with
1059the ``inherit`` directive. This technique leaves the recipe to describe
1060just the things that are specific to the software being built. A
1061:ref:`base <ref-classes-base>` class exists that
1062is implicitly inherited by all recipes and provides the functionality
1063that most recipes typically need.
1064
1065The remainder of this section presents information useful when working
1066with recipes.
1067
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001068Finding Logs and Work Files
1069---------------------------
1070
1071After the first run of the ``devtool build`` command, recipes that were
1072previously created using the ``devtool add`` command or whose sources
1073were modified using the ``devtool modify`` command contain symbolic
1074links created within the source tree:
1075
1076- ``oe-logs``: This link points to the directory in which log files and
1077 run scripts for each build step are created.
1078
1079- ``oe-workdir``: This link points to the temporary work area for the
1080 recipe. The following locations under ``oe-workdir`` are particularly
1081 useful:
1082
1083 - ``image/``: Contains all of the files installed during the
1084 :ref:`ref-tasks-install` stage.
1085 Within a recipe, this directory is referred to by the expression
1086 ``${``\ :term:`D`\ ``}``.
1087
1088 - ``sysroot-destdir/``: Contains a subset of files installed within
1089 ``do_install`` that have been put into the shared sysroot. For
1090 more information, see the "`Sharing Files Between
1091 Recipes <#sdk-sharing-files-between-recipes>`__" section.
1092
1093 - ``packages-split/``: Contains subdirectories for each package
1094 produced by the recipe. For more information, see the
1095 "`Packaging <#sdk-packaging>`__" section.
1096
1097You can use these links to get more information on what is happening at
1098each build step.
1099
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001100Setting Configure Arguments
1101---------------------------
1102
1103If the software your recipe is building uses GNU autoconf, then a fixed
1104set of arguments is passed to it to enable cross-compilation plus any
1105extras specified by
1106:term:`EXTRA_OECONF` or
1107:term:`PACKAGECONFIG_CONFARGS`
1108set within the recipe. If you wish to pass additional options, add them
1109to ``EXTRA_OECONF`` or ``PACKAGECONFIG_CONFARGS``. Other supported build
1110tools have similar variables (e.g.
1111:term:`EXTRA_OECMAKE` for
1112CMake, :term:`EXTRA_OESCONS`
1113for Scons, and so forth). If you need to pass anything on the ``make``
1114command line, you can use ``EXTRA_OEMAKE`` or the
1115:term:`PACKAGECONFIG_CONFARGS`
1116variables to do so.
1117
1118You can use the ``devtool configure-help`` command to help you set the
1119arguments listed in the previous paragraph. The command determines the
1120exact options being passed, and shows them to you along with any custom
1121arguments specified through ``EXTRA_OECONF`` or
1122``PACKAGECONFIG_CONFARGS``. If applicable, the command also shows you
1123the output of the configure script's "DASHDASHhelp" option as a
1124reference.
1125
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001126Sharing Files Between Recipes
1127-----------------------------
1128
1129Recipes often need to use files provided by other recipes on the
1130:term:`Build Host`. For example,
1131an application linking to a common library needs access to the library
1132itself and its associated headers. The way this access is accomplished
1133within the extensible SDK is through the sysroot. One sysroot exists per
1134"machine" for which the SDK is being built. In practical terms, this
1135means a sysroot exists for the target machine, and a sysroot exists for
1136the build host.
1137
1138Recipes should never write files directly into the sysroot. Instead,
1139files should be installed into standard locations during the
1140:ref:`ref-tasks-install` task within
1141the ``${``\ :term:`D`\ ``}`` directory. A
1142subset of these files automatically goes into the sysroot. The reason
1143for this limitation is that almost all files that go into the sysroot
1144are cataloged in manifests in order to ensure they can be removed later
1145when a recipe is modified or removed. Thus, the sysroot is able to
1146remain free from stale files.
1147
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001148Packaging
1149---------
1150
1151Packaging is not always particularly relevant within the extensible SDK.
1152However, if you examine how build output gets into the final image on
1153the target device, it is important to understand packaging because the
1154contents of the image are expressed in terms of packages and not
1155recipes.
1156
1157During the :ref:`ref-tasks-package`
1158task, files installed during the
1159:ref:`ref-tasks-install` task are
1160split into one main package, which is almost always named the same as
1161the recipe, and into several other packages. This separation exists
1162because not all of those installed files are useful in every image. For
1163example, you probably do not need any of the documentation installed in
1164a production image. Consequently, for each recipe the documentation
1165files are separated into a ``-doc`` package. Recipes that package
1166software containing optional modules or plugins might undergo additional
1167package splitting as well.
1168
1169After building a recipe, you can see where files have gone by looking in
1170the ``oe-workdir/packages-split`` directory, which contains a
1171subdirectory for each package. Apart from some advanced cases, the
1172:term:`PACKAGES` and
1173:term:`FILES` variables controls
1174splitting. The ``PACKAGES`` variable lists all of the packages to be
1175produced, while the ``FILES`` variable specifies which files to include
1176in each package by using an override to specify the package. For
1177example, ``FILES_${PN}`` specifies the files to go into the main package
1178(i.e. the main package has the same name as the recipe and
1179``${``\ :term:`PN`\ ``}`` evaluates to the
1180recipe name). The order of the ``PACKAGES`` value is significant. For
1181each installed file, the first package whose ``FILES`` value matches the
1182file is the package into which the file goes. Defaults exist for both
1183the ``PACKAGES`` and ``FILES`` variables. Consequently, you might find
1184you do not even need to set these variables in your recipe unless the
1185software the recipe is building installs files into non-standard
1186locations.
1187
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001188Restoring the Target Device to its Original State
1189=================================================
1190
1191If you use the ``devtool deploy-target`` command to write a recipe's
1192build output to the target, and you are working on an existing component
1193of the system, then you might find yourself in a situation where you
1194need to restore the original files that existed prior to running the
1195``devtool deploy-target`` command. Because the ``devtool deploy-target``
1196command backs up any files it overwrites, you can use the
1197``devtool undeploy-target`` command to restore those files and remove
1198any other files the recipe deployed. Consider the following example:
1199::
1200
1201 $ devtool undeploy-target lighttpd root@192.168.7.2
1202
1203If you have deployed
1204multiple applications, you can remove them all using the "-a" option
1205thus restoring the target device to its original state:
1206::
1207
1208 $ devtool undeploy-target -a root@192.168.7.2
1209
1210Information about files deployed to
1211the target as well as any backed up files are stored on the target
1212itself. This storage, of course, requires some additional space on the
1213target machine.
1214
1215.. note::
1216
1217 The
1218 devtool deploy-target
1219 and
1220 devtool undeploy-target
1221 commands do not currently interact with any package management system
1222 on the target device (e.g. RPM or OPKG). Consequently, you should not
1223 intermingle
1224 devtool deploy-target
1225 and package manager operations on the target device. Doing so could
1226 result in a conflicting set of files.
1227
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001228Installing Additional Items Into the Extensible SDK
1229===================================================
1230
1231Out of the box the extensible SDK typically only comes with a small
1232number of tools and libraries. A minimal SDK starts mostly empty and is
1233populated on-demand. Sometimes you must explicitly install extra items
1234into the SDK. If you need these extra items, you can first search for
1235the items using the ``devtool search`` command. For example, suppose you
1236need to link to libGL but you are not sure which recipe provides libGL.
1237You can use the following command to find out:
1238::
1239
1240 $ devtool search libGL mesa
1241
1242A free implementation of the OpenGL API Once you know the recipe
1243(i.e. ``mesa`` in this example), you can install it:
1244::
1245
1246 $ devtool sdk-install mesa
1247
1248By default, the ``devtool sdk-install`` command assumes
1249the item is available in pre-built form from your SDK provider. If the
1250item is not available and it is acceptable to build the item from
1251source, you can add the "-s" option as follows:
1252::
1253
1254 $ devtool sdk-install -s mesa
1255
1256It is important to remember that building the item from source
1257takes significantly longer than installing the pre-built artifact. Also,
1258if no recipe exists for the item you want to add to the SDK, you must
1259instead add the item using the ``devtool add`` command.
1260
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001261Applying Updates to an Installed Extensible SDK
1262===============================================
1263
1264If you are working with an installed extensible SDK that gets
1265occasionally updated (e.g. a third-party SDK), then you will need to
1266manually "pull down" the updates into the installed SDK.
1267
1268To update your installed SDK, use ``devtool`` as follows:
1269::
1270
1271 $ devtool sdk-update
1272
1273The previous command assumes your SDK provider has set the
1274default update URL for you through the
1275:term:`SDK_UPDATE_URL`
1276variable as described in the "`Providing Updates to the Extensible SDK
1277After
1278Installation <#sdk-providing-updates-to-the-extensible-sdk-after-installation>`__"
1279section. If the SDK provider has not set that default URL, you need to
1280specify it yourself in the command as follows: $ devtool sdk-update
1281path_to_update_directory
1282
1283.. note::
1284
1285 The URL needs to point specifically to a published SDK and not to an
1286 SDK installer that you would download and install.
1287
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001288Creating a Derivative SDK With Additional Components
1289====================================================
1290
1291You might need to produce an SDK that contains your own custom
1292libraries. A good example would be if you were a vendor with customers
1293that use your SDK to build their own platform-specific software and
1294those customers need an SDK that has custom libraries. In such a case,
1295you can produce a derivative SDK based on the currently installed SDK
1296fairly easily by following these steps:
1297
12981. If necessary, install an extensible SDK that you want to use as a
1299 base for your derivative SDK.
1300
13012. Source the environment script for the SDK.
1302
13033. Add the extra libraries or other components you want by using the
1304 ``devtool add`` command.
1305
13064. Run the ``devtool build-sdk`` command.
1307
1308The previous steps take the recipes added to the workspace and construct
1309a new SDK installer that contains those recipes and the resulting binary
1310artifacts. The recipes go into their own separate layer in the
1311constructed derivative SDK, which leaves the workspace clean and ready
1312for users to add their own recipes.