Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 1 | <!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" |
| 2 | "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" |
| 3 | [<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > |
| 4 | |
| 5 | <chapter id='usingpoky'> |
| 6 | <title>Using the Yocto Project</title> |
| 7 | |
| 8 | <para> |
| 9 | This chapter describes common usage for the Yocto Project. |
| 10 | The information is introductory in nature as other manuals in the Yocto Project |
| 11 | documentation set provide more details on how to use the Yocto Project. |
| 12 | </para> |
| 13 | |
| 14 | <section id='usingpoky-build'> |
| 15 | <title>Running a Build</title> |
| 16 | |
| 17 | <para> |
| 18 | This section provides a summary of the build process and provides information |
| 19 | for less obvious aspects of the build process. |
| 20 | For general information on how to build an image using the OpenEmbedded build |
| 21 | system, see the |
| 22 | "<ulink url='&YOCTO_DOCS_QS_URL;#qs-building-images'>Building Images</ulink>" |
| 23 | section of the Yocto Project Quick Start. |
| 24 | </para> |
| 25 | |
| 26 | <section id='build-overview'> |
| 27 | <title>Build Overview</title> |
| 28 | |
| 29 | <para> |
| 30 | In the development environment you will need to build an image whenever you change hardware |
| 31 | support, add or change system libraries, or add or change services that have dependencies. |
| 32 | </para> |
| 33 | |
| 34 | <mediaobject> |
| 35 | <imageobject> |
| 36 | <imagedata fileref="figures/building-an-image.png" format="PNG" align='center' scalefit='1'/> |
| 37 | </imageobject> |
| 38 | <caption> |
| 39 | <para>Building an Image</para> |
| 40 | </caption> |
| 41 | </mediaobject> |
| 42 | |
| 43 | <para> |
| 44 | The first thing you need to do is set up the OpenEmbedded build |
| 45 | environment by sourcing an environment setup script |
| 46 | (i.e. |
| 47 | <link linkend='structure-core-script'><filename>&OE_INIT_FILE;</filename></link> |
| 48 | or |
| 49 | <link linkend='structure-memres-core-script'><filename>oe-init-build-env-memres</filename></link>). |
| 50 | Here is an example: |
| 51 | <literallayout class='monospaced'> |
| 52 | $ source &OE_INIT_FILE; [<replaceable>build_dir</replaceable>] |
| 53 | </literallayout> |
| 54 | </para> |
| 55 | |
| 56 | <para> |
| 57 | The <replaceable>build_dir</replaceable> argument is optional and specifies the directory the |
| 58 | OpenEmbedded build system uses for the build - |
| 59 | the <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>. |
| 60 | If you do not specify a Build Directory, it defaults to a directory |
| 61 | named <filename>build</filename> in your current working directory. |
| 62 | A common practice is to use a different Build Directory for different targets. |
| 63 | For example, <filename>~/build/x86</filename> for a <filename>qemux86</filename> |
| 64 | target, and <filename>~/build/arm</filename> for a <filename>qemuarm</filename> target. |
| 65 | </para> |
| 66 | |
| 67 | <para> |
| 68 | Once the build environment is set up, you can build a target using: |
| 69 | <literallayout class='monospaced'> |
| 70 | $ bitbake <replaceable>target</replaceable> |
| 71 | </literallayout> |
Brad Bishop | 6e60e8b | 2018-02-01 10:27:11 -0500 | [diff] [blame^] | 72 | <note> |
| 73 | <para> |
| 74 | If you experience a build error due to resources |
| 75 | temporarily being unavailable and it appears you |
| 76 | should not be having this issue, it might be due |
| 77 | to the combination of a 4.3+ Linux kernel and |
| 78 | <filename>systemd</filename> version 228+ |
| 79 | (i.e. see this |
| 80 | <ulink url='http://unix.stackexchange.com/questions/253903/creating-threads-fails-with-resource-temporarily-unavailable-with-4-3-kernel'>link</ulink> |
| 81 | for information). |
| 82 | </para> |
| 83 | |
| 84 | <para> |
| 85 | To work around this issue, you can try either |
| 86 | of the following: |
| 87 | <itemizedlist> |
| 88 | <listitem><para> |
| 89 | Try the build again. |
| 90 | </para></listitem> |
| 91 | <listitem><para> |
| 92 | Modify the "DefaultTasksMax" |
| 93 | <filename>systemd</filename> parameter |
| 94 | by uncommenting it and setting it to |
| 95 | "infinity". |
| 96 | You can find this parameter in the |
| 97 | <filename>system.conf</filename> file |
| 98 | located in |
| 99 | <filename>/etc/systemd</filename> |
| 100 | on most systems. |
| 101 | </para></listitem> |
| 102 | </itemizedlist> |
| 103 | </para> |
| 104 | </note> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 105 | </para> |
| 106 | |
| 107 | <para> |
| 108 | The <replaceable>target</replaceable> is the name of the recipe you want to build. |
| 109 | Common targets are the images in <filename>meta/recipes-core/images</filename>, |
| 110 | <filename>meta/recipes-sato/images</filename>, etc. all found in the |
| 111 | <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>. |
| 112 | Or, the target can be the name of a recipe for a specific piece of software such as |
| 113 | BusyBox. |
| 114 | For more details about the images the OpenEmbedded build system supports, see the |
| 115 | "<link linkend="ref-images">Images</link>" chapter. |
| 116 | </para> |
| 117 | |
| 118 | <note> |
| 119 | Building an image without GNU General Public License Version |
| 120 | 3 (GPLv3), or similarly licensed, components is supported for |
| 121 | only minimal and base images. |
| 122 | See the "<link linkend='ref-images'>Images</link>" chapter for more information. |
| 123 | </note> |
| 124 | </section> |
| 125 | |
| 126 | <section id='building-an-image-using-gpl-components'> |
| 127 | <title>Building an Image Using GPL Components</title> |
| 128 | |
| 129 | <para> |
| 130 | When building an image using GPL components, you need to maintain your original |
| 131 | settings and not switch back and forth applying different versions of the GNU |
| 132 | General Public License. |
| 133 | If you rebuild using different versions of GPL, dependency errors might occur |
| 134 | due to some components not being rebuilt. |
| 135 | </para> |
| 136 | </section> |
| 137 | </section> |
| 138 | |
| 139 | <section id='usingpoky-install'> |
| 140 | <title>Installing and Using the Result</title> |
| 141 | |
| 142 | <para> |
| 143 | Once an image has been built, it often needs to be installed. |
| 144 | The images and kernels built by the OpenEmbedded build system are placed in the |
| 145 | <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink> in |
| 146 | <filename class="directory">tmp/deploy/images</filename>. |
| 147 | For information on how to run pre-built images such as <filename>qemux86</filename> |
| 148 | and <filename>qemuarm</filename>, see the |
Patrick Williams | d8c66bc | 2016-06-20 12:57:21 -0500 | [diff] [blame] | 149 | <ulink url='&YOCTO_DOCS_SDK_URL;#sdk-manual'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 150 | For information about how to install these images, see the documentation for your |
| 151 | particular board or machine. |
| 152 | </para> |
| 153 | </section> |
| 154 | |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 155 | <section id='usingpoky-debugging-tools-and-techniques'> |
| 156 | <title>Debugging Tools and Techniques</title> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 157 | |
| 158 | <para> |
| 159 | The exact method for debugging build failures depends on the nature of |
| 160 | the problem and on the system's area from which the bug originates. |
| 161 | Standard debugging practices such as comparison against the last |
| 162 | known working version with examination of the changes and the |
| 163 | re-application of steps to identify the one causing the problem are |
| 164 | valid for the Yocto Project just as they are for any other system. |
| 165 | Even though it is impossible to detail every possible potential failure, |
| 166 | this section provides some general tips to aid in debugging. |
| 167 | </para> |
| 168 | |
| 169 | <para> |
| 170 | A useful feature for debugging is the error reporting tool. |
| 171 | Configuring the Yocto Project to use this tool causes the |
| 172 | OpenEmbedded build system to produce error reporting commands as |
| 173 | part of the console output. |
| 174 | You can enter the commands after the build completes |
| 175 | to log error information |
| 176 | into a common database, that can help you figure out what might be |
| 177 | going wrong. |
| 178 | For information on how to enable and use this feature, see the |
| 179 | "<ulink url='&YOCTO_DOCS_DEV_URL;#using-the-error-reporting-tool'>Using the Error Reporting Tool</ulink>" |
| 180 | section in the Yocto Project Development Manual. |
| 181 | </para> |
| 182 | |
| 183 | <para> |
| 184 | For discussions on debugging, see the |
Patrick Williams | d8c66bc | 2016-06-20 12:57:21 -0500 | [diff] [blame] | 185 | "<ulink url='&YOCTO_DOCS_DEV_URL;#platdev-gdb-remotedebug'>Debugging With the GNU Project Debugger (GDB) Remotely</ulink>" section |
| 186 | in the Yocto Project Developer's Manual |
| 187 | and the |
| 188 | "<ulink url='&YOCTO_DOCS_SDK_URL;#adt-eclipse'>Working within Eclipse</ulink>" |
| 189 | section in the Yocto Project Software Development Kit (SDK) Developer's Guide. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 190 | </para> |
| 191 | |
| 192 | <note> |
| 193 | The remainder of this section presents many examples of the |
| 194 | <filename>bitbake</filename> command. |
| 195 | You can learn about BitBake by reading the |
| 196 | <ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual'>BitBake User Manual</ulink>. |
| 197 | </note> |
| 198 | |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 199 | <section id='usingpoky-debugging-viewing-logs-from-failed-tasks'> |
| 200 | <title>Viewing Logs from Failed Tasks</title> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 201 | |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 202 | <para> |
| 203 | You can find the log for a task in the file |
| 204 | <filename>${</filename><link linkend='var-WORKDIR'><filename>WORKDIR</filename></link><filename>}/temp/log.do_</filename><replaceable>taskname</replaceable>. |
| 205 | For example, the log for the |
| 206 | <link linkend='ref-tasks-compile'><filename>do_compile</filename></link> |
| 207 | task of the QEMU minimal image for the x86 machine |
| 208 | (<filename>qemux86</filename>) might be in |
| 209 | <filename>tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_compile</filename>. |
| 210 | To see the commands |
| 211 | <ulink url='&YOCTO_DOCS_DEV_URL;#bitbake-term'>BitBake</ulink> ran |
| 212 | to generate a log, look at the corresponding |
| 213 | <filename>run.do_</filename><replaceable>taskname</replaceable> |
| 214 | file in the same directory. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 215 | </para> |
| 216 | |
| 217 | <para> |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 218 | <filename>log.do_</filename><replaceable>taskname</replaceable> and |
| 219 | <filename>run.do_</filename><replaceable>taskname</replaceable> |
| 220 | are actually symbolic links to |
| 221 | <filename>log.do_</filename><replaceable>taskname</replaceable><filename>.</filename><replaceable>pid</replaceable> |
| 222 | and |
| 223 | <filename>log.run_</filename><replaceable>taskname</replaceable><filename>.</filename><replaceable>pid</replaceable>, |
| 224 | where <replaceable>pid</replaceable> is the PID the task had when |
| 225 | it ran. |
| 226 | The symlinks always point to the files corresponding to the most |
| 227 | recent run. |
| 228 | </para> |
| 229 | </section> |
| 230 | |
| 231 | <section id='usingpoky-debugging-viewing-variable-values'> |
| 232 | <title>Viewing Variable Values</title> |
| 233 | <para> |
| 234 | BitBake's <filename>-e</filename> option is used to display |
| 235 | variable values after parsing. |
| 236 | The following command displays the variable values after the |
| 237 | configuration files (i.e. <filename>local.conf</filename>, |
| 238 | <filename>bblayers.conf</filename>, |
| 239 | <filename>bitbake.conf</filename> and so forth) have been |
| 240 | parsed: |
| 241 | <literallayout class='monospaced'> |
| 242 | $ bitbake -e |
| 243 | </literallayout> |
| 244 | The following command displays variable values after a specific |
| 245 | recipe has been parsed. |
| 246 | The variables include those from the configuration as well: |
| 247 | <literallayout class='monospaced'> |
| 248 | $ bitbake -e recipename |
| 249 | </literallayout> |
| 250 | <note><para> |
| 251 | Each recipe has its own private set of variables (datastore). |
| 252 | Internally, after parsing the configuration, a copy of the |
| 253 | resulting datastore is made prior to parsing each recipe. |
| 254 | This copying implies that variables set in one recipe will |
| 255 | not be visible to other recipes.</para> |
| 256 | |
| 257 | <para>Likewise, each task within a recipe gets a private |
| 258 | datastore based on the recipe datastore, which means that |
| 259 | variables set within one task will not be visible to |
| 260 | other tasks.</para> |
| 261 | </note> |
| 262 | </para> |
| 263 | |
| 264 | <para> |
| 265 | In the output of <filename>bitbake -e</filename>, each variable is |
| 266 | preceded by a description of how the variable got its value, |
| 267 | including temporary values that were later overriden. |
| 268 | This description also includes variable flags (varflags) set on |
| 269 | the variable. |
| 270 | The output can be very helpful during debugging. |
| 271 | </para> |
| 272 | |
| 273 | <para> |
| 274 | Variables that are exported to the environment are preceded by |
| 275 | <filename>export</filename> in the output of |
| 276 | <filename>bitbake -e</filename>. |
| 277 | See the following example: |
| 278 | <literallayout class='monospaced'> |
| 279 | export CC="i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/ulf/poky/build/tmp/sysroots/qemux86" |
| 280 | </literallayout> |
| 281 | </para> |
| 282 | |
| 283 | <para> |
| 284 | In addition to variable values, the output of the |
| 285 | <filename>bitbake -e</filename> and |
| 286 | <filename>bitbake -e</filename> <replaceable>recipe</replaceable> |
| 287 | commands includes the following information: |
| 288 | <itemizedlist> |
| 289 | <listitem><para> |
| 290 | The output starts with a tree listing all configuration |
| 291 | files and classes included globally, recursively listing |
| 292 | the files they include or inherit in turn. |
| 293 | Much of the behavior of the OpenEmbedded build system |
| 294 | (including the behavior of the |
| 295 | <link linkend='normal-recipe-build-tasks'>normal recipe build tasks</link>) |
| 296 | is implemented in the |
| 297 | <link linkend='ref-classes-base'><filename>base</filename></link> |
| 298 | class and the classes it inherits, rather than being built |
| 299 | into BitBake itself. |
| 300 | </para></listitem> |
| 301 | <listitem><para> |
| 302 | After the variable values, all functions appear in the |
| 303 | output. |
| 304 | For shell functions, variables referenced within the |
| 305 | function body are expanded. |
| 306 | If a function has been modified using overrides or |
| 307 | using override-style operators like |
| 308 | <filename>_append</filename> and |
| 309 | <filename>_prepend</filename>, then the final assembled |
| 310 | function body appears in the output. |
| 311 | </para></listitem> |
| 312 | </itemizedlist> |
| 313 | </para> |
| 314 | </section> |
| 315 | |
| 316 | <section id='viewing-package-information-with-oe-pkgdata-util'> |
| 317 | <title>Viewing Package Information with <filename>oe-pkgdata-util</filename></title> |
| 318 | |
| 319 | <para> |
| 320 | You can use the <filename>oe-pkgdata-util</filename> command-line |
| 321 | utility to query |
| 322 | <link linkend='var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></link> |
| 323 | and display various package-related information. |
| 324 | When you use the utility, you must use it to view information |
| 325 | on packages that have already been built. |
| 326 | </para> |
| 327 | |
| 328 | <para> |
| 329 | Following are a few of the available |
| 330 | <filename>oe-pkgdata-util</filename> subcommands. |
| 331 | <note> |
| 332 | You can use the standard * and ? globbing wildcards as part of |
| 333 | package names and paths. |
| 334 | </note> |
| 335 | <itemizedlist> |
| 336 | <listitem><para> |
| 337 | <filename>oe-pkgdata-util list-pkgs [</filename><replaceable>pattern</replaceable><filename>]</filename>: |
| 338 | Lists all packages that have been built, optionally |
| 339 | limiting the match to packages that match |
| 340 | <replaceable>pattern</replaceable>. |
| 341 | </para></listitem> |
| 342 | <listitem><para> |
| 343 | <filename>oe-pkgdata-util list-pkg-files </filename><replaceable>package</replaceable><filename> ...</filename>: |
| 344 | Lists the files and directories contained in the given |
| 345 | packages. |
| 346 | <note> |
| 347 | <para> |
| 348 | A different way to view the contents of a package is |
| 349 | to look at the |
| 350 | <filename>${</filename><link linkend='var-WORKDIR'><filename>WORKDIR</filename></link><filename>}/packages-split</filename> |
| 351 | directory of the recipe that generates the |
| 352 | package. |
| 353 | This directory is created by the |
| 354 | <link linkend='ref-tasks-package'><filename>do_package</filename></link> |
| 355 | task and has one subdirectory for each package the |
| 356 | recipe generates, which contains the files stored in |
| 357 | that package.</para> |
| 358 | <para> |
| 359 | If you want to inspect the |
| 360 | <filename>${WORKDIR}/packages-split</filename> |
| 361 | directory, make sure that |
| 362 | <link linkend='ref-classes-rm-work'><filename>rm_work</filename></link> |
| 363 | is not enabled when you build the recipe. |
| 364 | </para> |
| 365 | </note> |
| 366 | </para></listitem> |
| 367 | <listitem><para> |
| 368 | <filename>oe-pkgdata-util find-path </filename><replaceable>path</replaceable><filename> ...</filename>: |
| 369 | Lists the names of the packages that contain the given |
| 370 | paths. |
| 371 | For example, the following tells us that |
| 372 | <filename>/usr/share/man/man1/make.1</filename> |
| 373 | is contained in the <filename>make-doc</filename> |
| 374 | package: |
| 375 | <literallayout class='monospaced'> |
| 376 | $ oe-pkgdata-util find-path /usr/share/man/man1/make.1 |
| 377 | make-doc: /usr/share/man/man1/make.1 |
| 378 | </literallayout> |
| 379 | </para></listitem> |
| 380 | <listitem><para> |
| 381 | <filename>oe-pkgdata-util lookup-recipe </filename><replaceable>package</replaceable><filename> ...</filename>: |
| 382 | Lists the name of the recipes that |
| 383 | produce the given packages. |
| 384 | </para></listitem> |
| 385 | </itemizedlist> |
| 386 | </para> |
| 387 | |
| 388 | <para> |
| 389 | For more information on the <filename>oe-pkgdata-util</filename> |
| 390 | command, use the help facility: |
| 391 | <literallayout class='monospaced'> |
| 392 | $ oe-pkgdata-util ‐‐help |
| 393 | $ oe-pkgdata-util <replaceable>subcommand</replaceable> --help |
| 394 | </literallayout> |
| 395 | </para> |
| 396 | </section> |
| 397 | |
| 398 | <section id='usingpoky-viewing-dependencies-between-recipes-and-tasks'> |
| 399 | <title>Viewing Dependencies Between Recipes and Tasks</title> |
| 400 | |
| 401 | <para> |
| 402 | Sometimes it can be hard to see why BitBake wants to build other |
| 403 | recipes before the one you have specified. |
| 404 | Dependency information can help you understand why a recipe is |
| 405 | built. |
| 406 | </para> |
| 407 | |
| 408 | <para> |
| 409 | To generate dependency information for a recipe, run the following |
| 410 | command: |
| 411 | <literallayout class='monospaced'> |
| 412 | $ bitbake -g <replaceable>recipename</replaceable> |
| 413 | </literallayout> |
| 414 | This command writes the following files in the current directory: |
| 415 | <itemizedlist> |
| 416 | <listitem><para> |
| 417 | <filename>pn-buildlist</filename>: A list of |
| 418 | recipes/targets involved in building |
| 419 | <replaceable>recipename</replaceable>. |
| 420 | "Involved" here means that at least one task from the |
| 421 | recipe needs to run when building |
| 422 | <replaceable>recipename</replaceable> from scratch. |
| 423 | Targets that are in |
| 424 | <link linkend='var-ASSUME_PROVIDED'><filename>ASSUME_PROVIDED</filename></link> |
| 425 | are not listed. |
| 426 | </para></listitem> |
| 427 | <listitem><para> |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 428 | <filename>task-depends.dot</filename>: A graph showing |
| 429 | dependencies between tasks. |
| 430 | </para></listitem> |
| 431 | </itemizedlist> |
| 432 | </para> |
| 433 | |
| 434 | <para> |
| 435 | The graphs are in |
| 436 | <ulink url='https://en.wikipedia.org/wiki/DOT_%28graph_description_language%29'>DOT</ulink> |
| 437 | format and can be converted to images (e.g. using the |
| 438 | <filename>dot</filename> tool from |
| 439 | <ulink url='http://www.graphviz.org/'>Graphviz</ulink>). |
| 440 | <note><title>Notes</title> |
| 441 | <itemizedlist> |
| 442 | <listitem><para> |
| 443 | DOT files use a plain text format. |
| 444 | The graphs generated using the |
| 445 | <filename>bitbake -g</filename> command are often so |
| 446 | large as to be difficult to read without special |
| 447 | pruning (e.g. with Bitbake's |
| 448 | <filename>-I</filename> option) and processing. |
| 449 | Despite the form and size of the graphs, the |
| 450 | corresponding <filename>.dot</filename> files can still |
| 451 | be possible to read and provide useful information. |
| 452 | </para> |
| 453 | |
| 454 | <para>As an example, the |
| 455 | <filename>task-depends.dot</filename> file contains |
| 456 | lines such as the following: |
| 457 | <literallayout class='monospaced'> |
| 458 | "libxslt.do_configure" -> "libxml2.do_populate_sysroot" |
| 459 | </literallayout> |
| 460 | The above example line reveals that the |
| 461 | <link linkend='ref-tasks-configure'><filename>do_configure</filename></link> |
| 462 | task in <filename>libxslt</filename> depends on the |
| 463 | <link linkend='ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></link> |
| 464 | task in <filename>libxml2</filename>, which is a normal |
| 465 | <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link> |
| 466 | dependency between the two recipes. |
| 467 | </para></listitem> |
| 468 | <listitem><para> |
| 469 | For an example of how <filename>.dot</filename> files |
| 470 | can be processed, see the |
| 471 | <filename>scripts/contrib/graph-tool</filename> Python |
| 472 | script, which finds and displays paths between graph |
| 473 | nodes. |
| 474 | </para></listitem> |
| 475 | </itemizedlist> |
| 476 | </note> |
| 477 | </para> |
| 478 | |
| 479 | <para> |
| 480 | You can use a different method to view dependency information |
| 481 | by using the following command: |
| 482 | <literallayout class='monospaced'> |
Brad Bishop | 6e60e8b | 2018-02-01 10:27:11 -0500 | [diff] [blame^] | 483 | $ bitbake -g -u taskexp <replaceable>recipename</replaceable> |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 484 | </literallayout> |
| 485 | This command displays a GUI window from which you can view |
| 486 | build-time and runtime dependencies for the recipes involved in |
| 487 | building <replaceable>recipename</replaceable>. |
| 488 | </para> |
| 489 | </section> |
| 490 | |
| 491 | <section id='usingpoky-viewing-task-variable-dependencies'> |
| 492 | <title>Viewing Task Variable Dependencies</title> |
| 493 | |
| 494 | <para> |
| 495 | As mentioned in the |
| 496 | "<ulink url='&YOCTO_DOCS_BB_URL;#checksums'>Checksums (Signatures)</ulink>" |
| 497 | section of the BitBake User Manual, BitBake tries to automatically |
| 498 | determine what variables a task depends on so that it can rerun |
| 499 | the task if any values of the variables change. |
| 500 | This determination is usually reliable. |
| 501 | However, if you do things like construct variable names at runtime, |
| 502 | then you might have to manually declare dependencies on those |
| 503 | variables using <filename>vardeps</filename> as described in the |
| 504 | "<ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'>Variable Flags</ulink>" |
| 505 | section of the BitBake User Manual. |
| 506 | </para> |
| 507 | |
| 508 | <para> |
| 509 | If you are unsure whether a variable dependency is being picked up |
| 510 | automatically for a given task, you can list the variable |
| 511 | dependencies BitBake has determined by doing the following: |
| 512 | <orderedlist> |
| 513 | <listitem><para> |
| 514 | Build the recipe containing the task: |
| 515 | <literallayout class='monospaced'> |
| 516 | $ bitbake <replaceable>recipename</replaceable> |
| 517 | </literallayout> |
| 518 | </para></listitem> |
| 519 | <listitem><para> |
| 520 | Inside the |
| 521 | <link linkend='var-STAMPS_DIR'><filename>STAMPS_DIR</filename></link> |
| 522 | directory, find the signature data |
| 523 | (<filename>sigdata</filename>) file that corresponds to the |
| 524 | task. |
| 525 | The <filename>sigdata</filename> files contain a pickled |
| 526 | Python database of all the metadata that went into creating |
| 527 | the input checksum for the task. |
| 528 | As an example, for the |
| 529 | <link linkend='ref-tasks-fetch'><filename>do_fetch</filename></link> |
| 530 | task of the <filename>db</filename> recipe, the |
| 531 | <filename>sigdata</filename> file might be found in the |
| 532 | following location: |
| 533 | <literallayout class='monospaced'> |
| 534 | ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1 |
| 535 | </literallayout> |
| 536 | For tasks that are accelerated through the shared state |
| 537 | (<link linkend='shared-state-cache'>sstate</link>) |
| 538 | cache, an additional <filename>siginfo</filename> file is |
| 539 | written into |
| 540 | <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> |
| 541 | along with the cached task output. |
| 542 | The <filename>siginfo</filename> files contain exactly the |
| 543 | same information as <filename>sigdata</filename> files. |
| 544 | </para></listitem> |
| 545 | <listitem><para> |
| 546 | Run <filename>bitbake-dumpsig</filename> on the |
| 547 | <filename>sigdata</filename> or |
| 548 | <filename>siginfo</filename> file. |
| 549 | Here is an example: |
| 550 | <literallayout class='monospaced'> |
| 551 | $ bitbake-dumpsig ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1 |
| 552 | </literallayout> |
| 553 | In the output of the above command, you will find a line |
| 554 | like the following, which lists all the (inferred) variable |
| 555 | dependencies for the task. |
| 556 | This list also includes indirect dependencies from |
| 557 | variables depending on other variables, recursively. |
| 558 | <literallayout class='monospaced'> |
| 559 | Task dependencies: ['PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]', 'base_do_fetch'] |
| 560 | </literallayout> |
| 561 | <note> |
| 562 | Functions (e.g. <filename>base_do_fetch</filename>) |
| 563 | also count as variable dependencies. |
| 564 | These functions in turn depend on the variables they |
| 565 | reference. |
| 566 | </note> |
| 567 | The output of <filename>bitbake-dumpsig</filename> also includes |
| 568 | the value each variable had, a list of dependencies for each |
| 569 | variable, and |
| 570 | <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_HASHBASE_WHITELIST'><filename>BB_HASHBASE_WHITELIST</filename></ulink> |
| 571 | information. |
| 572 | </para></listitem> |
| 573 | </orderedlist> |
| 574 | </para> |
| 575 | |
| 576 | <para> |
| 577 | There is also a <filename>bitbake-diffsigs</filename> command for |
| 578 | comparing two <filename>siginfo</filename> or |
| 579 | <filename>sigdata</filename> files. |
| 580 | This command can be helpful when trying to figure out what changed |
| 581 | between two versions of a task. |
| 582 | If you call <filename>bitbake-diffsigs</filename> with just one |
| 583 | file, the command behaves like |
| 584 | <filename>bitbake-dumpsig</filename>. |
| 585 | </para> |
| 586 | |
| 587 | <para> |
| 588 | You can also use BitBake to dump out the signature construction |
| 589 | information without executing tasks by using either of the |
| 590 | following BitBake command-line options: |
| 591 | <literallayout class='monospaced'> |
| 592 | ‐‐dump-signatures=<replaceable>SIGNATURE_HANDLER</replaceable> |
| 593 | -S <replaceable>SIGNATURE_HANDLER</replaceable> |
| 594 | </literallayout> |
| 595 | <note> |
| 596 | Two common values for |
| 597 | <replaceable>SIGNATURE_HANDLER</replaceable> are "none" and |
| 598 | "printdiff", which dump only the signature or compare the |
| 599 | dumped signature with the cached one, respectively. |
| 600 | </note> |
| 601 | Using BitBake with either of these options causes BitBake to dump |
| 602 | out <filename>sigdata</filename> files in the |
| 603 | <filename>stamps</filename> directory for every task it would have |
| 604 | executed instead of building the specified target package. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 605 | </para> |
| 606 | </section> |
| 607 | |
| 608 | <section id='usingpoky-debugging-taskrunning'> |
| 609 | <title>Running Specific Tasks</title> |
| 610 | |
| 611 | <para> |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 612 | Any given recipe consists of a set of tasks. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 613 | The standard BitBake behavior in most cases is: |
| 614 | <filename>do_fetch</filename>, |
| 615 | <filename>do_unpack</filename>, |
| 616 | <filename>do_patch</filename>, <filename>do_configure</filename>, |
| 617 | <filename>do_compile</filename>, <filename>do_install</filename>, |
| 618 | <filename>do_package</filename>, |
| 619 | <filename>do_package_write_*</filename>, and |
| 620 | <filename>do_build</filename>. |
| 621 | The default task is <filename>do_build</filename> and any tasks |
| 622 | on which it depends build first. |
| 623 | Some tasks, such as <filename>do_devshell</filename>, are not part |
| 624 | of the default build chain. |
| 625 | If you wish to run a task that is not part of the default build |
| 626 | chain, you can use the <filename>-c</filename> option in BitBake. |
| 627 | Here is an example: |
| 628 | <literallayout class='monospaced'> |
| 629 | $ bitbake matchbox-desktop -c devshell |
| 630 | </literallayout> |
| 631 | </para> |
| 632 | |
| 633 | <para> |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 634 | The <filename>-c</filename> option respects task dependencies, |
| 635 | which means that all other tasks (including tasks from other |
| 636 | recipes) that the specified task depends on will be run before the |
| 637 | task. |
| 638 | Even when you manually specify a task to run with |
| 639 | <filename>-c</filename>, BitBake will only run the task if it |
| 640 | considers it "out of date". |
| 641 | See the |
| 642 | "<link linkend='stamp-files-and-the-rerunning-of-tasks'>Stamp Files and the Rerunning of Tasks</link>" |
| 643 | section for how BitBake determines whether a task is "out of date". |
| 644 | </para> |
| 645 | |
| 646 | <para> |
| 647 | If you want to force an up-to-date task to be rerun (e.g. |
| 648 | because you made manual modifications to the recipe's |
| 649 | <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link> |
| 650 | that you want to try out), then you can use the |
| 651 | <filename>-f</filename> option. |
| 652 | <note> |
| 653 | The reason <filename>-f</filename> is never required when |
| 654 | running the |
| 655 | <link linkend='ref-tasks-devshell'><filename>do_devshell</filename></link> |
| 656 | task is because the |
| 657 | <filename>[</filename><ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'><filename>nostamp</filename></ulink><filename>]</filename> |
| 658 | variable flag is already set for the task. |
| 659 | </note> |
| 660 | The following example shows one way you can use the |
| 661 | <filename>-f</filename> option: |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 662 | <literallayout class='monospaced'> |
| 663 | $ bitbake matchbox-desktop |
| 664 | . |
| 665 | . |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 666 | make some changes to the source code in the work directory |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 667 | . |
| 668 | . |
| 669 | $ bitbake matchbox-desktop -c compile -f |
| 670 | $ bitbake matchbox-desktop |
| 671 | </literallayout> |
| 672 | </para> |
| 673 | |
| 674 | <para> |
| 675 | This sequence first builds and then recompiles |
| 676 | <filename>matchbox-desktop</filename>. |
| 677 | The last command reruns all tasks (basically the packaging tasks) |
| 678 | after the compile. |
| 679 | BitBake recognizes that the <filename>do_compile</filename> |
| 680 | task was rerun and therefore understands that the other tasks |
| 681 | also need to be run again. |
| 682 | </para> |
| 683 | |
| 684 | <para> |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 685 | Another, shorter way to rerun a task and all |
| 686 | <link linkend='normal-recipe-build-tasks'>normal recipe build tasks</link> |
| 687 | that depend on it is to use the <filename>-C</filename> |
| 688 | option. |
| 689 | <note> |
| 690 | This option is upper-cased and is separate from the |
| 691 | <filename>-c</filename> option, which is lower-cased. |
| 692 | </note> |
| 693 | Using this option invalidates the given task and then runs the |
| 694 | <link linkend='ref-tasks-build'><filename>do_build</filename></link> |
| 695 | task, which is the default task if no task is given, and the |
| 696 | tasks on which it depends. |
| 697 | You could replace the final two commands in the previous example |
| 698 | with the following single command: |
| 699 | <literallayout class='monospaced'> |
| 700 | $ bitbake matchbox-desktop -C compile |
| 701 | </literallayout> |
| 702 | Internally, the <filename>-f</filename> and |
| 703 | <filename>-C</filename> options work by tainting (modifying) the |
| 704 | input checksum of the specified task. |
| 705 | This tainting indirectly causes the task and its |
| 706 | dependent tasks to be rerun through the normal task dependency |
| 707 | mechanisms. |
| 708 | <note> |
| 709 | BitBake explicitly keeps track of which tasks have been |
| 710 | tainted in this fashion, and will print warnings such as the |
| 711 | following for builds involving such tasks: |
| 712 | <literallayout class='monospaced'> |
| 713 | WARNING: /home/ulf/poky/meta/recipes-sato/matchbox-desktop/matchbox-desktop_2.1.bb.do_compile is tainted from a forced run |
| 714 | </literallayout> |
| 715 | The purpose of the warning is to let you know that the work |
| 716 | directory and build output might not be in the clean state they |
| 717 | would be in for a "normal" build, depending on what actions |
| 718 | you took. |
| 719 | To get rid of such warnings, you can remove the work directory |
| 720 | and rebuild the recipe, as follows: |
| 721 | <literallayout class='monospaced'> |
| 722 | $ bitbake matchbox-desktop -c clean |
| 723 | $ bitbake matchbox-desktop |
| 724 | </literallayout> |
| 725 | </note> |
| 726 | </para> |
| 727 | |
| 728 | <para> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 729 | You can view a list of tasks in a given package by running the |
| 730 | <filename>do_listtasks</filename> task as follows: |
| 731 | <literallayout class='monospaced'> |
| 732 | $ bitbake matchbox-desktop -c listtasks |
| 733 | </literallayout> |
| 734 | The results appear as output to the console and are also in the |
| 735 | file <filename>${WORKDIR}/temp/log.do_listtasks</filename>. |
| 736 | </para> |
| 737 | </section> |
| 738 | |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 739 | <section id='usingpoky-debugging-bitbake'> |
| 740 | <title>General BitBake Problems</title> |
| 741 | |
| 742 | <para> |
| 743 | You can see debug output from BitBake by using the <filename>-D</filename> option. |
| 744 | The debug output gives more information about what BitBake |
| 745 | is doing and the reason behind it. |
| 746 | Each <filename>-D</filename> option you use increases the logging level. |
| 747 | The most common usage is <filename>-DDD</filename>. |
| 748 | </para> |
| 749 | |
| 750 | <para> |
| 751 | The output from <filename>bitbake -DDD -v</filename> <replaceable>targetname</replaceable> can reveal why |
| 752 | BitBake chose a certain version of a package or why BitBake |
| 753 | picked a certain provider. |
| 754 | This command could also help you in a situation where you think BitBake did something |
| 755 | unexpected. |
| 756 | </para> |
| 757 | </section> |
| 758 | |
| 759 | <section id='development-host-system-issues'> |
| 760 | <title>Development Host System Issues</title> |
| 761 | |
| 762 | <para> |
| 763 | Sometimes issues on the host development system can cause your |
| 764 | build to fail. |
| 765 | Following are known, host-specific problems. |
| 766 | Be sure to always consult the |
| 767 | <ulink url='&YOCTO_RELEASE_NOTES;'>Release Notes</ulink> |
| 768 | for a look at all release-related issues. |
| 769 | <itemizedlist> |
| 770 | <listitem><para><emphasis><filename>glibc-initial</filename> fails to build</emphasis>: |
| 771 | If your development host system has the unpatched |
| 772 | <filename>GNU Make 3.82</filename>, |
| 773 | the |
| 774 | <link linkend='ref-tasks-install'><filename>do_install</filename></link> |
| 775 | task fails for <filename>glibc-initial</filename> during |
| 776 | the build.</para> |
| 777 | <para>Typically, every distribution that ships |
| 778 | <filename>GNU Make 3.82</filename> as |
| 779 | the default already has the patched version. |
| 780 | However, some distributions, such as Debian, have |
| 781 | <filename>GNU Make 3.82</filename> as an option, which |
| 782 | is unpatched. |
| 783 | You will see this error on these types of distributions. |
| 784 | Switch to <filename>GNU Make 3.81</filename> or patch |
| 785 | your <filename>make</filename> to solve the problem. |
| 786 | </para></listitem> |
| 787 | </itemizedlist> |
| 788 | </para> |
| 789 | </section> |
| 790 | |
| 791 | <section id='usingpoky-debugging-buildfile'> |
| 792 | <title>Building with No Dependencies</title> |
| 793 | <para> |
| 794 | To build a specific recipe (<filename>.bb</filename> file), |
| 795 | you can use the following command form: |
| 796 | <literallayout class='monospaced'> |
| 797 | $ bitbake -b <replaceable>somepath</replaceable>/<replaceable>somerecipe</replaceable>.bb |
| 798 | </literallayout> |
| 799 | This command form does not check for dependencies. |
| 800 | Consequently, you should use it |
| 801 | only when you know existing dependencies have been met. |
| 802 | <note> |
| 803 | You can also specify fragments of the filename. |
| 804 | In this case, BitBake checks for a unique match. |
| 805 | </note> |
| 806 | </para> |
| 807 | </section> |
| 808 | |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 809 | <section id='recipe-logging-mechanisms'> |
| 810 | <title>Recipe Logging Mechanisms</title> |
| 811 | <para> |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 812 | The Yocto Project provides several logging functions for producing |
| 813 | debugging output and reporting errors and warnings. |
| 814 | For Python functions, the following logging functions exist. |
| 815 | All of these functions log to |
| 816 | <filename>${T}/log.do_</filename><replaceable>task</replaceable>, |
| 817 | and can also log to standard output (stdout) with the right |
| 818 | settings: |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 819 | <itemizedlist> |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 820 | <listitem><para> |
| 821 | <filename>bb.plain(</filename><replaceable>msg</replaceable><filename>)</filename>: |
| 822 | Writes <replaceable>msg</replaceable> as is to the log while |
| 823 | also logging to stdout. |
| 824 | </para></listitem> |
| 825 | <listitem><para> |
| 826 | <filename>bb.note(</filename><replaceable>msg</replaceable><filename>)</filename>: |
| 827 | Writes "NOTE: <replaceable>msg</replaceable>" to the log. |
| 828 | Also logs to stdout if BitBake is called with "-v". |
| 829 | </para></listitem> |
| 830 | <listitem><para> |
| 831 | <filename>bb.debug(</filename><replaceable>level</replaceable><filename>, </filename><replaceable>msg</replaceable><filename>)</filename>: |
| 832 | Writes "DEBUG: <replaceable>msg</replaceable>" to the log. |
| 833 | Also logs to stdout if the log level is greater than or |
| 834 | equal to <replaceable>level</replaceable>. |
| 835 | See the |
| 836 | "<ulink url='&YOCTO_DOCS_BB_URL;#usage-and-syntax'>-D</ulink>" |
| 837 | option in the BitBake User Manual for more information. |
| 838 | </para></listitem> |
| 839 | <listitem><para> |
| 840 | <filename>bb.warn(</filename><replaceable>msg</replaceable><filename>)</filename>: |
| 841 | Writes "WARNING: <replaceable>msg</replaceable>" to the log |
| 842 | while also logging to stdout. |
| 843 | </para></listitem> |
| 844 | <listitem><para> |
| 845 | <filename>bb.error(</filename><replaceable>msg</replaceable><filename>)</filename>: |
| 846 | Writes "ERROR: <replaceable>msg</replaceable>" to the log |
| 847 | while also logging to stdout. |
| 848 | <note> |
| 849 | Calling this function does not cause the task to fail. |
| 850 | </note> |
| 851 | </para></listitem> |
| 852 | <listitem><para> |
| 853 | <filename>bb.fatal(</filename><replaceable>msg</replaceable><filename>)</filename>: |
| 854 | This logging function is similar to |
| 855 | <filename>bb.error(</filename><replaceable>msg</replaceable><filename>)</filename> |
| 856 | but also causes the calling task to fail. |
| 857 | <note> |
| 858 | <filename>bb.fatal()</filename> raises an exception, |
| 859 | which means you do not need to put a "return" |
| 860 | statement after the function. |
| 861 | </note> |
| 862 | </para></listitem> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 863 | </itemizedlist> |
| 864 | </para> |
| 865 | |
| 866 | <para> |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 867 | The same logging functions are also available in shell functions, |
| 868 | under the names |
| 869 | <filename>bbplain</filename>, <filename>bbnote</filename>, |
| 870 | <filename>bbdebug</filename>, <filename>bbwarn</filename>, |
| 871 | <filename>bberror</filename>, and <filename>bbfatal</filename>. |
| 872 | The |
| 873 | <link linkend='ref-classes-logging'><filename>logging</filename></link> |
| 874 | class implements these functions. |
| 875 | See that class in the |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 876 | <filename>meta/classes</filename> folder of the |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 877 | <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink> |
| 878 | for information. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 879 | </para> |
| 880 | |
| 881 | <section id='logging-with-python'> |
| 882 | <title>Logging With Python</title> |
| 883 | <para> |
| 884 | When creating recipes using Python and inserting code that handles build logs, |
| 885 | keep in mind the goal is to have informative logs while keeping the console as |
| 886 | "silent" as possible. |
| 887 | Also, if you want status messages in the log, use the "debug" loglevel. |
| 888 | </para> |
| 889 | |
| 890 | <para> |
| 891 | Following is an example written in Python. |
| 892 | The code handles logging for a function that determines the |
| 893 | number of tasks needed to be run. |
| 894 | See the |
| 895 | "<link linkend='ref-tasks-listtasks'><filename>do_listtasks</filename></link>" |
| 896 | section for additional information: |
| 897 | <literallayout class='monospaced'> |
| 898 | python do_listtasks() { |
| 899 | bb.debug(2, "Starting to figure out the task list") |
| 900 | if noteworthy_condition: |
| 901 | bb.note("There are 47 tasks to run") |
| 902 | bb.debug(2, "Got to point xyz") |
| 903 | if warning_trigger: |
| 904 | bb.warn("Detected warning_trigger, this might be a problem later.") |
| 905 | if recoverable_error: |
| 906 | bb.error("Hit recoverable_error, you really need to fix this!") |
| 907 | if fatal_error: |
| 908 | bb.fatal("fatal_error detected, unable to print the task list") |
| 909 | bb.plain("The tasks present are abc") |
| 910 | bb.debug(2, "Finished figuring out the tasklist") |
| 911 | } |
| 912 | </literallayout> |
| 913 | </para> |
| 914 | </section> |
| 915 | |
| 916 | <section id='logging-with-bash'> |
| 917 | <title>Logging With Bash</title> |
| 918 | <para> |
| 919 | When creating recipes using Bash and inserting code that handles build |
| 920 | logs, you have the same goals - informative with minimal console output. |
| 921 | The syntax you use for recipes written in Bash is similar to that of |
| 922 | recipes written in Python described in the previous section. |
| 923 | </para> |
| 924 | |
| 925 | <para> |
| 926 | Following is an example written in Bash. |
| 927 | The code logs the progress of the <filename>do_my_function</filename> function. |
| 928 | <literallayout class='monospaced'> |
| 929 | do_my_function() { |
| 930 | bbdebug 2 "Running do_my_function" |
| 931 | if [ exceptional_condition ]; then |
| 932 | bbnote "Hit exceptional_condition" |
| 933 | fi |
| 934 | bbdebug 2 "Got to point xyz" |
| 935 | if [ warning_trigger ]; then |
| 936 | bbwarn "Detected warning_trigger, this might cause a problem later." |
| 937 | fi |
| 938 | if [ recoverable_error ]; then |
| 939 | bberror "Hit recoverable_error, correcting" |
| 940 | fi |
| 941 | if [ fatal_error ]; then |
| 942 | bbfatal "fatal_error detected" |
| 943 | fi |
| 944 | bbdebug 2 "Completed do_my_function" |
| 945 | } |
| 946 | </literallayout> |
| 947 | </para> |
| 948 | </section> |
| 949 | </section> |
| 950 | |
| 951 | <section id='usingpoky-debugging-others'> |
| 952 | <title>Other Tips</title> |
| 953 | |
| 954 | <para> |
| 955 | Here are some other tips that you might find useful: |
| 956 | <itemizedlist> |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 957 | <listitem><para> |
| 958 | When adding new packages, it is worth watching for |
| 959 | undesirable items making their way into compiler command |
| 960 | lines. |
| 961 | For example, you do not want references to local system |
| 962 | files like |
| 963 | <filename>/usr/lib/</filename> or |
| 964 | <filename>/usr/include/</filename>. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 965 | </para></listitem> |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 966 | <listitem><para> |
| 967 | If you want to remove the <filename>psplash</filename> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 968 | boot splashscreen, |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 969 | add <filename>psplash=false</filename> to the kernel |
| 970 | command line. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 971 | Doing so prevents <filename>psplash</filename> from loading |
| 972 | and thus allows you to see the console. |
| 973 | It is also possible to switch out of the splashscreen by |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 974 | switching the virtual console (e.g. Fn+Left or Fn+Right |
| 975 | on a Zaurus). |
| 976 | </para></listitem> |
| 977 | <listitem><para> |
| 978 | Removing |
| 979 | <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link> |
| 980 | (usually <filename>tmp/</filename>, within the |
| 981 | <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>) |
| 982 | can often fix temporary build issues. |
| 983 | Removing <filename>TMPDIR</filename> is usually a |
| 984 | relatively cheap operation, because task output will be |
| 985 | cached in |
| 986 | <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> |
| 987 | (usually <filename>sstate-cache/</filename>, which is |
| 988 | also in the Build Directory). |
| 989 | <note> |
| 990 | Removing <filename>TMPDIR</filename> might be a |
| 991 | workaround rather than a fix. |
| 992 | Consequently, trying to determine the underlying cause |
| 993 | of an issue before removing the directory is a good |
| 994 | idea. |
| 995 | </note> |
| 996 | </para></listitem> |
| 997 | <listitem><para> |
| 998 | Understanding how a feature is used in practice within |
| 999 | existing recipes can be very helpful. |
| 1000 | It is recommended that you configure some method that |
| 1001 | allows you to quickly search through files.</para> |
| 1002 | |
| 1003 | <para>Using GNU Grep, you can use the following shell |
| 1004 | function to recursively search through common |
| 1005 | recipe-related files, skipping binary files, |
| 1006 | <filename>.git</filename> directories, and the |
| 1007 | Build Directory (assuming its name starts with |
| 1008 | "build"): |
| 1009 | <literallayout class='monospaced'> |
| 1010 | g() { |
| 1011 | grep -Ir \ |
| 1012 | --exclude-dir=.git \ |
| 1013 | --exclude-dir='build*' \ |
| 1014 | --include='*.bb*' \ |
| 1015 | --include='*.inc*' \ |
| 1016 | --include='*.conf*' \ |
| 1017 | --include='*.py*' \ |
| 1018 | "$@" |
| 1019 | } |
| 1020 | </literallayout> |
| 1021 | Following are some usage examples: |
| 1022 | <literallayout class='monospaced'> |
| 1023 | $ g FOO # Search recursively for "FOO" |
| 1024 | $ g -i foo # Search recursively for "foo", ignoring case |
| 1025 | $ g -w FOO # Search recursively for "FOO" as a word, ignoring e.g. "FOOBAR" |
| 1026 | </literallayout> |
| 1027 | If figuring out how some feature works requires a lot of |
| 1028 | searching, it might indicate that the documentation should |
| 1029 | be extended or improved. |
| 1030 | In such cases, consider filing a documentation bug using |
| 1031 | the Yocto Project implementation of |
| 1032 | <ulink url='https://bugzilla.yoctoproject.org/'>Bugzilla</ulink>. |
| 1033 | For general information on how to submit a bug against |
| 1034 | the Yocto Project, see the |
| 1035 | "<ulink url='&YOCTO_DOCS_DEV_URL;#tracking-bugs'>Tracking Bugs</ulink>" |
| 1036 | section in the Yocto Project Development Manual. |
| 1037 | <note> |
| 1038 | The manuals might not be the right place to document |
| 1039 | variables that are purely internal and have a limited |
| 1040 | scope (e.g. internal variables used to implement a |
| 1041 | single <filename>.bbclass</filename> file). |
| 1042 | </note> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 1043 | </para></listitem> |
| 1044 | </itemizedlist> |
| 1045 | </para> |
| 1046 | </section> |
| 1047 | </section> |
| 1048 | |
| 1049 | <section id='maintaining-build-output-quality'> |
| 1050 | <title>Maintaining Build Output Quality</title> |
| 1051 | |
| 1052 | <para> |
| 1053 | Many factors can influence the quality of a build. |
| 1054 | For example, if you upgrade a recipe to use a new version of an upstream software |
| 1055 | package or you experiment with some new configuration options, subtle changes |
| 1056 | can occur that you might not detect until later. |
| 1057 | Consider the case where your recipe is using a newer version of an upstream package. |
| 1058 | In this case, a new version of a piece of software might introduce an optional |
| 1059 | dependency on another library, which is auto-detected. |
| 1060 | If that library has already been built when the software is building, |
| 1061 | the software will link to the built library and that library will be pulled |
| 1062 | into your image along with the new software even if you did not want the |
| 1063 | library. |
| 1064 | </para> |
| 1065 | |
| 1066 | <para> |
| 1067 | The |
| 1068 | <link linkend='ref-classes-buildhistory'><filename>buildhistory</filename></link> |
| 1069 | class exists to help you maintain |
| 1070 | the quality of your build output. |
| 1071 | You can use the class to highlight unexpected and possibly unwanted |
| 1072 | changes in the build output. |
| 1073 | When you enable build history, it records information about the contents of |
| 1074 | each package and image and then commits that information to a local Git |
| 1075 | repository where you can examine the information. |
| 1076 | </para> |
| 1077 | |
| 1078 | <para> |
| 1079 | The remainder of this section describes the following: |
| 1080 | <itemizedlist> |
| 1081 | <listitem><para>How you can enable and disable |
| 1082 | build history</para></listitem> |
| 1083 | <listitem><para>How to understand what the build history contains |
| 1084 | </para></listitem> |
| 1085 | <listitem><para>How to limit the information used for build history |
| 1086 | </para></listitem> |
| 1087 | <listitem><para>How to examine the build history from both a |
| 1088 | command-line and web interface</para></listitem> |
| 1089 | </itemizedlist> |
| 1090 | </para> |
| 1091 | |
| 1092 | <section id='enabling-and-disabling-build-history'> |
| 1093 | <title>Enabling and Disabling Build History</title> |
| 1094 | |
| 1095 | <para> |
| 1096 | Build history is disabled by default. |
| 1097 | To enable it, add the following <filename>INHERIT</filename> |
| 1098 | statement and set the |
| 1099 | <link linkend='var-BUILDHISTORY_COMMIT'><filename>BUILDHISTORY_COMMIT</filename></link> |
| 1100 | variable to "1" at the end of your |
| 1101 | <filename>conf/local.conf</filename> file found in the |
| 1102 | <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>: |
| 1103 | <literallayout class='monospaced'> |
| 1104 | INHERIT += "buildhistory" |
| 1105 | BUILDHISTORY_COMMIT = "1" |
| 1106 | </literallayout> |
| 1107 | Enabling build history as previously described |
| 1108 | causes the build process to collect build |
| 1109 | output information and commit it to a local |
| 1110 | <ulink url='&YOCTO_DOCS_DEV_URL;#git'>Git</ulink> repository. |
| 1111 | <note> |
| 1112 | Enabling build history increases your build times slightly, |
| 1113 | particularly for images, and increases the amount of disk |
| 1114 | space used during the build. |
| 1115 | </note> |
| 1116 | </para> |
| 1117 | |
| 1118 | <para> |
| 1119 | You can disable build history by removing the previous statements |
| 1120 | from your <filename>conf/local.conf</filename> file. |
| 1121 | </para> |
| 1122 | </section> |
| 1123 | |
| 1124 | <section id='understanding-what-the-build-history-contains'> |
| 1125 | <title>Understanding What the Build History Contains</title> |
| 1126 | |
| 1127 | <para> |
| 1128 | Build history information is kept in |
| 1129 | <filename>${</filename><link linkend='var-TOPDIR'><filename>TOPDIR</filename></link><filename>}/buildhistory</filename> |
| 1130 | in the Build Directory as defined by the |
| 1131 | <link linkend='var-BUILDHISTORY_DIR'><filename>BUILDHISTORY_DIR</filename></link> |
| 1132 | variable. |
| 1133 | The following is an example abbreviated listing: |
| 1134 | <imagedata fileref="figures/buildhistory.png" align="center" width="6in" depth="4in" /> |
| 1135 | </para> |
| 1136 | |
| 1137 | <para> |
| 1138 | At the top level, there is a <filename>metadata-revs</filename> file |
| 1139 | that lists the revisions of the repositories for the layers enabled |
| 1140 | when the build was produced. |
| 1141 | The rest of the data splits into separate |
| 1142 | <filename>packages</filename>, <filename>images</filename> and |
| 1143 | <filename>sdk</filename> directories, the contents of which are |
| 1144 | described below. |
| 1145 | </para> |
| 1146 | |
| 1147 | <section id='build-history-package-information'> |
| 1148 | <title>Build History Package Information</title> |
| 1149 | |
| 1150 | <para> |
| 1151 | The history for each package contains a text file that has |
| 1152 | name-value pairs with information about the package. |
| 1153 | For example, <filename>buildhistory/packages/i586-poky-linux/busybox/busybox/latest</filename> |
| 1154 | contains the following: |
| 1155 | <literallayout class='monospaced'> |
| 1156 | PV = 1.22.1 |
| 1157 | PR = r32 |
| 1158 | RPROVIDES = |
| 1159 | RDEPENDS = glibc (>= 2.20) update-alternatives-opkg |
| 1160 | RRECOMMENDS = busybox-syslog busybox-udhcpc update-rc.d |
| 1161 | PKGSIZE = 540168 |
| 1162 | FILES = /usr/bin/* /usr/sbin/* /usr/lib/busybox/* /usr/lib/lib*.so.* \ |
| 1163 | /etc /com /var /bin/* /sbin/* /lib/*.so.* /lib/udev/rules.d \ |
| 1164 | /usr/lib/udev/rules.d /usr/share/busybox /usr/lib/busybox/* \ |
| 1165 | /usr/share/pixmaps /usr/share/applications /usr/share/idl \ |
| 1166 | /usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers |
| 1167 | FILELIST = /bin/busybox /bin/busybox.nosuid /bin/busybox.suid /bin/sh \ |
| 1168 | /etc/busybox.links.nosuid /etc/busybox.links.suid |
| 1169 | </literallayout> |
| 1170 | Most of these name-value pairs correspond to variables used |
| 1171 | to produce the package. |
| 1172 | The exceptions are <filename>FILELIST</filename>, which is the |
| 1173 | actual list of files in the package, and |
| 1174 | <filename>PKGSIZE</filename>, which is the total size of files |
| 1175 | in the package in bytes. |
| 1176 | </para> |
| 1177 | |
| 1178 | <para> |
| 1179 | There is also a file corresponding to the recipe from which the |
| 1180 | package came (e.g. |
| 1181 | <filename>buildhistory/packages/i586-poky-linux/busybox/latest</filename>): |
| 1182 | <literallayout class='monospaced'> |
| 1183 | PV = 1.22.1 |
| 1184 | PR = r32 |
| 1185 | DEPENDS = initscripts kern-tools-native update-rc.d-native \ |
| 1186 | virtual/i586-poky-linux-compilerlibs virtual/i586-poky-linux-gcc \ |
| 1187 | virtual/libc virtual/update-alternatives |
| 1188 | PACKAGES = busybox-ptest busybox-httpd busybox-udhcpd busybox-udhcpc \ |
| 1189 | busybox-syslog busybox-mdev busybox-hwclock busybox-dbg \ |
| 1190 | busybox-staticdev busybox-dev busybox-doc busybox-locale busybox |
| 1191 | </literallayout> |
| 1192 | </para> |
| 1193 | |
| 1194 | <para> |
| 1195 | Finally, for those recipes fetched from a version control |
| 1196 | system (e.g., Git), a file exists that lists source revisions |
| 1197 | that are specified in the recipe and lists the actual revisions |
| 1198 | used during the build. |
| 1199 | Listed and actual revisions might differ when |
| 1200 | <link linkend='var-SRCREV'><filename>SRCREV</filename></link> |
| 1201 | is set to |
| 1202 | <filename>${<link linkend='var-AUTOREV'>AUTOREV</link>}</filename>. |
| 1203 | Here is an example assuming |
| 1204 | <filename>buildhistory/packages/qemux86-poky-linux/linux-yocto/latest_srcrev</filename>): |
| 1205 | <literallayout class='monospaced'> |
| 1206 | # SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" |
| 1207 | SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" |
| 1208 | # SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f" |
| 1209 | SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f" |
| 1210 | </literallayout> |
| 1211 | You can use the <filename>buildhistory-collect-srcrevs</filename> |
| 1212 | command with the <filename>-a</filename> option to |
| 1213 | collect the stored <filename>SRCREV</filename> values |
| 1214 | from build history and report them in a format suitable for |
| 1215 | use in global configuration (e.g., |
| 1216 | <filename>local.conf</filename> or a distro include file) to |
| 1217 | override floating <filename>AUTOREV</filename> values to a |
| 1218 | fixed set of revisions. |
| 1219 | Here is some example output from this command: |
| 1220 | <literallayout class='monospaced'> |
| 1221 | $ buildhistory-collect-srcrevs -a |
| 1222 | # i586-poky-linux |
| 1223 | SRCREV_pn-glibc = "b8079dd0d360648e4e8de48656c5c38972621072" |
| 1224 | SRCREV_pn-glibc-initial = "b8079dd0d360648e4e8de48656c5c38972621072" |
| 1225 | SRCREV_pn-opkg-utils = "53274f087565fd45d8452c5367997ba6a682a37a" |
| 1226 | SRCREV_pn-kmod = "fd56638aed3fe147015bfa10ed4a5f7491303cb4" |
| 1227 | # x86_64-linux |
| 1228 | SRCREV_pn-gtk-doc-stub-native = "1dea266593edb766d6d898c79451ef193eb17cfa" |
| 1229 | SRCREV_pn-dtc-native = "65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf" |
| 1230 | SRCREV_pn-update-rc.d-native = "eca680ddf28d024954895f59a241a622dd575c11" |
| 1231 | SRCREV_glibc_pn-cross-localedef-native = "b8079dd0d360648e4e8de48656c5c38972621072" |
| 1232 | SRCREV_localedef_pn-cross-localedef-native = "c833367348d39dad7ba018990bfdaffaec8e9ed3" |
| 1233 | SRCREV_pn-prelink-native = "faa069deec99bf61418d0bab831c83d7c1b797ca" |
| 1234 | SRCREV_pn-opkg-utils-native = "53274f087565fd45d8452c5367997ba6a682a37a" |
| 1235 | SRCREV_pn-kern-tools-native = "23345b8846fe4bd167efdf1bd8a1224b2ba9a5ff" |
| 1236 | SRCREV_pn-kmod-native = "fd56638aed3fe147015bfa10ed4a5f7491303cb4" |
| 1237 | # qemux86-poky-linux |
| 1238 | SRCREV_machine_pn-linux-yocto = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" |
| 1239 | SRCREV_meta_pn-linux-yocto = "a227f20eff056e511d504b2e490f3774ab260d6f" |
| 1240 | # all-poky-linux |
| 1241 | SRCREV_pn-update-rc.d = "eca680ddf28d024954895f59a241a622dd575c11" |
| 1242 | </literallayout> |
| 1243 | <note> |
| 1244 | Here are some notes on using the |
| 1245 | <filename>buildhistory-collect-srcrevs</filename> command: |
| 1246 | <itemizedlist> |
| 1247 | <listitem><para>By default, only values where the |
| 1248 | <filename>SRCREV</filename> was |
| 1249 | not hardcoded (usually when <filename>AUTOREV</filename> |
| 1250 | was used) are reported. |
| 1251 | Use the <filename>-a</filename> option to see all |
| 1252 | <filename>SRCREV</filename> values. |
| 1253 | </para></listitem> |
| 1254 | <listitem><para>The output statements might not have any effect |
| 1255 | if overrides are applied elsewhere in the build system |
| 1256 | configuration. |
| 1257 | Use the <filename>-f</filename> option to add the |
| 1258 | <filename>forcevariable</filename> override to each output line |
| 1259 | if you need to work around this restriction. |
| 1260 | </para></listitem> |
| 1261 | <listitem><para>The script does apply special handling when |
| 1262 | building for multiple machines. |
| 1263 | However, the script does place a |
| 1264 | comment before each set of values that specifies |
| 1265 | which triplet to which they belong as shown above |
| 1266 | (e.g., <filename>i586-poky-linux</filename>). |
| 1267 | </para></listitem> |
| 1268 | </itemizedlist> |
| 1269 | </note> |
| 1270 | </para> |
| 1271 | </section> |
| 1272 | |
| 1273 | <section id='build-history-image-information'> |
| 1274 | <title>Build History Image Information</title> |
| 1275 | |
| 1276 | <para> |
| 1277 | The files produced for each image are as follows: |
| 1278 | <itemizedlist> |
| 1279 | <listitem><para><filename>image-files:</filename> |
| 1280 | A directory containing selected files from the root |
| 1281 | filesystem. |
| 1282 | The files are defined by |
| 1283 | <link linkend='var-BUILDHISTORY_IMAGE_FILES'><filename>BUILDHISTORY_IMAGE_FILES</filename></link>. |
| 1284 | </para></listitem> |
| 1285 | <listitem><para><filename>build-id.txt:</filename> |
| 1286 | Human-readable information about the build configuration |
| 1287 | and metadata source revisions. |
| 1288 | This file contains the full build header as printed |
| 1289 | by BitBake.</para></listitem> |
| 1290 | <listitem><para><filename>*.dot:</filename> |
| 1291 | Dependency graphs for the image that are |
| 1292 | compatible with <filename>graphviz</filename>. |
| 1293 | </para></listitem> |
| 1294 | <listitem><para><filename>files-in-image.txt:</filename> |
| 1295 | A list of files in the image with permissions, |
| 1296 | owner, group, size, and symlink information. |
| 1297 | </para></listitem> |
| 1298 | <listitem><para><filename>image-info.txt:</filename> |
| 1299 | A text file containing name-value pairs with information |
| 1300 | about the image. |
| 1301 | See the following listing example for more information. |
| 1302 | </para></listitem> |
| 1303 | <listitem><para><filename>installed-package-names.txt:</filename> |
| 1304 | A list of installed packages by name only.</para></listitem> |
| 1305 | <listitem><para><filename>installed-package-sizes.txt:</filename> |
| 1306 | A list of installed packages ordered by size. |
| 1307 | </para></listitem> |
| 1308 | <listitem><para><filename>installed-packages.txt:</filename> |
| 1309 | A list of installed packages with full package |
| 1310 | filenames.</para></listitem> |
| 1311 | </itemizedlist> |
| 1312 | <note> |
| 1313 | Installed package information is able to be gathered and |
| 1314 | produced even if package management is disabled for the final |
| 1315 | image. |
| 1316 | </note> |
| 1317 | </para> |
| 1318 | |
| 1319 | <para> |
| 1320 | Here is an example of <filename>image-info.txt</filename>: |
| 1321 | <literallayout class='monospaced'> |
| 1322 | DISTRO = poky |
| 1323 | DISTRO_VERSION = 1.7 |
| 1324 | USER_CLASSES = buildstats image-mklibs image-prelink |
| 1325 | IMAGE_CLASSES = image_types |
| 1326 | IMAGE_FEATURES = debug-tweaks |
| 1327 | IMAGE_LINGUAS = |
| 1328 | IMAGE_INSTALL = packagegroup-core-boot run-postinsts |
| 1329 | BAD_RECOMMENDATIONS = |
| 1330 | NO_RECOMMENDATIONS = |
| 1331 | PACKAGE_EXCLUDE = |
| 1332 | ROOTFS_POSTPROCESS_COMMAND = write_package_manifest; license_create_manifest; \ |
| 1333 | write_image_manifest ; buildhistory_list_installed_image ; \ |
| 1334 | buildhistory_get_image_installed ; ssh_allow_empty_password; \ |
| 1335 | postinst_enable_logging; rootfs_update_timestamp ; ssh_disable_dns_lookup ; |
| 1336 | IMAGE_POSTPROCESS_COMMAND = buildhistory_get_imageinfo ; |
| 1337 | IMAGESIZE = 6900 |
| 1338 | </literallayout> |
| 1339 | Other than <filename>IMAGESIZE</filename>, which is the |
| 1340 | total size of the files in the image in Kbytes, the |
| 1341 | name-value pairs are variables that may have influenced the |
| 1342 | content of the image. |
| 1343 | This information is often useful when you are trying to determine |
| 1344 | why a change in the package or file listings has occurred. |
| 1345 | </para> |
| 1346 | </section> |
| 1347 | |
| 1348 | <section id='using-build-history-to-gather-image-information-only'> |
| 1349 | <title>Using Build History to Gather Image Information Only</title> |
| 1350 | |
| 1351 | <para> |
| 1352 | As you can see, build history produces image information, |
| 1353 | including dependency graphs, so you can see why something |
| 1354 | was pulled into the image. |
| 1355 | If you are just interested in this information and not |
| 1356 | interested in collecting specific package or SDK information, |
| 1357 | you can enable writing only image information without |
| 1358 | any history by adding the following to your |
| 1359 | <filename>conf/local.conf</filename> file found in the |
| 1360 | <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>: |
| 1361 | <literallayout class='monospaced'> |
| 1362 | INHERIT += "buildhistory" |
| 1363 | BUILDHISTORY_COMMIT = "0" |
| 1364 | BUILDHISTORY_FEATURES = "image" |
| 1365 | </literallayout> |
| 1366 | Here, you set the |
| 1367 | <link linkend='var-BUILDHISTORY_FEATURES'><filename>BUILDHISTORY_FEATURES</filename></link> |
| 1368 | variable to use the image feature only. |
| 1369 | </para> |
| 1370 | </section> |
| 1371 | |
| 1372 | <section id='build-history-sdk-information'> |
| 1373 | <title>Build History SDK Information</title> |
Patrick Williams | d8c66bc | 2016-06-20 12:57:21 -0500 | [diff] [blame] | 1374 | |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 1375 | <para> |
| 1376 | Build history collects similar information on the contents |
Patrick Williams | d8c66bc | 2016-06-20 12:57:21 -0500 | [diff] [blame] | 1377 | of SDKs |
| 1378 | (e.g. <filename>bitbake -c populate_sdk imagename</filename>) |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 1379 | as compared to information it collects for images. |
Patrick Williams | d8c66bc | 2016-06-20 12:57:21 -0500 | [diff] [blame] | 1380 | Furthermore, this information differs depending on whether an |
| 1381 | extensible or standard SDK is being produced. |
| 1382 | </para> |
| 1383 | |
| 1384 | <para> |
| 1385 | The following list shows the files produced for SDKs: |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 1386 | <itemizedlist> |
| 1387 | <listitem><para><filename>files-in-sdk.txt:</filename> |
| 1388 | A list of files in the SDK with permissions, |
| 1389 | owner, group, size, and symlink information. |
| 1390 | This list includes both the host and target parts |
| 1391 | of the SDK. |
| 1392 | </para></listitem> |
| 1393 | <listitem><para><filename>sdk-info.txt:</filename> |
| 1394 | A text file containing name-value pairs with information |
| 1395 | about the SDK. |
| 1396 | See the following listing example for more information. |
| 1397 | </para></listitem> |
Patrick Williams | d8c66bc | 2016-06-20 12:57:21 -0500 | [diff] [blame] | 1398 | <listitem><para><filename>sstate-task-sizes.txt:</filename> |
| 1399 | A text file containing name-value pairs with information |
| 1400 | about task group sizes |
| 1401 | (e.g. <filename>do_populate_sysroot</filename> tasks |
| 1402 | have a total size). |
| 1403 | The <filename>sstate-task-sizes.txt</filename> file |
| 1404 | exists only when an extensible SDK is created. |
| 1405 | </para></listitem> |
| 1406 | <listitem><para><filename>sstate-package-sizes.txt:</filename> |
| 1407 | A text file containing name-value pairs with information |
| 1408 | for the shared-state packages and sizes in the SDK. |
| 1409 | The <filename>sstate-package-sizes.txt</filename> file |
| 1410 | exists only when an extensible SDK is created. |
| 1411 | </para></listitem> |
| 1412 | <listitem><para><filename>sdk-files:</filename> |
| 1413 | A folder that contains copies of the files mentioned in |
| 1414 | <filename>BUILDHISTORY_SDK_FILES</filename> if the |
| 1415 | files are present in the output. |
| 1416 | Additionally, the default value of |
| 1417 | <filename>BUILDHISTORY_SDK_FILES</filename> is specific |
| 1418 | to the extensible SDK although you can set it |
| 1419 | differently if you would like to pull in specific files |
| 1420 | from the standard SDK.</para> |
| 1421 | <para>The default files are |
| 1422 | <filename>conf/local.conf</filename>, |
| 1423 | <filename>conf/bblayers.conf</filename>, |
| 1424 | <filename>conf/auto.conf</filename>, |
| 1425 | <filename>conf/locked-sigs.inc</filename>, and |
| 1426 | <filename>conf/devtool.conf</filename>. |
| 1427 | Thus, for an extensible SDK, these files get copied |
| 1428 | into the <filename>sdk-files</filename> directory. |
| 1429 | </para></listitem> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 1430 | <listitem><para>The following information appears under |
| 1431 | each of the <filename>host</filename> |
| 1432 | and <filename>target</filename> directories |
| 1433 | for the portions of the SDK that run on the host and |
| 1434 | on the target, respectively: |
Patrick Williams | d8c66bc | 2016-06-20 12:57:21 -0500 | [diff] [blame] | 1435 | <note> |
| 1436 | The following files for the most part are empty |
| 1437 | when producing an extensible SDK because this |
| 1438 | type of SDK is not constructed from packages as is |
| 1439 | the standard SDK. |
| 1440 | </note> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 1441 | <itemizedlist> |
| 1442 | <listitem><para><filename>depends.dot:</filename> |
| 1443 | Dependency graph for the SDK that is |
| 1444 | compatible with <filename>graphviz</filename>. |
| 1445 | </para></listitem> |
| 1446 | <listitem><para><filename>installed-package-names.txt:</filename> |
| 1447 | A list of installed packages by name only. |
| 1448 | </para></listitem> |
| 1449 | <listitem><para><filename>installed-package-sizes.txt:</filename> |
| 1450 | A list of installed packages ordered by size. |
| 1451 | </para></listitem> |
| 1452 | <listitem><para><filename>installed-packages.txt:</filename> |
| 1453 | A list of installed packages with full package |
| 1454 | filenames.</para></listitem> |
| 1455 | </itemizedlist> |
| 1456 | </para></listitem> |
| 1457 | </itemizedlist> |
| 1458 | </para> |
| 1459 | |
| 1460 | <para> |
| 1461 | Here is an example of <filename>sdk-info.txt</filename>: |
| 1462 | <literallayout class='monospaced'> |
| 1463 | DISTRO = poky |
| 1464 | DISTRO_VERSION = 1.3+snapshot-20130327 |
| 1465 | SDK_NAME = poky-glibc-i686-arm |
| 1466 | SDK_VERSION = 1.3+snapshot |
| 1467 | SDKMACHINE = |
| 1468 | SDKIMAGE_FEATURES = dev-pkgs dbg-pkgs |
| 1469 | BAD_RECOMMENDATIONS = |
| 1470 | SDKSIZE = 352712 |
| 1471 | </literallayout> |
| 1472 | Other than <filename>SDKSIZE</filename>, which is the |
| 1473 | total size of the files in the SDK in Kbytes, the |
| 1474 | name-value pairs are variables that might have influenced the |
| 1475 | content of the SDK. |
| 1476 | This information is often useful when you are trying to |
| 1477 | determine why a change in the package or file listings |
| 1478 | has occurred. |
| 1479 | </para> |
| 1480 | </section> |
| 1481 | |
| 1482 | <section id='examining-build-history-information'> |
| 1483 | <title>Examining Build History Information</title> |
| 1484 | |
| 1485 | <para> |
| 1486 | You can examine build history output from the command line or |
| 1487 | from a web interface. |
| 1488 | </para> |
| 1489 | |
| 1490 | <para> |
| 1491 | To see any changes that have occurred (assuming you have |
| 1492 | <link linkend='var-BUILDHISTORY_COMMIT'><filename>BUILDHISTORY_COMMIT = "1"</filename></link>), |
| 1493 | you can simply |
| 1494 | use any Git command that allows you to view the history of |
| 1495 | a repository. |
| 1496 | Here is one method: |
| 1497 | <literallayout class='monospaced'> |
| 1498 | $ git log -p |
| 1499 | </literallayout> |
| 1500 | You need to realize, however, that this method does show |
| 1501 | changes that are not significant (e.g. a package's size |
| 1502 | changing by a few bytes). |
| 1503 | </para> |
| 1504 | |
| 1505 | <para> |
| 1506 | A command-line tool called <filename>buildhistory-diff</filename> |
| 1507 | does exist, though, that queries the Git repository and prints just |
| 1508 | the differences that might be significant in human-readable form. |
| 1509 | Here is an example: |
| 1510 | <literallayout class='monospaced'> |
| 1511 | $ ~/poky/poky/scripts/buildhistory-diff . HEAD^ |
| 1512 | Changes to images/qemux86_64/glibc/core-image-minimal (files-in-image.txt): |
| 1513 | /etc/anotherpkg.conf was added |
| 1514 | /sbin/anotherpkg was added |
| 1515 | * (installed-package-names.txt): |
| 1516 | * anotherpkg was added |
| 1517 | Changes to images/qemux86_64/glibc/core-image-minimal (installed-package-names.txt): |
| 1518 | anotherpkg was added |
| 1519 | packages/qemux86_64-poky-linux/v86d: PACKAGES: added "v86d-extras" |
| 1520 | * PR changed from "r0" to "r1" |
| 1521 | * PV changed from "0.1.10" to "0.1.12" |
| 1522 | packages/qemux86_64-poky-linux/v86d/v86d: PKGSIZE changed from 110579 to 144381 (+30%) |
| 1523 | * PR changed from "r0" to "r1" |
| 1524 | * PV changed from "0.1.10" to "0.1.12" |
| 1525 | </literallayout> |
Patrick Williams | c0f7c04 | 2017-02-23 20:41:17 -0600 | [diff] [blame] | 1526 | <note> |
| 1527 | The <filename>buildhistory-diff</filename> tool requires |
| 1528 | the <filename>GitPython</filename> package. |
| 1529 | Be sure to install it using Pip3 as follows: |
| 1530 | <literallayout class='monospaced'> |
| 1531 | $ pip3 install GitPython --user |
| 1532 | </literallayout> |
| 1533 | Alternatively, you can install |
| 1534 | <filename>python3-git</filename> using the appropriate |
| 1535 | distribution package manager (e.g. |
| 1536 | <filename>apt-get</filename>, <filename>dnf</filename>, or |
| 1537 | <filename>zipper</filename>). |
| 1538 | </note> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 1539 | </para> |
| 1540 | |
| 1541 | <para> |
| 1542 | To see changes to the build history using a web interface, follow |
| 1543 | the instruction in the <filename>README</filename> file here. |
| 1544 | <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/buildhistory-web/'></ulink>. |
| 1545 | </para> |
| 1546 | |
| 1547 | <para> |
| 1548 | Here is a sample screenshot of the interface: |
| 1549 | <imagedata fileref="figures/buildhistory-web.png" align="center" scalefit="1" width="130%" contentdepth="130%" /> |
| 1550 | </para> |
| 1551 | </section> |
| 1552 | </section> |
| 1553 | </section> |
| 1554 | |
| 1555 | <section id='speeding-up-the-build'> |
| 1556 | <title>Speeding Up the Build</title> |
| 1557 | |
| 1558 | <para> |
| 1559 | Build time can be an issue. |
| 1560 | By default, the build system uses simple controls to try and maximize |
| 1561 | build efficiency. |
| 1562 | In general, the default settings for all the following variables |
| 1563 | result in the most efficient build times when dealing with single |
| 1564 | socket systems (i.e. a single CPU). |
| 1565 | If you have multiple CPUs, you might try increasing the default |
| 1566 | values to gain more speed. |
| 1567 | See the descriptions in the glossary for each variable for more |
| 1568 | information: |
| 1569 | <itemizedlist> |
| 1570 | <listitem><para> |
| 1571 | <link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename>:</link> |
| 1572 | The maximum number of threads BitBake simultaneously executes. |
| 1573 | </para></listitem> |
| 1574 | <listitem><para> |
| 1575 | <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_NUMBER_PARSE_THREADS'><filename>BB_NUMBER_PARSE_THREADS</filename>:</ulink> |
| 1576 | The number of threads BitBake uses during parsing. |
| 1577 | </para></listitem> |
| 1578 | <listitem><para> |
| 1579 | <link linkend='var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename>:</link> |
| 1580 | Extra options passed to the <filename>make</filename> command |
| 1581 | during the |
| 1582 | <link linkend='ref-tasks-compile'><filename>do_compile</filename></link> |
| 1583 | task in order to specify parallel compilation on the |
| 1584 | local build host. |
| 1585 | </para></listitem> |
| 1586 | <listitem><para> |
| 1587 | <link linkend='var-PARALLEL_MAKEINST'><filename>PARALLEL_MAKEINST</filename>:</link> |
| 1588 | Extra options passed to the <filename>make</filename> command |
| 1589 | during the |
| 1590 | <link linkend='ref-tasks-install'><filename>do_install</filename></link> |
| 1591 | task in order to specify parallel installation on the |
| 1592 | local build host. |
| 1593 | </para></listitem> |
| 1594 | </itemizedlist> |
| 1595 | As mentioned, these variables all scale to the number of processor |
| 1596 | cores available on the build system. |
| 1597 | For single socket systems, this auto-scaling ensures that the build |
| 1598 | system fundamentally takes advantage of potential parallel operations |
| 1599 | during the build based on the build machine's capabilities. |
| 1600 | </para> |
| 1601 | |
| 1602 | <para> |
| 1603 | Following are additional factors that can affect build speed: |
| 1604 | <itemizedlist> |
| 1605 | <listitem><para> |
| 1606 | File system type: |
| 1607 | The file system type that the build is being performed on can |
| 1608 | also influence performance. |
| 1609 | Using <filename>ext4</filename> is recommended as compared |
| 1610 | to <filename>ext2</filename> and <filename>ext3</filename> |
| 1611 | due to <filename>ext4</filename> improved features |
| 1612 | such as extents. |
| 1613 | </para></listitem> |
| 1614 | <listitem><para> |
| 1615 | Disabling the updating of access time using |
| 1616 | <filename>noatime</filename>: |
| 1617 | The <filename>noatime</filename> mount option prevents the |
| 1618 | build system from updating file and directory access times. |
| 1619 | </para></listitem> |
| 1620 | <listitem><para> |
| 1621 | Setting a longer commit: |
| 1622 | Using the "commit=" mount option increases the interval |
| 1623 | in seconds between disk cache writes. |
| 1624 | Changing this interval from the five second default to |
| 1625 | something longer increases the risk of data loss but decreases |
| 1626 | the need to write to the disk, thus increasing the build |
| 1627 | performance. |
| 1628 | </para></listitem> |
| 1629 | <listitem><para> |
| 1630 | Choosing the packaging backend: |
| 1631 | Of the available packaging backends, IPK is the fastest. |
| 1632 | Additionally, selecting a singular packaging backend also |
| 1633 | helps. |
| 1634 | </para></listitem> |
| 1635 | <listitem><para> |
| 1636 | Using <filename>tmpfs</filename> for |
| 1637 | <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link> |
| 1638 | as a temporary file system: |
| 1639 | While this can help speed up the build, the benefits are |
| 1640 | limited due to the compiler using |
| 1641 | <filename>-pipe</filename>. |
| 1642 | The build system goes to some lengths to avoid |
| 1643 | <filename>sync()</filename> calls into the |
| 1644 | file system on the principle that if there was a significant |
| 1645 | failure, the |
| 1646 | <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink> |
| 1647 | contents could easily be rebuilt. |
| 1648 | </para></listitem> |
| 1649 | <listitem><para> |
| 1650 | Inheriting the |
| 1651 | <link linkend='ref-classes-rm-work'><filename>rm_work</filename></link> |
| 1652 | class: |
| 1653 | Inheriting this class has shown to speed up builds due to |
| 1654 | significantly lower amounts of data stored in the data |
| 1655 | cache as well as on disk. |
| 1656 | Inheriting this class also makes cleanup of |
| 1657 | <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link> |
| 1658 | faster, at the expense of being easily able to dive into the |
| 1659 | source code. |
| 1660 | File system maintainers have recommended that the fastest way |
| 1661 | to clean up large numbers of files is to reformat partitions |
| 1662 | rather than delete files due to the linear nature of partitions. |
| 1663 | This, of course, assumes you structure the disk partitions and |
| 1664 | file systems in a way that this is practical. |
| 1665 | </para></listitem> |
| 1666 | </itemizedlist> |
| 1667 | Aside from the previous list, you should keep some trade offs in |
| 1668 | mind that can help you speed up the build: |
| 1669 | <itemizedlist> |
| 1670 | <listitem><para> |
| 1671 | Remove items from |
| 1672 | <link linkend='var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></link> |
| 1673 | that you might not need. |
| 1674 | </para></listitem> |
| 1675 | <listitem><para> |
| 1676 | Exclude debug symbols and other debug information: |
| 1677 | If you do not need these symbols and other debug information, |
| 1678 | disabling the <filename>*-dbg</filename> package generation |
| 1679 | can speed up the build. |
| 1680 | You can disable this generation by setting the |
| 1681 | <link linkend='var-INHIBIT_PACKAGE_DEBUG_SPLIT'><filename>INHIBIT_PACKAGE_DEBUG_SPLIT</filename></link> |
| 1682 | variable to "1". |
| 1683 | </para></listitem> |
| 1684 | <listitem><para> |
| 1685 | Disable static library generation for recipes derived from |
| 1686 | <filename>autoconf</filename> or <filename>libtool</filename>: |
| 1687 | Following is an example showing how to disable static |
| 1688 | libraries and still provide an override to handle exceptions: |
| 1689 | <literallayout class='monospaced'> |
| 1690 | STATICLIBCONF = "--disable-static" |
| 1691 | STATICLIBCONF_sqlite3-native = "" |
| 1692 | EXTRA_OECONF += "${STATICLIBCONF}" |
| 1693 | </literallayout> |
| 1694 | <note><title>Notes</title> |
| 1695 | <itemizedlist> |
| 1696 | <listitem><para> |
| 1697 | Some recipes need static libraries in order to work |
| 1698 | correctly (e.g. <filename>pseudo-native</filename> |
| 1699 | needs <filename>sqlite3-native</filename>). |
| 1700 | Overrides, as in the previous example, account for |
| 1701 | these kinds of exceptions. |
| 1702 | </para></listitem> |
| 1703 | <listitem><para> |
| 1704 | Some packages have packaging code that assumes the |
| 1705 | presence of the static libraries. |
| 1706 | If so, you might need to exclude them as well. |
| 1707 | </para></listitem> |
| 1708 | </itemizedlist> |
| 1709 | </note> |
| 1710 | </para></listitem> |
| 1711 | </itemizedlist> |
| 1712 | </para> |
| 1713 | </section> |
| 1714 | </chapter> |
| 1715 | <!-- |
| 1716 | vim: expandtab tw=80 ts=4 |
| 1717 | --> |