Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 1 | .. SPDX-License-Identifier: CC-BY-2.5 |
| 2 | |
| 3 | ==================== |
| 4 | Syntax and Operators |
| 5 | ==================== |
| 6 | |
| 7 | | |
| 8 | |
| 9 | BitBake files have their own syntax. The syntax has similarities to |
| 10 | several other languages but also has some unique features. This section |
| 11 | describes the available syntax and operators as well as provides |
| 12 | examples. |
| 13 | |
| 14 | Basic Syntax |
| 15 | ============ |
| 16 | |
| 17 | This section provides some basic syntax examples. |
| 18 | |
| 19 | Basic Variable Setting |
| 20 | ---------------------- |
| 21 | |
| 22 | The following example sets ``VARIABLE`` to "value". This assignment |
| 23 | occurs immediately as the statement is parsed. It is a "hard" |
| 24 | assignment. :: |
| 25 | |
| 26 | VARIABLE = "value" |
| 27 | |
| 28 | As expected, if you include leading or |
| 29 | trailing spaces as part of an assignment, the spaces are retained: :: |
| 30 | |
| 31 | VARIABLE = " value" |
| 32 | VARIABLE = "value " |
| 33 | |
| 34 | Setting ``VARIABLE`` to "" sets |
| 35 | it to an empty string, while setting the variable to " " sets it to a |
| 36 | blank space (i.e. these are not the same values). :: |
| 37 | |
| 38 | VARIABLE = "" |
| 39 | VARIABLE = " " |
| 40 | |
| 41 | You can use single quotes instead of double quotes when setting a |
| 42 | variable's value. Doing so allows you to use values that contain the |
| 43 | double quote character: :: |
| 44 | |
| 45 | VARIABLE = 'I have a " in my value' |
| 46 | |
| 47 | .. note:: |
| 48 | |
| 49 | Unlike in Bourne shells, single quotes work identically to double |
| 50 | quotes in all other ways. They do not suppress variable expansions. |
| 51 | |
| 52 | Modifying Existing Variables |
| 53 | ---------------------------- |
| 54 | |
| 55 | Sometimes you need to modify existing variables. Following are some |
| 56 | cases where you might find you want to modify an existing variable: |
| 57 | |
| 58 | - Customize a recipe that uses the variable. |
| 59 | |
| 60 | - Change a variable's default value used in a ``*.bbclass`` file. |
| 61 | |
| 62 | - Change the variable in a ``*.bbappend`` file to override the variable |
| 63 | in the original recipe. |
| 64 | |
| 65 | - Change the variable in a configuration file so that the value |
| 66 | overrides an existing configuration. |
| 67 | |
| 68 | Changing a variable value can sometimes depend on how the value was |
| 69 | originally assigned and also on the desired intent of the change. In |
| 70 | particular, when you append a value to a variable that has a default |
| 71 | value, the resulting value might not be what you expect. In this case, |
| 72 | the value you provide might replace the value rather than append to the |
| 73 | default value. |
| 74 | |
| 75 | If after you have changed a variable's value and something unexplained |
| 76 | occurs, you can use BitBake to check the actual value of the suspect |
| 77 | variable. You can make these checks for both configuration and recipe |
| 78 | level changes: |
| 79 | |
| 80 | - For configuration changes, use the following: :: |
| 81 | |
| 82 | $ bitbake -e |
| 83 | |
| 84 | This |
| 85 | command displays variable values after the configuration files (i.e. |
| 86 | ``local.conf``, ``bblayers.conf``, ``bitbake.conf`` and so forth) |
| 87 | have been parsed. |
| 88 | |
| 89 | .. note:: |
| 90 | |
| 91 | Variables that are exported to the environment are preceded by the |
| 92 | string "export" in the command's output. |
| 93 | |
| 94 | - For recipe changes, use the following: :: |
| 95 | |
| 96 | $ bitbake recipe -e \| grep VARIABLE=" |
| 97 | |
| 98 | This command checks to see if the variable actually makes |
| 99 | it into a specific recipe. |
| 100 | |
| 101 | Line Joining |
| 102 | ------------ |
| 103 | |
| 104 | Outside of :ref:`functions <bitbake-user-manual/bitbake-user-manual-metadata:functions>`, |
| 105 | BitBake joins any line ending in |
| 106 | a backslash character ("\") with the following line before parsing |
| 107 | statements. The most common use for the "\" character is to split |
| 108 | variable assignments over multiple lines, as in the following example: :: |
| 109 | |
| 110 | FOO = "bar \ |
| 111 | baz \ |
| 112 | qaz" |
| 113 | |
| 114 | Both the "\" character and the newline |
| 115 | character that follow it are removed when joining lines. Thus, no |
| 116 | newline characters end up in the value of ``FOO``. |
| 117 | |
| 118 | Consider this additional example where the two assignments both assign |
| 119 | "barbaz" to ``FOO``: :: |
| 120 | |
| 121 | FOO = "barbaz" |
| 122 | FOO = "bar\ |
| 123 | baz" |
| 124 | |
| 125 | .. note:: |
| 126 | |
| 127 | BitBake does not interpret escape sequences like "\n" in variable |
| 128 | values. For these to have an effect, the value must be passed to some |
| 129 | utility that interprets escape sequences, such as |
| 130 | ``printf`` or ``echo -n``. |
| 131 | |
| 132 | Variable Expansion |
| 133 | ------------------ |
| 134 | |
| 135 | Variables can reference the contents of other variables using a syntax |
| 136 | that is similar to variable expansion in Bourne shells. The following |
| 137 | assignments result in A containing "aval" and B evaluating to |
| 138 | "preavalpost". :: |
| 139 | |
| 140 | A = "aval" |
| 141 | B = "pre${A}post" |
| 142 | |
| 143 | .. note:: |
| 144 | |
| 145 | Unlike in Bourne shells, the curly braces are mandatory: Only ``${FOO}`` and not |
| 146 | ``$FOO`` is recognized as an expansion of ``FOO``. |
| 147 | |
| 148 | The "=" operator does not immediately expand variable references in the |
| 149 | right-hand side. Instead, expansion is deferred until the variable |
| 150 | assigned to is actually used. The result depends on the current values |
| 151 | of the referenced variables. The following example should clarify this |
| 152 | behavior: :: |
| 153 | |
| 154 | A = "${B} baz" |
| 155 | B = "${C} bar" |
| 156 | C = "foo" |
| 157 | *At this point, ${A} equals "foo bar baz"* |
| 158 | C = "qux" |
| 159 | *At this point, ${A} equals "qux bar baz"* |
| 160 | B = "norf" |
| 161 | *At this point, ${A} equals "norf baz"\* |
| 162 | |
| 163 | Contrast this behavior with the |
| 164 | :ref:`bitbake-user-manual/bitbake-user-manual-metadata:immediate variable |
| 165 | expansion (:=)` operator. |
| 166 | |
| 167 | If the variable expansion syntax is used on a variable that does not |
| 168 | exist, the string is kept as is. For example, given the following |
| 169 | assignment, ``BAR`` expands to the literal string "${FOO}" as long as |
| 170 | ``FOO`` does not exist. :: |
| 171 | |
| 172 | BAR = "${FOO}" |
| 173 | |
| 174 | Setting a default value (?=) |
| 175 | ---------------------------- |
| 176 | |
| 177 | You can use the "?=" operator to achieve a "softer" assignment for a |
| 178 | variable. This type of assignment allows you to define a variable if it |
| 179 | is undefined when the statement is parsed, but to leave the value alone |
| 180 | if the variable has a value. Here is an example: :: |
| 181 | |
| 182 | A ?= "aval" |
| 183 | |
| 184 | If ``A`` is |
| 185 | set at the time this statement is parsed, the variable retains its |
| 186 | value. However, if ``A`` is not set, the variable is set to "aval". |
| 187 | |
| 188 | .. note:: |
| 189 | |
| 190 | This assignment is immediate. Consequently, if multiple "?=" |
| 191 | assignments to a single variable exist, the first of those ends up |
| 192 | getting used. |
| 193 | |
| 194 | Setting a weak default value (??=) |
| 195 | ---------------------------------- |
| 196 | |
| 197 | It is possible to use a "weaker" assignment than in the previous section |
| 198 | by using the "??=" operator. This assignment behaves identical to "?=" |
| 199 | except that the assignment is made at the end of the parsing process |
| 200 | rather than immediately. Consequently, when multiple "??=" assignments |
| 201 | exist, the last one is used. Also, any "=" or "?=" assignment will |
| 202 | override the value set with "??=". Here is an example: :: |
| 203 | |
| 204 | A ??= "somevalue" |
| 205 | A ??= "someothervalue" |
| 206 | |
| 207 | If ``A`` is set before the above statements are |
| 208 | parsed, the variable retains its value. If ``A`` is not set, the |
| 209 | variable is set to "someothervalue". |
| 210 | |
| 211 | Again, this assignment is a "lazy" or "weak" assignment because it does |
| 212 | not occur until the end of the parsing process. |
| 213 | |
| 214 | Immediate variable expansion (:=) |
| 215 | --------------------------------- |
| 216 | |
| 217 | The ":=" operator results in a variable's contents being expanded |
| 218 | immediately, rather than when the variable is actually used: :: |
| 219 | |
| 220 | T = "123" |
| 221 | A := "test ${T}" |
| 222 | T = "456" |
| 223 | B := "${T} ${C}" |
| 224 | C = "cval" |
| 225 | C := "${C}append" |
| 226 | |
| 227 | In this example, ``A`` contains "test 123", even though the final value |
| 228 | of ``T`` is "456". The variable ``B`` will end up containing "456 |
| 229 | cvalappend". This is because references to undefined variables are |
| 230 | preserved as is during (immediate)expansion. This is in contrast to GNU |
| 231 | Make, where undefined variables expand to nothing. The variable ``C`` |
| 232 | contains "cvalappend" since ``${C}`` immediately expands to "cval". |
| 233 | |
| 234 | .. _appending-and-prepending: |
| 235 | |
| 236 | Appending (+=) and prepending (=+) With Spaces |
| 237 | ---------------------------------------------- |
| 238 | |
| 239 | Appending and prepending values is common and can be accomplished using |
| 240 | the "+=" and "=+" operators. These operators insert a space between the |
| 241 | current value and prepended or appended value. |
| 242 | |
| 243 | These operators take immediate effect during parsing. Here are some |
| 244 | examples: :: |
| 245 | |
| 246 | B = "bval" |
| 247 | B += "additionaldata" |
| 248 | C = "cval" |
| 249 | C =+ "test" |
| 250 | |
| 251 | The variable ``B`` contains "bval additionaldata" and ``C`` contains "test |
| 252 | cval". |
| 253 | |
| 254 | .. _appending-and-prepending-without-spaces: |
| 255 | |
| 256 | Appending (.=) and Prepending (=.) Without Spaces |
| 257 | ------------------------------------------------- |
| 258 | |
| 259 | If you want to append or prepend values without an inserted space, use |
| 260 | the ".=" and "=." operators. |
| 261 | |
| 262 | These operators take immediate effect during parsing. Here are some |
| 263 | examples: :: |
| 264 | |
| 265 | B = "bval" |
| 266 | B .= "additionaldata" |
| 267 | C = "cval" |
| 268 | C =. "test" |
| 269 | |
| 270 | The variable ``B`` contains "bvaladditionaldata" and ``C`` contains |
| 271 | "testcval". |
| 272 | |
| 273 | Appending and Prepending (Override Style Syntax) |
| 274 | ------------------------------------------------ |
| 275 | |
| 276 | You can also append and prepend a variable's value using an override |
| 277 | style syntax. When you use this syntax, no spaces are inserted. |
| 278 | |
| 279 | These operators differ from the ":=", ".=", "=.", "+=", and "=+" |
| 280 | operators in that their effects are applied at variable expansion time |
| 281 | rather than being immediately applied. Here are some examples: :: |
| 282 | |
| 283 | B = "bval" |
| 284 | B_append = " additional data" |
| 285 | C = "cval" |
| 286 | C_prepend = "additional data " |
| 287 | D = "dval" |
| 288 | D_append = "additional data" |
| 289 | |
| 290 | The variable ``B`` |
| 291 | becomes "bval additional data" and ``C`` becomes "additional data cval". |
| 292 | The variable ``D`` becomes "dvaladditional data". |
| 293 | |
| 294 | .. note:: |
| 295 | |
| 296 | You must control all spacing when you use the override syntax. |
| 297 | |
| 298 | It is also possible to append and prepend to shell functions and |
| 299 | BitBake-style Python functions. See the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:shell functions`" and ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:bitbake-style python functions`" |
| 300 | sections for examples. |
| 301 | |
| 302 | .. _removing-override-style-syntax: |
| 303 | |
| 304 | Removal (Override Style Syntax) |
| 305 | ------------------------------- |
| 306 | |
| 307 | You can remove values from lists using the removal override style |
| 308 | syntax. Specifying a value for removal causes all occurrences of that |
| 309 | value to be removed from the variable. |
| 310 | |
| 311 | When you use this syntax, BitBake expects one or more strings. |
| 312 | Surrounding spaces and spacing are preserved. Here is an example: :: |
| 313 | |
| 314 | FOO = "123 456 789 123456 123 456 123 456" |
| 315 | FOO_remove = "123" |
| 316 | FOO_remove = "456" |
| 317 | FOO2 = " abc def ghi abcdef abc def abc def def" |
| 318 | FOO2_remove = "\ |
| 319 | def \ |
| 320 | abc \ |
| 321 | ghi \ |
| 322 | " |
| 323 | |
| 324 | The variable ``FOO`` becomes |
| 325 | " 789 123456 " and ``FOO2`` becomes " abcdef ". |
| 326 | |
| 327 | Like "_append" and "_prepend", "_remove" is applied at variable |
| 328 | expansion time. |
| 329 | |
| 330 | Override Style Operation Advantages |
| 331 | ----------------------------------- |
| 332 | |
| 333 | An advantage of the override style operations "_append", "_prepend", and |
| 334 | "_remove" as compared to the "+=" and "=+" operators is that the |
| 335 | override style operators provide guaranteed operations. For example, |
| 336 | consider a class ``foo.bbclass`` that needs to add the value "val" to |
| 337 | the variable ``FOO``, and a recipe that uses ``foo.bbclass`` as follows: :: |
| 338 | |
| 339 | inherit foo |
| 340 | FOO = "initial" |
| 341 | |
| 342 | If ``foo.bbclass`` uses the "+=" operator, |
| 343 | as follows, then the final value of ``FOO`` will be "initial", which is |
| 344 | not what is desired: :: |
| 345 | |
| 346 | FOO += "val" |
| 347 | |
| 348 | If, on the other hand, ``foo.bbclass`` |
| 349 | uses the "_append" operator, then the final value of ``FOO`` will be |
| 350 | "initial val", as intended: :: |
| 351 | |
| 352 | FOO_append = " val" |
| 353 | |
| 354 | .. note:: |
| 355 | |
| 356 | It is never necessary to use "+=" together with "_append". The following |
| 357 | sequence of assignments appends "barbaz" to FOO: :: |
| 358 | |
| 359 | FOO_append = "bar" |
| 360 | FOO_append = "baz" |
| 361 | |
| 362 | |
| 363 | The only effect of changing the second assignment in the previous |
| 364 | example to use "+=" would be to add a space before "baz" in the |
| 365 | appended value (due to how the "+=" operator works). |
| 366 | |
| 367 | Another advantage of the override style operations is that you can |
| 368 | combine them with other overrides as described in the |
| 369 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:conditional syntax (overrides)`" section. |
| 370 | |
| 371 | Variable Flag Syntax |
| 372 | -------------------- |
| 373 | |
| 374 | Variable flags are BitBake's implementation of variable properties or |
| 375 | attributes. It is a way of tagging extra information onto a variable. |
| 376 | You can find more out about variable flags in general in the |
| 377 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:variable flags`" section. |
| 378 | |
| 379 | You can define, append, and prepend values to variable flags. All the |
| 380 | standard syntax operations previously mentioned work for variable flags |
| 381 | except for override style syntax (i.e. "_prepend", "_append", and |
| 382 | "_remove"). |
| 383 | |
| 384 | Here are some examples showing how to set variable flags: :: |
| 385 | |
| 386 | FOO[a] = "abc" |
| 387 | FOO[b] = "123" |
| 388 | FOO[a] += "456" |
| 389 | |
| 390 | The variable ``FOO`` has two flags: |
| 391 | ``[a]`` and ``[b]``. The flags are immediately set to "abc" and "123", |
| 392 | respectively. The ``[a]`` flag becomes "abc 456". |
| 393 | |
| 394 | No need exists to pre-define variable flags. You can simply start using |
| 395 | them. One extremely common application is to attach some brief |
| 396 | documentation to a BitBake variable as follows: :: |
| 397 | |
| 398 | CACHE[doc] = "The directory holding the cache of the metadata." |
| 399 | |
| 400 | Inline Python Variable Expansion |
| 401 | -------------------------------- |
| 402 | |
| 403 | You can use inline Python variable expansion to set variables. Here is |
| 404 | an example: :: |
| 405 | |
| 406 | DATE = "${@time.strftime('%Y%m%d',time.gmtime())}" |
| 407 | |
| 408 | This example results in the ``DATE`` variable being set to the current date. |
| 409 | |
| 410 | Probably the most common use of this feature is to extract the value of |
| 411 | variables from BitBake's internal data dictionary, ``d``. The following |
| 412 | lines select the values of a package name and its version number, |
| 413 | respectively: :: |
| 414 | |
| 415 | PN = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'}" |
| 416 | PV = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[1] or '1.0'}" |
| 417 | |
| 418 | .. note:: |
| 419 | |
| 420 | Inline Python expressions work just like variable expansions insofar as the |
| 421 | "=" and ":=" operators are concerned. Given the following assignment, foo() |
| 422 | is called each time FOO is expanded: :: |
| 423 | |
| 424 | FOO = "${@foo()}" |
| 425 | |
| 426 | Contrast this with the following immediate assignment, where foo() is only |
| 427 | called once, while the assignment is parsed: :: |
| 428 | |
| 429 | FOO := "${@foo()}" |
| 430 | |
| 431 | For a different way to set variables with Python code during parsing, |
| 432 | see the |
| 433 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:anonymous python functions`" section. |
| 434 | |
| 435 | Unsetting variables |
| 436 | ------------------- |
| 437 | |
| 438 | It is possible to completely remove a variable or a variable flag from |
| 439 | BitBake's internal data dictionary by using the "unset" keyword. Here is |
| 440 | an example: :: |
| 441 | |
| 442 | unset DATE |
| 443 | unset do_fetch[noexec] |
| 444 | |
| 445 | These two statements remove the ``DATE`` and the ``do_fetch[noexec]`` flag. |
| 446 | |
| 447 | Providing Pathnames |
| 448 | ------------------- |
| 449 | |
| 450 | When specifying pathnames for use with BitBake, do not use the tilde |
| 451 | ("~") character as a shortcut for your home directory. Doing so might |
| 452 | cause BitBake to not recognize the path since BitBake does not expand |
| 453 | this character in the same way a shell would. |
| 454 | |
| 455 | Instead, provide a fuller path as the following example illustrates: :: |
| 456 | |
| 457 | BBLAYERS ?= " \ |
| 458 | /home/scott-lenovo/LayerA \ |
| 459 | " |
| 460 | |
| 461 | Exporting Variables to the Environment |
| 462 | ====================================== |
| 463 | |
| 464 | You can export variables to the environment of running tasks by using |
| 465 | the ``export`` keyword. For example, in the following example, the |
| 466 | ``do_foo`` task prints "value from the environment" when run: :: |
| 467 | |
| 468 | export ENV_VARIABLE |
| 469 | ENV_VARIABLE = "value from the environment" |
| 470 | |
| 471 | do_foo() { |
| 472 | bbplain "$ENV_VARIABLE" |
| 473 | } |
| 474 | |
| 475 | .. note:: |
| 476 | |
| 477 | BitBake does not expand ``$ENV_VARIABLE`` in this case because it lacks the |
| 478 | obligatory ``{}`` . Rather, ``$ENV_VARIABLE`` is expanded by the shell. |
| 479 | |
| 480 | It does not matter whether ``export ENV_VARIABLE`` appears before or |
| 481 | after assignments to ``ENV_VARIABLE``. |
| 482 | |
| 483 | It is also possible to combine ``export`` with setting a value for the |
| 484 | variable. Here is an example: :: |
| 485 | |
| 486 | export ENV_VARIABLE = "variable-value" |
| 487 | |
| 488 | In the output of ``bitbake -e``, variables that are exported to the |
| 489 | environment are preceded by "export". |
| 490 | |
| 491 | Among the variables commonly exported to the environment are ``CC`` and |
| 492 | ``CFLAGS``, which are picked up by many build systems. |
| 493 | |
| 494 | Conditional Syntax (Overrides) |
| 495 | ============================== |
| 496 | |
| 497 | BitBake uses :term:`OVERRIDES` to control what |
| 498 | variables are overridden after BitBake parses recipes and configuration |
| 499 | files. This section describes how you can use ``OVERRIDES`` as |
| 500 | conditional metadata, talks about key expansion in relationship to |
| 501 | ``OVERRIDES``, and provides some examples to help with understanding. |
| 502 | |
| 503 | Conditional Metadata |
| 504 | -------------------- |
| 505 | |
| 506 | You can use ``OVERRIDES`` to conditionally select a specific version of |
| 507 | a variable and to conditionally append or prepend the value of a |
| 508 | variable. |
| 509 | |
| 510 | .. note:: |
| 511 | |
| 512 | Overrides can only use lower-case characters. Additionally, |
| 513 | underscores are not permitted in override names as they are used to |
| 514 | separate overrides from each other and from the variable name. |
| 515 | |
| 516 | - *Selecting a Variable:* The ``OVERRIDES`` variable is a |
| 517 | colon-character-separated list that contains items for which you want |
| 518 | to satisfy conditions. Thus, if you have a variable that is |
Andrew Geissler | f034379 | 2020-11-18 10:42:21 -0600 | [diff] [blame] | 519 | conditional on "arm", and "arm" is in ``OVERRIDES``, then the |
| 520 | "arm"-specific version of the variable is used rather than the |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 521 | non-conditional version. Here is an example: :: |
| 522 | |
| 523 | OVERRIDES = "architecture:os:machine" |
| 524 | TEST = "default" |
| 525 | TEST_os = "osspecific" |
| 526 | TEST_nooverride = "othercondvalue" |
| 527 | |
| 528 | In this example, the ``OVERRIDES`` |
| 529 | variable lists three overrides: "architecture", "os", and "machine". |
| 530 | The variable ``TEST`` by itself has a default value of "default". You |
| 531 | select the os-specific version of the ``TEST`` variable by appending |
| 532 | the "os" override to the variable (i.e. ``TEST_os``). |
| 533 | |
| 534 | To better understand this, consider a practical example that assumes |
| 535 | an OpenEmbedded metadata-based Linux kernel recipe file. The |
| 536 | following lines from the recipe file first set the kernel branch |
| 537 | variable ``KBRANCH`` to a default value, then conditionally override |
| 538 | that value based on the architecture of the build: :: |
| 539 | |
| 540 | KBRANCH = "standard/base" |
| 541 | KBRANCH_qemuarm = "standard/arm-versatile-926ejs" |
| 542 | KBRANCH_qemumips = "standard/mti-malta32" |
| 543 | KBRANCH_qemuppc = "standard/qemuppc" |
| 544 | KBRANCH_qemux86 = "standard/common-pc/base" |
| 545 | KBRANCH_qemux86-64 = "standard/common-pc-64/base" |
| 546 | KBRANCH_qemumips64 = "standard/mti-malta64" |
| 547 | |
| 548 | - *Appending and Prepending:* BitBake also supports append and prepend |
| 549 | operations to variable values based on whether a specific item is |
| 550 | listed in ``OVERRIDES``. Here is an example: :: |
| 551 | |
| 552 | DEPENDS = "glibc ncurses" |
| 553 | OVERRIDES = "machine:local" |
| 554 | DEPENDS_append_machine = "libmad" |
| 555 | |
| 556 | In this example, ``DEPENDS`` becomes "glibc ncurses libmad". |
| 557 | |
| 558 | Again, using an OpenEmbedded metadata-based kernel recipe file as an |
| 559 | example, the following lines will conditionally append to the |
| 560 | ``KERNEL_FEATURES`` variable based on the architecture: :: |
| 561 | |
| 562 | KERNEL_FEATURES_append = " ${KERNEL_EXTRA_FEATURES}" |
| 563 | KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc" |
| 564 | KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc" |
| 565 | |
| 566 | - *Setting a Variable for a Single Task:* BitBake supports setting a |
| 567 | variable just for the duration of a single task. Here is an example: :: |
| 568 | |
| 569 | FOO_task-configure = "val 1" |
| 570 | FOO_task-compile = "val 2" |
| 571 | |
| 572 | In the |
| 573 | previous example, ``FOO`` has the value "val 1" while the |
| 574 | ``do_configure`` task is executed, and the value "val 2" while the |
| 575 | ``do_compile`` task is executed. |
| 576 | |
| 577 | Internally, this is implemented by prepending the task (e.g. |
| 578 | "task-compile:") to the value of |
| 579 | :term:`OVERRIDES` for the local datastore of the |
| 580 | ``do_compile`` task. |
| 581 | |
| 582 | You can also use this syntax with other combinations (e.g. |
| 583 | "``_prepend``") as shown in the following example: :: |
| 584 | |
| 585 | EXTRA_OEMAKE_prepend_task-compile = "${PARALLEL_MAKE} " |
| 586 | |
| 587 | Key Expansion |
| 588 | ------------- |
| 589 | |
| 590 | Key expansion happens when the BitBake datastore is finalized. To better |
| 591 | understand this, consider the following example: :: |
| 592 | |
| 593 | A${B} = "X" |
| 594 | B = "2" |
| 595 | A2 = "Y" |
| 596 | |
| 597 | In this case, after all the parsing is complete, BitBake expands |
| 598 | ``${B}`` into "2". This expansion causes ``A2``, which was set to "Y" |
| 599 | before the expansion, to become "X". |
| 600 | |
| 601 | .. _variable-interaction-worked-examples: |
| 602 | |
| 603 | Examples |
| 604 | -------- |
| 605 | |
| 606 | Despite the previous explanations that show the different forms of |
| 607 | variable definitions, it can be hard to work out exactly what happens |
| 608 | when variable operators, conditional overrides, and unconditional |
| 609 | overrides are combined. This section presents some common scenarios |
| 610 | along with explanations for variable interactions that typically confuse |
| 611 | users. |
| 612 | |
| 613 | There is often confusion concerning the order in which overrides and |
| 614 | various "append" operators take effect. Recall that an append or prepend |
| 615 | operation using "_append" and "_prepend" does not result in an immediate |
| 616 | assignment as would "+=", ".=", "=+", or "=.". Consider the following |
| 617 | example: :: |
| 618 | |
| 619 | OVERRIDES = "foo" |
| 620 | A = "Z" |
| 621 | A_foo_append = "X" |
| 622 | |
| 623 | For this case, |
| 624 | ``A`` is unconditionally set to "Z" and "X" is unconditionally and |
| 625 | immediately appended to the variable ``A_foo``. Because overrides have |
| 626 | not been applied yet, ``A_foo`` is set to "X" due to the append and |
| 627 | ``A`` simply equals "Z". |
| 628 | |
| 629 | Applying overrides, however, changes things. Since "foo" is listed in |
| 630 | ``OVERRIDES``, the conditional variable ``A`` is replaced with the "foo" |
| 631 | version, which is equal to "X". So effectively, ``A_foo`` replaces |
| 632 | ``A``. |
| 633 | |
| 634 | This next example changes the order of the override and the append: :: |
| 635 | |
| 636 | OVERRIDES = "foo" |
| 637 | A = "Z" |
| 638 | A_append_foo = "X" |
| 639 | |
| 640 | For this case, before |
| 641 | overrides are handled, ``A`` is set to "Z" and ``A_append_foo`` is set |
| 642 | to "X". Once the override for "foo" is applied, however, ``A`` gets |
| 643 | appended with "X". Consequently, ``A`` becomes "ZX". Notice that spaces |
| 644 | are not appended. |
| 645 | |
| 646 | This next example has the order of the appends and overrides reversed |
| 647 | back as in the first example: :: |
| 648 | |
| 649 | OVERRIDES = "foo" |
| 650 | A = "Y" |
| 651 | A_foo_append = "Z" |
| 652 | A_foo_append = "X" |
| 653 | |
| 654 | For this case, before any overrides are resolved, |
| 655 | ``A`` is set to "Y" using an immediate assignment. After this immediate |
| 656 | assignment, ``A_foo`` is set to "Z", and then further appended with "X" |
| 657 | leaving the variable set to "ZX". Finally, applying the override for |
| 658 | "foo" results in the conditional variable ``A`` becoming "ZX" (i.e. |
| 659 | ``A`` is replaced with ``A_foo``). |
| 660 | |
| 661 | This final example mixes in some varying operators: :: |
| 662 | |
| 663 | A = "1" |
| 664 | A_append = "2" |
| 665 | A_append = "3" |
| 666 | A += "4" |
| 667 | A .= "5" |
| 668 | |
| 669 | For this case, the type of append |
| 670 | operators are affecting the order of assignments as BitBake passes |
| 671 | through the code multiple times. Initially, ``A`` is set to "1 45" |
| 672 | because of the three statements that use immediate operators. After |
| 673 | these assignments are made, BitBake applies the "_append" operations. |
| 674 | Those operations result in ``A`` becoming "1 4523". |
| 675 | |
| 676 | Sharing Functionality |
| 677 | ===================== |
| 678 | |
| 679 | BitBake allows for metadata sharing through include files (``.inc``) and |
| 680 | class files (``.bbclass``). For example, suppose you have a piece of |
| 681 | common functionality such as a task definition that you want to share |
| 682 | between more than one recipe. In this case, creating a ``.bbclass`` file |
| 683 | that contains the common functionality and then using the ``inherit`` |
| 684 | directive in your recipes to inherit the class would be a common way to |
| 685 | share the task. |
| 686 | |
| 687 | This section presents the mechanisms BitBake provides to allow you to |
| 688 | share functionality between recipes. Specifically, the mechanisms |
| 689 | include ``include``, ``inherit``, ``INHERIT``, and ``require`` |
| 690 | directives. |
| 691 | |
| 692 | Locating Include and Class Files |
| 693 | -------------------------------- |
| 694 | |
| 695 | BitBake uses the :term:`BBPATH` variable to locate |
| 696 | needed include and class files. Additionally, BitBake searches the |
| 697 | current directory for ``include`` and ``require`` directives. |
| 698 | |
| 699 | .. note:: |
| 700 | |
| 701 | The BBPATH variable is analogous to the environment variable PATH . |
| 702 | |
| 703 | In order for include and class files to be found by BitBake, they need |
| 704 | to be located in a "classes" subdirectory that can be found in |
| 705 | ``BBPATH``. |
| 706 | |
| 707 | ``inherit`` Directive |
| 708 | --------------------- |
| 709 | |
| 710 | When writing a recipe or class file, you can use the ``inherit`` |
| 711 | directive to inherit the functionality of a class (``.bbclass``). |
| 712 | BitBake only supports this directive when used within recipe and class |
| 713 | files (i.e. ``.bb`` and ``.bbclass``). |
| 714 | |
| 715 | The ``inherit`` directive is a rudimentary means of specifying |
| 716 | functionality contained in class files that your recipes require. For |
| 717 | example, you can easily abstract out the tasks involved in building a |
| 718 | package that uses Autoconf and Automake and put those tasks into a class |
| 719 | file and then have your recipe inherit that class file. |
| 720 | |
| 721 | As an example, your recipes could use the following directive to inherit |
| 722 | an ``autotools.bbclass`` file. The class file would contain common |
| 723 | functionality for using Autotools that could be shared across recipes: :: |
| 724 | |
| 725 | inherit autotools |
| 726 | |
| 727 | In this case, BitBake would search for the directory |
| 728 | ``classes/autotools.bbclass`` in ``BBPATH``. |
| 729 | |
| 730 | .. note:: |
| 731 | |
| 732 | You can override any values and functions of the inherited class |
| 733 | within your recipe by doing so after the "inherit" statement. |
| 734 | |
| 735 | If you want to use the directive to inherit multiple classes, separate |
| 736 | them with spaces. The following example shows how to inherit both the |
| 737 | ``buildhistory`` and ``rm_work`` classes: :: |
| 738 | |
| 739 | inherit buildhistory rm_work |
| 740 | |
| 741 | An advantage with the inherit directive as compared to both the |
| 742 | :ref:`include <bitbake-user-manual/bitbake-user-manual-metadata:\`\`include\`\` directive>` and :ref:`require <bitbake-user-manual/bitbake-user-manual-metadata:\`\`require\`\` directive>` |
| 743 | directives is that you can inherit class files conditionally. You can |
| 744 | accomplish this by using a variable expression after the ``inherit`` |
| 745 | statement. Here is an example: :: |
| 746 | |
| 747 | inherit ${VARNAME} |
| 748 | |
| 749 | If ``VARNAME`` is |
| 750 | going to be set, it needs to be set before the ``inherit`` statement is |
| 751 | parsed. One way to achieve a conditional inherit in this case is to use |
| 752 | overrides: :: |
| 753 | |
| 754 | VARIABLE = "" |
| 755 | VARIABLE_someoverride = "myclass" |
| 756 | |
| 757 | Another method is by using anonymous Python. Here is an example: :: |
| 758 | |
| 759 | python () { |
| 760 | if condition == value: |
| 761 | d.setVar('VARIABLE', 'myclass') |
| 762 | else: |
| 763 | d.setVar('VARIABLE', '') |
| 764 | } |
| 765 | |
| 766 | Alternatively, you could use an in-line Python expression in the |
| 767 | following form: :: |
| 768 | |
| 769 | inherit ${@'classname' if condition else ''} |
| 770 | inherit ${@functionname(params)} |
| 771 | |
| 772 | In all cases, if the expression evaluates to an |
| 773 | empty string, the statement does not trigger a syntax error because it |
| 774 | becomes a no-op. |
| 775 | |
| 776 | ``include`` Directive |
| 777 | --------------------- |
| 778 | |
| 779 | BitBake understands the ``include`` directive. This directive causes |
| 780 | BitBake to parse whatever file you specify, and to insert that file at |
| 781 | that location. The directive is much like its equivalent in Make except |
| 782 | that if the path specified on the include line is a relative path, |
| 783 | BitBake locates the first file it can find within ``BBPATH``. |
| 784 | |
| 785 | The include directive is a more generic method of including |
| 786 | functionality as compared to the :ref:`inherit <bitbake-user-manual/bitbake-user-manual-metadata:\`\`inherit\`\` directive>` |
| 787 | directive, which is restricted to class (i.e. ``.bbclass``) files. The |
| 788 | include directive is applicable for any other kind of shared or |
| 789 | encapsulated functionality or configuration that does not suit a |
| 790 | ``.bbclass`` file. |
| 791 | |
| 792 | As an example, suppose you needed a recipe to include some self-test |
| 793 | definitions: :: |
| 794 | |
| 795 | include test_defs.inc |
| 796 | |
| 797 | .. note:: |
| 798 | |
| 799 | The include directive does not produce an error when the file cannot be |
| 800 | found. Consequently, it is recommended that if the file you are including is |
| 801 | expected to exist, you should use :ref:`require <require-inclusion>` instead |
| 802 | of include . Doing so makes sure that an error is produced if the file cannot |
| 803 | be found. |
| 804 | |
| 805 | .. _require-inclusion: |
| 806 | |
| 807 | ``require`` Directive |
| 808 | --------------------- |
| 809 | |
| 810 | BitBake understands the ``require`` directive. This directive behaves |
| 811 | just like the ``include`` directive with the exception that BitBake |
| 812 | raises a parsing error if the file to be included cannot be found. Thus, |
| 813 | any file you require is inserted into the file that is being parsed at |
| 814 | the location of the directive. |
| 815 | |
| 816 | The require directive, like the include directive previously described, |
| 817 | is a more generic method of including functionality as compared to the |
| 818 | :ref:`inherit <bitbake-user-manual/bitbake-user-manual-metadata:\`\`inherit\`\` directive>` directive, which is restricted to class |
| 819 | (i.e. ``.bbclass``) files. The require directive is applicable for any |
| 820 | other kind of shared or encapsulated functionality or configuration that |
| 821 | does not suit a ``.bbclass`` file. |
| 822 | |
| 823 | Similar to how BitBake handles :ref:`include <bitbake-user-manual/bitbake-user-manual-metadata:\`\`include\`\` directive>`, if |
| 824 | the path specified on the require line is a relative path, BitBake |
| 825 | locates the first file it can find within ``BBPATH``. |
| 826 | |
| 827 | As an example, suppose you have two versions of a recipe (e.g. |
| 828 | ``foo_1.2.2.bb`` and ``foo_2.0.0.bb``) where each version contains some |
| 829 | identical functionality that could be shared. You could create an |
| 830 | include file named ``foo.inc`` that contains the common definitions |
| 831 | needed to build "foo". You need to be sure ``foo.inc`` is located in the |
| 832 | same directory as your two recipe files as well. Once these conditions |
| 833 | are set up, you can share the functionality using a ``require`` |
| 834 | directive from within each recipe: :: |
| 835 | |
| 836 | require foo.inc |
| 837 | |
| 838 | ``INHERIT`` Configuration Directive |
| 839 | ----------------------------------- |
| 840 | |
| 841 | When creating a configuration file (``.conf``), you can use the |
| 842 | :term:`INHERIT` configuration directive to inherit a |
| 843 | class. BitBake only supports this directive when used within a |
| 844 | configuration file. |
| 845 | |
| 846 | As an example, suppose you needed to inherit a class file called |
| 847 | ``abc.bbclass`` from a configuration file as follows: :: |
| 848 | |
| 849 | INHERIT += "abc" |
| 850 | |
| 851 | This configuration directive causes the named class to be inherited at |
| 852 | the point of the directive during parsing. As with the ``inherit`` |
| 853 | directive, the ``.bbclass`` file must be located in a "classes" |
| 854 | subdirectory in one of the directories specified in ``BBPATH``. |
| 855 | |
| 856 | .. note:: |
| 857 | |
| 858 | Because .conf files are parsed first during BitBake's execution, using |
| 859 | INHERIT to inherit a class effectively inherits the class globally (i.e. for |
| 860 | all recipes). |
| 861 | |
| 862 | If you want to use the directive to inherit multiple classes, you can |
| 863 | provide them on the same line in the ``local.conf`` file. Use spaces to |
| 864 | separate the classes. The following example shows how to inherit both |
| 865 | the ``autotools`` and ``pkgconfig`` classes: :: |
| 866 | |
| 867 | INHERIT += "autotools pkgconfig" |
| 868 | |
| 869 | Functions |
| 870 | ========= |
| 871 | |
| 872 | As with most languages, functions are the building blocks that are used |
| 873 | to build up operations into tasks. BitBake supports these types of |
| 874 | functions: |
| 875 | |
| 876 | - *Shell Functions:* Functions written in shell script and executed |
| 877 | either directly as functions, tasks, or both. They can also be called |
| 878 | by other shell functions. |
| 879 | |
| 880 | - *BitBake-Style Python Functions:* Functions written in Python and |
| 881 | executed by BitBake or other Python functions using |
| 882 | ``bb.build.exec_func()``. |
| 883 | |
| 884 | - *Python Functions:* Functions written in Python and executed by |
| 885 | Python. |
| 886 | |
| 887 | - *Anonymous Python Functions:* Python functions executed automatically |
| 888 | during parsing. |
| 889 | |
| 890 | Regardless of the type of function, you can only define them in class |
| 891 | (``.bbclass``) and recipe (``.bb`` or ``.inc``) files. |
| 892 | |
| 893 | Shell Functions |
| 894 | --------------- |
| 895 | |
| 896 | Functions written in shell script and executed either directly as |
| 897 | functions, tasks, or both. They can also be called by other shell |
| 898 | functions. Here is an example shell function definition: :: |
| 899 | |
| 900 | some_function () { |
| 901 | echo "Hello World" |
| 902 | } |
| 903 | |
| 904 | When you create these types of functions in |
| 905 | your recipe or class files, you need to follow the shell programming |
| 906 | rules. The scripts are executed by ``/bin/sh``, which may not be a bash |
| 907 | shell but might be something such as ``dash``. You should not use |
| 908 | Bash-specific script (bashisms). |
| 909 | |
| 910 | Overrides and override-style operators like ``_append`` and ``_prepend`` |
| 911 | can also be applied to shell functions. Most commonly, this application |
| 912 | would be used in a ``.bbappend`` file to modify functions in the main |
| 913 | recipe. It can also be used to modify functions inherited from classes. |
| 914 | |
| 915 | As an example, consider the following: :: |
| 916 | |
| 917 | do_foo() { |
| 918 | bbplain first |
| 919 | fn |
| 920 | } |
| 921 | |
| 922 | fn_prepend() { |
| 923 | bbplain second |
| 924 | } |
| 925 | |
| 926 | fn() { |
| 927 | bbplain third |
| 928 | } |
| 929 | |
| 930 | do_foo_append() { |
| 931 | bbplain fourth |
| 932 | } |
| 933 | |
| 934 | Running ``do_foo`` prints the following: :: |
| 935 | |
| 936 | recipename do_foo: first |
| 937 | recipename do_foo: second |
| 938 | recipename do_foo: third |
| 939 | recipename do_foo: fourth |
| 940 | |
| 941 | .. note:: |
| 942 | |
| 943 | Overrides and override-style operators can be applied to any shell |
| 944 | function, not just :ref:`tasks <bitbake-user-manual/bitbake-user-manual-metadata:tasks>`. |
| 945 | |
| 946 | You can use the ``bitbake -e``Â recipename command to view the final |
| 947 | assembled function after all overrides have been applied. |
| 948 | |
| 949 | BitBake-Style Python Functions |
| 950 | ------------------------------ |
| 951 | |
| 952 | These functions are written in Python and executed by BitBake or other |
| 953 | Python functions using ``bb.build.exec_func()``. |
| 954 | |
| 955 | An example BitBake function is: :: |
| 956 | |
| 957 | python some_python_function () { |
| 958 | d.setVar("TEXT", "Hello World") |
| 959 | print d.getVar("TEXT") |
| 960 | } |
| 961 | |
| 962 | Because the |
| 963 | Python "bb" and "os" modules are already imported, you do not need to |
| 964 | import these modules. Also in these types of functions, the datastore |
| 965 | ("d") is a global variable and is always automatically available. |
| 966 | |
| 967 | .. note:: |
| 968 | |
| 969 | Variable expressions (e.g. ``${X}`` ) are no longer expanded within Python |
| 970 | functions. This behavior is intentional in order to allow you to freely set |
| 971 | variable values to expandable expressions without having them expanded |
| 972 | prematurely. If you do wish to expand a variable within a Python function, |
| 973 | use ``d.getVar("X")`` . Or, for more complicated expressions, use ``d.expand()``. |
| 974 | |
| 975 | Similar to shell functions, you can also apply overrides and |
| 976 | override-style operators to BitBake-style Python functions. |
| 977 | |
| 978 | As an example, consider the following: :: |
| 979 | |
| 980 | python do_foo_prepend() { |
| 981 | bb.plain("first") |
| 982 | } |
| 983 | |
| 984 | python do_foo() { |
| 985 | bb.plain("second") |
| 986 | } |
| 987 | |
| 988 | python do_foo_append() { |
| 989 | bb.plain("third") |
| 990 | } |
| 991 | |
| 992 | Running ``do_foo`` prints the following: :: |
| 993 | |
| 994 | recipename do_foo: first |
| 995 | recipename do_foo: second |
| 996 | recipename do_foo: third |
| 997 | |
| 998 | You can use the ``bitbake -e``Â recipename command to view |
| 999 | the final assembled function after all overrides have been applied. |
| 1000 | |
| 1001 | Python Functions |
| 1002 | ---------------- |
| 1003 | |
| 1004 | These functions are written in Python and are executed by other Python |
| 1005 | code. Examples of Python functions are utility functions that you intend |
| 1006 | to call from in-line Python or from within other Python functions. Here |
| 1007 | is an example: :: |
| 1008 | |
| 1009 | def get_depends(d): |
| 1010 | if d.getVar('SOMECONDITION'): |
| 1011 | return "dependencywithcond" |
| 1012 | else: |
| 1013 | return "dependency" |
| 1014 | |
| 1015 | SOMECONDITION = "1" |
| 1016 | DEPENDS = "${@get_depends(d)}" |
| 1017 | |
| 1018 | This would result in ``DEPENDS`` containing ``dependencywithcond``. |
| 1019 | |
| 1020 | Here are some things to know about Python functions: |
| 1021 | |
| 1022 | - Python functions can take parameters. |
| 1023 | |
| 1024 | - The BitBake datastore is not automatically available. Consequently, |
| 1025 | you must pass it in as a parameter to the function. |
| 1026 | |
| 1027 | - The "bb" and "os" Python modules are automatically available. You do |
| 1028 | not need to import them. |
| 1029 | |
| 1030 | BitBake-Style Python Functions Versus Python Functions |
| 1031 | ------------------------------------------------------ |
| 1032 | |
| 1033 | Following are some important differences between BitBake-style Python |
| 1034 | functions and regular Python functions defined with "def": |
| 1035 | |
| 1036 | - Only BitBake-style Python functions can be :ref:`tasks <bitbake-user-manual/bitbake-user-manual-metadata:tasks>`. |
| 1037 | |
| 1038 | - Overrides and override-style operators can only be applied to |
| 1039 | BitBake-style Python functions. |
| 1040 | |
| 1041 | - Only regular Python functions can take arguments and return values. |
| 1042 | |
| 1043 | - :ref:`Variable flags <bitbake-user-manual/bitbake-user-manual-metadata:variable flags>` such as |
| 1044 | ``[dirs]``, ``[cleandirs]``, and ``[lockfiles]`` can be used on BitBake-style |
| 1045 | Python functions, but not on regular Python functions. |
| 1046 | |
| 1047 | - BitBake-style Python functions generate a separate |
| 1048 | ``${``\ :term:`T`\ ``}/run.``\ function-name\ ``.``\ pid |
| 1049 | script that is executed to run the function, and also generate a log |
| 1050 | file in ``${T}/log.``\ function-name\ ``.``\ pid if they are executed |
| 1051 | as tasks. |
| 1052 | |
| 1053 | Regular Python functions execute "inline" and do not generate any |
| 1054 | files in ``${T}``. |
| 1055 | |
| 1056 | - Regular Python functions are called with the usual Python syntax. |
| 1057 | BitBake-style Python functions are usually tasks and are called |
| 1058 | directly by BitBake, but can also be called manually from Python code |
| 1059 | by using the ``bb.build.exec_func()`` function. Here is an example: :: |
| 1060 | |
| 1061 | bb.build.exec_func("my_bitbake_style_function", d) |
| 1062 | |
| 1063 | .. note:: |
| 1064 | |
| 1065 | ``bb.build.exec_func()`` can also be used to run shell functions from Python |
| 1066 | code. If you want to run a shell function before a Python function within |
| 1067 | the same task, then you can use a parent helper Python function that |
| 1068 | starts by running the shell function with ``bb.build.exec_func()`` and then |
| 1069 | runs the Python code. |
| 1070 | |
| 1071 | To detect errors from functions executed with |
| 1072 | ``bb.build.exec_func()``, you can catch the ``bb.build.FuncFailed`` |
| 1073 | exception. |
| 1074 | |
| 1075 | .. note:: |
| 1076 | |
| 1077 | Functions in metadata (recipes and classes) should not themselves raise |
| 1078 | ``bb.build.FuncFailed``. Rather, ``bb.build.FuncFailed`` should be viewed as a |
| 1079 | general indicator that the called function failed by raising an |
| 1080 | exception. For example, an exception raised by ``bb.fatal()`` will be caught |
| 1081 | inside ``bb.build.exec_func()``, and a ``bb.build.FuncFailed`` will be raised in |
| 1082 | response. |
| 1083 | |
| 1084 | Due to their simplicity, you should prefer regular Python functions over |
| 1085 | BitBake-style Python functions unless you need a feature specific to |
| 1086 | BitBake-style Python functions. Regular Python functions in metadata are |
| 1087 | a more recent invention than BitBake-style Python functions, and older |
| 1088 | code tends to use ``bb.build.exec_func()`` more often. |
| 1089 | |
| 1090 | Anonymous Python Functions |
| 1091 | -------------------------- |
| 1092 | |
| 1093 | Sometimes it is useful to set variables or perform other operations |
| 1094 | programmatically during parsing. To do this, you can define special |
| 1095 | Python functions, called anonymous Python functions, that run at the end |
| 1096 | of parsing. For example, the following conditionally sets a variable |
| 1097 | based on the value of another variable: :: |
| 1098 | |
| 1099 | python () { |
| 1100 | if d.getVar('SOMEVAR') == 'value': |
| 1101 | d.setVar('ANOTHERVAR', 'value2') |
| 1102 | } |
| 1103 | |
| 1104 | An equivalent way to mark a function as an anonymous function is to give it |
| 1105 | the name "__anonymous", rather than no name. |
| 1106 | |
| 1107 | Anonymous Python functions always run at the end of parsing, regardless |
| 1108 | of where they are defined. If a recipe contains many anonymous |
| 1109 | functions, they run in the same order as they are defined within the |
| 1110 | recipe. As an example, consider the following snippet: :: |
| 1111 | |
| 1112 | python () { |
| 1113 | d.setVar('FOO', 'foo 2') |
| 1114 | } |
| 1115 | |
| 1116 | FOO = "foo 1" |
| 1117 | |
| 1118 | python () { |
| 1119 | d.appendVar('BAR',' bar 2') |
| 1120 | } |
| 1121 | |
| 1122 | BAR = "bar 1" |
| 1123 | |
| 1124 | The previous example is conceptually |
| 1125 | equivalent to the following snippet: :: |
| 1126 | |
| 1127 | FOO = "foo 1" |
| 1128 | BAR = "bar 1" |
| 1129 | FOO = "foo 2" |
| 1130 | BAR += "bar 2" |
| 1131 | |
| 1132 | ``FOO`` ends up with the value "foo 2", and |
| 1133 | ``BAR`` with the value "bar 1 bar 2". Just as in the second snippet, the |
| 1134 | values set for the variables within the anonymous functions become |
| 1135 | available to tasks, which always run after parsing. |
| 1136 | |
| 1137 | Overrides and override-style operators such as "``_append``" are applied |
| 1138 | before anonymous functions run. In the following example, ``FOO`` ends |
| 1139 | up with the value "foo from anonymous": :: |
| 1140 | |
| 1141 | FOO = "foo" |
| 1142 | FOO_append = " from outside" |
| 1143 | |
| 1144 | python () { |
| 1145 | d.setVar("FOO", "foo from anonymous") |
| 1146 | } |
| 1147 | |
| 1148 | For methods |
| 1149 | you can use with anonymous Python functions, see the |
| 1150 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:functions you can call from within python`" |
| 1151 | section. For a different method to run Python code during parsing, see |
| 1152 | the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:inline python variable expansion`" section. |
| 1153 | |
| 1154 | Flexible Inheritance for Class Functions |
| 1155 | ---------------------------------------- |
| 1156 | |
| 1157 | Through coding techniques and the use of ``EXPORT_FUNCTIONS``, BitBake |
| 1158 | supports exporting a function from a class such that the class function |
| 1159 | appears as the default implementation of the function, but can still be |
| 1160 | called if a recipe inheriting the class needs to define its own version |
| 1161 | of the function. |
| 1162 | |
| 1163 | To understand the benefits of this feature, consider the basic scenario |
| 1164 | where a class defines a task function and your recipe inherits the |
| 1165 | class. In this basic scenario, your recipe inherits the task function as |
| 1166 | defined in the class. If desired, your recipe can add to the start and |
| 1167 | end of the function by using the "_prepend" or "_append" operations |
| 1168 | respectively, or it can redefine the function completely. However, if it |
| 1169 | redefines the function, there is no means for it to call the class |
| 1170 | version of the function. ``EXPORT_FUNCTIONS`` provides a mechanism that |
| 1171 | enables the recipe's version of the function to call the original |
| 1172 | version of the function. |
| 1173 | |
| 1174 | To make use of this technique, you need the following things in place: |
| 1175 | |
| 1176 | - The class needs to define the function as follows: :: |
| 1177 | |
| 1178 | classname_functionname |
| 1179 | |
| 1180 | For example, if you have a class file |
| 1181 | ``bar.bbclass`` and a function named ``do_foo``, the class must |
| 1182 | define the function as follows: :: |
| 1183 | |
| 1184 | bar_do_foo |
| 1185 | |
| 1186 | - The class needs to contain the ``EXPORT_FUNCTIONS`` statement as |
| 1187 | follows: :: |
| 1188 | |
| 1189 | EXPORT_FUNCTIONS functionname |
| 1190 | |
| 1191 | For example, continuing with |
| 1192 | the same example, the statement in the ``bar.bbclass`` would be as |
| 1193 | follows: :: |
| 1194 | |
| 1195 | EXPORT_FUNCTIONS do_foo |
| 1196 | |
| 1197 | - You need to call the function appropriately from within your recipe. |
| 1198 | Continuing with the same example, if your recipe needs to call the |
| 1199 | class version of the function, it should call ``bar_do_foo``. |
| 1200 | Assuming ``do_foo`` was a shell function and ``EXPORT_FUNCTIONS`` was |
| 1201 | used as above, the recipe's function could conditionally call the |
| 1202 | class version of the function as follows: :: |
| 1203 | |
| 1204 | do_foo() { |
| 1205 | if [ somecondition ] ; then |
| 1206 | bar_do_foo |
| 1207 | else |
| 1208 | # Do something else |
| 1209 | fi |
| 1210 | } |
| 1211 | |
| 1212 | To call your modified version of the function as defined in your recipe, |
| 1213 | call it as ``do_foo``. |
| 1214 | |
| 1215 | With these conditions met, your single recipe can freely choose between |
| 1216 | the original function as defined in the class file and the modified |
| 1217 | function in your recipe. If you do not set up these conditions, you are |
| 1218 | limited to using one function or the other. |
| 1219 | |
| 1220 | Tasks |
| 1221 | ===== |
| 1222 | |
| 1223 | Tasks are BitBake execution units that make up the steps that BitBake |
| 1224 | can run for a given recipe. Tasks are only supported in recipes and |
| 1225 | classes (i.e. in ``.bb`` files and files included or inherited from |
| 1226 | ``.bb`` files). By convention, tasks have names that start with "do\_". |
| 1227 | |
| 1228 | Promoting a Function to a Task |
| 1229 | ------------------------------ |
| 1230 | |
| 1231 | Tasks are either :ref:`shell functions <bitbake-user-manual/bitbake-user-manual-metadata:shell functions>` or |
| 1232 | :ref:`BitBake-style Python functions <bitbake-user-manual/bitbake-user-manual-metadata:bitbake-style python functions>` |
| 1233 | that have been promoted to tasks by using the ``addtask`` command. The |
| 1234 | ``addtask`` command can also optionally describe dependencies between |
| 1235 | the task and other tasks. Here is an example that shows how to define a |
| 1236 | task and declare some dependencies: :: |
| 1237 | |
| 1238 | python do_printdate () { |
| 1239 | import time |
| 1240 | print time.strftime('%Y%m%d', time.gmtime()) |
| 1241 | } |
| 1242 | addtask printdate after do_fetch before do_build |
| 1243 | |
| 1244 | The first argument to ``addtask`` is the name |
| 1245 | of the function to promote to a task. If the name does not start with |
| 1246 | "do\_", "do\_" is implicitly added, which enforces the convention that all |
| 1247 | task names start with "do\_". |
| 1248 | |
| 1249 | In the previous example, the ``do_printdate`` task becomes a dependency |
| 1250 | of the ``do_build`` task, which is the default task (i.e. the task run |
| 1251 | by the ``bitbake`` command unless another task is specified explicitly). |
| 1252 | Additionally, the ``do_printdate`` task becomes dependent upon the |
| 1253 | ``do_fetch`` task. Running the ``do_build`` task results in the |
| 1254 | ``do_printdate`` task running first. |
| 1255 | |
| 1256 | .. note:: |
| 1257 | |
| 1258 | If you try out the previous example, you might see that the |
| 1259 | ``do_printdate`` |
| 1260 | task is only run the first time you build the recipe with the |
| 1261 | ``bitbake`` |
| 1262 | command. This is because BitBake considers the task "up-to-date" |
| 1263 | after that initial run. If you want to force the task to always be |
| 1264 | rerun for experimentation purposes, you can make BitBake always |
| 1265 | consider the task "out-of-date" by using the |
| 1266 | :ref:`[nostamp] <bitbake-user-manual/bitbake-user-manual-metadata:Variable Flags>` |
| 1267 | variable flag, as follows: :: |
| 1268 | |
| 1269 | do_printdate[nostamp] = "1" |
| 1270 | |
| 1271 | You can also explicitly run the task and provide the |
| 1272 | -f option as follows: :: |
| 1273 | |
| 1274 | $ bitbake recipe -c printdate -f |
| 1275 | |
| 1276 | When manually selecting a task to run with the bitbake ``recipe |
| 1277 | -c task`` command, you can omit the "do\_" prefix as part of the task |
| 1278 | name. |
| 1279 | |
| 1280 | You might wonder about the practical effects of using ``addtask`` |
| 1281 | without specifying any dependencies as is done in the following example: :: |
| 1282 | |
| 1283 | addtask printdate |
| 1284 | |
| 1285 | In this example, assuming dependencies have not been |
| 1286 | added through some other means, the only way to run the task is by |
| 1287 | explicitly selecting it with ``bitbake`` recipe ``-c printdate``. You |
| 1288 | can use the ``do_listtasks`` task to list all tasks defined in a recipe |
| 1289 | as shown in the following example: :: |
| 1290 | |
| 1291 | $ bitbake recipe -c listtasks |
| 1292 | |
| 1293 | For more information on task dependencies, see the |
| 1294 | ":ref:`bitbake-user-manual/bitbake-user-manual-execution:dependencies`" section. |
| 1295 | |
| 1296 | See the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:variable flags`" section for information |
| 1297 | on variable flags you can use with tasks. |
| 1298 | |
| 1299 | Deleting a Task |
| 1300 | --------------- |
| 1301 | |
| 1302 | As well as being able to add tasks, you can delete them. Simply use the |
| 1303 | ``deltask`` command to delete a task. For example, to delete the example |
| 1304 | task used in the previous sections, you would use: :: |
| 1305 | |
| 1306 | deltask printdate |
| 1307 | |
| 1308 | If you delete a task using the ``deltask`` command and the task has |
| 1309 | dependencies, the dependencies are not reconnected. For example, suppose |
| 1310 | you have three tasks named ``do_a``, ``do_b``, and ``do_c``. |
| 1311 | Furthermore, ``do_c`` is dependent on ``do_b``, which in turn is |
| 1312 | dependent on ``do_a``. Given this scenario, if you use ``deltask`` to |
| 1313 | delete ``do_b``, the implicit dependency relationship between ``do_c`` |
| 1314 | and ``do_a`` through ``do_b`` no longer exists, and ``do_c`` |
| 1315 | dependencies are not updated to include ``do_a``. Thus, ``do_c`` is free |
| 1316 | to run before ``do_a``. |
| 1317 | |
| 1318 | If you want dependencies such as these to remain intact, use the |
| 1319 | ``[noexec]`` varflag to disable the task instead of using the |
| 1320 | ``deltask`` command to delete it: :: |
| 1321 | |
| 1322 | do_b[noexec] = "1" |
| 1323 | |
| 1324 | Passing Information Into the Build Task Environment |
| 1325 | --------------------------------------------------- |
| 1326 | |
| 1327 | When running a task, BitBake tightly controls the shell execution |
| 1328 | environment of the build tasks to make sure unwanted contamination from |
| 1329 | the build machine cannot influence the build. |
| 1330 | |
| 1331 | .. note:: |
| 1332 | |
| 1333 | By default, BitBake cleans the environment to include only those |
| 1334 | things exported or listed in its whitelist to ensure that the build |
| 1335 | environment is reproducible and consistent. You can prevent this |
| 1336 | "cleaning" by setting the :term:`BB_PRESERVE_ENV` variable. |
| 1337 | |
| 1338 | Consequently, if you do want something to get passed into the build task |
| 1339 | environment, you must take these two steps: |
| 1340 | |
| 1341 | #. Tell BitBake to load what you want from the environment into the |
| 1342 | datastore. You can do so through the |
| 1343 | :term:`BB_ENV_WHITELIST` and |
| 1344 | :term:`BB_ENV_EXTRAWHITE` variables. For |
| 1345 | example, assume you want to prevent the build system from accessing |
| 1346 | your ``$HOME/.ccache`` directory. The following command "whitelists" |
| 1347 | the environment variable ``CCACHE_DIR`` causing BitBake to allow that |
| 1348 | variable into the datastore: :: |
| 1349 | |
| 1350 | export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE CCACHE_DIR" |
| 1351 | |
| 1352 | #. Tell BitBake to export what you have loaded into the datastore to the |
| 1353 | task environment of every running task. Loading something from the |
| 1354 | environment into the datastore (previous step) only makes it |
| 1355 | available in the datastore. To export it to the task environment of |
| 1356 | every running task, use a command similar to the following in your |
| 1357 | local configuration file ``local.conf`` or your distribution |
| 1358 | configuration file: :: |
| 1359 | |
| 1360 | export CCACHE_DIR |
| 1361 | |
| 1362 | .. note:: |
| 1363 | |
| 1364 | A side effect of the previous steps is that BitBake records the |
| 1365 | variable as a dependency of the build process in things like the |
| 1366 | setscene checksums. If doing so results in unnecessary rebuilds of |
| 1367 | tasks, you can whitelist the variable so that the setscene code |
| 1368 | ignores the dependency when it creates checksums. |
| 1369 | |
| 1370 | Sometimes, it is useful to be able to obtain information from the |
| 1371 | original execution environment. BitBake saves a copy of the original |
| 1372 | environment into a special variable named :term:`BB_ORIGENV`. |
| 1373 | |
| 1374 | The ``BB_ORIGENV`` variable returns a datastore object that can be |
| 1375 | queried using the standard datastore operators such as |
| 1376 | ``getVar(, False)``. The datastore object is useful, for example, to |
| 1377 | find the original ``DISPLAY`` variable. Here is an example: :: |
| 1378 | |
| 1379 | origenv = d.getVar("BB_ORIGENV", False) |
| 1380 | bar = origenv.getVar("BAR", False) |
| 1381 | |
| 1382 | The previous example returns ``BAR`` from the original execution |
| 1383 | environment. |
| 1384 | |
| 1385 | Variable Flags |
| 1386 | ============== |
| 1387 | |
| 1388 | Variable flags (varflags) help control a task's functionality and |
| 1389 | dependencies. BitBake reads and writes varflags to the datastore using |
| 1390 | the following command forms: :: |
| 1391 | |
| 1392 | variable = d.getVarFlags("variable") |
| 1393 | self.d.setVarFlags("FOO", {"func": True}) |
| 1394 | |
| 1395 | When working with varflags, the same syntax, with the exception of |
| 1396 | overrides, applies. In other words, you can set, append, and prepend |
| 1397 | varflags just like variables. See the |
| 1398 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:variable flag syntax`" section for details. |
| 1399 | |
| 1400 | BitBake has a defined set of varflags available for recipes and classes. |
| 1401 | Tasks support a number of these flags which control various |
| 1402 | functionality of the task: |
| 1403 | |
| 1404 | - ``[cleandirs]``: Empty directories that should be created before |
| 1405 | the task runs. Directories that already exist are removed and |
| 1406 | recreated to empty them. |
| 1407 | |
| 1408 | - ``[depends]``: Controls inter-task dependencies. See the |
| 1409 | :term:`DEPENDS` variable and the |
| 1410 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:inter-task |
| 1411 | dependencies`" section for more information. |
| 1412 | |
| 1413 | - ``[deptask]``: Controls task build-time dependencies. See the |
| 1414 | :term:`DEPENDS` variable and the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:build dependencies`" section for more information. |
| 1415 | |
| 1416 | - ``[dirs]``: Directories that should be created before the task |
| 1417 | runs. Directories that already exist are left as is. The last |
| 1418 | directory listed is used as the current working directory for the |
| 1419 | task. |
| 1420 | |
| 1421 | - ``[lockfiles]``: Specifies one or more lockfiles to lock while the |
| 1422 | task executes. Only one task may hold a lockfile, and any task that |
| 1423 | attempts to lock an already locked file will block until the lock is |
| 1424 | released. You can use this variable flag to accomplish mutual |
| 1425 | exclusion. |
| 1426 | |
| 1427 | - ``[noexec]``: When set to "1", marks the task as being empty, with |
| 1428 | no execution required. You can use the ``[noexec]`` flag to set up |
| 1429 | tasks as dependency placeholders, or to disable tasks defined |
| 1430 | elsewhere that are not needed in a particular recipe. |
| 1431 | |
| 1432 | - ``[nostamp]``: When set to "1", tells BitBake to not generate a |
| 1433 | stamp file for a task, which implies the task should always be |
| 1434 | executed. |
| 1435 | |
| 1436 | .. caution:: |
| 1437 | |
| 1438 | Any task that depends (possibly indirectly) on a ``[nostamp]`` task will |
| 1439 | always be executed as well. This can cause unnecessary rebuilding if you |
| 1440 | are not careful. |
| 1441 | |
| 1442 | - ``[number_threads]``: Limits tasks to a specific number of |
| 1443 | simultaneous threads during execution. This varflag is useful when |
| 1444 | your build host has a large number of cores but certain tasks need to |
| 1445 | be rate-limited due to various kinds of resource constraints (e.g. to |
| 1446 | avoid network throttling). ``number_threads`` works similarly to the |
| 1447 | :term:`BB_NUMBER_THREADS` variable but is task-specific. |
| 1448 | |
| 1449 | Set the value globally. For example, the following makes sure the |
| 1450 | ``do_fetch`` task uses no more than two simultaneous execution |
| 1451 | threads: do_fetch[number_threads] = "2" |
| 1452 | |
| 1453 | .. warning:: |
| 1454 | |
| 1455 | - Setting the varflag in individual recipes rather than globally |
| 1456 | can result in unpredictable behavior. |
| 1457 | |
| 1458 | - Setting the varflag to a value greater than the value used in |
| 1459 | the ``BB_NUMBER_THREADS`` variable causes ``number_threads`` to |
| 1460 | have no effect. |
| 1461 | |
| 1462 | - ``[postfuncs]``: List of functions to call after the completion of |
| 1463 | the task. |
| 1464 | |
| 1465 | - ``[prefuncs]``: List of functions to call before the task executes. |
| 1466 | |
| 1467 | - ``[rdepends]``: Controls inter-task runtime dependencies. See the |
| 1468 | :term:`RDEPENDS` variable, the |
| 1469 | :term:`RRECOMMENDS` variable, and the |
| 1470 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:inter-task dependencies`" section for |
| 1471 | more information. |
| 1472 | |
| 1473 | - ``[rdeptask]``: Controls task runtime dependencies. See the |
| 1474 | :term:`RDEPENDS` variable, the |
| 1475 | :term:`RRECOMMENDS` variable, and the |
| 1476 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:runtime dependencies`" section for more |
| 1477 | information. |
| 1478 | |
| 1479 | - ``[recideptask]``: When set in conjunction with ``recrdeptask``, |
| 1480 | specifies a task that should be inspected for additional |
| 1481 | dependencies. |
| 1482 | |
| 1483 | - ``[recrdeptask]``: Controls task recursive runtime dependencies. |
| 1484 | See the :term:`RDEPENDS` variable, the |
| 1485 | :term:`RRECOMMENDS` variable, and the |
| 1486 | ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:recursive dependencies`" section for |
| 1487 | more information. |
| 1488 | |
| 1489 | - ``[stamp-extra-info]``: Extra stamp information to append to the |
| 1490 | task's stamp. As an example, OpenEmbedded uses this flag to allow |
| 1491 | machine-specific tasks. |
| 1492 | |
| 1493 | - ``[umask]``: The umask to run the task under. |
| 1494 | |
| 1495 | Several varflags are useful for controlling how signatures are |
| 1496 | calculated for variables. For more information on this process, see the |
| 1497 | ":ref:`bitbake-user-manual/bitbake-user-manual-execution:checksums (signatures)`" section. |
| 1498 | |
| 1499 | - ``[vardeps]``: Specifies a space-separated list of additional |
| 1500 | variables to add to a variable's dependencies for the purposes of |
| 1501 | calculating its signature. Adding variables to this list is useful, |
| 1502 | for example, when a function refers to a variable in a manner that |
| 1503 | does not allow BitBake to automatically determine that the variable |
| 1504 | is referred to. |
| 1505 | |
| 1506 | - ``[vardepsexclude]``: Specifies a space-separated list of variables |
| 1507 | that should be excluded from a variable's dependencies for the |
| 1508 | purposes of calculating its signature. |
| 1509 | |
| 1510 | - ``[vardepvalue]``: If set, instructs BitBake to ignore the actual |
| 1511 | value of the variable and instead use the specified value when |
| 1512 | calculating the variable's signature. |
| 1513 | |
| 1514 | - ``[vardepvalueexclude]``: Specifies a pipe-separated list of |
| 1515 | strings to exclude from the variable's value when calculating the |
| 1516 | variable's signature. |
| 1517 | |
| 1518 | Events |
| 1519 | ====== |
| 1520 | |
| 1521 | BitBake allows installation of event handlers within recipe and class |
| 1522 | files. Events are triggered at certain points during operation, such as |
| 1523 | the beginning of operation against a given recipe (i.e. ``*.bb``), the |
| 1524 | start of a given task, a task failure, a task success, and so forth. The |
| 1525 | intent is to make it easy to do things like email notification on build |
| 1526 | failures. |
| 1527 | |
| 1528 | Following is an example event handler that prints the name of the event |
| 1529 | and the content of the ``FILE`` variable: :: |
| 1530 | |
| 1531 | addhandler myclass_eventhandler |
| 1532 | python myclass_eventhandler() { |
| 1533 | from bb.event import getName |
| 1534 | print("The name of the Event is %s" % getName(e)) |
| 1535 | print("The file we run for is %s" % d.getVar('FILE')) |
| 1536 | } |
| 1537 | myclass_eventhandler[eventmask] = "bb.event.BuildStarted |
| 1538 | bb.event.BuildCompleted" |
| 1539 | |
| 1540 | In the previous example, an eventmask has been |
| 1541 | set so that the handler only sees the "BuildStarted" and |
| 1542 | "BuildCompleted" events. This event handler gets called every time an |
| 1543 | event matching the eventmask is triggered. A global variable "e" is |
| 1544 | defined, which represents the current event. With the ``getName(e)`` |
| 1545 | method, you can get the name of the triggered event. The global |
| 1546 | datastore is available as "d". In legacy code, you might see "e.data" |
| 1547 | used to get the datastore. However, realize that "e.data" is deprecated |
| 1548 | and you should use "d" going forward. |
| 1549 | |
| 1550 | The context of the datastore is appropriate to the event in question. |
| 1551 | For example, "BuildStarted" and "BuildCompleted" events run before any |
| 1552 | tasks are executed so would be in the global configuration datastore |
| 1553 | namespace. No recipe-specific metadata exists in that namespace. The |
| 1554 | "BuildStarted" and "BuildCompleted" events also run in the main |
| 1555 | cooker/server process rather than any worker context. Thus, any changes |
| 1556 | made to the datastore would be seen by other cooker/server events within |
| 1557 | the current build but not seen outside of that build or in any worker |
| 1558 | context. Task events run in the actual tasks in question consequently |
| 1559 | have recipe-specific and task-specific contents. These events run in the |
| 1560 | worker context and are discarded at the end of task execution. |
| 1561 | |
| 1562 | During a standard build, the following common events might occur. The |
| 1563 | following events are the most common kinds of events that most metadata |
| 1564 | might have an interest in viewing: |
| 1565 | |
| 1566 | - ``bb.event.ConfigParsed()``: Fired when the base configuration; which |
| 1567 | consists of ``bitbake.conf``, ``base.bbclass`` and any global |
| 1568 | ``INHERIT`` statements; has been parsed. You can see multiple such |
| 1569 | events when each of the workers parse the base configuration or if |
| 1570 | the server changes configuration and reparses. Any given datastore |
| 1571 | only has one such event executed against it, however. If |
| 1572 | ```BB_INVALIDCONF`` <#>`__ is set in the datastore by the event |
| 1573 | handler, the configuration is reparsed and a new event triggered, |
| 1574 | allowing the metadata to update configuration. |
| 1575 | |
| 1576 | - ``bb.event.HeartbeatEvent()``: Fires at regular time intervals of one |
| 1577 | second. You can configure the interval time using the |
| 1578 | ``BB_HEARTBEAT_EVENT`` variable. The event's "time" attribute is the |
| 1579 | ``time.time()`` value when the event is triggered. This event is |
| 1580 | useful for activities such as system state monitoring. |
| 1581 | |
| 1582 | - ``bb.event.ParseStarted()``: Fired when BitBake is about to start |
| 1583 | parsing recipes. This event's "total" attribute represents the number |
| 1584 | of recipes BitBake plans to parse. |
| 1585 | |
| 1586 | - ``bb.event.ParseProgress()``: Fired as parsing progresses. This |
| 1587 | event's "current" attribute is the number of recipes parsed as well |
| 1588 | as the "total" attribute. |
| 1589 | |
| 1590 | - ``bb.event.ParseCompleted()``: Fired when parsing is complete. This |
| 1591 | event's "cached", "parsed", "skipped", "virtuals", "masked", and |
| 1592 | "errors" attributes provide statistics for the parsing results. |
| 1593 | |
| 1594 | - ``bb.event.BuildStarted()``: Fired when a new build starts. BitBake |
| 1595 | fires multiple "BuildStarted" events (one per configuration) when |
| 1596 | multiple configuration (multiconfig) is enabled. |
| 1597 | |
| 1598 | - ``bb.build.TaskStarted()``: Fired when a task starts. This event's |
| 1599 | "taskfile" attribute points to the recipe from which the task |
| 1600 | originates. The "taskname" attribute, which is the task's name, |
| 1601 | includes the ``do_`` prefix, and the "logfile" attribute point to |
| 1602 | where the task's output is stored. Finally, the "time" attribute is |
| 1603 | the task's execution start time. |
| 1604 | |
| 1605 | - ``bb.build.TaskInvalid()``: Fired if BitBake tries to execute a task |
| 1606 | that does not exist. |
| 1607 | |
| 1608 | - ``bb.build.TaskFailedSilent()``: Fired for setscene tasks that fail |
| 1609 | and should not be presented to the user verbosely. |
| 1610 | |
| 1611 | - ``bb.build.TaskFailed()``: Fired for normal tasks that fail. |
| 1612 | |
| 1613 | - ``bb.build.TaskSucceeded()``: Fired when a task successfully |
| 1614 | completes. |
| 1615 | |
| 1616 | - ``bb.event.BuildCompleted()``: Fired when a build finishes. |
| 1617 | |
| 1618 | - ``bb.cooker.CookerExit()``: Fired when the BitBake server/cooker |
| 1619 | shuts down. This event is usually only seen by the UIs as a sign they |
| 1620 | should also shutdown. |
| 1621 | |
| 1622 | This next list of example events occur based on specific requests to the |
| 1623 | server. These events are often used to communicate larger pieces of |
| 1624 | information from the BitBake server to other parts of BitBake such as |
| 1625 | user interfaces: |
| 1626 | |
| 1627 | - ``bb.event.TreeDataPreparationStarted()`` |
| 1628 | - ``bb.event.TreeDataPreparationProgress()`` |
| 1629 | - ``bb.event.TreeDataPreparationCompleted()`` |
| 1630 | - ``bb.event.DepTreeGenerated()`` |
| 1631 | - ``bb.event.CoreBaseFilesFound()`` |
| 1632 | - ``bb.event.ConfigFilePathFound()`` |
| 1633 | - ``bb.event.FilesMatchingFound()`` |
| 1634 | - ``bb.event.ConfigFilesFound()`` |
| 1635 | - ``bb.event.TargetsTreeGenerated()`` |
| 1636 | |
| 1637 | .. _variants-class-extension-mechanism: |
| 1638 | |
| 1639 | Variants - Class Extension Mechanism |
| 1640 | ==================================== |
| 1641 | |
| 1642 | BitBake supports two features that facilitate creating from a single |
| 1643 | recipe file multiple incarnations of that recipe file where all |
| 1644 | incarnations are buildable. These features are enabled through the |
| 1645 | :term:`BBCLASSEXTEND` and :term:`BBVERSIONS` variables. |
| 1646 | |
| 1647 | .. note:: |
| 1648 | |
| 1649 | The mechanism for this class extension is extremely specific to the |
| 1650 | implementation. Usually, the recipe's :term:`PROVIDES` , :term:`PN` , and |
| 1651 | :term:`DEPENDS` variables would need to be modified by the extension |
| 1652 | class. For specific examples, see the OE-Core native , nativesdk , and |
| 1653 | multilib classes. |
| 1654 | |
| 1655 | - ``BBCLASSEXTEND``: This variable is a space separated list of |
| 1656 | classes used to "extend" the recipe for each variant. Here is an |
| 1657 | example that results in a second incarnation of the current recipe |
| 1658 | being available. This second incarnation will have the "native" class |
| 1659 | inherited. :: |
| 1660 | |
| 1661 | BBCLASSEXTEND = "native" |
| 1662 | |
| 1663 | - ``BBVERSIONS``: This variable allows a single recipe to build |
| 1664 | multiple versions of a project from a single recipe file. You can |
| 1665 | also specify conditional metadata (using the |
| 1666 | :term:`OVERRIDES` mechanism) for a single |
| 1667 | version, or an optionally named range of versions. Here is an |
| 1668 | example: :: |
| 1669 | |
| 1670 | BBVERSIONS = "1.0 2.0 git" |
| 1671 | SRC_URI_git = "git://someurl/somepath.git" |
| 1672 | |
| 1673 | BBVERSIONS = "1.0.[0-6]:1.0.0+ 1.0.[7-9]:1.0.7+" |
| 1674 | SRC_URI_append_1.0.7+ = "file://some_patch_which_the_new_versions_need.patch;patch=1" |
| 1675 | |
| 1676 | The name of the range defaults to the original version of the recipe. For |
| 1677 | example, in OpenEmbedded, the recipe file ``foo_1.0.0+.bb`` creates a default |
| 1678 | name range of ``1.0.0+``. This is useful because the range name is not only |
| 1679 | placed into overrides, but it is also made available for the metadata to use |
| 1680 | in the variable that defines the base recipe versions for use in ``file://`` |
| 1681 | search paths (:term:`FILESPATH`). |
| 1682 | |
| 1683 | Dependencies |
| 1684 | ============ |
| 1685 | |
| 1686 | To allow for efficient parallel processing, BitBake handles dependencies |
| 1687 | at the task level. Dependencies can exist both between tasks within a |
| 1688 | single recipe and between tasks in different recipes. Following are |
| 1689 | examples of each: |
| 1690 | |
| 1691 | - For tasks within a single recipe, a recipe's ``do_configure`` task |
| 1692 | might need to complete before its ``do_compile`` task can run. |
| 1693 | |
| 1694 | - For tasks in different recipes, one recipe's ``do_configure`` task |
| 1695 | might require another recipe's ``do_populate_sysroot`` task to finish |
| 1696 | first such that the libraries and headers provided by the other |
| 1697 | recipe are available. |
| 1698 | |
| 1699 | This section describes several ways to declare dependencies. Remember, |
| 1700 | even though dependencies are declared in different ways, they are all |
| 1701 | simply dependencies between tasks. |
| 1702 | |
| 1703 | .. _dependencies-internal-to-the-bb-file: |
| 1704 | |
| 1705 | Dependencies Internal to the ``.bb`` File |
| 1706 | ----------------------------------------- |
| 1707 | |
| 1708 | BitBake uses the ``addtask`` directive to manage dependencies that are |
| 1709 | internal to a given recipe file. You can use the ``addtask`` directive |
| 1710 | to indicate when a task is dependent on other tasks or when other tasks |
| 1711 | depend on that recipe. Here is an example: :: |
| 1712 | |
| 1713 | addtask printdate after do_fetch before do_build |
| 1714 | |
| 1715 | In this example, the ``do_printdate`` task |
| 1716 | depends on the completion of the ``do_fetch`` task, and the ``do_build`` |
| 1717 | task depends on the completion of the ``do_printdate`` task. |
| 1718 | |
| 1719 | .. note:: |
| 1720 | |
| 1721 | For a task to run, it must be a direct or indirect dependency of some |
| 1722 | other task that is scheduled to run. |
| 1723 | |
| 1724 | For illustration, here are some examples: |
| 1725 | |
| 1726 | - The directive ``addtask mytask before do_configure`` causes |
| 1727 | ``do_mytask`` to run before ``do_configure`` runs. Be aware that |
| 1728 | ``do_mytask`` still only runs if its :ref:`input |
| 1729 | checksum <bitbake-user-manual/bitbake-user-manual-execution:checksums (signatures)>` has changed since the last time it was |
| 1730 | run. Changes to the input checksum of ``do_mytask`` also |
| 1731 | indirectly cause ``do_configure`` to run. |
| 1732 | |
| 1733 | - The directive ``addtask mytask after do_configure`` by itself |
| 1734 | never causes ``do_mytask`` to run. ``do_mytask`` can still be run |
| 1735 | manually as follows: :: |
| 1736 | |
| 1737 | $ bitbake recipe -c mytask |
| 1738 | |
| 1739 | Declaring ``do_mytask`` as a dependency of some other task that is |
| 1740 | scheduled to run also causes it to run. Regardless, the task runs after |
| 1741 | ``do_configure``. |
| 1742 | |
| 1743 | Build Dependencies |
| 1744 | ------------------ |
| 1745 | |
| 1746 | BitBake uses the :term:`DEPENDS` variable to manage |
| 1747 | build time dependencies. The ``[deptask]`` varflag for tasks signifies |
| 1748 | the task of each item listed in ``DEPENDS`` that must complete before |
| 1749 | that task can be executed. Here is an example: :: |
| 1750 | |
| 1751 | do_configure[deptask] = "do_populate_sysroot" |
| 1752 | |
| 1753 | In this example, the ``do_populate_sysroot`` task |
| 1754 | of each item in ``DEPENDS`` must complete before ``do_configure`` can |
| 1755 | execute. |
| 1756 | |
| 1757 | Runtime Dependencies |
| 1758 | -------------------- |
| 1759 | |
| 1760 | BitBake uses the :term:`PACKAGES`, :term:`RDEPENDS`, and :term:`RRECOMMENDS` |
| 1761 | variables to manage runtime dependencies. |
| 1762 | |
| 1763 | The ``PACKAGES`` variable lists runtime packages. Each of those packages |
| 1764 | can have ``RDEPENDS`` and ``RRECOMMENDS`` runtime dependencies. The |
| 1765 | ``[rdeptask]`` flag for tasks is used to signify the task of each item |
| 1766 | runtime dependency which must have completed before that task can be |
| 1767 | executed. :: |
| 1768 | |
| 1769 | do_package_qa[rdeptask] = "do_packagedata" |
| 1770 | |
| 1771 | In the previous |
| 1772 | example, the ``do_packagedata`` task of each item in ``RDEPENDS`` must |
| 1773 | have completed before ``do_package_qa`` can execute. |
| 1774 | Although ``RDEPENDS`` contains entries from the |
| 1775 | runtime dependency namespace, BitBake knows how to map them back |
| 1776 | to the build-time dependency namespace, in which the tasks are defined. |
| 1777 | |
| 1778 | Recursive Dependencies |
| 1779 | ---------------------- |
| 1780 | |
| 1781 | BitBake uses the ``[recrdeptask]`` flag to manage recursive task |
| 1782 | dependencies. BitBake looks through the build-time and runtime |
| 1783 | dependencies of the current recipe, looks through the task's inter-task |
| 1784 | dependencies, and then adds dependencies for the listed task. Once |
| 1785 | BitBake has accomplished this, it recursively works through the |
| 1786 | dependencies of those tasks. Iterative passes continue until all |
| 1787 | dependencies are discovered and added. |
| 1788 | |
| 1789 | The ``[recrdeptask]`` flag is most commonly used in high-level recipes |
| 1790 | that need to wait for some task to finish "globally". For example, |
| 1791 | ``image.bbclass`` has the following: :: |
| 1792 | |
| 1793 | do_rootfs[recrdeptask] += "do_packagedata" |
| 1794 | |
| 1795 | This statement says that the ``do_packagedata`` task of |
| 1796 | the current recipe and all recipes reachable (by way of dependencies) |
| 1797 | from the image recipe must run before the ``do_rootfs`` task can run. |
| 1798 | |
| 1799 | BitBake allows a task to recursively depend on itself by |
| 1800 | referencing itself in the task list: :: |
| 1801 | |
| 1802 | do_a[recrdeptask] = "do_a do_b" |
| 1803 | |
| 1804 | In the same way as before, this means that the ``do_a`` |
| 1805 | and ``do_b`` tasks of the current recipe and all |
| 1806 | recipes reachable (by way of dependencies) from the recipe |
| 1807 | must run before the ``do_a`` task can run. In this |
| 1808 | case BitBake will ignore the current recipe's ``do_a`` |
| 1809 | task circular dependency on itself. |
| 1810 | |
| 1811 | Inter-Task Dependencies |
| 1812 | ----------------------- |
| 1813 | |
| 1814 | BitBake uses the ``[depends]`` flag in a more generic form to manage |
| 1815 | inter-task dependencies. This more generic form allows for |
| 1816 | inter-dependency checks for specific tasks rather than checks for the |
| 1817 | data in ``DEPENDS``. Here is an example: :: |
| 1818 | |
| 1819 | do_patch[depends] = "quilt-native:do_populate_sysroot" |
| 1820 | |
| 1821 | In this example, the ``do_populate_sysroot`` task of the target ``quilt-native`` |
| 1822 | must have completed before the ``do_patch`` task can execute. |
| 1823 | |
| 1824 | The ``[rdepends]`` flag works in a similar way but takes targets in the |
| 1825 | runtime namespace instead of the build-time dependency namespace. |
| 1826 | |
| 1827 | Functions You Can Call From Within Python |
| 1828 | ========================================= |
| 1829 | |
| 1830 | BitBake provides many functions you can call from within Python |
| 1831 | functions. This section lists the most commonly used functions, and |
| 1832 | mentions where to find others. |
| 1833 | |
| 1834 | Functions for Accessing Datastore Variables |
| 1835 | ------------------------------------------- |
| 1836 | |
| 1837 | It is often necessary to access variables in the BitBake datastore using |
| 1838 | Python functions. The BitBake datastore has an API that allows you this |
| 1839 | access. Here is a list of available operations: |
| 1840 | |
| 1841 | .. list-table:: |
| 1842 | :widths: auto |
| 1843 | :header-rows: 1 |
| 1844 | |
| 1845 | * - *Operation* |
| 1846 | - *Description* |
| 1847 | * - ``d.getVar("X", expand)`` |
| 1848 | - Returns the value of variable "X". Using "expand=True" expands the |
| 1849 | value. Returns "None" if the variable "X" does not exist. |
| 1850 | * - ``d.setVar("X", "value")`` |
| 1851 | - Sets the variable "X" to "value" |
| 1852 | * - ``d.appendVar("X", "value")`` |
| 1853 | - Adds "value" to the end of the variable "X". Acts like ``d.setVar("X", |
| 1854 | "value")`` if the variable "X" does not exist. |
| 1855 | * - ``d.prependVar("X", "value")`` |
| 1856 | - Adds "value" to the start of the variable "X". Acts like |
| 1857 | ``d.setVar("X","value")`` if the variable "X" does not exist. |
| 1858 | * - ``d.delVar("X")`` |
| 1859 | - Deletes the variable "X" from the datastore. Does nothing if the variable |
| 1860 | "X" does not exist. |
| 1861 | * - ``d.renameVar("X", "Y")`` |
| 1862 | - Renames the variable "X" to "Y". Does nothing if the variable "X" does |
| 1863 | not exist. |
| 1864 | * - ``d.getVarFlag("X", flag, expand)`` |
| 1865 | - Returns the value of variable "X". Using "expand=True" expands the |
| 1866 | value. Returns "None" if either the variable "X" or the named flag does |
| 1867 | not exist. |
| 1868 | * - ``d.setVarFlag("X", flag, "value")`` |
| 1869 | - Sets the named flag for variable "X" to "value". |
| 1870 | * - ``d.appendVarFlag("X", flag, "value")`` |
| 1871 | - Appends "value" to the named flag on the variable "X". Acts like |
| 1872 | ``d.setVarFlag("X", flag, "value")`` if the named flag does not exist. |
| 1873 | * - ``d.prependVarFlag("X", flag, "value")`` |
| 1874 | - Prepends "value" to the named flag on the variable "X". Acts like |
| 1875 | ``d.setVarFlag("X", flag, "value")`` if the named flag does not exist. |
| 1876 | * - ``d.delVarFlag("X", flag)`` |
| 1877 | - Deletes the named flag on the variable "X" from the datastore. |
| 1878 | * - ``d.setVarFlags("X", flagsdict)`` |
| 1879 | - Sets the flags specified in the ``flagsdict()`` |
| 1880 | parameter. ``setVarFlags`` does not clear previous flags. Think of this |
| 1881 | operation as ``addVarFlags``. |
| 1882 | * - ``d.getVarFlags("X")`` |
| 1883 | - Returns a ``flagsdict`` of the flags for the variable "X". Returns "None" |
| 1884 | if the variable "X" does not exist. |
| 1885 | * - ``d.delVarFlags("X")`` |
| 1886 | - Deletes all the flags for the variable "X". Does nothing if the variable |
| 1887 | "X" does not exist. |
| 1888 | * - ``d.expand(expression)`` |
| 1889 | - Expands variable references in the specified string |
| 1890 | expression. References to variables that do not exist are left as is. For |
| 1891 | example, ``d.expand("foo ${X}")`` expands to the literal string "foo |
| 1892 | ${X}" if the variable "X" does not exist. |
| 1893 | |
| 1894 | Other Functions |
| 1895 | --------------- |
| 1896 | |
| 1897 | You can find many other functions that can be called from Python by |
| 1898 | looking at the source code of the ``bb`` module, which is in |
| 1899 | ``bitbake/lib/bb``. For example, ``bitbake/lib/bb/utils.py`` includes |
| 1900 | the commonly used functions ``bb.utils.contains()`` and |
| 1901 | ``bb.utils.mkdirhier()``, which come with docstrings. |
| 1902 | |
| 1903 | Task Checksums and Setscene |
| 1904 | =========================== |
| 1905 | |
| 1906 | BitBake uses checksums (or signatures) along with the setscene to |
| 1907 | determine if a task needs to be run. This section describes the process. |
| 1908 | To help understand how BitBake does this, the section assumes an |
| 1909 | OpenEmbedded metadata-based example. |
| 1910 | |
| 1911 | These checksums are stored in :term:`STAMP`. You can |
| 1912 | examine the checksums using the following BitBake command: :: |
| 1913 | |
| 1914 | $ bitbake-dumpsigs |
| 1915 | |
| 1916 | This command returns the signature data in a readable |
| 1917 | format that allows you to examine the inputs used when the OpenEmbedded |
| 1918 | build system generates signatures. For example, using |
| 1919 | ``bitbake-dumpsigs`` allows you to examine the ``do_compile`` task's |
Andrew Geissler | f034379 | 2020-11-18 10:42:21 -0600 | [diff] [blame] | 1920 | "sigdata" for a C application (e.g. ``bash``). Running the command also |
| 1921 | reveals that the "CC" variable is part of the inputs that are hashed. |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 1922 | Any changes to this variable would invalidate the stamp and cause the |
| 1923 | ``do_compile`` task to run. |
| 1924 | |
| 1925 | The following list describes related variables: |
| 1926 | |
| 1927 | - :term:`BB_HASHCHECK_FUNCTION`: |
| 1928 | Specifies the name of the function to call during the "setscene" part |
| 1929 | of the task's execution in order to validate the list of task hashes. |
| 1930 | |
| 1931 | - :term:`BB_SETSCENE_DEPVALID`: |
| 1932 | Specifies a function BitBake calls that determines whether BitBake |
| 1933 | requires a setscene dependency to be met. |
| 1934 | |
| 1935 | - :term:`BB_SETSCENE_VERIFY_FUNCTION2`: |
| 1936 | Specifies a function to call that verifies the list of planned task |
| 1937 | execution before the main task execution happens. |
| 1938 | |
| 1939 | - :term:`BB_STAMP_POLICY`: Defines the mode |
| 1940 | for comparing timestamps of stamp files. |
| 1941 | |
| 1942 | - :term:`BB_STAMP_WHITELIST`: Lists stamp |
| 1943 | files that are looked at when the stamp policy is "whitelist". |
| 1944 | |
| 1945 | - :term:`BB_TASKHASH`: Within an executing task, |
| 1946 | this variable holds the hash of the task as returned by the currently |
| 1947 | enabled signature generator. |
| 1948 | |
| 1949 | - :term:`STAMP`: The base path to create stamp files. |
| 1950 | |
| 1951 | - :term:`STAMPCLEAN`: Again, the base path to |
| 1952 | create stamp files but can use wildcards for matching a range of |
| 1953 | files for clean operations. |
| 1954 | |
| 1955 | Wildcard Support in Variables |
| 1956 | ============================= |
| 1957 | |
| 1958 | Support for wildcard use in variables varies depending on the context in |
| 1959 | which it is used. For example, some variables and file names allow |
| 1960 | limited use of wildcards through the "``%``" and "``*``" characters. |
| 1961 | Other variables or names support Python's |
| 1962 | `glob <https://docs.python.org/3/library/glob.html>`_ syntax, |
| 1963 | `fnmatch <https://docs.python.org/3/library/fnmatch.html#module-fnmatch>`_ |
| 1964 | syntax, or |
| 1965 | `Regular Expression (re) <https://docs.python.org/3/library/re.html>`_ |
| 1966 | syntax. |
| 1967 | |
| 1968 | For variables that have wildcard suport, the documentation describes |
| 1969 | which form of wildcard, its use, and its limitations. |