blob: 119720d527d5be95fb96068367e7722ceaed5304 [file] [log] [blame]
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001.. SPDX-License-Identifier: CC-BY-2.5
2
3====================
4Syntax and Operators
5====================
6
7|
8
9BitBake files have their own syntax. The syntax has similarities to
10several other languages but also has some unique features. This section
11describes the available syntax and operators as well as provides
12examples.
13
14Basic Syntax
15============
16
17This section provides some basic syntax examples.
18
19Basic Variable Setting
20----------------------
21
22The following example sets ``VARIABLE`` to "value". This assignment
23occurs immediately as the statement is parsed. It is a "hard"
24assignment. ::
25
26 VARIABLE = "value"
27
28As expected, if you include leading or
Andrew Geisslerc926e172021-05-07 16:11:35 -050029trailing spaces as part of an assignment, the spaces are retained::
Andrew Geisslerc9f78652020-09-18 14:11:35 -050030
31 VARIABLE = " value"
32 VARIABLE = "value "
33
34Setting ``VARIABLE`` to "" sets
35it to an empty string, while setting the variable to " " sets it to a
36blank space (i.e. these are not the same values). ::
37
38 VARIABLE = ""
39 VARIABLE = " "
40
41You can use single quotes instead of double quotes when setting a
42variable's value. Doing so allows you to use values that contain the
Andrew Geisslerc926e172021-05-07 16:11:35 -050043double quote character::
Andrew Geisslerc9f78652020-09-18 14:11:35 -050044
45 VARIABLE = 'I have a " in my value'
46
47.. note::
48
49 Unlike in Bourne shells, single quotes work identically to double
50 quotes in all other ways. They do not suppress variable expansions.
51
52Modifying Existing Variables
53----------------------------
54
55Sometimes you need to modify existing variables. Following are some
56cases where you might find you want to modify an existing variable:
57
58- Customize a recipe that uses the variable.
59
60- Change a variable's default value used in a ``*.bbclass`` file.
61
62- Change the variable in a ``*.bbappend`` file to override the variable
63 in the original recipe.
64
65- Change the variable in a configuration file so that the value
66 overrides an existing configuration.
67
68Changing a variable value can sometimes depend on how the value was
69originally assigned and also on the desired intent of the change. In
70particular, when you append a value to a variable that has a default
71value, the resulting value might not be what you expect. In this case,
72the value you provide might replace the value rather than append to the
73default value.
74
75If after you have changed a variable's value and something unexplained
76occurs, you can use BitBake to check the actual value of the suspect
77variable. You can make these checks for both configuration and recipe
78level changes:
79
Andrew Geisslerc926e172021-05-07 16:11:35 -050080- For configuration changes, use the following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -050081
82 $ bitbake -e
83
84 This
85 command displays variable values after the configuration files (i.e.
86 ``local.conf``, ``bblayers.conf``, ``bitbake.conf`` and so forth)
87 have been parsed.
88
89 .. note::
90
91 Variables that are exported to the environment are preceded by the
92 string "export" in the command's output.
93
Patrick Williams213cb262021-08-07 19:21:33 -050094- To find changes to a given variable in a specific recipe, use the
95 following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -050096
Patrick Williams213cb262021-08-07 19:21:33 -050097 $ bitbake recipename -e | grep VARIABLENAME=\"
Andrew Geisslerc9f78652020-09-18 14:11:35 -050098
99 This command checks to see if the variable actually makes
100 it into a specific recipe.
101
102Line Joining
103------------
104
105Outside of :ref:`functions <bitbake-user-manual/bitbake-user-manual-metadata:functions>`,
106BitBake joins any line ending in
107a backslash character ("\") with the following line before parsing
108statements. The most common use for the "\" character is to split
Andrew Geisslerc926e172021-05-07 16:11:35 -0500109variable assignments over multiple lines, as in the following example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500110
111 FOO = "bar \
112 baz \
113 qaz"
114
115Both the "\" character and the newline
116character that follow it are removed when joining lines. Thus, no
117newline characters end up in the value of ``FOO``.
118
119Consider this additional example where the two assignments both assign
Andrew Geisslerc926e172021-05-07 16:11:35 -0500120"barbaz" to ``FOO``::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500121
122 FOO = "barbaz"
123 FOO = "bar\
124 baz"
125
126.. note::
127
128 BitBake does not interpret escape sequences like "\n" in variable
129 values. For these to have an effect, the value must be passed to some
130 utility that interprets escape sequences, such as
131 ``printf`` or ``echo -n``.
132
133Variable Expansion
134------------------
135
136Variables can reference the contents of other variables using a syntax
137that is similar to variable expansion in Bourne shells. The following
138assignments result in A containing "aval" and B evaluating to
139"preavalpost". ::
140
141 A = "aval"
142 B = "pre${A}post"
143
144.. note::
145
146 Unlike in Bourne shells, the curly braces are mandatory: Only ``${FOO}`` and not
147 ``$FOO`` is recognized as an expansion of ``FOO``.
148
149The "=" operator does not immediately expand variable references in the
150right-hand side. Instead, expansion is deferred until the variable
151assigned to is actually used. The result depends on the current values
152of the referenced variables. The following example should clarify this
Andrew Geisslerc926e172021-05-07 16:11:35 -0500153behavior::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500154
155 A = "${B} baz"
156 B = "${C} bar"
157 C = "foo"
158 *At this point, ${A} equals "foo bar baz"*
159 C = "qux"
160 *At this point, ${A} equals "qux bar baz"*
161 B = "norf"
162 *At this point, ${A} equals "norf baz"\*
163
164Contrast this behavior with the
165:ref:`bitbake-user-manual/bitbake-user-manual-metadata:immediate variable
166expansion (:=)` operator.
167
168If the variable expansion syntax is used on a variable that does not
169exist, the string is kept as is. For example, given the following
170assignment, ``BAR`` expands to the literal string "${FOO}" as long as
171``FOO`` does not exist. ::
172
173 BAR = "${FOO}"
174
175Setting a default value (?=)
176----------------------------
177
178You can use the "?=" operator to achieve a "softer" assignment for a
179variable. This type of assignment allows you to define a variable if it
180is undefined when the statement is parsed, but to leave the value alone
Andrew Geisslerc926e172021-05-07 16:11:35 -0500181if the variable has a value. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500182
183 A ?= "aval"
184
185If ``A`` is
186set at the time this statement is parsed, the variable retains its
187value. However, if ``A`` is not set, the variable is set to "aval".
188
189.. note::
190
191 This assignment is immediate. Consequently, if multiple "?="
192 assignments to a single variable exist, the first of those ends up
193 getting used.
194
195Setting a weak default value (??=)
196----------------------------------
197
198It is possible to use a "weaker" assignment than in the previous section
199by using the "??=" operator. This assignment behaves identical to "?="
200except that the assignment is made at the end of the parsing process
201rather than immediately. Consequently, when multiple "??=" assignments
202exist, the last one is used. Also, any "=" or "?=" assignment will
Andrew Geisslerc926e172021-05-07 16:11:35 -0500203override the value set with "??=". Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500204
205 A ??= "somevalue"
206 A ??= "someothervalue"
207
208If ``A`` is set before the above statements are
209parsed, the variable retains its value. If ``A`` is not set, the
210variable is set to "someothervalue".
211
212Again, this assignment is a "lazy" or "weak" assignment because it does
213not occur until the end of the parsing process.
214
215Immediate variable expansion (:=)
216---------------------------------
217
218The ":=" operator results in a variable's contents being expanded
Andrew Geisslerc926e172021-05-07 16:11:35 -0500219immediately, rather than when the variable is actually used::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500220
221 T = "123"
222 A := "test ${T}"
223 T = "456"
224 B := "${T} ${C}"
225 C = "cval"
226 C := "${C}append"
227
228In this example, ``A`` contains "test 123", even though the final value
Patrick Williams213cb262021-08-07 19:21:33 -0500229of :term:`T` is "456". The variable :term:`B` will end up containing "456
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500230cvalappend". This is because references to undefined variables are
231preserved as is during (immediate)expansion. This is in contrast to GNU
232Make, where undefined variables expand to nothing. The variable ``C``
233contains "cvalappend" since ``${C}`` immediately expands to "cval".
234
235.. _appending-and-prepending:
236
237Appending (+=) and prepending (=+) With Spaces
238----------------------------------------------
239
240Appending and prepending values is common and can be accomplished using
241the "+=" and "=+" operators. These operators insert a space between the
242current value and prepended or appended value.
243
244These operators take immediate effect during parsing. Here are some
Andrew Geisslerc926e172021-05-07 16:11:35 -0500245examples::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500246
247 B = "bval"
248 B += "additionaldata"
249 C = "cval"
250 C =+ "test"
251
Patrick Williams213cb262021-08-07 19:21:33 -0500252The variable :term:`B` contains "bval additionaldata" and ``C`` contains "test
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500253cval".
254
255.. _appending-and-prepending-without-spaces:
256
257Appending (.=) and Prepending (=.) Without Spaces
258-------------------------------------------------
259
260If you want to append or prepend values without an inserted space, use
261the ".=" and "=." operators.
262
263These operators take immediate effect during parsing. Here are some
Andrew Geisslerc926e172021-05-07 16:11:35 -0500264examples::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500265
266 B = "bval"
267 B .= "additionaldata"
268 C = "cval"
269 C =. "test"
270
Patrick Williams213cb262021-08-07 19:21:33 -0500271The variable :term:`B` contains "bvaladditionaldata" and ``C`` contains
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500272"testcval".
273
274Appending and Prepending (Override Style Syntax)
275------------------------------------------------
276
277You can also append and prepend a variable's value using an override
278style syntax. When you use this syntax, no spaces are inserted.
279
280These operators differ from the ":=", ".=", "=.", "+=", and "=+"
281operators in that their effects are applied at variable expansion time
Andrew Geisslerc926e172021-05-07 16:11:35 -0500282rather than being immediately applied. Here are some examples::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500283
284 B = "bval"
Patrick Williams213cb262021-08-07 19:21:33 -0500285 B:append = " additional data"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500286 C = "cval"
Patrick Williams213cb262021-08-07 19:21:33 -0500287 C:prepend = "additional data "
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500288 D = "dval"
Patrick Williams213cb262021-08-07 19:21:33 -0500289 D:append = "additional data"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500290
Patrick Williams213cb262021-08-07 19:21:33 -0500291The variable :term:`B`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500292becomes "bval additional data" and ``C`` becomes "additional data cval".
293The variable ``D`` becomes "dvaladditional data".
294
295.. note::
296
297 You must control all spacing when you use the override syntax.
298
299It is also possible to append and prepend to shell functions and
300BitBake-style Python functions. See the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:shell functions`" and ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:bitbake-style python functions`"
301sections for examples.
302
303.. _removing-override-style-syntax:
304
305Removal (Override Style Syntax)
306-------------------------------
307
308You can remove values from lists using the removal override style
309syntax. Specifying a value for removal causes all occurrences of that
310value to be removed from the variable.
311
312When you use this syntax, BitBake expects one or more strings.
Andrew Geisslerc926e172021-05-07 16:11:35 -0500313Surrounding spaces and spacing are preserved. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500314
315 FOO = "123 456 789 123456 123 456 123 456"
Patrick Williams213cb262021-08-07 19:21:33 -0500316 FOO:remove = "123"
317 FOO:remove = "456"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500318 FOO2 = " abc def ghi abcdef abc def abc def def"
Patrick Williams213cb262021-08-07 19:21:33 -0500319 FOO2:remove = "\
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500320 def \
321 abc \
322 ghi \
323 "
324
325The variable ``FOO`` becomes
326" 789 123456 " and ``FOO2`` becomes " abcdef ".
327
Patrick Williams213cb262021-08-07 19:21:33 -0500328Like ":append" and ":prepend", ":remove" is applied at variable
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500329expansion time.
330
331Override Style Operation Advantages
332-----------------------------------
333
Patrick Williams213cb262021-08-07 19:21:33 -0500334An advantage of the override style operations ":append", ":prepend", and
335":remove" as compared to the "+=" and "=+" operators is that the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500336override style operators provide guaranteed operations. For example,
337consider a class ``foo.bbclass`` that needs to add the value "val" to
Andrew Geisslerc926e172021-05-07 16:11:35 -0500338the variable ``FOO``, and a recipe that uses ``foo.bbclass`` as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500339
340 inherit foo
341 FOO = "initial"
342
343If ``foo.bbclass`` uses the "+=" operator,
344as follows, then the final value of ``FOO`` will be "initial", which is
Andrew Geisslerc926e172021-05-07 16:11:35 -0500345not what is desired::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500346
347 FOO += "val"
348
349If, on the other hand, ``foo.bbclass``
Patrick Williams213cb262021-08-07 19:21:33 -0500350uses the ":append" operator, then the final value of ``FOO`` will be
Andrew Geisslerc926e172021-05-07 16:11:35 -0500351"initial val", as intended::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500352
Patrick Williams213cb262021-08-07 19:21:33 -0500353 FOO:append = " val"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500354
355.. note::
356
Patrick Williams213cb262021-08-07 19:21:33 -0500357 It is never necessary to use "+=" together with ":append". The following
Andrew Geisslerc926e172021-05-07 16:11:35 -0500358 sequence of assignments appends "barbaz" to FOO::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500359
Patrick Williams213cb262021-08-07 19:21:33 -0500360 FOO:append = "bar"
361 FOO:append = "baz"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500362
363
364 The only effect of changing the second assignment in the previous
365 example to use "+=" would be to add a space before "baz" in the
366 appended value (due to how the "+=" operator works).
367
368Another advantage of the override style operations is that you can
369combine them with other overrides as described in the
370":ref:`bitbake-user-manual/bitbake-user-manual-metadata:conditional syntax (overrides)`" section.
371
372Variable Flag Syntax
373--------------------
374
375Variable flags are BitBake's implementation of variable properties or
376attributes. It is a way of tagging extra information onto a variable.
377You can find more out about variable flags in general in the
378":ref:`bitbake-user-manual/bitbake-user-manual-metadata:variable flags`" section.
379
380You can define, append, and prepend values to variable flags. All the
381standard syntax operations previously mentioned work for variable flags
Patrick Williams213cb262021-08-07 19:21:33 -0500382except for override style syntax (i.e. ":prepend", ":append", and
383":remove").
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500384
Andrew Geisslerc926e172021-05-07 16:11:35 -0500385Here are some examples showing how to set variable flags::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500386
387 FOO[a] = "abc"
388 FOO[b] = "123"
389 FOO[a] += "456"
390
391The variable ``FOO`` has two flags:
392``[a]`` and ``[b]``. The flags are immediately set to "abc" and "123",
393respectively. The ``[a]`` flag becomes "abc 456".
394
395No need exists to pre-define variable flags. You can simply start using
396them. One extremely common application is to attach some brief
Andrew Geisslerc926e172021-05-07 16:11:35 -0500397documentation to a BitBake variable as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500398
399 CACHE[doc] = "The directory holding the cache of the metadata."
400
401Inline Python Variable Expansion
402--------------------------------
403
404You can use inline Python variable expansion to set variables. Here is
Andrew Geisslerc926e172021-05-07 16:11:35 -0500405an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500406
407 DATE = "${@time.strftime('%Y%m%d',time.gmtime())}"
408
409This example results in the ``DATE`` variable being set to the current date.
410
411Probably the most common use of this feature is to extract the value of
412variables from BitBake's internal data dictionary, ``d``. The following
413lines select the values of a package name and its version number,
Andrew Geisslerc926e172021-05-07 16:11:35 -0500414respectively::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500415
Andrew Geissler5199d832021-09-24 16:47:35 -0500416 PN = "${@bb.parse.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'}"
417 PV = "${@bb.parse.vars_from_file(d.getVar('FILE', False),d)[1] or '1.0'}"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500418
419.. note::
420
421 Inline Python expressions work just like variable expansions insofar as the
422 "=" and ":=" operators are concerned. Given the following assignment, foo()
Andrew Geisslerc926e172021-05-07 16:11:35 -0500423 is called each time FOO is expanded::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500424
425 FOO = "${@foo()}"
426
427 Contrast this with the following immediate assignment, where foo() is only
Andrew Geisslerc926e172021-05-07 16:11:35 -0500428 called once, while the assignment is parsed::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500429
430 FOO := "${@foo()}"
431
432For a different way to set variables with Python code during parsing,
433see the
434":ref:`bitbake-user-manual/bitbake-user-manual-metadata:anonymous python functions`" section.
435
436Unsetting variables
437-------------------
438
439It is possible to completely remove a variable or a variable flag from
440BitBake's internal data dictionary by using the "unset" keyword. Here is
Andrew Geisslerc926e172021-05-07 16:11:35 -0500441an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500442
443 unset DATE
444 unset do_fetch[noexec]
445
446These two statements remove the ``DATE`` and the ``do_fetch[noexec]`` flag.
447
448Providing Pathnames
449-------------------
450
451When specifying pathnames for use with BitBake, do not use the tilde
452("~") character as a shortcut for your home directory. Doing so might
453cause BitBake to not recognize the path since BitBake does not expand
454this character in the same way a shell would.
455
Andrew Geisslerc926e172021-05-07 16:11:35 -0500456Instead, provide a fuller path as the following example illustrates::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500457
458 BBLAYERS ?= " \
459 /home/scott-lenovo/LayerA \
460 "
461
462Exporting Variables to the Environment
463======================================
464
465You can export variables to the environment of running tasks by using
466the ``export`` keyword. For example, in the following example, the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500467``do_foo`` task prints "value from the environment" when run::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500468
469 export ENV_VARIABLE
470 ENV_VARIABLE = "value from the environment"
471
472 do_foo() {
473 bbplain "$ENV_VARIABLE"
474 }
475
476.. note::
477
478 BitBake does not expand ``$ENV_VARIABLE`` in this case because it lacks the
479 obligatory ``{}`` . Rather, ``$ENV_VARIABLE`` is expanded by the shell.
480
481It does not matter whether ``export ENV_VARIABLE`` appears before or
482after assignments to ``ENV_VARIABLE``.
483
484It is also possible to combine ``export`` with setting a value for the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500485variable. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500486
487 export ENV_VARIABLE = "variable-value"
488
489In the output of ``bitbake -e``, variables that are exported to the
490environment are preceded by "export".
491
492Among the variables commonly exported to the environment are ``CC`` and
493``CFLAGS``, which are picked up by many build systems.
494
495Conditional Syntax (Overrides)
496==============================
497
498BitBake uses :term:`OVERRIDES` to control what
499variables are overridden after BitBake parses recipes and configuration
Patrick Williams213cb262021-08-07 19:21:33 -0500500files. This section describes how you can use :term:`OVERRIDES` as
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500501conditional metadata, talks about key expansion in relationship to
Patrick Williams213cb262021-08-07 19:21:33 -0500502:term:`OVERRIDES`, and provides some examples to help with understanding.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500503
504Conditional Metadata
505--------------------
506
Patrick Williams213cb262021-08-07 19:21:33 -0500507You can use :term:`OVERRIDES` to conditionally select a specific version of
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500508a variable and to conditionally append or prepend the value of a
509variable.
510
511.. note::
512
513 Overrides can only use lower-case characters. Additionally,
514 underscores are not permitted in override names as they are used to
515 separate overrides from each other and from the variable name.
516
Patrick Williams213cb262021-08-07 19:21:33 -0500517- *Selecting a Variable:* The :term:`OVERRIDES` variable is a
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500518 colon-character-separated list that contains items for which you want
519 to satisfy conditions. Thus, if you have a variable that is
Patrick Williams213cb262021-08-07 19:21:33 -0500520 conditional on "arm", and "arm" is in :term:`OVERRIDES`, then the
Andrew Geisslerf0343792020-11-18 10:42:21 -0600521 "arm"-specific version of the variable is used rather than the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500522 non-conditional version. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500523
524 OVERRIDES = "architecture:os:machine"
525 TEST = "default"
526 TEST_os = "osspecific"
527 TEST_nooverride = "othercondvalue"
528
Patrick Williams213cb262021-08-07 19:21:33 -0500529 In this example, the :term:`OVERRIDES`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500530 variable lists three overrides: "architecture", "os", and "machine".
531 The variable ``TEST`` by itself has a default value of "default". You
532 select the os-specific version of the ``TEST`` variable by appending
533 the "os" override to the variable (i.e. ``TEST_os``).
534
535 To better understand this, consider a practical example that assumes
536 an OpenEmbedded metadata-based Linux kernel recipe file. The
537 following lines from the recipe file first set the kernel branch
538 variable ``KBRANCH`` to a default value, then conditionally override
Andrew Geisslerc926e172021-05-07 16:11:35 -0500539 that value based on the architecture of the build::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500540
541 KBRANCH = "standard/base"
Patrick Williams213cb262021-08-07 19:21:33 -0500542 KBRANCH:qemuarm = "standard/arm-versatile-926ejs"
543 KBRANCH:qemumips = "standard/mti-malta32"
544 KBRANCH:qemuppc = "standard/qemuppc"
545 KBRANCH:qemux86 = "standard/common-pc/base"
546 KBRANCH:qemux86-64 = "standard/common-pc-64/base"
547 KBRANCH:qemumips64 = "standard/mti-malta64"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500548
549- *Appending and Prepending:* BitBake also supports append and prepend
550 operations to variable values based on whether a specific item is
Patrick Williams213cb262021-08-07 19:21:33 -0500551 listed in :term:`OVERRIDES`. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500552
553 DEPENDS = "glibc ncurses"
554 OVERRIDES = "machine:local"
Patrick Williams213cb262021-08-07 19:21:33 -0500555 DEPENDS:append:machine = "libmad"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500556
Patrick Williams213cb262021-08-07 19:21:33 -0500557 In this example, :term:`DEPENDS` becomes "glibc ncurses libmad".
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500558
559 Again, using an OpenEmbedded metadata-based kernel recipe file as an
560 example, the following lines will conditionally append to the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500561 ``KERNEL_FEATURES`` variable based on the architecture::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500562
Patrick Williams213cb262021-08-07 19:21:33 -0500563 KERNEL_FEATURES:append = " ${KERNEL_EXTRA_FEATURES}"
564 KERNEL_FEATURES:append:qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
565 KERNEL_FEATURES:append:qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500566
567- *Setting a Variable for a Single Task:* BitBake supports setting a
Andrew Geisslerc926e172021-05-07 16:11:35 -0500568 variable just for the duration of a single task. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500569
570 FOO_task-configure = "val 1"
Patrick Williams213cb262021-08-07 19:21:33 -0500571 FOO:task-compile = "val 2"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500572
573 In the
574 previous example, ``FOO`` has the value "val 1" while the
575 ``do_configure`` task is executed, and the value "val 2" while the
576 ``do_compile`` task is executed.
577
578 Internally, this is implemented by prepending the task (e.g.
579 "task-compile:") to the value of
580 :term:`OVERRIDES` for the local datastore of the
581 ``do_compile`` task.
582
583 You can also use this syntax with other combinations (e.g.
Patrick Williams213cb262021-08-07 19:21:33 -0500584 "``:prepend``") as shown in the following example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500585
Patrick Williams213cb262021-08-07 19:21:33 -0500586 EXTRA_OEMAKE:prepend:task-compile = "${PARALLEL_MAKE} "
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500587
588Key Expansion
589-------------
590
591Key expansion happens when the BitBake datastore is finalized. To better
Andrew Geisslerc926e172021-05-07 16:11:35 -0500592understand this, consider the following example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500593
594 A${B} = "X"
595 B = "2"
596 A2 = "Y"
597
598In this case, after all the parsing is complete, BitBake expands
599``${B}`` into "2". This expansion causes ``A2``, which was set to "Y"
600before the expansion, to become "X".
601
602.. _variable-interaction-worked-examples:
603
604Examples
605--------
606
607Despite the previous explanations that show the different forms of
608variable definitions, it can be hard to work out exactly what happens
609when variable operators, conditional overrides, and unconditional
610overrides are combined. This section presents some common scenarios
611along with explanations for variable interactions that typically confuse
612users.
613
614There is often confusion concerning the order in which overrides and
615various "append" operators take effect. Recall that an append or prepend
Patrick Williams213cb262021-08-07 19:21:33 -0500616operation using ":append" and ":prepend" does not result in an immediate
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500617assignment as would "+=", ".=", "=+", or "=.". Consider the following
Andrew Geisslerc926e172021-05-07 16:11:35 -0500618example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500619
620 OVERRIDES = "foo"
621 A = "Z"
Patrick Williams213cb262021-08-07 19:21:33 -0500622 A:foo:append = "X"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500623
624For this case,
625``A`` is unconditionally set to "Z" and "X" is unconditionally and
Patrick Williams213cb262021-08-07 19:21:33 -0500626immediately appended to the variable ``A:foo``. Because overrides have
627not been applied yet, ``A:foo`` is set to "X" due to the append and
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500628``A`` simply equals "Z".
629
630Applying overrides, however, changes things. Since "foo" is listed in
Patrick Williams213cb262021-08-07 19:21:33 -0500631:term:`OVERRIDES`, the conditional variable ``A`` is replaced with the "foo"
632version, which is equal to "X". So effectively, ``A:foo`` replaces
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500633``A``.
634
Andrew Geisslerc926e172021-05-07 16:11:35 -0500635This next example changes the order of the override and the append::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500636
637 OVERRIDES = "foo"
638 A = "Z"
Patrick Williams213cb262021-08-07 19:21:33 -0500639 A:append:foo = "X"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500640
641For this case, before
Patrick Williams213cb262021-08-07 19:21:33 -0500642overrides are handled, ``A`` is set to "Z" and ``A:append:foo`` is set
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500643to "X". Once the override for "foo" is applied, however, ``A`` gets
644appended with "X". Consequently, ``A`` becomes "ZX". Notice that spaces
645are not appended.
646
647This next example has the order of the appends and overrides reversed
Andrew Geisslerc926e172021-05-07 16:11:35 -0500648back as in the first example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500649
650 OVERRIDES = "foo"
651 A = "Y"
Patrick Williams213cb262021-08-07 19:21:33 -0500652 A:foo:append = "Z"
653 A:foo:append = "X"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500654
655For this case, before any overrides are resolved,
656``A`` is set to "Y" using an immediate assignment. After this immediate
Patrick Williams213cb262021-08-07 19:21:33 -0500657assignment, ``A:foo`` is set to "Z", and then further appended with "X"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500658leaving the variable set to "ZX". Finally, applying the override for
659"foo" results in the conditional variable ``A`` becoming "ZX" (i.e.
Patrick Williams213cb262021-08-07 19:21:33 -0500660``A`` is replaced with ``A:foo``).
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500661
Andrew Geisslerc926e172021-05-07 16:11:35 -0500662This final example mixes in some varying operators::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500663
664 A = "1"
Patrick Williams213cb262021-08-07 19:21:33 -0500665 A:append = "2"
666 A:append = "3"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500667 A += "4"
668 A .= "5"
669
670For this case, the type of append
671operators are affecting the order of assignments as BitBake passes
672through the code multiple times. Initially, ``A`` is set to "1 45"
673because of the three statements that use immediate operators. After
Patrick Williams213cb262021-08-07 19:21:33 -0500674these assignments are made, BitBake applies the ":append" operations.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500675Those operations result in ``A`` becoming "1 4523".
676
677Sharing Functionality
678=====================
679
680BitBake allows for metadata sharing through include files (``.inc``) and
681class files (``.bbclass``). For example, suppose you have a piece of
682common functionality such as a task definition that you want to share
683between more than one recipe. In this case, creating a ``.bbclass`` file
684that contains the common functionality and then using the ``inherit``
685directive in your recipes to inherit the class would be a common way to
686share the task.
687
688This section presents the mechanisms BitBake provides to allow you to
689share functionality between recipes. Specifically, the mechanisms
Patrick Williams213cb262021-08-07 19:21:33 -0500690include ``include``, ``inherit``, :term:`INHERIT`, and ``require``
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500691directives.
692
693Locating Include and Class Files
694--------------------------------
695
696BitBake uses the :term:`BBPATH` variable to locate
697needed include and class files. Additionally, BitBake searches the
698current directory for ``include`` and ``require`` directives.
699
700.. note::
701
702 The BBPATH variable is analogous to the environment variable PATH .
703
704In order for include and class files to be found by BitBake, they need
705to be located in a "classes" subdirectory that can be found in
Patrick Williams213cb262021-08-07 19:21:33 -0500706:term:`BBPATH`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500707
708``inherit`` Directive
709---------------------
710
711When writing a recipe or class file, you can use the ``inherit``
712directive to inherit the functionality of a class (``.bbclass``).
713BitBake only supports this directive when used within recipe and class
714files (i.e. ``.bb`` and ``.bbclass``).
715
716The ``inherit`` directive is a rudimentary means of specifying
717functionality contained in class files that your recipes require. For
718example, you can easily abstract out the tasks involved in building a
719package that uses Autoconf and Automake and put those tasks into a class
720file and then have your recipe inherit that class file.
721
722As an example, your recipes could use the following directive to inherit
723an ``autotools.bbclass`` file. The class file would contain common
Andrew Geisslerc926e172021-05-07 16:11:35 -0500724functionality for using Autotools that could be shared across recipes::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500725
726 inherit autotools
727
728In this case, BitBake would search for the directory
Patrick Williams213cb262021-08-07 19:21:33 -0500729``classes/autotools.bbclass`` in :term:`BBPATH`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500730
731.. note::
732
733 You can override any values and functions of the inherited class
734 within your recipe by doing so after the "inherit" statement.
735
736If you want to use the directive to inherit multiple classes, separate
737them with spaces. The following example shows how to inherit both the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500738``buildhistory`` and ``rm_work`` classes::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500739
740 inherit buildhistory rm_work
741
742An advantage with the inherit directive as compared to both the
743:ref:`include <bitbake-user-manual/bitbake-user-manual-metadata:\`\`include\`\` directive>` and :ref:`require <bitbake-user-manual/bitbake-user-manual-metadata:\`\`require\`\` directive>`
744directives is that you can inherit class files conditionally. You can
745accomplish this by using a variable expression after the ``inherit``
Andrew Geisslerc926e172021-05-07 16:11:35 -0500746statement. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500747
748 inherit ${VARNAME}
749
750If ``VARNAME`` is
751going to be set, it needs to be set before the ``inherit`` statement is
752parsed. One way to achieve a conditional inherit in this case is to use
Andrew Geisslerc926e172021-05-07 16:11:35 -0500753overrides::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500754
755 VARIABLE = ""
Patrick Williams213cb262021-08-07 19:21:33 -0500756 VARIABLE:someoverride = "myclass"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500757
Andrew Geisslerc926e172021-05-07 16:11:35 -0500758Another method is by using anonymous Python. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500759
760 python () {
761 if condition == value:
762 d.setVar('VARIABLE', 'myclass')
763 else:
764 d.setVar('VARIABLE', '')
765 }
766
767Alternatively, you could use an in-line Python expression in the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500768following form::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500769
770 inherit ${@'classname' if condition else ''}
771 inherit ${@functionname(params)}
772
773In all cases, if the expression evaluates to an
774empty string, the statement does not trigger a syntax error because it
775becomes a no-op.
776
777``include`` Directive
778---------------------
779
780BitBake understands the ``include`` directive. This directive causes
781BitBake to parse whatever file you specify, and to insert that file at
782that location. The directive is much like its equivalent in Make except
783that if the path specified on the include line is a relative path,
Patrick Williams213cb262021-08-07 19:21:33 -0500784BitBake locates the first file it can find within :term:`BBPATH`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500785
786The include directive is a more generic method of including
787functionality as compared to the :ref:`inherit <bitbake-user-manual/bitbake-user-manual-metadata:\`\`inherit\`\` directive>`
788directive, which is restricted to class (i.e. ``.bbclass``) files. The
789include directive is applicable for any other kind of shared or
790encapsulated functionality or configuration that does not suit a
791``.bbclass`` file.
792
793As an example, suppose you needed a recipe to include some self-test
Andrew Geisslerc926e172021-05-07 16:11:35 -0500794definitions::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500795
796 include test_defs.inc
797
798.. note::
799
800 The include directive does not produce an error when the file cannot be
801 found. Consequently, it is recommended that if the file you are including is
802 expected to exist, you should use :ref:`require <require-inclusion>` instead
803 of include . Doing so makes sure that an error is produced if the file cannot
804 be found.
805
806.. _require-inclusion:
807
808``require`` Directive
809---------------------
810
811BitBake understands the ``require`` directive. This directive behaves
812just like the ``include`` directive with the exception that BitBake
813raises a parsing error if the file to be included cannot be found. Thus,
814any file you require is inserted into the file that is being parsed at
815the location of the directive.
816
817The require directive, like the include directive previously described,
818is a more generic method of including functionality as compared to the
819:ref:`inherit <bitbake-user-manual/bitbake-user-manual-metadata:\`\`inherit\`\` directive>` directive, which is restricted to class
820(i.e. ``.bbclass``) files. The require directive is applicable for any
821other kind of shared or encapsulated functionality or configuration that
822does not suit a ``.bbclass`` file.
823
824Similar to how BitBake handles :ref:`include <bitbake-user-manual/bitbake-user-manual-metadata:\`\`include\`\` directive>`, if
825the path specified on the require line is a relative path, BitBake
Patrick Williams213cb262021-08-07 19:21:33 -0500826locates the first file it can find within :term:`BBPATH`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500827
828As an example, suppose you have two versions of a recipe (e.g.
829``foo_1.2.2.bb`` and ``foo_2.0.0.bb``) where each version contains some
830identical functionality that could be shared. You could create an
831include file named ``foo.inc`` that contains the common definitions
832needed to build "foo". You need to be sure ``foo.inc`` is located in the
833same directory as your two recipe files as well. Once these conditions
834are set up, you can share the functionality using a ``require``
Andrew Geisslerc926e172021-05-07 16:11:35 -0500835directive from within each recipe::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500836
837 require foo.inc
838
839``INHERIT`` Configuration Directive
840-----------------------------------
841
842When creating a configuration file (``.conf``), you can use the
843:term:`INHERIT` configuration directive to inherit a
844class. BitBake only supports this directive when used within a
845configuration file.
846
847As an example, suppose you needed to inherit a class file called
Andrew Geisslerc926e172021-05-07 16:11:35 -0500848``abc.bbclass`` from a configuration file as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500849
850 INHERIT += "abc"
851
852This configuration directive causes the named class to be inherited at
853the point of the directive during parsing. As with the ``inherit``
854directive, the ``.bbclass`` file must be located in a "classes"
Patrick Williams213cb262021-08-07 19:21:33 -0500855subdirectory in one of the directories specified in :term:`BBPATH`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500856
857.. note::
858
859 Because .conf files are parsed first during BitBake's execution, using
860 INHERIT to inherit a class effectively inherits the class globally (i.e. for
861 all recipes).
862
863If you want to use the directive to inherit multiple classes, you can
864provide them on the same line in the ``local.conf`` file. Use spaces to
865separate the classes. The following example shows how to inherit both
Andrew Geisslerc926e172021-05-07 16:11:35 -0500866the ``autotools`` and ``pkgconfig`` classes::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500867
868 INHERIT += "autotools pkgconfig"
869
870Functions
871=========
872
873As with most languages, functions are the building blocks that are used
874to build up operations into tasks. BitBake supports these types of
875functions:
876
877- *Shell Functions:* Functions written in shell script and executed
878 either directly as functions, tasks, or both. They can also be called
879 by other shell functions.
880
881- *BitBake-Style Python Functions:* Functions written in Python and
882 executed by BitBake or other Python functions using
883 ``bb.build.exec_func()``.
884
885- *Python Functions:* Functions written in Python and executed by
886 Python.
887
888- *Anonymous Python Functions:* Python functions executed automatically
889 during parsing.
890
891Regardless of the type of function, you can only define them in class
892(``.bbclass``) and recipe (``.bb`` or ``.inc``) files.
893
894Shell Functions
895---------------
896
897Functions written in shell script and executed either directly as
898functions, tasks, or both. They can also be called by other shell
Andrew Geisslerc926e172021-05-07 16:11:35 -0500899functions. Here is an example shell function definition::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500900
901 some_function () {
902 echo "Hello World"
903 }
904
905When you create these types of functions in
906your recipe or class files, you need to follow the shell programming
907rules. The scripts are executed by ``/bin/sh``, which may not be a bash
908shell but might be something such as ``dash``. You should not use
909Bash-specific script (bashisms).
910
Patrick Williams213cb262021-08-07 19:21:33 -0500911Overrides and override-style operators like ``:append`` and ``:prepend``
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500912can also be applied to shell functions. Most commonly, this application
913would be used in a ``.bbappend`` file to modify functions in the main
914recipe. It can also be used to modify functions inherited from classes.
915
Andrew Geisslerc926e172021-05-07 16:11:35 -0500916As an example, consider the following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500917
918 do_foo() {
919 bbplain first
920 fn
921 }
922
Patrick Williams213cb262021-08-07 19:21:33 -0500923 fn:prepend() {
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500924 bbplain second
925 }
926
927 fn() {
928 bbplain third
929 }
930
Patrick Williams213cb262021-08-07 19:21:33 -0500931 do_foo:append() {
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500932 bbplain fourth
933 }
934
Andrew Geisslerc926e172021-05-07 16:11:35 -0500935Running ``do_foo`` prints the following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500936
937 recipename do_foo: first
938 recipename do_foo: second
939 recipename do_foo: third
940 recipename do_foo: fourth
941
942.. note::
943
944 Overrides and override-style operators can be applied to any shell
945 function, not just :ref:`tasks <bitbake-user-manual/bitbake-user-manual-metadata:tasks>`.
946
947You can use the ``bitbake -e`` recipename command to view the final
948assembled function after all overrides have been applied.
949
950BitBake-Style Python Functions
951------------------------------
952
953These functions are written in Python and executed by BitBake or other
954Python functions using ``bb.build.exec_func()``.
955
Andrew Geisslerc926e172021-05-07 16:11:35 -0500956An example BitBake function is::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500957
958 python some_python_function () {
959 d.setVar("TEXT", "Hello World")
960 print d.getVar("TEXT")
961 }
962
963Because the
964Python "bb" and "os" modules are already imported, you do not need to
965import these modules. Also in these types of functions, the datastore
966("d") is a global variable and is always automatically available.
967
968.. note::
969
970 Variable expressions (e.g. ``${X}`` ) are no longer expanded within Python
971 functions. This behavior is intentional in order to allow you to freely set
972 variable values to expandable expressions without having them expanded
973 prematurely. If you do wish to expand a variable within a Python function,
974 use ``d.getVar("X")`` . Or, for more complicated expressions, use ``d.expand()``.
975
976Similar to shell functions, you can also apply overrides and
977override-style operators to BitBake-style Python functions.
978
Andrew Geisslerc926e172021-05-07 16:11:35 -0500979As an example, consider the following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500980
Patrick Williams213cb262021-08-07 19:21:33 -0500981 python do_foo:prepend() {
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500982 bb.plain("first")
983 }
984
985 python do_foo() {
986 bb.plain("second")
987 }
988
Patrick Williams213cb262021-08-07 19:21:33 -0500989 python do_foo:append() {
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500990 bb.plain("third")
991 }
992
Andrew Geisslerc926e172021-05-07 16:11:35 -0500993Running ``do_foo`` prints the following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500994
995 recipename do_foo: first
996 recipename do_foo: second
997 recipename do_foo: third
998
999You can use the ``bitbake -e`` recipename command to view
1000the final assembled function after all overrides have been applied.
1001
1002Python Functions
1003----------------
1004
1005These functions are written in Python and are executed by other Python
1006code. Examples of Python functions are utility functions that you intend
1007to call from in-line Python or from within other Python functions. Here
Andrew Geisslerc926e172021-05-07 16:11:35 -05001008is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001009
1010 def get_depends(d):
1011 if d.getVar('SOMECONDITION'):
1012 return "dependencywithcond"
1013 else:
1014 return "dependency"
1015
1016 SOMECONDITION = "1"
1017 DEPENDS = "${@get_depends(d)}"
1018
Patrick Williams213cb262021-08-07 19:21:33 -05001019This would result in :term:`DEPENDS` containing ``dependencywithcond``.
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001020
1021Here are some things to know about Python functions:
1022
1023- Python functions can take parameters.
1024
1025- The BitBake datastore is not automatically available. Consequently,
1026 you must pass it in as a parameter to the function.
1027
1028- The "bb" and "os" Python modules are automatically available. You do
1029 not need to import them.
1030
1031BitBake-Style Python Functions Versus Python Functions
1032------------------------------------------------------
1033
1034Following are some important differences between BitBake-style Python
1035functions and regular Python functions defined with "def":
1036
1037- Only BitBake-style Python functions can be :ref:`tasks <bitbake-user-manual/bitbake-user-manual-metadata:tasks>`.
1038
1039- Overrides and override-style operators can only be applied to
1040 BitBake-style Python functions.
1041
1042- Only regular Python functions can take arguments and return values.
1043
1044- :ref:`Variable flags <bitbake-user-manual/bitbake-user-manual-metadata:variable flags>` such as
1045 ``[dirs]``, ``[cleandirs]``, and ``[lockfiles]`` can be used on BitBake-style
1046 Python functions, but not on regular Python functions.
1047
1048- BitBake-style Python functions generate a separate
1049 ``${``\ :term:`T`\ ``}/run.``\ function-name\ ``.``\ pid
1050 script that is executed to run the function, and also generate a log
1051 file in ``${T}/log.``\ function-name\ ``.``\ pid if they are executed
1052 as tasks.
1053
1054 Regular Python functions execute "inline" and do not generate any
1055 files in ``${T}``.
1056
1057- Regular Python functions are called with the usual Python syntax.
1058 BitBake-style Python functions are usually tasks and are called
1059 directly by BitBake, but can also be called manually from Python code
Andrew Geisslerc926e172021-05-07 16:11:35 -05001060 by using the ``bb.build.exec_func()`` function. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001061
1062 bb.build.exec_func("my_bitbake_style_function", d)
1063
1064 .. note::
1065
1066 ``bb.build.exec_func()`` can also be used to run shell functions from Python
1067 code. If you want to run a shell function before a Python function within
1068 the same task, then you can use a parent helper Python function that
1069 starts by running the shell function with ``bb.build.exec_func()`` and then
1070 runs the Python code.
1071
1072 To detect errors from functions executed with
1073 ``bb.build.exec_func()``, you can catch the ``bb.build.FuncFailed``
1074 exception.
1075
1076 .. note::
1077
1078 Functions in metadata (recipes and classes) should not themselves raise
1079 ``bb.build.FuncFailed``. Rather, ``bb.build.FuncFailed`` should be viewed as a
1080 general indicator that the called function failed by raising an
1081 exception. For example, an exception raised by ``bb.fatal()`` will be caught
1082 inside ``bb.build.exec_func()``, and a ``bb.build.FuncFailed`` will be raised in
1083 response.
1084
1085Due to their simplicity, you should prefer regular Python functions over
1086BitBake-style Python functions unless you need a feature specific to
1087BitBake-style Python functions. Regular Python functions in metadata are
1088a more recent invention than BitBake-style Python functions, and older
1089code tends to use ``bb.build.exec_func()`` more often.
1090
1091Anonymous Python Functions
1092--------------------------
1093
1094Sometimes it is useful to set variables or perform other operations
1095programmatically during parsing. To do this, you can define special
1096Python functions, called anonymous Python functions, that run at the end
1097of parsing. For example, the following conditionally sets a variable
Andrew Geisslerc926e172021-05-07 16:11:35 -05001098based on the value of another variable::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001099
1100 python () {
1101 if d.getVar('SOMEVAR') == 'value':
1102 d.setVar('ANOTHERVAR', 'value2')
1103 }
1104
1105An equivalent way to mark a function as an anonymous function is to give it
1106the name "__anonymous", rather than no name.
1107
1108Anonymous Python functions always run at the end of parsing, regardless
1109of where they are defined. If a recipe contains many anonymous
1110functions, they run in the same order as they are defined within the
Andrew Geisslerc926e172021-05-07 16:11:35 -05001111recipe. As an example, consider the following snippet::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001112
1113 python () {
1114 d.setVar('FOO', 'foo 2')
1115 }
1116
1117 FOO = "foo 1"
1118
1119 python () {
1120 d.appendVar('BAR',' bar 2')
1121 }
1122
1123 BAR = "bar 1"
1124
1125The previous example is conceptually
Andrew Geisslerc926e172021-05-07 16:11:35 -05001126equivalent to the following snippet::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001127
1128 FOO = "foo 1"
1129 BAR = "bar 1"
1130 FOO = "foo 2"
1131 BAR += "bar 2"
1132
1133``FOO`` ends up with the value "foo 2", and
1134``BAR`` with the value "bar 1 bar 2". Just as in the second snippet, the
1135values set for the variables within the anonymous functions become
1136available to tasks, which always run after parsing.
1137
Patrick Williams213cb262021-08-07 19:21:33 -05001138Overrides and override-style operators such as "``:append``" are applied
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001139before anonymous functions run. In the following example, ``FOO`` ends
Andrew Geisslerc926e172021-05-07 16:11:35 -05001140up with the value "foo from anonymous"::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001141
1142 FOO = "foo"
Patrick Williams213cb262021-08-07 19:21:33 -05001143 FOO:append = " from outside"
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001144
1145 python () {
1146 d.setVar("FOO", "foo from anonymous")
1147 }
1148
1149For methods
1150you can use with anonymous Python functions, see the
1151":ref:`bitbake-user-manual/bitbake-user-manual-metadata:functions you can call from within python`"
1152section. For a different method to run Python code during parsing, see
1153the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:inline python variable expansion`" section.
1154
1155Flexible Inheritance for Class Functions
1156----------------------------------------
1157
1158Through coding techniques and the use of ``EXPORT_FUNCTIONS``, BitBake
1159supports exporting a function from a class such that the class function
1160appears as the default implementation of the function, but can still be
1161called if a recipe inheriting the class needs to define its own version
1162of the function.
1163
1164To understand the benefits of this feature, consider the basic scenario
1165where a class defines a task function and your recipe inherits the
1166class. In this basic scenario, your recipe inherits the task function as
1167defined in the class. If desired, your recipe can add to the start and
Patrick Williams213cb262021-08-07 19:21:33 -05001168end of the function by using the ":prepend" or ":append" operations
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001169respectively, or it can redefine the function completely. However, if it
1170redefines the function, there is no means for it to call the class
1171version of the function. ``EXPORT_FUNCTIONS`` provides a mechanism that
1172enables the recipe's version of the function to call the original
1173version of the function.
1174
1175To make use of this technique, you need the following things in place:
1176
Andrew Geisslerc926e172021-05-07 16:11:35 -05001177- The class needs to define the function as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001178
1179 classname_functionname
1180
1181 For example, if you have a class file
1182 ``bar.bbclass`` and a function named ``do_foo``, the class must
Andrew Geisslerc926e172021-05-07 16:11:35 -05001183 define the function as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001184
1185 bar_do_foo
1186
1187- The class needs to contain the ``EXPORT_FUNCTIONS`` statement as
Andrew Geisslerc926e172021-05-07 16:11:35 -05001188 follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001189
1190 EXPORT_FUNCTIONS functionname
1191
1192 For example, continuing with
1193 the same example, the statement in the ``bar.bbclass`` would be as
Andrew Geisslerc926e172021-05-07 16:11:35 -05001194 follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001195
1196 EXPORT_FUNCTIONS do_foo
1197
1198- You need to call the function appropriately from within your recipe.
1199 Continuing with the same example, if your recipe needs to call the
1200 class version of the function, it should call ``bar_do_foo``.
1201 Assuming ``do_foo`` was a shell function and ``EXPORT_FUNCTIONS`` was
1202 used as above, the recipe's function could conditionally call the
Andrew Geisslerc926e172021-05-07 16:11:35 -05001203 class version of the function as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001204
1205 do_foo() {
1206 if [ somecondition ] ; then
1207 bar_do_foo
1208 else
1209 # Do something else
1210 fi
1211 }
1212
1213 To call your modified version of the function as defined in your recipe,
1214 call it as ``do_foo``.
1215
1216With these conditions met, your single recipe can freely choose between
1217the original function as defined in the class file and the modified
1218function in your recipe. If you do not set up these conditions, you are
1219limited to using one function or the other.
1220
1221Tasks
1222=====
1223
1224Tasks are BitBake execution units that make up the steps that BitBake
1225can run for a given recipe. Tasks are only supported in recipes and
1226classes (i.e. in ``.bb`` files and files included or inherited from
1227``.bb`` files). By convention, tasks have names that start with "do\_".
1228
1229Promoting a Function to a Task
1230------------------------------
1231
1232Tasks are either :ref:`shell functions <bitbake-user-manual/bitbake-user-manual-metadata:shell functions>` or
1233:ref:`BitBake-style Python functions <bitbake-user-manual/bitbake-user-manual-metadata:bitbake-style python functions>`
1234that have been promoted to tasks by using the ``addtask`` command. The
1235``addtask`` command can also optionally describe dependencies between
1236the task and other tasks. Here is an example that shows how to define a
Andrew Geisslerc926e172021-05-07 16:11:35 -05001237task and declare some dependencies::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001238
1239 python do_printdate () {
1240 import time
1241 print time.strftime('%Y%m%d', time.gmtime())
1242 }
1243 addtask printdate after do_fetch before do_build
1244
1245The first argument to ``addtask`` is the name
1246of the function to promote to a task. If the name does not start with
1247"do\_", "do\_" is implicitly added, which enforces the convention that all
1248task names start with "do\_".
1249
1250In the previous example, the ``do_printdate`` task becomes a dependency
1251of the ``do_build`` task, which is the default task (i.e. the task run
1252by the ``bitbake`` command unless another task is specified explicitly).
1253Additionally, the ``do_printdate`` task becomes dependent upon the
1254``do_fetch`` task. Running the ``do_build`` task results in the
1255``do_printdate`` task running first.
1256
1257.. note::
1258
1259 If you try out the previous example, you might see that the
1260 ``do_printdate``
1261 task is only run the first time you build the recipe with the
1262 ``bitbake``
1263 command. This is because BitBake considers the task "up-to-date"
1264 after that initial run. If you want to force the task to always be
1265 rerun for experimentation purposes, you can make BitBake always
1266 consider the task "out-of-date" by using the
1267 :ref:`[nostamp] <bitbake-user-manual/bitbake-user-manual-metadata:Variable Flags>`
Andrew Geisslerc926e172021-05-07 16:11:35 -05001268 variable flag, as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001269
1270 do_printdate[nostamp] = "1"
1271
1272 You can also explicitly run the task and provide the
Andrew Geisslerc926e172021-05-07 16:11:35 -05001273 -f option as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001274
1275 $ bitbake recipe -c printdate -f
1276
1277 When manually selecting a task to run with the bitbake ``recipe
1278 -c task`` command, you can omit the "do\_" prefix as part of the task
1279 name.
1280
1281You might wonder about the practical effects of using ``addtask``
Andrew Geisslerc926e172021-05-07 16:11:35 -05001282without specifying any dependencies as is done in the following example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001283
1284 addtask printdate
1285
1286In this example, assuming dependencies have not been
1287added through some other means, the only way to run the task is by
1288explicitly selecting it with ``bitbake`` recipe ``-c printdate``. You
1289can use the ``do_listtasks`` task to list all tasks defined in a recipe
Andrew Geisslerc926e172021-05-07 16:11:35 -05001290as shown in the following example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001291
1292 $ bitbake recipe -c listtasks
1293
1294For more information on task dependencies, see the
1295":ref:`bitbake-user-manual/bitbake-user-manual-execution:dependencies`" section.
1296
1297See the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:variable flags`" section for information
1298on variable flags you can use with tasks.
1299
Andrew Geissler95ac1b82021-03-31 14:34:31 -05001300.. note::
1301
1302 While it's infrequent, it's possible to define multiple tasks as
1303 dependencies when calling ``addtask``. For example, here's a snippet
1304 from the OpenEmbedded class file ``package_tar.bbclass``::
1305
1306 addtask package_write_tar before do_build after do_packagedata do_package
1307
1308 Note how the ``package_write_tar`` task has to wait until both of
1309 ``do_packagedata`` and ``do_package`` complete.
1310
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001311Deleting a Task
1312---------------
1313
1314As well as being able to add tasks, you can delete them. Simply use the
1315``deltask`` command to delete a task. For example, to delete the example
Andrew Geisslerc926e172021-05-07 16:11:35 -05001316task used in the previous sections, you would use::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001317
1318 deltask printdate
1319
1320If you delete a task using the ``deltask`` command and the task has
1321dependencies, the dependencies are not reconnected. For example, suppose
1322you have three tasks named ``do_a``, ``do_b``, and ``do_c``.
1323Furthermore, ``do_c`` is dependent on ``do_b``, which in turn is
1324dependent on ``do_a``. Given this scenario, if you use ``deltask`` to
1325delete ``do_b``, the implicit dependency relationship between ``do_c``
1326and ``do_a`` through ``do_b`` no longer exists, and ``do_c``
1327dependencies are not updated to include ``do_a``. Thus, ``do_c`` is free
1328to run before ``do_a``.
1329
1330If you want dependencies such as these to remain intact, use the
1331``[noexec]`` varflag to disable the task instead of using the
Andrew Geisslerc926e172021-05-07 16:11:35 -05001332``deltask`` command to delete it::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001333
1334 do_b[noexec] = "1"
1335
1336Passing Information Into the Build Task Environment
1337---------------------------------------------------
1338
1339When running a task, BitBake tightly controls the shell execution
1340environment of the build tasks to make sure unwanted contamination from
1341the build machine cannot influence the build.
1342
1343.. note::
1344
1345 By default, BitBake cleans the environment to include only those
1346 things exported or listed in its whitelist to ensure that the build
1347 environment is reproducible and consistent. You can prevent this
1348 "cleaning" by setting the :term:`BB_PRESERVE_ENV` variable.
1349
1350Consequently, if you do want something to get passed into the build task
1351environment, you must take these two steps:
1352
1353#. Tell BitBake to load what you want from the environment into the
1354 datastore. You can do so through the
1355 :term:`BB_ENV_WHITELIST` and
1356 :term:`BB_ENV_EXTRAWHITE` variables. For
1357 example, assume you want to prevent the build system from accessing
1358 your ``$HOME/.ccache`` directory. The following command "whitelists"
1359 the environment variable ``CCACHE_DIR`` causing BitBake to allow that
Andrew Geisslerc926e172021-05-07 16:11:35 -05001360 variable into the datastore::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001361
1362 export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE CCACHE_DIR"
1363
1364#. Tell BitBake to export what you have loaded into the datastore to the
1365 task environment of every running task. Loading something from the
1366 environment into the datastore (previous step) only makes it
1367 available in the datastore. To export it to the task environment of
1368 every running task, use a command similar to the following in your
1369 local configuration file ``local.conf`` or your distribution
Andrew Geisslerc926e172021-05-07 16:11:35 -05001370 configuration file::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001371
1372 export CCACHE_DIR
1373
1374 .. note::
1375
1376 A side effect of the previous steps is that BitBake records the
1377 variable as a dependency of the build process in things like the
1378 setscene checksums. If doing so results in unnecessary rebuilds of
1379 tasks, you can whitelist the variable so that the setscene code
1380 ignores the dependency when it creates checksums.
1381
1382Sometimes, it is useful to be able to obtain information from the
1383original execution environment. BitBake saves a copy of the original
1384environment into a special variable named :term:`BB_ORIGENV`.
1385
Patrick Williams213cb262021-08-07 19:21:33 -05001386The :term:`BB_ORIGENV` variable returns a datastore object that can be
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001387queried using the standard datastore operators such as
1388``getVar(, False)``. The datastore object is useful, for example, to
Andrew Geisslerc926e172021-05-07 16:11:35 -05001389find the original ``DISPLAY`` variable. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001390
1391 origenv = d.getVar("BB_ORIGENV", False)
1392 bar = origenv.getVar("BAR", False)
1393
1394The previous example returns ``BAR`` from the original execution
1395environment.
1396
1397Variable Flags
1398==============
1399
1400Variable flags (varflags) help control a task's functionality and
1401dependencies. BitBake reads and writes varflags to the datastore using
Andrew Geisslerc926e172021-05-07 16:11:35 -05001402the following command forms::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001403
1404 variable = d.getVarFlags("variable")
1405 self.d.setVarFlags("FOO", {"func": True})
1406
1407When working with varflags, the same syntax, with the exception of
1408overrides, applies. In other words, you can set, append, and prepend
1409varflags just like variables. See the
1410":ref:`bitbake-user-manual/bitbake-user-manual-metadata:variable flag syntax`" section for details.
1411
1412BitBake has a defined set of varflags available for recipes and classes.
1413Tasks support a number of these flags which control various
1414functionality of the task:
1415
1416- ``[cleandirs]``: Empty directories that should be created before
1417 the task runs. Directories that already exist are removed and
1418 recreated to empty them.
1419
1420- ``[depends]``: Controls inter-task dependencies. See the
1421 :term:`DEPENDS` variable and the
1422 ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:inter-task
1423 dependencies`" section for more information.
1424
1425- ``[deptask]``: Controls task build-time dependencies. See the
1426 :term:`DEPENDS` variable and the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:build dependencies`" section for more information.
1427
1428- ``[dirs]``: Directories that should be created before the task
1429 runs. Directories that already exist are left as is. The last
1430 directory listed is used as the current working directory for the
1431 task.
1432
1433- ``[lockfiles]``: Specifies one or more lockfiles to lock while the
1434 task executes. Only one task may hold a lockfile, and any task that
1435 attempts to lock an already locked file will block until the lock is
1436 released. You can use this variable flag to accomplish mutual
1437 exclusion.
1438
1439- ``[noexec]``: When set to "1", marks the task as being empty, with
1440 no execution required. You can use the ``[noexec]`` flag to set up
1441 tasks as dependency placeholders, or to disable tasks defined
1442 elsewhere that are not needed in a particular recipe.
1443
1444- ``[nostamp]``: When set to "1", tells BitBake to not generate a
1445 stamp file for a task, which implies the task should always be
1446 executed.
1447
1448 .. caution::
1449
1450 Any task that depends (possibly indirectly) on a ``[nostamp]`` task will
1451 always be executed as well. This can cause unnecessary rebuilding if you
1452 are not careful.
1453
1454- ``[number_threads]``: Limits tasks to a specific number of
1455 simultaneous threads during execution. This varflag is useful when
1456 your build host has a large number of cores but certain tasks need to
1457 be rate-limited due to various kinds of resource constraints (e.g. to
1458 avoid network throttling). ``number_threads`` works similarly to the
1459 :term:`BB_NUMBER_THREADS` variable but is task-specific.
1460
1461 Set the value globally. For example, the following makes sure the
1462 ``do_fetch`` task uses no more than two simultaneous execution
1463 threads: do_fetch[number_threads] = "2"
1464
1465 .. warning::
1466
1467 - Setting the varflag in individual recipes rather than globally
1468 can result in unpredictable behavior.
1469
1470 - Setting the varflag to a value greater than the value used in
Patrick Williams213cb262021-08-07 19:21:33 -05001471 the :term:`BB_NUMBER_THREADS` variable causes ``number_threads`` to
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001472 have no effect.
1473
1474- ``[postfuncs]``: List of functions to call after the completion of
1475 the task.
1476
1477- ``[prefuncs]``: List of functions to call before the task executes.
1478
1479- ``[rdepends]``: Controls inter-task runtime dependencies. See the
1480 :term:`RDEPENDS` variable, the
1481 :term:`RRECOMMENDS` variable, and the
1482 ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:inter-task dependencies`" section for
1483 more information.
1484
1485- ``[rdeptask]``: Controls task runtime dependencies. See the
1486 :term:`RDEPENDS` variable, the
1487 :term:`RRECOMMENDS` variable, and the
1488 ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:runtime dependencies`" section for more
1489 information.
1490
1491- ``[recideptask]``: When set in conjunction with ``recrdeptask``,
1492 specifies a task that should be inspected for additional
1493 dependencies.
1494
1495- ``[recrdeptask]``: Controls task recursive runtime dependencies.
1496 See the :term:`RDEPENDS` variable, the
1497 :term:`RRECOMMENDS` variable, and the
1498 ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:recursive dependencies`" section for
1499 more information.
1500
1501- ``[stamp-extra-info]``: Extra stamp information to append to the
1502 task's stamp. As an example, OpenEmbedded uses this flag to allow
1503 machine-specific tasks.
1504
1505- ``[umask]``: The umask to run the task under.
1506
1507Several varflags are useful for controlling how signatures are
1508calculated for variables. For more information on this process, see the
1509":ref:`bitbake-user-manual/bitbake-user-manual-execution:checksums (signatures)`" section.
1510
1511- ``[vardeps]``: Specifies a space-separated list of additional
1512 variables to add to a variable's dependencies for the purposes of
1513 calculating its signature. Adding variables to this list is useful,
1514 for example, when a function refers to a variable in a manner that
1515 does not allow BitBake to automatically determine that the variable
1516 is referred to.
1517
1518- ``[vardepsexclude]``: Specifies a space-separated list of variables
1519 that should be excluded from a variable's dependencies for the
1520 purposes of calculating its signature.
1521
1522- ``[vardepvalue]``: If set, instructs BitBake to ignore the actual
1523 value of the variable and instead use the specified value when
1524 calculating the variable's signature.
1525
1526- ``[vardepvalueexclude]``: Specifies a pipe-separated list of
1527 strings to exclude from the variable's value when calculating the
1528 variable's signature.
1529
1530Events
1531======
1532
1533BitBake allows installation of event handlers within recipe and class
1534files. Events are triggered at certain points during operation, such as
1535the beginning of operation against a given recipe (i.e. ``*.bb``), the
1536start of a given task, a task failure, a task success, and so forth. The
1537intent is to make it easy to do things like email notification on build
1538failures.
1539
1540Following is an example event handler that prints the name of the event
Patrick Williams213cb262021-08-07 19:21:33 -05001541and the content of the :term:`FILE` variable::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001542
1543 addhandler myclass_eventhandler
1544 python myclass_eventhandler() {
1545 from bb.event import getName
1546 print("The name of the Event is %s" % getName(e))
1547 print("The file we run for is %s" % d.getVar('FILE'))
1548 }
1549 myclass_eventhandler[eventmask] = "bb.event.BuildStarted
1550 bb.event.BuildCompleted"
1551
1552In the previous example, an eventmask has been
1553set so that the handler only sees the "BuildStarted" and
1554"BuildCompleted" events. This event handler gets called every time an
1555event matching the eventmask is triggered. A global variable "e" is
1556defined, which represents the current event. With the ``getName(e)``
1557method, you can get the name of the triggered event. The global
1558datastore is available as "d". In legacy code, you might see "e.data"
1559used to get the datastore. However, realize that "e.data" is deprecated
1560and you should use "d" going forward.
1561
1562The context of the datastore is appropriate to the event in question.
1563For example, "BuildStarted" and "BuildCompleted" events run before any
1564tasks are executed so would be in the global configuration datastore
1565namespace. No recipe-specific metadata exists in that namespace. The
1566"BuildStarted" and "BuildCompleted" events also run in the main
1567cooker/server process rather than any worker context. Thus, any changes
1568made to the datastore would be seen by other cooker/server events within
1569the current build but not seen outside of that build or in any worker
1570context. Task events run in the actual tasks in question consequently
1571have recipe-specific and task-specific contents. These events run in the
1572worker context and are discarded at the end of task execution.
1573
1574During a standard build, the following common events might occur. The
1575following events are the most common kinds of events that most metadata
1576might have an interest in viewing:
1577
1578- ``bb.event.ConfigParsed()``: Fired when the base configuration; which
1579 consists of ``bitbake.conf``, ``base.bbclass`` and any global
Patrick Williams213cb262021-08-07 19:21:33 -05001580 :term:`INHERIT` statements; has been parsed. You can see multiple such
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001581 events when each of the workers parse the base configuration or if
1582 the server changes configuration and reparses. Any given datastore
1583 only has one such event executed against it, however. If
Andrew Geissler95ac1b82021-03-31 14:34:31 -05001584 :term:`BB_INVALIDCONF` is set in the datastore by the event
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001585 handler, the configuration is reparsed and a new event triggered,
1586 allowing the metadata to update configuration.
1587
1588- ``bb.event.HeartbeatEvent()``: Fires at regular time intervals of one
1589 second. You can configure the interval time using the
1590 ``BB_HEARTBEAT_EVENT`` variable. The event's "time" attribute is the
1591 ``time.time()`` value when the event is triggered. This event is
1592 useful for activities such as system state monitoring.
1593
1594- ``bb.event.ParseStarted()``: Fired when BitBake is about to start
1595 parsing recipes. This event's "total" attribute represents the number
1596 of recipes BitBake plans to parse.
1597
1598- ``bb.event.ParseProgress()``: Fired as parsing progresses. This
1599 event's "current" attribute is the number of recipes parsed as well
1600 as the "total" attribute.
1601
1602- ``bb.event.ParseCompleted()``: Fired when parsing is complete. This
1603 event's "cached", "parsed", "skipped", "virtuals", "masked", and
1604 "errors" attributes provide statistics for the parsing results.
1605
1606- ``bb.event.BuildStarted()``: Fired when a new build starts. BitBake
1607 fires multiple "BuildStarted" events (one per configuration) when
1608 multiple configuration (multiconfig) is enabled.
1609
1610- ``bb.build.TaskStarted()``: Fired when a task starts. This event's
1611 "taskfile" attribute points to the recipe from which the task
1612 originates. The "taskname" attribute, which is the task's name,
1613 includes the ``do_`` prefix, and the "logfile" attribute point to
1614 where the task's output is stored. Finally, the "time" attribute is
1615 the task's execution start time.
1616
1617- ``bb.build.TaskInvalid()``: Fired if BitBake tries to execute a task
1618 that does not exist.
1619
1620- ``bb.build.TaskFailedSilent()``: Fired for setscene tasks that fail
1621 and should not be presented to the user verbosely.
1622
1623- ``bb.build.TaskFailed()``: Fired for normal tasks that fail.
1624
1625- ``bb.build.TaskSucceeded()``: Fired when a task successfully
1626 completes.
1627
1628- ``bb.event.BuildCompleted()``: Fired when a build finishes.
1629
1630- ``bb.cooker.CookerExit()``: Fired when the BitBake server/cooker
1631 shuts down. This event is usually only seen by the UIs as a sign they
1632 should also shutdown.
1633
1634This next list of example events occur based on specific requests to the
1635server. These events are often used to communicate larger pieces of
1636information from the BitBake server to other parts of BitBake such as
1637user interfaces:
1638
1639- ``bb.event.TreeDataPreparationStarted()``
1640- ``bb.event.TreeDataPreparationProgress()``
1641- ``bb.event.TreeDataPreparationCompleted()``
1642- ``bb.event.DepTreeGenerated()``
1643- ``bb.event.CoreBaseFilesFound()``
1644- ``bb.event.ConfigFilePathFound()``
1645- ``bb.event.FilesMatchingFound()``
1646- ``bb.event.ConfigFilesFound()``
1647- ``bb.event.TargetsTreeGenerated()``
1648
1649.. _variants-class-extension-mechanism:
1650
1651Variants - Class Extension Mechanism
1652====================================
1653
Andrew Geissler09036742021-06-25 14:25:14 -05001654BitBake supports multiple incarnations of a recipe file via the
1655:term:`BBCLASSEXTEND` variable.
1656
1657The :term:`BBCLASSEXTEND` variable is a space separated list of classes used
1658to "extend" the recipe for each variant. Here is an example that results in a
1659second incarnation of the current recipe being available. This second
1660incarnation will have the "native" class inherited. ::
1661
1662 BBCLASSEXTEND = "native"
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001663
1664.. note::
1665
1666 The mechanism for this class extension is extremely specific to the
1667 implementation. Usually, the recipe's :term:`PROVIDES` , :term:`PN` , and
1668 :term:`DEPENDS` variables would need to be modified by the extension
1669 class. For specific examples, see the OE-Core native , nativesdk , and
1670 multilib classes.
1671
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001672Dependencies
1673============
1674
1675To allow for efficient parallel processing, BitBake handles dependencies
1676at the task level. Dependencies can exist both between tasks within a
1677single recipe and between tasks in different recipes. Following are
1678examples of each:
1679
1680- For tasks within a single recipe, a recipe's ``do_configure`` task
1681 might need to complete before its ``do_compile`` task can run.
1682
1683- For tasks in different recipes, one recipe's ``do_configure`` task
1684 might require another recipe's ``do_populate_sysroot`` task to finish
1685 first such that the libraries and headers provided by the other
1686 recipe are available.
1687
1688This section describes several ways to declare dependencies. Remember,
1689even though dependencies are declared in different ways, they are all
1690simply dependencies between tasks.
1691
1692.. _dependencies-internal-to-the-bb-file:
1693
1694Dependencies Internal to the ``.bb`` File
1695-----------------------------------------
1696
1697BitBake uses the ``addtask`` directive to manage dependencies that are
1698internal to a given recipe file. You can use the ``addtask`` directive
1699to indicate when a task is dependent on other tasks or when other tasks
Andrew Geisslerc926e172021-05-07 16:11:35 -05001700depend on that recipe. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001701
1702 addtask printdate after do_fetch before do_build
1703
1704In this example, the ``do_printdate`` task
1705depends on the completion of the ``do_fetch`` task, and the ``do_build``
1706task depends on the completion of the ``do_printdate`` task.
1707
1708.. note::
1709
1710 For a task to run, it must be a direct or indirect dependency of some
1711 other task that is scheduled to run.
1712
1713 For illustration, here are some examples:
1714
1715 - The directive ``addtask mytask before do_configure`` causes
1716 ``do_mytask`` to run before ``do_configure`` runs. Be aware that
1717 ``do_mytask`` still only runs if its :ref:`input
1718 checksum <bitbake-user-manual/bitbake-user-manual-execution:checksums (signatures)>` has changed since the last time it was
1719 run. Changes to the input checksum of ``do_mytask`` also
1720 indirectly cause ``do_configure`` to run.
1721
1722 - The directive ``addtask mytask after do_configure`` by itself
1723 never causes ``do_mytask`` to run. ``do_mytask`` can still be run
Andrew Geisslerc926e172021-05-07 16:11:35 -05001724 manually as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001725
1726 $ bitbake recipe -c mytask
1727
1728 Declaring ``do_mytask`` as a dependency of some other task that is
1729 scheduled to run also causes it to run. Regardless, the task runs after
1730 ``do_configure``.
1731
1732Build Dependencies
1733------------------
1734
1735BitBake uses the :term:`DEPENDS` variable to manage
1736build time dependencies. The ``[deptask]`` varflag for tasks signifies
Patrick Williams213cb262021-08-07 19:21:33 -05001737the task of each item listed in :term:`DEPENDS` that must complete before
Andrew Geisslerc926e172021-05-07 16:11:35 -05001738that task can be executed. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001739
1740 do_configure[deptask] = "do_populate_sysroot"
1741
1742In this example, the ``do_populate_sysroot`` task
Patrick Williams213cb262021-08-07 19:21:33 -05001743of each item in :term:`DEPENDS` must complete before ``do_configure`` can
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001744execute.
1745
1746Runtime Dependencies
1747--------------------
1748
1749BitBake uses the :term:`PACKAGES`, :term:`RDEPENDS`, and :term:`RRECOMMENDS`
1750variables to manage runtime dependencies.
1751
Patrick Williams213cb262021-08-07 19:21:33 -05001752The :term:`PACKAGES` variable lists runtime packages. Each of those packages
1753can have :term:`RDEPENDS` and :term:`RRECOMMENDS` runtime dependencies. The
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001754``[rdeptask]`` flag for tasks is used to signify the task of each item
1755runtime dependency which must have completed before that task can be
1756executed. ::
1757
1758 do_package_qa[rdeptask] = "do_packagedata"
1759
1760In the previous
Patrick Williams213cb262021-08-07 19:21:33 -05001761example, the ``do_packagedata`` task of each item in :term:`RDEPENDS` must
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001762have completed before ``do_package_qa`` can execute.
Patrick Williams213cb262021-08-07 19:21:33 -05001763Although :term:`RDEPENDS` contains entries from the
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001764runtime dependency namespace, BitBake knows how to map them back
1765to the build-time dependency namespace, in which the tasks are defined.
1766
1767Recursive Dependencies
1768----------------------
1769
1770BitBake uses the ``[recrdeptask]`` flag to manage recursive task
1771dependencies. BitBake looks through the build-time and runtime
1772dependencies of the current recipe, looks through the task's inter-task
1773dependencies, and then adds dependencies for the listed task. Once
1774BitBake has accomplished this, it recursively works through the
1775dependencies of those tasks. Iterative passes continue until all
1776dependencies are discovered and added.
1777
1778The ``[recrdeptask]`` flag is most commonly used in high-level recipes
1779that need to wait for some task to finish "globally". For example,
Andrew Geisslerc926e172021-05-07 16:11:35 -05001780``image.bbclass`` has the following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001781
1782 do_rootfs[recrdeptask] += "do_packagedata"
1783
1784This statement says that the ``do_packagedata`` task of
1785the current recipe and all recipes reachable (by way of dependencies)
1786from the image recipe must run before the ``do_rootfs`` task can run.
1787
1788BitBake allows a task to recursively depend on itself by
Andrew Geisslerc926e172021-05-07 16:11:35 -05001789referencing itself in the task list::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001790
1791 do_a[recrdeptask] = "do_a do_b"
1792
1793In the same way as before, this means that the ``do_a``
1794and ``do_b`` tasks of the current recipe and all
1795recipes reachable (by way of dependencies) from the recipe
1796must run before the ``do_a`` task can run. In this
1797case BitBake will ignore the current recipe's ``do_a``
1798task circular dependency on itself.
1799
1800Inter-Task Dependencies
1801-----------------------
1802
1803BitBake uses the ``[depends]`` flag in a more generic form to manage
1804inter-task dependencies. This more generic form allows for
1805inter-dependency checks for specific tasks rather than checks for the
Patrick Williams213cb262021-08-07 19:21:33 -05001806data in :term:`DEPENDS`. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001807
1808 do_patch[depends] = "quilt-native:do_populate_sysroot"
1809
1810In this example, the ``do_populate_sysroot`` task of the target ``quilt-native``
1811must have completed before the ``do_patch`` task can execute.
1812
1813The ``[rdepends]`` flag works in a similar way but takes targets in the
1814runtime namespace instead of the build-time dependency namespace.
1815
1816Functions You Can Call From Within Python
1817=========================================
1818
1819BitBake provides many functions you can call from within Python
1820functions. This section lists the most commonly used functions, and
1821mentions where to find others.
1822
1823Functions for Accessing Datastore Variables
1824-------------------------------------------
1825
1826It is often necessary to access variables in the BitBake datastore using
1827Python functions. The BitBake datastore has an API that allows you this
1828access. Here is a list of available operations:
1829
1830.. list-table::
1831 :widths: auto
1832 :header-rows: 1
1833
1834 * - *Operation*
1835 - *Description*
1836 * - ``d.getVar("X", expand)``
1837 - Returns the value of variable "X". Using "expand=True" expands the
1838 value. Returns "None" if the variable "X" does not exist.
1839 * - ``d.setVar("X", "value")``
1840 - Sets the variable "X" to "value"
1841 * - ``d.appendVar("X", "value")``
1842 - Adds "value" to the end of the variable "X". Acts like ``d.setVar("X",
1843 "value")`` if the variable "X" does not exist.
1844 * - ``d.prependVar("X", "value")``
1845 - Adds "value" to the start of the variable "X". Acts like
1846 ``d.setVar("X","value")`` if the variable "X" does not exist.
1847 * - ``d.delVar("X")``
1848 - Deletes the variable "X" from the datastore. Does nothing if the variable
1849 "X" does not exist.
1850 * - ``d.renameVar("X", "Y")``
1851 - Renames the variable "X" to "Y". Does nothing if the variable "X" does
1852 not exist.
1853 * - ``d.getVarFlag("X", flag, expand)``
1854 - Returns the value of variable "X". Using "expand=True" expands the
1855 value. Returns "None" if either the variable "X" or the named flag does
1856 not exist.
1857 * - ``d.setVarFlag("X", flag, "value")``
1858 - Sets the named flag for variable "X" to "value".
1859 * - ``d.appendVarFlag("X", flag, "value")``
1860 - Appends "value" to the named flag on the variable "X". Acts like
1861 ``d.setVarFlag("X", flag, "value")`` if the named flag does not exist.
1862 * - ``d.prependVarFlag("X", flag, "value")``
1863 - Prepends "value" to the named flag on the variable "X". Acts like
1864 ``d.setVarFlag("X", flag, "value")`` if the named flag does not exist.
1865 * - ``d.delVarFlag("X", flag)``
1866 - Deletes the named flag on the variable "X" from the datastore.
1867 * - ``d.setVarFlags("X", flagsdict)``
1868 - Sets the flags specified in the ``flagsdict()``
1869 parameter. ``setVarFlags`` does not clear previous flags. Think of this
1870 operation as ``addVarFlags``.
1871 * - ``d.getVarFlags("X")``
1872 - Returns a ``flagsdict`` of the flags for the variable "X". Returns "None"
1873 if the variable "X" does not exist.
1874 * - ``d.delVarFlags("X")``
1875 - Deletes all the flags for the variable "X". Does nothing if the variable
1876 "X" does not exist.
1877 * - ``d.expand(expression)``
1878 - Expands variable references in the specified string
1879 expression. References to variables that do not exist are left as is. For
1880 example, ``d.expand("foo ${X}")`` expands to the literal string "foo
1881 ${X}" if the variable "X" does not exist.
1882
1883Other Functions
1884---------------
1885
1886You can find many other functions that can be called from Python by
1887looking at the source code of the ``bb`` module, which is in
1888``bitbake/lib/bb``. For example, ``bitbake/lib/bb/utils.py`` includes
1889the commonly used functions ``bb.utils.contains()`` and
1890``bb.utils.mkdirhier()``, which come with docstrings.
1891
1892Task Checksums and Setscene
1893===========================
1894
1895BitBake uses checksums (or signatures) along with the setscene to
1896determine if a task needs to be run. This section describes the process.
1897To help understand how BitBake does this, the section assumes an
1898OpenEmbedded metadata-based example.
1899
1900These checksums are stored in :term:`STAMP`. You can
Andrew Geisslerc926e172021-05-07 16:11:35 -05001901examine the checksums using the following BitBake command::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001902
1903 $ bitbake-dumpsigs
1904
1905This command returns the signature data in a readable
1906format that allows you to examine the inputs used when the OpenEmbedded
1907build system generates signatures. For example, using
1908``bitbake-dumpsigs`` allows you to examine the ``do_compile`` task's
Andrew Geisslerf0343792020-11-18 10:42:21 -06001909"sigdata" for a C application (e.g. ``bash``). Running the command also
1910reveals that the "CC" variable is part of the inputs that are hashed.
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001911Any changes to this variable would invalidate the stamp and cause the
1912``do_compile`` task to run.
1913
1914The following list describes related variables:
1915
1916- :term:`BB_HASHCHECK_FUNCTION`:
1917 Specifies the name of the function to call during the "setscene" part
1918 of the task's execution in order to validate the list of task hashes.
1919
1920- :term:`BB_SETSCENE_DEPVALID`:
1921 Specifies a function BitBake calls that determines whether BitBake
1922 requires a setscene dependency to be met.
1923
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001924- :term:`BB_STAMP_POLICY`: Defines the mode
1925 for comparing timestamps of stamp files.
1926
1927- :term:`BB_STAMP_WHITELIST`: Lists stamp
1928 files that are looked at when the stamp policy is "whitelist".
1929
1930- :term:`BB_TASKHASH`: Within an executing task,
1931 this variable holds the hash of the task as returned by the currently
1932 enabled signature generator.
1933
1934- :term:`STAMP`: The base path to create stamp files.
1935
1936- :term:`STAMPCLEAN`: Again, the base path to
1937 create stamp files but can use wildcards for matching a range of
1938 files for clean operations.
1939
1940Wildcard Support in Variables
1941=============================
1942
1943Support for wildcard use in variables varies depending on the context in
Andrew Geissler5199d832021-09-24 16:47:35 -05001944which it is used. For example, some variables and filenames allow
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001945limited use of wildcards through the "``%``" and "``*``" characters.
1946Other variables or names support Python's
1947`glob <https://docs.python.org/3/library/glob.html>`_ syntax,
1948`fnmatch <https://docs.python.org/3/library/fnmatch.html#module-fnmatch>`_
1949syntax, or
1950`Regular Expression (re) <https://docs.python.org/3/library/re.html>`_
1951syntax.
1952
1953For variables that have wildcard suport, the documentation describes
1954which form of wildcard, its use, and its limitations.