blob: deb7afad435125009e4cc9abff5e4396430d3770 [file] [log] [blame]
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001.. SPDX-License-Identifier: CC-BY-2.5
2
3====================
4Syntax and Operators
5====================
6
7|
8
9BitBake files have their own syntax. The syntax has similarities to
10several other languages but also has some unique features. This section
11describes the available syntax and operators as well as provides
12examples.
13
14Basic Syntax
15============
16
17This section provides some basic syntax examples.
18
19Basic Variable Setting
20----------------------
21
22The following example sets ``VARIABLE`` to "value". This assignment
23occurs immediately as the statement is parsed. It is a "hard"
24assignment. ::
25
26 VARIABLE = "value"
27
28As expected, if you include leading or
Andrew Geisslerc926e172021-05-07 16:11:35 -050029trailing spaces as part of an assignment, the spaces are retained::
Andrew Geisslerc9f78652020-09-18 14:11:35 -050030
31 VARIABLE = " value"
32 VARIABLE = "value "
33
34Setting ``VARIABLE`` to "" sets
35it to an empty string, while setting the variable to " " sets it to a
36blank space (i.e. these are not the same values). ::
37
38 VARIABLE = ""
39 VARIABLE = " "
40
41You can use single quotes instead of double quotes when setting a
42variable's value. Doing so allows you to use values that contain the
Andrew Geisslerc926e172021-05-07 16:11:35 -050043double quote character::
Andrew Geisslerc9f78652020-09-18 14:11:35 -050044
45 VARIABLE = 'I have a " in my value'
46
47.. note::
48
49 Unlike in Bourne shells, single quotes work identically to double
50 quotes in all other ways. They do not suppress variable expansions.
51
52Modifying Existing Variables
53----------------------------
54
55Sometimes you need to modify existing variables. Following are some
56cases where you might find you want to modify an existing variable:
57
58- Customize a recipe that uses the variable.
59
60- Change a variable's default value used in a ``*.bbclass`` file.
61
62- Change the variable in a ``*.bbappend`` file to override the variable
63 in the original recipe.
64
65- Change the variable in a configuration file so that the value
66 overrides an existing configuration.
67
68Changing a variable value can sometimes depend on how the value was
69originally assigned and also on the desired intent of the change. In
70particular, when you append a value to a variable that has a default
71value, the resulting value might not be what you expect. In this case,
72the value you provide might replace the value rather than append to the
73default value.
74
75If after you have changed a variable's value and something unexplained
76occurs, you can use BitBake to check the actual value of the suspect
77variable. You can make these checks for both configuration and recipe
78level changes:
79
Andrew Geisslerc926e172021-05-07 16:11:35 -050080- For configuration changes, use the following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -050081
82 $ bitbake -e
83
84 This
85 command displays variable values after the configuration files (i.e.
86 ``local.conf``, ``bblayers.conf``, ``bitbake.conf`` and so forth)
87 have been parsed.
88
89 .. note::
90
91 Variables that are exported to the environment are preceded by the
92 string "export" in the command's output.
93
Patrick Williams213cb262021-08-07 19:21:33 -050094- To find changes to a given variable in a specific recipe, use the
95 following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -050096
Patrick Williams213cb262021-08-07 19:21:33 -050097 $ bitbake recipename -e | grep VARIABLENAME=\"
Andrew Geisslerc9f78652020-09-18 14:11:35 -050098
99 This command checks to see if the variable actually makes
100 it into a specific recipe.
101
102Line Joining
103------------
104
105Outside of :ref:`functions <bitbake-user-manual/bitbake-user-manual-metadata:functions>`,
106BitBake joins any line ending in
Andrew Geissler595f6302022-01-24 19:11:47 +0000107a backslash character ("\\") with the following line before parsing
108statements. The most common use for the "\\" character is to split
Andrew Geisslerc926e172021-05-07 16:11:35 -0500109variable assignments over multiple lines, as in the following example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500110
111 FOO = "bar \
112 baz \
113 qaz"
114
Andrew Geissler595f6302022-01-24 19:11:47 +0000115Both the "\\" character and the newline
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500116character that follow it are removed when joining lines. Thus, no
117newline characters end up in the value of ``FOO``.
118
119Consider this additional example where the two assignments both assign
Andrew Geisslerc926e172021-05-07 16:11:35 -0500120"barbaz" to ``FOO``::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500121
122 FOO = "barbaz"
123 FOO = "bar\
124 baz"
125
126.. note::
127
Andrew Geissler595f6302022-01-24 19:11:47 +0000128 BitBake does not interpret escape sequences like "\\n" in variable
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500129 values. For these to have an effect, the value must be passed to some
130 utility that interprets escape sequences, such as
131 ``printf`` or ``echo -n``.
132
133Variable Expansion
134------------------
135
136Variables can reference the contents of other variables using a syntax
137that is similar to variable expansion in Bourne shells. The following
138assignments result in A containing "aval" and B evaluating to
139"preavalpost". ::
140
141 A = "aval"
142 B = "pre${A}post"
143
144.. note::
145
146 Unlike in Bourne shells, the curly braces are mandatory: Only ``${FOO}`` and not
147 ``$FOO`` is recognized as an expansion of ``FOO``.
148
149The "=" operator does not immediately expand variable references in the
150right-hand side. Instead, expansion is deferred until the variable
151assigned to is actually used. The result depends on the current values
152of the referenced variables. The following example should clarify this
Andrew Geisslerc926e172021-05-07 16:11:35 -0500153behavior::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500154
155 A = "${B} baz"
156 B = "${C} bar"
157 C = "foo"
158 *At this point, ${A} equals "foo bar baz"*
159 C = "qux"
160 *At this point, ${A} equals "qux bar baz"*
161 B = "norf"
Andrew Geissler595f6302022-01-24 19:11:47 +0000162 *At this point, ${A} equals "norf baz"*
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500163
164Contrast this behavior with the
165:ref:`bitbake-user-manual/bitbake-user-manual-metadata:immediate variable
166expansion (:=)` operator.
167
168If the variable expansion syntax is used on a variable that does not
169exist, the string is kept as is. For example, given the following
170assignment, ``BAR`` expands to the literal string "${FOO}" as long as
171``FOO`` does not exist. ::
172
173 BAR = "${FOO}"
174
175Setting a default value (?=)
176----------------------------
177
178You can use the "?=" operator to achieve a "softer" assignment for a
179variable. This type of assignment allows you to define a variable if it
180is undefined when the statement is parsed, but to leave the value alone
Andrew Geisslerc926e172021-05-07 16:11:35 -0500181if the variable has a value. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500182
183 A ?= "aval"
184
185If ``A`` is
186set at the time this statement is parsed, the variable retains its
187value. However, if ``A`` is not set, the variable is set to "aval".
188
189.. note::
190
191 This assignment is immediate. Consequently, if multiple "?="
192 assignments to a single variable exist, the first of those ends up
193 getting used.
194
195Setting a weak default value (??=)
196----------------------------------
197
Andrew Geissler615f2f12022-07-15 14:00:58 -0500198The weak default value of a variable is the value which that variable
199will expand to if no value has been assigned to it via any of the other
200assignment operators. The "??=" operator takes effect immediately, replacing
201any previously defined weak default value. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500202
Andrew Geissler615f2f12022-07-15 14:00:58 -0500203 W ??= "x"
204 A := "${W}" # Immediate variable expansion
205 W ??= "y"
206 B := "${W}" # Immediate variable expansion
207 W ??= "z"
208 C = "${W}"
209 W ?= "i"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500210
Andrew Geissler615f2f12022-07-15 14:00:58 -0500211After parsing we will have::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500212
Andrew Geissler615f2f12022-07-15 14:00:58 -0500213 A = "x"
214 B = "y"
215 C = "i"
216 W = "i"
217
218Appending and prepending non-override style will not substitute the weak
219default value, which means that after parsing::
220
221 W ??= "x"
222 W += "y"
223
224we will have::
225
226 W = " y"
227
228On the other hand, override-style appends/prepends/removes are applied after
229any active weak default value has been substituted::
230
231 W ??= "x"
232 W:append = "y"
233
234After parsing we will have::
235
236 W = "xy"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500237
238Immediate variable expansion (:=)
239---------------------------------
240
241The ":=" operator results in a variable's contents being expanded
Andrew Geisslerc926e172021-05-07 16:11:35 -0500242immediately, rather than when the variable is actually used::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500243
244 T = "123"
245 A := "test ${T}"
246 T = "456"
247 B := "${T} ${C}"
248 C = "cval"
249 C := "${C}append"
250
251In this example, ``A`` contains "test 123", even though the final value
Patrick Williams213cb262021-08-07 19:21:33 -0500252of :term:`T` is "456". The variable :term:`B` will end up containing "456
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500253cvalappend". This is because references to undefined variables are
254preserved as is during (immediate)expansion. This is in contrast to GNU
255Make, where undefined variables expand to nothing. The variable ``C``
256contains "cvalappend" since ``${C}`` immediately expands to "cval".
257
258.. _appending-and-prepending:
259
260Appending (+=) and prepending (=+) With Spaces
261----------------------------------------------
262
263Appending and prepending values is common and can be accomplished using
264the "+=" and "=+" operators. These operators insert a space between the
265current value and prepended or appended value.
266
267These operators take immediate effect during parsing. Here are some
Andrew Geisslerc926e172021-05-07 16:11:35 -0500268examples::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500269
270 B = "bval"
271 B += "additionaldata"
272 C = "cval"
273 C =+ "test"
274
Patrick Williams213cb262021-08-07 19:21:33 -0500275The variable :term:`B` contains "bval additionaldata" and ``C`` contains "test
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500276cval".
277
278.. _appending-and-prepending-without-spaces:
279
280Appending (.=) and Prepending (=.) Without Spaces
281-------------------------------------------------
282
283If you want to append or prepend values without an inserted space, use
284the ".=" and "=." operators.
285
286These operators take immediate effect during parsing. Here are some
Andrew Geisslerc926e172021-05-07 16:11:35 -0500287examples::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500288
289 B = "bval"
290 B .= "additionaldata"
291 C = "cval"
292 C =. "test"
293
Patrick Williams213cb262021-08-07 19:21:33 -0500294The variable :term:`B` contains "bvaladditionaldata" and ``C`` contains
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500295"testcval".
296
297Appending and Prepending (Override Style Syntax)
298------------------------------------------------
299
300You can also append and prepend a variable's value using an override
301style syntax. When you use this syntax, no spaces are inserted.
302
303These operators differ from the ":=", ".=", "=.", "+=", and "=+"
304operators in that their effects are applied at variable expansion time
Andrew Geisslerc926e172021-05-07 16:11:35 -0500305rather than being immediately applied. Here are some examples::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500306
307 B = "bval"
Patrick Williams213cb262021-08-07 19:21:33 -0500308 B:append = " additional data"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500309 C = "cval"
Patrick Williams213cb262021-08-07 19:21:33 -0500310 C:prepend = "additional data "
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500311 D = "dval"
Patrick Williams213cb262021-08-07 19:21:33 -0500312 D:append = "additional data"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500313
Patrick Williams213cb262021-08-07 19:21:33 -0500314The variable :term:`B`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500315becomes "bval additional data" and ``C`` becomes "additional data cval".
316The variable ``D`` becomes "dvaladditional data".
317
318.. note::
319
320 You must control all spacing when you use the override syntax.
321
Andrew Geissler517393d2023-01-13 08:55:19 -0600322.. note::
323
324 The overrides are applied in this order, ":append", ":prepend", ":remove".
325
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500326It is also possible to append and prepend to shell functions and
327BitBake-style Python functions. See the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:shell functions`" and ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:bitbake-style python functions`"
328sections for examples.
329
330.. _removing-override-style-syntax:
331
332Removal (Override Style Syntax)
333-------------------------------
334
335You can remove values from lists using the removal override style
336syntax. Specifying a value for removal causes all occurrences of that
Patrick Williams2390b1b2022-11-03 13:47:49 -0500337value to be removed from the variable. Unlike ":append" and ":prepend",
338there is no need to add a leading or trailing space to the value.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500339
340When you use this syntax, BitBake expects one or more strings.
Andrew Geisslerc926e172021-05-07 16:11:35 -0500341Surrounding spaces and spacing are preserved. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500342
343 FOO = "123 456 789 123456 123 456 123 456"
Patrick Williams213cb262021-08-07 19:21:33 -0500344 FOO:remove = "123"
345 FOO:remove = "456"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500346 FOO2 = " abc def ghi abcdef abc def abc def def"
Patrick Williams213cb262021-08-07 19:21:33 -0500347 FOO2:remove = "\
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500348 def \
349 abc \
350 ghi \
351 "
352
353The variable ``FOO`` becomes
354" 789 123456 " and ``FOO2`` becomes " abcdef ".
355
Patrick Williams213cb262021-08-07 19:21:33 -0500356Like ":append" and ":prepend", ":remove" is applied at variable
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500357expansion time.
358
Andrew Geissler517393d2023-01-13 08:55:19 -0600359.. note::
360
361 The overrides are applied in this order, ":append", ":prepend", ":remove".
362 This implies it is not possible to re-append previously removed strings.
363 However, one can undo a ":remove" by using an intermediate variable whose
364 content is passed to the ":remove" so that modifying the intermediate
365 variable equals to keeping the string in::
366
367 FOOREMOVE = "123 456 789"
368 FOO:remove = "${FOOREMOVE}"
369 ...
370 FOOREMOVE = "123 789"
371
372 This expands to ``FOO:remove = "123 789"``.
373
374.. note::
375
376 Override application order may not match variable parse history, i.e.
377 the output of ``bitbake -e`` may contain ":remove" before ":append",
378 but the result will be removed string, because ":remove" is handled
379 last.
380
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500381Override Style Operation Advantages
382-----------------------------------
383
Patrick Williams213cb262021-08-07 19:21:33 -0500384An advantage of the override style operations ":append", ":prepend", and
385":remove" as compared to the "+=" and "=+" operators is that the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500386override style operators provide guaranteed operations. For example,
387consider a class ``foo.bbclass`` that needs to add the value "val" to
Andrew Geisslerc926e172021-05-07 16:11:35 -0500388the variable ``FOO``, and a recipe that uses ``foo.bbclass`` as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500389
390 inherit foo
391 FOO = "initial"
392
393If ``foo.bbclass`` uses the "+=" operator,
394as follows, then the final value of ``FOO`` will be "initial", which is
Andrew Geisslerc926e172021-05-07 16:11:35 -0500395not what is desired::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500396
397 FOO += "val"
398
399If, on the other hand, ``foo.bbclass``
Patrick Williams213cb262021-08-07 19:21:33 -0500400uses the ":append" operator, then the final value of ``FOO`` will be
Andrew Geisslerc926e172021-05-07 16:11:35 -0500401"initial val", as intended::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500402
Patrick Williams213cb262021-08-07 19:21:33 -0500403 FOO:append = " val"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500404
405.. note::
406
Patrick Williams213cb262021-08-07 19:21:33 -0500407 It is never necessary to use "+=" together with ":append". The following
Andrew Geisslerc926e172021-05-07 16:11:35 -0500408 sequence of assignments appends "barbaz" to FOO::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500409
Patrick Williams213cb262021-08-07 19:21:33 -0500410 FOO:append = "bar"
411 FOO:append = "baz"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500412
413
414 The only effect of changing the second assignment in the previous
415 example to use "+=" would be to add a space before "baz" in the
416 appended value (due to how the "+=" operator works).
417
418Another advantage of the override style operations is that you can
419combine them with other overrides as described in the
420":ref:`bitbake-user-manual/bitbake-user-manual-metadata:conditional syntax (overrides)`" section.
421
422Variable Flag Syntax
423--------------------
424
425Variable flags are BitBake's implementation of variable properties or
426attributes. It is a way of tagging extra information onto a variable.
427You can find more out about variable flags in general in the
428":ref:`bitbake-user-manual/bitbake-user-manual-metadata:variable flags`" section.
429
430You can define, append, and prepend values to variable flags. All the
431standard syntax operations previously mentioned work for variable flags
Patrick Williams213cb262021-08-07 19:21:33 -0500432except for override style syntax (i.e. ":prepend", ":append", and
433":remove").
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500434
Andrew Geisslerc926e172021-05-07 16:11:35 -0500435Here are some examples showing how to set variable flags::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500436
437 FOO[a] = "abc"
438 FOO[b] = "123"
439 FOO[a] += "456"
440
441The variable ``FOO`` has two flags:
442``[a]`` and ``[b]``. The flags are immediately set to "abc" and "123",
443respectively. The ``[a]`` flag becomes "abc 456".
444
445No need exists to pre-define variable flags. You can simply start using
446them. One extremely common application is to attach some brief
Andrew Geisslerc926e172021-05-07 16:11:35 -0500447documentation to a BitBake variable as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500448
449 CACHE[doc] = "The directory holding the cache of the metadata."
450
Patrick Williams2390b1b2022-11-03 13:47:49 -0500451.. note::
452
453 Variable flag names starting with an underscore (``_``) character
454 are allowed but are ignored by ``d.getVarFlags("VAR")``
455 in Python code. Such flag names are used internally by BitBake.
456
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500457Inline Python Variable Expansion
458--------------------------------
459
460You can use inline Python variable expansion to set variables. Here is
Andrew Geisslerc926e172021-05-07 16:11:35 -0500461an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500462
463 DATE = "${@time.strftime('%Y%m%d',time.gmtime())}"
464
465This example results in the ``DATE`` variable being set to the current date.
466
467Probably the most common use of this feature is to extract the value of
468variables from BitBake's internal data dictionary, ``d``. The following
469lines select the values of a package name and its version number,
Andrew Geisslerc926e172021-05-07 16:11:35 -0500470respectively::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500471
Andrew Geissler5199d832021-09-24 16:47:35 -0500472 PN = "${@bb.parse.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'}"
473 PV = "${@bb.parse.vars_from_file(d.getVar('FILE', False),d)[1] or '1.0'}"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500474
475.. note::
476
477 Inline Python expressions work just like variable expansions insofar as the
478 "=" and ":=" operators are concerned. Given the following assignment, foo()
Andrew Geisslerc926e172021-05-07 16:11:35 -0500479 is called each time FOO is expanded::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500480
481 FOO = "${@foo()}"
482
483 Contrast this with the following immediate assignment, where foo() is only
Andrew Geisslerc926e172021-05-07 16:11:35 -0500484 called once, while the assignment is parsed::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500485
486 FOO := "${@foo()}"
487
488For a different way to set variables with Python code during parsing,
489see the
490":ref:`bitbake-user-manual/bitbake-user-manual-metadata:anonymous python functions`" section.
491
492Unsetting variables
493-------------------
494
495It is possible to completely remove a variable or a variable flag from
496BitBake's internal data dictionary by using the "unset" keyword. Here is
Andrew Geisslerc926e172021-05-07 16:11:35 -0500497an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500498
499 unset DATE
500 unset do_fetch[noexec]
501
502These two statements remove the ``DATE`` and the ``do_fetch[noexec]`` flag.
503
504Providing Pathnames
505-------------------
506
507When specifying pathnames for use with BitBake, do not use the tilde
508("~") character as a shortcut for your home directory. Doing so might
509cause BitBake to not recognize the path since BitBake does not expand
510this character in the same way a shell would.
511
Andrew Geisslerc926e172021-05-07 16:11:35 -0500512Instead, provide a fuller path as the following example illustrates::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500513
514 BBLAYERS ?= " \
515 /home/scott-lenovo/LayerA \
516 "
517
518Exporting Variables to the Environment
519======================================
520
521You can export variables to the environment of running tasks by using
522the ``export`` keyword. For example, in the following example, the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500523``do_foo`` task prints "value from the environment" when run::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500524
525 export ENV_VARIABLE
526 ENV_VARIABLE = "value from the environment"
527
528 do_foo() {
529 bbplain "$ENV_VARIABLE"
530 }
531
532.. note::
533
534 BitBake does not expand ``$ENV_VARIABLE`` in this case because it lacks the
535 obligatory ``{}`` . Rather, ``$ENV_VARIABLE`` is expanded by the shell.
536
537It does not matter whether ``export ENV_VARIABLE`` appears before or
538after assignments to ``ENV_VARIABLE``.
539
540It is also possible to combine ``export`` with setting a value for the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500541variable. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500542
543 export ENV_VARIABLE = "variable-value"
544
545In the output of ``bitbake -e``, variables that are exported to the
546environment are preceded by "export".
547
548Among the variables commonly exported to the environment are ``CC`` and
549``CFLAGS``, which are picked up by many build systems.
550
551Conditional Syntax (Overrides)
552==============================
553
554BitBake uses :term:`OVERRIDES` to control what
555variables are overridden after BitBake parses recipes and configuration
Patrick Williams213cb262021-08-07 19:21:33 -0500556files. This section describes how you can use :term:`OVERRIDES` as
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500557conditional metadata, talks about key expansion in relationship to
Patrick Williams213cb262021-08-07 19:21:33 -0500558:term:`OVERRIDES`, and provides some examples to help with understanding.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500559
560Conditional Metadata
561--------------------
562
Patrick Williams213cb262021-08-07 19:21:33 -0500563You can use :term:`OVERRIDES` to conditionally select a specific version of
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500564a variable and to conditionally append or prepend the value of a
565variable.
566
567.. note::
568
Andrew Geissler9aee5002022-03-30 16:27:02 +0000569 Overrides can only use lower-case characters, digits and dashes.
570 In particular, colons are not permitted in override names as they are used to
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500571 separate overrides from each other and from the variable name.
572
Patrick Williams213cb262021-08-07 19:21:33 -0500573- *Selecting a Variable:* The :term:`OVERRIDES` variable is a
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500574 colon-character-separated list that contains items for which you want
575 to satisfy conditions. Thus, if you have a variable that is
Patrick Williams213cb262021-08-07 19:21:33 -0500576 conditional on "arm", and "arm" is in :term:`OVERRIDES`, then the
Andrew Geisslerf0343792020-11-18 10:42:21 -0600577 "arm"-specific version of the variable is used rather than the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500578 non-conditional version. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500579
580 OVERRIDES = "architecture:os:machine"
581 TEST = "default"
Andrew Geissler9aee5002022-03-30 16:27:02 +0000582 TEST:os = "osspecific"
583 TEST:nooverride = "othercondvalue"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500584
Patrick Williams213cb262021-08-07 19:21:33 -0500585 In this example, the :term:`OVERRIDES`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500586 variable lists three overrides: "architecture", "os", and "machine".
587 The variable ``TEST`` by itself has a default value of "default". You
588 select the os-specific version of the ``TEST`` variable by appending
Andrew Geissler9aee5002022-03-30 16:27:02 +0000589 the "os" override to the variable (i.e. ``TEST:os``).
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500590
591 To better understand this, consider a practical example that assumes
592 an OpenEmbedded metadata-based Linux kernel recipe file. The
593 following lines from the recipe file first set the kernel branch
594 variable ``KBRANCH`` to a default value, then conditionally override
Andrew Geisslerc926e172021-05-07 16:11:35 -0500595 that value based on the architecture of the build::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500596
597 KBRANCH = "standard/base"
Patrick Williams213cb262021-08-07 19:21:33 -0500598 KBRANCH:qemuarm = "standard/arm-versatile-926ejs"
599 KBRANCH:qemumips = "standard/mti-malta32"
600 KBRANCH:qemuppc = "standard/qemuppc"
601 KBRANCH:qemux86 = "standard/common-pc/base"
602 KBRANCH:qemux86-64 = "standard/common-pc-64/base"
603 KBRANCH:qemumips64 = "standard/mti-malta64"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500604
605- *Appending and Prepending:* BitBake also supports append and prepend
606 operations to variable values based on whether a specific item is
Patrick Williams213cb262021-08-07 19:21:33 -0500607 listed in :term:`OVERRIDES`. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500608
609 DEPENDS = "glibc ncurses"
610 OVERRIDES = "machine:local"
Patrick Williams213cb262021-08-07 19:21:33 -0500611 DEPENDS:append:machine = "libmad"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500612
Patrick Williams213cb262021-08-07 19:21:33 -0500613 In this example, :term:`DEPENDS` becomes "glibc ncurses libmad".
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500614
615 Again, using an OpenEmbedded metadata-based kernel recipe file as an
616 example, the following lines will conditionally append to the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500617 ``KERNEL_FEATURES`` variable based on the architecture::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500618
Patrick Williams213cb262021-08-07 19:21:33 -0500619 KERNEL_FEATURES:append = " ${KERNEL_EXTRA_FEATURES}"
620 KERNEL_FEATURES:append:qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
621 KERNEL_FEATURES:append:qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500622
623- *Setting a Variable for a Single Task:* BitBake supports setting a
Andrew Geisslerc926e172021-05-07 16:11:35 -0500624 variable just for the duration of a single task. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500625
Andrew Geissler9aee5002022-03-30 16:27:02 +0000626 FOO:task-configure = "val 1"
Patrick Williams213cb262021-08-07 19:21:33 -0500627 FOO:task-compile = "val 2"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500628
629 In the
630 previous example, ``FOO`` has the value "val 1" while the
631 ``do_configure`` task is executed, and the value "val 2" while the
632 ``do_compile`` task is executed.
633
634 Internally, this is implemented by prepending the task (e.g.
635 "task-compile:") to the value of
636 :term:`OVERRIDES` for the local datastore of the
637 ``do_compile`` task.
638
639 You can also use this syntax with other combinations (e.g.
Patrick Williams213cb262021-08-07 19:21:33 -0500640 "``:prepend``") as shown in the following example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500641
Patrick Williams213cb262021-08-07 19:21:33 -0500642 EXTRA_OEMAKE:prepend:task-compile = "${PARALLEL_MAKE} "
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500643
Andrew Geissler9aee5002022-03-30 16:27:02 +0000644.. note::
645
646 Before BitBake 1.52 (Honister 3.4), the syntax for :term:`OVERRIDES`
647 used ``_`` instead of ``:``, so you will still find a lot of documentation
648 using ``_append``, ``_prepend``, and ``_remove``, for example.
649
650 For details, see the
651 :yocto_docs:`Overrides Syntax Changes </migration-guides/migration-3.4.html#override-syntax-changes>`
652 section in the Yocto Project manual migration notes.
653
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500654Key Expansion
655-------------
656
657Key expansion happens when the BitBake datastore is finalized. To better
Andrew Geisslerc926e172021-05-07 16:11:35 -0500658understand this, consider the following example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500659
660 A${B} = "X"
661 B = "2"
662 A2 = "Y"
663
664In this case, after all the parsing is complete, BitBake expands
665``${B}`` into "2". This expansion causes ``A2``, which was set to "Y"
666before the expansion, to become "X".
667
668.. _variable-interaction-worked-examples:
669
670Examples
671--------
672
673Despite the previous explanations that show the different forms of
674variable definitions, it can be hard to work out exactly what happens
675when variable operators, conditional overrides, and unconditional
676overrides are combined. This section presents some common scenarios
677along with explanations for variable interactions that typically confuse
678users.
679
680There is often confusion concerning the order in which overrides and
681various "append" operators take effect. Recall that an append or prepend
Patrick Williams213cb262021-08-07 19:21:33 -0500682operation using ":append" and ":prepend" does not result in an immediate
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500683assignment as would "+=", ".=", "=+", or "=.". Consider the following
Andrew Geisslerc926e172021-05-07 16:11:35 -0500684example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500685
686 OVERRIDES = "foo"
687 A = "Z"
Patrick Williams213cb262021-08-07 19:21:33 -0500688 A:foo:append = "X"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500689
690For this case,
691``A`` is unconditionally set to "Z" and "X" is unconditionally and
Patrick Williams213cb262021-08-07 19:21:33 -0500692immediately appended to the variable ``A:foo``. Because overrides have
693not been applied yet, ``A:foo`` is set to "X" due to the append and
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500694``A`` simply equals "Z".
695
696Applying overrides, however, changes things. Since "foo" is listed in
Patrick Williams213cb262021-08-07 19:21:33 -0500697:term:`OVERRIDES`, the conditional variable ``A`` is replaced with the "foo"
698version, which is equal to "X". So effectively, ``A:foo`` replaces
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500699``A``.
700
Andrew Geisslerc926e172021-05-07 16:11:35 -0500701This next example changes the order of the override and the append::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500702
703 OVERRIDES = "foo"
704 A = "Z"
Patrick Williams213cb262021-08-07 19:21:33 -0500705 A:append:foo = "X"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500706
707For this case, before
Patrick Williams213cb262021-08-07 19:21:33 -0500708overrides are handled, ``A`` is set to "Z" and ``A:append:foo`` is set
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500709to "X". Once the override for "foo" is applied, however, ``A`` gets
710appended with "X". Consequently, ``A`` becomes "ZX". Notice that spaces
711are not appended.
712
713This next example has the order of the appends and overrides reversed
Andrew Geisslerc926e172021-05-07 16:11:35 -0500714back as in the first example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500715
716 OVERRIDES = "foo"
717 A = "Y"
Patrick Williams213cb262021-08-07 19:21:33 -0500718 A:foo:append = "Z"
719 A:foo:append = "X"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500720
721For this case, before any overrides are resolved,
722``A`` is set to "Y" using an immediate assignment. After this immediate
Patrick Williams213cb262021-08-07 19:21:33 -0500723assignment, ``A:foo`` is set to "Z", and then further appended with "X"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500724leaving the variable set to "ZX". Finally, applying the override for
725"foo" results in the conditional variable ``A`` becoming "ZX" (i.e.
Patrick Williams213cb262021-08-07 19:21:33 -0500726``A`` is replaced with ``A:foo``).
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500727
Andrew Geisslerc926e172021-05-07 16:11:35 -0500728This final example mixes in some varying operators::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500729
730 A = "1"
Patrick Williams213cb262021-08-07 19:21:33 -0500731 A:append = "2"
732 A:append = "3"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500733 A += "4"
734 A .= "5"
735
736For this case, the type of append
737operators are affecting the order of assignments as BitBake passes
738through the code multiple times. Initially, ``A`` is set to "1 45"
739because of the three statements that use immediate operators. After
Patrick Williams213cb262021-08-07 19:21:33 -0500740these assignments are made, BitBake applies the ":append" operations.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500741Those operations result in ``A`` becoming "1 4523".
742
743Sharing Functionality
744=====================
745
746BitBake allows for metadata sharing through include files (``.inc``) and
747class files (``.bbclass``). For example, suppose you have a piece of
748common functionality such as a task definition that you want to share
749between more than one recipe. In this case, creating a ``.bbclass`` file
750that contains the common functionality and then using the ``inherit``
751directive in your recipes to inherit the class would be a common way to
752share the task.
753
754This section presents the mechanisms BitBake provides to allow you to
755share functionality between recipes. Specifically, the mechanisms
Patrick Williams213cb262021-08-07 19:21:33 -0500756include ``include``, ``inherit``, :term:`INHERIT`, and ``require``
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500757directives.
758
759Locating Include and Class Files
760--------------------------------
761
762BitBake uses the :term:`BBPATH` variable to locate
763needed include and class files. Additionally, BitBake searches the
764current directory for ``include`` and ``require`` directives.
765
766.. note::
767
768 The BBPATH variable is analogous to the environment variable PATH .
769
770In order for include and class files to be found by BitBake, they need
771to be located in a "classes" subdirectory that can be found in
Patrick Williams213cb262021-08-07 19:21:33 -0500772:term:`BBPATH`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500773
774``inherit`` Directive
775---------------------
776
777When writing a recipe or class file, you can use the ``inherit``
778directive to inherit the functionality of a class (``.bbclass``).
779BitBake only supports this directive when used within recipe and class
780files (i.e. ``.bb`` and ``.bbclass``).
781
782The ``inherit`` directive is a rudimentary means of specifying
783functionality contained in class files that your recipes require. For
784example, you can easily abstract out the tasks involved in building a
785package that uses Autoconf and Automake and put those tasks into a class
786file and then have your recipe inherit that class file.
787
788As an example, your recipes could use the following directive to inherit
789an ``autotools.bbclass`` file. The class file would contain common
Andrew Geisslerc926e172021-05-07 16:11:35 -0500790functionality for using Autotools that could be shared across recipes::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500791
792 inherit autotools
793
794In this case, BitBake would search for the directory
Patrick Williams213cb262021-08-07 19:21:33 -0500795``classes/autotools.bbclass`` in :term:`BBPATH`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500796
797.. note::
798
799 You can override any values and functions of the inherited class
800 within your recipe by doing so after the "inherit" statement.
801
802If you want to use the directive to inherit multiple classes, separate
803them with spaces. The following example shows how to inherit both the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500804``buildhistory`` and ``rm_work`` classes::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500805
806 inherit buildhistory rm_work
807
808An advantage with the inherit directive as compared to both the
809:ref:`include <bitbake-user-manual/bitbake-user-manual-metadata:\`\`include\`\` directive>` and :ref:`require <bitbake-user-manual/bitbake-user-manual-metadata:\`\`require\`\` directive>`
810directives is that you can inherit class files conditionally. You can
811accomplish this by using a variable expression after the ``inherit``
Andrew Geisslerc926e172021-05-07 16:11:35 -0500812statement. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500813
814 inherit ${VARNAME}
815
816If ``VARNAME`` is
817going to be set, it needs to be set before the ``inherit`` statement is
818parsed. One way to achieve a conditional inherit in this case is to use
Andrew Geisslerc926e172021-05-07 16:11:35 -0500819overrides::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500820
821 VARIABLE = ""
Patrick Williams213cb262021-08-07 19:21:33 -0500822 VARIABLE:someoverride = "myclass"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500823
Andrew Geisslerc926e172021-05-07 16:11:35 -0500824Another method is by using anonymous Python. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500825
826 python () {
827 if condition == value:
828 d.setVar('VARIABLE', 'myclass')
829 else:
830 d.setVar('VARIABLE', '')
831 }
832
833Alternatively, you could use an in-line Python expression in the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500834following form::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500835
836 inherit ${@'classname' if condition else ''}
837 inherit ${@functionname(params)}
838
839In all cases, if the expression evaluates to an
840empty string, the statement does not trigger a syntax error because it
841becomes a no-op.
842
843``include`` Directive
844---------------------
845
846BitBake understands the ``include`` directive. This directive causes
847BitBake to parse whatever file you specify, and to insert that file at
848that location. The directive is much like its equivalent in Make except
849that if the path specified on the include line is a relative path,
Patrick Williams213cb262021-08-07 19:21:33 -0500850BitBake locates the first file it can find within :term:`BBPATH`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500851
852The include directive is a more generic method of including
853functionality as compared to the :ref:`inherit <bitbake-user-manual/bitbake-user-manual-metadata:\`\`inherit\`\` directive>`
854directive, which is restricted to class (i.e. ``.bbclass``) files. The
855include directive is applicable for any other kind of shared or
856encapsulated functionality or configuration that does not suit a
857``.bbclass`` file.
858
859As an example, suppose you needed a recipe to include some self-test
Andrew Geisslerc926e172021-05-07 16:11:35 -0500860definitions::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500861
862 include test_defs.inc
863
864.. note::
865
866 The include directive does not produce an error when the file cannot be
867 found. Consequently, it is recommended that if the file you are including is
868 expected to exist, you should use :ref:`require <require-inclusion>` instead
869 of include . Doing so makes sure that an error is produced if the file cannot
870 be found.
871
872.. _require-inclusion:
873
874``require`` Directive
875---------------------
876
877BitBake understands the ``require`` directive. This directive behaves
878just like the ``include`` directive with the exception that BitBake
879raises a parsing error if the file to be included cannot be found. Thus,
880any file you require is inserted into the file that is being parsed at
881the location of the directive.
882
883The require directive, like the include directive previously described,
884is a more generic method of including functionality as compared to the
885:ref:`inherit <bitbake-user-manual/bitbake-user-manual-metadata:\`\`inherit\`\` directive>` directive, which is restricted to class
886(i.e. ``.bbclass``) files. The require directive is applicable for any
887other kind of shared or encapsulated functionality or configuration that
888does not suit a ``.bbclass`` file.
889
890Similar to how BitBake handles :ref:`include <bitbake-user-manual/bitbake-user-manual-metadata:\`\`include\`\` directive>`, if
891the path specified on the require line is a relative path, BitBake
Patrick Williams213cb262021-08-07 19:21:33 -0500892locates the first file it can find within :term:`BBPATH`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500893
894As an example, suppose you have two versions of a recipe (e.g.
895``foo_1.2.2.bb`` and ``foo_2.0.0.bb``) where each version contains some
896identical functionality that could be shared. You could create an
897include file named ``foo.inc`` that contains the common definitions
898needed to build "foo". You need to be sure ``foo.inc`` is located in the
899same directory as your two recipe files as well. Once these conditions
900are set up, you can share the functionality using a ``require``
Andrew Geisslerc926e172021-05-07 16:11:35 -0500901directive from within each recipe::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500902
903 require foo.inc
904
905``INHERIT`` Configuration Directive
906-----------------------------------
907
908When creating a configuration file (``.conf``), you can use the
909:term:`INHERIT` configuration directive to inherit a
910class. BitBake only supports this directive when used within a
911configuration file.
912
913As an example, suppose you needed to inherit a class file called
Andrew Geisslerc926e172021-05-07 16:11:35 -0500914``abc.bbclass`` from a configuration file as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500915
916 INHERIT += "abc"
917
918This configuration directive causes the named class to be inherited at
919the point of the directive during parsing. As with the ``inherit``
920directive, the ``.bbclass`` file must be located in a "classes"
Patrick Williams213cb262021-08-07 19:21:33 -0500921subdirectory in one of the directories specified in :term:`BBPATH`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500922
923.. note::
924
925 Because .conf files are parsed first during BitBake's execution, using
926 INHERIT to inherit a class effectively inherits the class globally (i.e. for
927 all recipes).
928
929If you want to use the directive to inherit multiple classes, you can
930provide them on the same line in the ``local.conf`` file. Use spaces to
931separate the classes. The following example shows how to inherit both
Andrew Geisslerc926e172021-05-07 16:11:35 -0500932the ``autotools`` and ``pkgconfig`` classes::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500933
934 INHERIT += "autotools pkgconfig"
935
936Functions
937=========
938
939As with most languages, functions are the building blocks that are used
940to build up operations into tasks. BitBake supports these types of
941functions:
942
943- *Shell Functions:* Functions written in shell script and executed
944 either directly as functions, tasks, or both. They can also be called
945 by other shell functions.
946
947- *BitBake-Style Python Functions:* Functions written in Python and
948 executed by BitBake or other Python functions using
949 ``bb.build.exec_func()``.
950
951- *Python Functions:* Functions written in Python and executed by
952 Python.
953
954- *Anonymous Python Functions:* Python functions executed automatically
955 during parsing.
956
957Regardless of the type of function, you can only define them in class
958(``.bbclass``) and recipe (``.bb`` or ``.inc``) files.
959
960Shell Functions
961---------------
962
Andrew Geissler595f6302022-01-24 19:11:47 +0000963Functions written in shell script are executed either directly as
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500964functions, tasks, or both. They can also be called by other shell
Andrew Geisslerc926e172021-05-07 16:11:35 -0500965functions. Here is an example shell function definition::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500966
967 some_function () {
968 echo "Hello World"
969 }
970
971When you create these types of functions in
972your recipe or class files, you need to follow the shell programming
973rules. The scripts are executed by ``/bin/sh``, which may not be a bash
974shell but might be something such as ``dash``. You should not use
975Bash-specific script (bashisms).
976
Patrick Williams213cb262021-08-07 19:21:33 -0500977Overrides and override-style operators like ``:append`` and ``:prepend``
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500978can also be applied to shell functions. Most commonly, this application
979would be used in a ``.bbappend`` file to modify functions in the main
980recipe. It can also be used to modify functions inherited from classes.
981
Andrew Geisslerc926e172021-05-07 16:11:35 -0500982As an example, consider the following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500983
984 do_foo() {
985 bbplain first
986 fn
987 }
988
Patrick Williams213cb262021-08-07 19:21:33 -0500989 fn:prepend() {
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500990 bbplain second
991 }
992
993 fn() {
994 bbplain third
995 }
996
Patrick Williams213cb262021-08-07 19:21:33 -0500997 do_foo:append() {
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500998 bbplain fourth
999 }
1000
Andrew Geisslerc926e172021-05-07 16:11:35 -05001001Running ``do_foo`` prints the following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001002
1003 recipename do_foo: first
1004 recipename do_foo: second
1005 recipename do_foo: third
1006 recipename do_foo: fourth
1007
1008.. note::
1009
1010 Overrides and override-style operators can be applied to any shell
1011 function, not just :ref:`tasks <bitbake-user-manual/bitbake-user-manual-metadata:tasks>`.
1012
Andrew Geissler595f6302022-01-24 19:11:47 +00001013You can use the ``bitbake -e recipename`` command to view the final
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001014assembled function after all overrides have been applied.
1015
1016BitBake-Style Python Functions
1017------------------------------
1018
1019These functions are written in Python and executed by BitBake or other
1020Python functions using ``bb.build.exec_func()``.
1021
Andrew Geisslerc926e172021-05-07 16:11:35 -05001022An example BitBake function is::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001023
1024 python some_python_function () {
1025 d.setVar("TEXT", "Hello World")
1026 print d.getVar("TEXT")
1027 }
1028
1029Because the
1030Python "bb" and "os" modules are already imported, you do not need to
1031import these modules. Also in these types of functions, the datastore
1032("d") is a global variable and is always automatically available.
1033
1034.. note::
1035
1036 Variable expressions (e.g. ``${X}`` ) are no longer expanded within Python
1037 functions. This behavior is intentional in order to allow you to freely set
1038 variable values to expandable expressions without having them expanded
1039 prematurely. If you do wish to expand a variable within a Python function,
1040 use ``d.getVar("X")`` . Or, for more complicated expressions, use ``d.expand()``.
1041
1042Similar to shell functions, you can also apply overrides and
1043override-style operators to BitBake-style Python functions.
1044
Andrew Geisslerc926e172021-05-07 16:11:35 -05001045As an example, consider the following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001046
Patrick Williams213cb262021-08-07 19:21:33 -05001047 python do_foo:prepend() {
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001048 bb.plain("first")
1049 }
1050
1051 python do_foo() {
1052 bb.plain("second")
1053 }
1054
Patrick Williams213cb262021-08-07 19:21:33 -05001055 python do_foo:append() {
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001056 bb.plain("third")
1057 }
1058
Andrew Geisslerc926e172021-05-07 16:11:35 -05001059Running ``do_foo`` prints the following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001060
1061 recipename do_foo: first
1062 recipename do_foo: second
1063 recipename do_foo: third
1064
Andrew Geissler595f6302022-01-24 19:11:47 +00001065You can use the ``bitbake -e recipename`` command to view
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001066the final assembled function after all overrides have been applied.
1067
1068Python Functions
1069----------------
1070
1071These functions are written in Python and are executed by other Python
1072code. Examples of Python functions are utility functions that you intend
1073to call from in-line Python or from within other Python functions. Here
Andrew Geisslerc926e172021-05-07 16:11:35 -05001074is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001075
1076 def get_depends(d):
1077 if d.getVar('SOMECONDITION'):
1078 return "dependencywithcond"
1079 else:
1080 return "dependency"
1081
1082 SOMECONDITION = "1"
1083 DEPENDS = "${@get_depends(d)}"
1084
Patrick Williams213cb262021-08-07 19:21:33 -05001085This would result in :term:`DEPENDS` containing ``dependencywithcond``.
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001086
1087Here are some things to know about Python functions:
1088
1089- Python functions can take parameters.
1090
1091- The BitBake datastore is not automatically available. Consequently,
1092 you must pass it in as a parameter to the function.
1093
1094- The "bb" and "os" Python modules are automatically available. You do
1095 not need to import them.
1096
1097BitBake-Style Python Functions Versus Python Functions
1098------------------------------------------------------
1099
1100Following are some important differences between BitBake-style Python
1101functions and regular Python functions defined with "def":
1102
1103- Only BitBake-style Python functions can be :ref:`tasks <bitbake-user-manual/bitbake-user-manual-metadata:tasks>`.
1104
1105- Overrides and override-style operators can only be applied to
1106 BitBake-style Python functions.
1107
1108- Only regular Python functions can take arguments and return values.
1109
1110- :ref:`Variable flags <bitbake-user-manual/bitbake-user-manual-metadata:variable flags>` such as
1111 ``[dirs]``, ``[cleandirs]``, and ``[lockfiles]`` can be used on BitBake-style
1112 Python functions, but not on regular Python functions.
1113
1114- BitBake-style Python functions generate a separate
1115 ``${``\ :term:`T`\ ``}/run.``\ function-name\ ``.``\ pid
1116 script that is executed to run the function, and also generate a log
1117 file in ``${T}/log.``\ function-name\ ``.``\ pid if they are executed
1118 as tasks.
1119
1120 Regular Python functions execute "inline" and do not generate any
1121 files in ``${T}``.
1122
1123- Regular Python functions are called with the usual Python syntax.
1124 BitBake-style Python functions are usually tasks and are called
1125 directly by BitBake, but can also be called manually from Python code
Andrew Geisslerc926e172021-05-07 16:11:35 -05001126 by using the ``bb.build.exec_func()`` function. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001127
1128 bb.build.exec_func("my_bitbake_style_function", d)
1129
1130 .. note::
1131
1132 ``bb.build.exec_func()`` can also be used to run shell functions from Python
1133 code. If you want to run a shell function before a Python function within
1134 the same task, then you can use a parent helper Python function that
1135 starts by running the shell function with ``bb.build.exec_func()`` and then
1136 runs the Python code.
1137
1138 To detect errors from functions executed with
1139 ``bb.build.exec_func()``, you can catch the ``bb.build.FuncFailed``
1140 exception.
1141
1142 .. note::
1143
1144 Functions in metadata (recipes and classes) should not themselves raise
1145 ``bb.build.FuncFailed``. Rather, ``bb.build.FuncFailed`` should be viewed as a
1146 general indicator that the called function failed by raising an
1147 exception. For example, an exception raised by ``bb.fatal()`` will be caught
1148 inside ``bb.build.exec_func()``, and a ``bb.build.FuncFailed`` will be raised in
1149 response.
1150
1151Due to their simplicity, you should prefer regular Python functions over
1152BitBake-style Python functions unless you need a feature specific to
1153BitBake-style Python functions. Regular Python functions in metadata are
1154a more recent invention than BitBake-style Python functions, and older
1155code tends to use ``bb.build.exec_func()`` more often.
1156
1157Anonymous Python Functions
1158--------------------------
1159
1160Sometimes it is useful to set variables or perform other operations
1161programmatically during parsing. To do this, you can define special
1162Python functions, called anonymous Python functions, that run at the end
1163of parsing. For example, the following conditionally sets a variable
Andrew Geisslerc926e172021-05-07 16:11:35 -05001164based on the value of another variable::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001165
1166 python () {
1167 if d.getVar('SOMEVAR') == 'value':
1168 d.setVar('ANOTHERVAR', 'value2')
1169 }
1170
1171An equivalent way to mark a function as an anonymous function is to give it
1172the name "__anonymous", rather than no name.
1173
1174Anonymous Python functions always run at the end of parsing, regardless
1175of where they are defined. If a recipe contains many anonymous
1176functions, they run in the same order as they are defined within the
Andrew Geisslerc926e172021-05-07 16:11:35 -05001177recipe. As an example, consider the following snippet::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001178
1179 python () {
1180 d.setVar('FOO', 'foo 2')
1181 }
1182
1183 FOO = "foo 1"
1184
1185 python () {
1186 d.appendVar('BAR',' bar 2')
1187 }
1188
1189 BAR = "bar 1"
1190
1191The previous example is conceptually
Andrew Geisslerc926e172021-05-07 16:11:35 -05001192equivalent to the following snippet::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001193
1194 FOO = "foo 1"
1195 BAR = "bar 1"
1196 FOO = "foo 2"
1197 BAR += "bar 2"
1198
1199``FOO`` ends up with the value "foo 2", and
1200``BAR`` with the value "bar 1 bar 2". Just as in the second snippet, the
1201values set for the variables within the anonymous functions become
1202available to tasks, which always run after parsing.
1203
Patrick Williams213cb262021-08-07 19:21:33 -05001204Overrides and override-style operators such as "``:append``" are applied
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001205before anonymous functions run. In the following example, ``FOO`` ends
Andrew Geisslerc926e172021-05-07 16:11:35 -05001206up with the value "foo from anonymous"::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001207
1208 FOO = "foo"
Patrick Williams213cb262021-08-07 19:21:33 -05001209 FOO:append = " from outside"
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001210
1211 python () {
1212 d.setVar("FOO", "foo from anonymous")
1213 }
1214
1215For methods
1216you can use with anonymous Python functions, see the
1217":ref:`bitbake-user-manual/bitbake-user-manual-metadata:functions you can call from within python`"
1218section. For a different method to run Python code during parsing, see
1219the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:inline python variable expansion`" section.
1220
1221Flexible Inheritance for Class Functions
1222----------------------------------------
1223
1224Through coding techniques and the use of ``EXPORT_FUNCTIONS``, BitBake
1225supports exporting a function from a class such that the class function
1226appears as the default implementation of the function, but can still be
1227called if a recipe inheriting the class needs to define its own version
1228of the function.
1229
1230To understand the benefits of this feature, consider the basic scenario
1231where a class defines a task function and your recipe inherits the
1232class. In this basic scenario, your recipe inherits the task function as
1233defined in the class. If desired, your recipe can add to the start and
Patrick Williams213cb262021-08-07 19:21:33 -05001234end of the function by using the ":prepend" or ":append" operations
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001235respectively, or it can redefine the function completely. However, if it
1236redefines the function, there is no means for it to call the class
1237version of the function. ``EXPORT_FUNCTIONS`` provides a mechanism that
1238enables the recipe's version of the function to call the original
1239version of the function.
1240
1241To make use of this technique, you need the following things in place:
1242
Andrew Geisslerc926e172021-05-07 16:11:35 -05001243- The class needs to define the function as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001244
1245 classname_functionname
1246
1247 For example, if you have a class file
1248 ``bar.bbclass`` and a function named ``do_foo``, the class must
Andrew Geisslerc926e172021-05-07 16:11:35 -05001249 define the function as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001250
1251 bar_do_foo
1252
1253- The class needs to contain the ``EXPORT_FUNCTIONS`` statement as
Andrew Geisslerc926e172021-05-07 16:11:35 -05001254 follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001255
1256 EXPORT_FUNCTIONS functionname
1257
1258 For example, continuing with
1259 the same example, the statement in the ``bar.bbclass`` would be as
Andrew Geisslerc926e172021-05-07 16:11:35 -05001260 follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001261
1262 EXPORT_FUNCTIONS do_foo
1263
1264- You need to call the function appropriately from within your recipe.
1265 Continuing with the same example, if your recipe needs to call the
1266 class version of the function, it should call ``bar_do_foo``.
1267 Assuming ``do_foo`` was a shell function and ``EXPORT_FUNCTIONS`` was
1268 used as above, the recipe's function could conditionally call the
Andrew Geisslerc926e172021-05-07 16:11:35 -05001269 class version of the function as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001270
1271 do_foo() {
1272 if [ somecondition ] ; then
1273 bar_do_foo
1274 else
1275 # Do something else
1276 fi
1277 }
1278
1279 To call your modified version of the function as defined in your recipe,
1280 call it as ``do_foo``.
1281
1282With these conditions met, your single recipe can freely choose between
1283the original function as defined in the class file and the modified
1284function in your recipe. If you do not set up these conditions, you are
1285limited to using one function or the other.
1286
1287Tasks
1288=====
1289
1290Tasks are BitBake execution units that make up the steps that BitBake
1291can run for a given recipe. Tasks are only supported in recipes and
1292classes (i.e. in ``.bb`` files and files included or inherited from
1293``.bb`` files). By convention, tasks have names that start with "do\_".
1294
1295Promoting a Function to a Task
1296------------------------------
1297
1298Tasks are either :ref:`shell functions <bitbake-user-manual/bitbake-user-manual-metadata:shell functions>` or
1299:ref:`BitBake-style Python functions <bitbake-user-manual/bitbake-user-manual-metadata:bitbake-style python functions>`
1300that have been promoted to tasks by using the ``addtask`` command. The
1301``addtask`` command can also optionally describe dependencies between
1302the task and other tasks. Here is an example that shows how to define a
Andrew Geisslerc926e172021-05-07 16:11:35 -05001303task and declare some dependencies::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001304
1305 python do_printdate () {
1306 import time
1307 print time.strftime('%Y%m%d', time.gmtime())
1308 }
1309 addtask printdate after do_fetch before do_build
1310
1311The first argument to ``addtask`` is the name
1312of the function to promote to a task. If the name does not start with
1313"do\_", "do\_" is implicitly added, which enforces the convention that all
1314task names start with "do\_".
1315
1316In the previous example, the ``do_printdate`` task becomes a dependency
1317of the ``do_build`` task, which is the default task (i.e. the task run
1318by the ``bitbake`` command unless another task is specified explicitly).
1319Additionally, the ``do_printdate`` task becomes dependent upon the
1320``do_fetch`` task. Running the ``do_build`` task results in the
1321``do_printdate`` task running first.
1322
1323.. note::
1324
1325 If you try out the previous example, you might see that the
1326 ``do_printdate``
1327 task is only run the first time you build the recipe with the
1328 ``bitbake``
1329 command. This is because BitBake considers the task "up-to-date"
1330 after that initial run. If you want to force the task to always be
1331 rerun for experimentation purposes, you can make BitBake always
1332 consider the task "out-of-date" by using the
1333 :ref:`[nostamp] <bitbake-user-manual/bitbake-user-manual-metadata:Variable Flags>`
Andrew Geisslerc926e172021-05-07 16:11:35 -05001334 variable flag, as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001335
1336 do_printdate[nostamp] = "1"
1337
1338 You can also explicitly run the task and provide the
Andrew Geisslerc926e172021-05-07 16:11:35 -05001339 -f option as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001340
1341 $ bitbake recipe -c printdate -f
1342
1343 When manually selecting a task to run with the bitbake ``recipe
1344 -c task`` command, you can omit the "do\_" prefix as part of the task
1345 name.
1346
1347You might wonder about the practical effects of using ``addtask``
Andrew Geisslerc926e172021-05-07 16:11:35 -05001348without specifying any dependencies as is done in the following example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001349
1350 addtask printdate
1351
1352In this example, assuming dependencies have not been
1353added through some other means, the only way to run the task is by
1354explicitly selecting it with ``bitbake`` recipe ``-c printdate``. You
1355can use the ``do_listtasks`` task to list all tasks defined in a recipe
Andrew Geisslerc926e172021-05-07 16:11:35 -05001356as shown in the following example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001357
1358 $ bitbake recipe -c listtasks
1359
1360For more information on task dependencies, see the
1361":ref:`bitbake-user-manual/bitbake-user-manual-execution:dependencies`" section.
1362
1363See the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:variable flags`" section for information
1364on variable flags you can use with tasks.
1365
Andrew Geissler95ac1b82021-03-31 14:34:31 -05001366.. note::
1367
1368 While it's infrequent, it's possible to define multiple tasks as
1369 dependencies when calling ``addtask``. For example, here's a snippet
1370 from the OpenEmbedded class file ``package_tar.bbclass``::
1371
1372 addtask package_write_tar before do_build after do_packagedata do_package
1373
1374 Note how the ``package_write_tar`` task has to wait until both of
1375 ``do_packagedata`` and ``do_package`` complete.
1376
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001377Deleting a Task
1378---------------
1379
1380As well as being able to add tasks, you can delete them. Simply use the
1381``deltask`` command to delete a task. For example, to delete the example
Andrew Geisslerc926e172021-05-07 16:11:35 -05001382task used in the previous sections, you would use::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001383
1384 deltask printdate
1385
1386If you delete a task using the ``deltask`` command and the task has
1387dependencies, the dependencies are not reconnected. For example, suppose
1388you have three tasks named ``do_a``, ``do_b``, and ``do_c``.
1389Furthermore, ``do_c`` is dependent on ``do_b``, which in turn is
1390dependent on ``do_a``. Given this scenario, if you use ``deltask`` to
1391delete ``do_b``, the implicit dependency relationship between ``do_c``
1392and ``do_a`` through ``do_b`` no longer exists, and ``do_c``
1393dependencies are not updated to include ``do_a``. Thus, ``do_c`` is free
1394to run before ``do_a``.
1395
1396If you want dependencies such as these to remain intact, use the
1397``[noexec]`` varflag to disable the task instead of using the
Andrew Geisslerc926e172021-05-07 16:11:35 -05001398``deltask`` command to delete it::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001399
1400 do_b[noexec] = "1"
1401
1402Passing Information Into the Build Task Environment
1403---------------------------------------------------
1404
1405When running a task, BitBake tightly controls the shell execution
1406environment of the build tasks to make sure unwanted contamination from
1407the build machine cannot influence the build.
1408
1409.. note::
1410
1411 By default, BitBake cleans the environment to include only those
Andrew Geissler7e0e3c02022-02-25 20:34:39 +00001412 things exported or listed in its passthrough list to ensure that the
1413 build environment is reproducible and consistent. You can prevent this
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001414 "cleaning" by setting the :term:`BB_PRESERVE_ENV` variable.
1415
1416Consequently, if you do want something to get passed into the build task
1417environment, you must take these two steps:
1418
1419#. Tell BitBake to load what you want from the environment into the
1420 datastore. You can do so through the
Andrew Geissler7e0e3c02022-02-25 20:34:39 +00001421 :term:`BB_ENV_PASSTHROUGH` and
1422 :term:`BB_ENV_PASSTHROUGH_ADDITIONS` variables. For
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001423 example, assume you want to prevent the build system from accessing
Andrew Geissler7e0e3c02022-02-25 20:34:39 +00001424 your ``$HOME/.ccache`` directory. The following command adds the
1425 the environment variable ``CCACHE_DIR`` to BitBake's passthrough
1426 list to allow that variable into the datastore::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001427
Andrew Geissler7e0e3c02022-02-25 20:34:39 +00001428 export BB_ENV_PASSTHROUGH_ADDITIONS="$BB_ENV_PASSTHROUGH_ADDITIONS CCACHE_DIR"
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001429
1430#. Tell BitBake to export what you have loaded into the datastore to the
1431 task environment of every running task. Loading something from the
1432 environment into the datastore (previous step) only makes it
1433 available in the datastore. To export it to the task environment of
1434 every running task, use a command similar to the following in your
1435 local configuration file ``local.conf`` or your distribution
Andrew Geisslerc926e172021-05-07 16:11:35 -05001436 configuration file::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001437
1438 export CCACHE_DIR
1439
1440 .. note::
1441
1442 A side effect of the previous steps is that BitBake records the
1443 variable as a dependency of the build process in things like the
1444 setscene checksums. If doing so results in unnecessary rebuilds of
Andrew Geissler7e0e3c02022-02-25 20:34:39 +00001445 tasks, you can also flag the variable so that the setscene code
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001446 ignores the dependency when it creates checksums.
1447
1448Sometimes, it is useful to be able to obtain information from the
1449original execution environment. BitBake saves a copy of the original
1450environment into a special variable named :term:`BB_ORIGENV`.
1451
Patrick Williams213cb262021-08-07 19:21:33 -05001452The :term:`BB_ORIGENV` variable returns a datastore object that can be
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001453queried using the standard datastore operators such as
1454``getVar(, False)``. The datastore object is useful, for example, to
Andrew Geisslerc926e172021-05-07 16:11:35 -05001455find the original ``DISPLAY`` variable. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001456
1457 origenv = d.getVar("BB_ORIGENV", False)
1458 bar = origenv.getVar("BAR", False)
1459
1460The previous example returns ``BAR`` from the original execution
1461environment.
1462
1463Variable Flags
1464==============
1465
1466Variable flags (varflags) help control a task's functionality and
1467dependencies. BitBake reads and writes varflags to the datastore using
Andrew Geisslerc926e172021-05-07 16:11:35 -05001468the following command forms::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001469
1470 variable = d.getVarFlags("variable")
1471 self.d.setVarFlags("FOO", {"func": True})
1472
1473When working with varflags, the same syntax, with the exception of
1474overrides, applies. In other words, you can set, append, and prepend
1475varflags just like variables. See the
1476":ref:`bitbake-user-manual/bitbake-user-manual-metadata:variable flag syntax`" section for details.
1477
1478BitBake has a defined set of varflags available for recipes and classes.
1479Tasks support a number of these flags which control various
1480functionality of the task:
1481
1482- ``[cleandirs]``: Empty directories that should be created before
1483 the task runs. Directories that already exist are removed and
1484 recreated to empty them.
1485
1486- ``[depends]``: Controls inter-task dependencies. See the
1487 :term:`DEPENDS` variable and the
1488 ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:inter-task
1489 dependencies`" section for more information.
1490
1491- ``[deptask]``: Controls task build-time dependencies. See the
1492 :term:`DEPENDS` variable and the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:build dependencies`" section for more information.
1493
1494- ``[dirs]``: Directories that should be created before the task
1495 runs. Directories that already exist are left as is. The last
1496 directory listed is used as the current working directory for the
1497 task.
1498
1499- ``[lockfiles]``: Specifies one or more lockfiles to lock while the
1500 task executes. Only one task may hold a lockfile, and any task that
1501 attempts to lock an already locked file will block until the lock is
1502 released. You can use this variable flag to accomplish mutual
1503 exclusion.
1504
1505- ``[noexec]``: When set to "1", marks the task as being empty, with
1506 no execution required. You can use the ``[noexec]`` flag to set up
1507 tasks as dependency placeholders, or to disable tasks defined
1508 elsewhere that are not needed in a particular recipe.
1509
1510- ``[nostamp]``: When set to "1", tells BitBake to not generate a
1511 stamp file for a task, which implies the task should always be
1512 executed.
1513
1514 .. caution::
1515
1516 Any task that depends (possibly indirectly) on a ``[nostamp]`` task will
1517 always be executed as well. This can cause unnecessary rebuilding if you
1518 are not careful.
1519
1520- ``[number_threads]``: Limits tasks to a specific number of
1521 simultaneous threads during execution. This varflag is useful when
1522 your build host has a large number of cores but certain tasks need to
1523 be rate-limited due to various kinds of resource constraints (e.g. to
1524 avoid network throttling). ``number_threads`` works similarly to the
1525 :term:`BB_NUMBER_THREADS` variable but is task-specific.
1526
1527 Set the value globally. For example, the following makes sure the
1528 ``do_fetch`` task uses no more than two simultaneous execution
1529 threads: do_fetch[number_threads] = "2"
1530
1531 .. warning::
1532
1533 - Setting the varflag in individual recipes rather than globally
1534 can result in unpredictable behavior.
1535
1536 - Setting the varflag to a value greater than the value used in
Patrick Williams213cb262021-08-07 19:21:33 -05001537 the :term:`BB_NUMBER_THREADS` variable causes ``number_threads`` to
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001538 have no effect.
1539
1540- ``[postfuncs]``: List of functions to call after the completion of
1541 the task.
1542
1543- ``[prefuncs]``: List of functions to call before the task executes.
1544
1545- ``[rdepends]``: Controls inter-task runtime dependencies. See the
1546 :term:`RDEPENDS` variable, the
1547 :term:`RRECOMMENDS` variable, and the
1548 ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:inter-task dependencies`" section for
1549 more information.
1550
1551- ``[rdeptask]``: Controls task runtime dependencies. See the
1552 :term:`RDEPENDS` variable, the
1553 :term:`RRECOMMENDS` variable, and the
1554 ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:runtime dependencies`" section for more
1555 information.
1556
1557- ``[recideptask]``: When set in conjunction with ``recrdeptask``,
1558 specifies a task that should be inspected for additional
1559 dependencies.
1560
1561- ``[recrdeptask]``: Controls task recursive runtime dependencies.
1562 See the :term:`RDEPENDS` variable, the
1563 :term:`RRECOMMENDS` variable, and the
1564 ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:recursive dependencies`" section for
1565 more information.
1566
1567- ``[stamp-extra-info]``: Extra stamp information to append to the
1568 task's stamp. As an example, OpenEmbedded uses this flag to allow
1569 machine-specific tasks.
1570
1571- ``[umask]``: The umask to run the task under.
1572
1573Several varflags are useful for controlling how signatures are
1574calculated for variables. For more information on this process, see the
1575":ref:`bitbake-user-manual/bitbake-user-manual-execution:checksums (signatures)`" section.
1576
1577- ``[vardeps]``: Specifies a space-separated list of additional
1578 variables to add to a variable's dependencies for the purposes of
1579 calculating its signature. Adding variables to this list is useful,
1580 for example, when a function refers to a variable in a manner that
1581 does not allow BitBake to automatically determine that the variable
1582 is referred to.
1583
1584- ``[vardepsexclude]``: Specifies a space-separated list of variables
1585 that should be excluded from a variable's dependencies for the
1586 purposes of calculating its signature.
1587
1588- ``[vardepvalue]``: If set, instructs BitBake to ignore the actual
1589 value of the variable and instead use the specified value when
1590 calculating the variable's signature.
1591
1592- ``[vardepvalueexclude]``: Specifies a pipe-separated list of
1593 strings to exclude from the variable's value when calculating the
1594 variable's signature.
1595
1596Events
1597======
1598
1599BitBake allows installation of event handlers within recipe and class
1600files. Events are triggered at certain points during operation, such as
1601the beginning of operation against a given recipe (i.e. ``*.bb``), the
1602start of a given task, a task failure, a task success, and so forth. The
1603intent is to make it easy to do things like email notification on build
1604failures.
1605
1606Following is an example event handler that prints the name of the event
Patrick Williams213cb262021-08-07 19:21:33 -05001607and the content of the :term:`FILE` variable::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001608
1609 addhandler myclass_eventhandler
1610 python myclass_eventhandler() {
1611 from bb.event import getName
1612 print("The name of the Event is %s" % getName(e))
1613 print("The file we run for is %s" % d.getVar('FILE'))
1614 }
1615 myclass_eventhandler[eventmask] = "bb.event.BuildStarted
1616 bb.event.BuildCompleted"
1617
1618In the previous example, an eventmask has been
1619set so that the handler only sees the "BuildStarted" and
1620"BuildCompleted" events. This event handler gets called every time an
1621event matching the eventmask is triggered. A global variable "e" is
1622defined, which represents the current event. With the ``getName(e)``
1623method, you can get the name of the triggered event. The global
1624datastore is available as "d". In legacy code, you might see "e.data"
1625used to get the datastore. However, realize that "e.data" is deprecated
1626and you should use "d" going forward.
1627
1628The context of the datastore is appropriate to the event in question.
1629For example, "BuildStarted" and "BuildCompleted" events run before any
1630tasks are executed so would be in the global configuration datastore
1631namespace. No recipe-specific metadata exists in that namespace. The
1632"BuildStarted" and "BuildCompleted" events also run in the main
1633cooker/server process rather than any worker context. Thus, any changes
1634made to the datastore would be seen by other cooker/server events within
1635the current build but not seen outside of that build or in any worker
1636context. Task events run in the actual tasks in question consequently
1637have recipe-specific and task-specific contents. These events run in the
1638worker context and are discarded at the end of task execution.
1639
1640During a standard build, the following common events might occur. The
1641following events are the most common kinds of events that most metadata
1642might have an interest in viewing:
1643
1644- ``bb.event.ConfigParsed()``: Fired when the base configuration; which
1645 consists of ``bitbake.conf``, ``base.bbclass`` and any global
Patrick Williams213cb262021-08-07 19:21:33 -05001646 :term:`INHERIT` statements; has been parsed. You can see multiple such
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001647 events when each of the workers parse the base configuration or if
1648 the server changes configuration and reparses. Any given datastore
1649 only has one such event executed against it, however. If
Andrew Geissler95ac1b82021-03-31 14:34:31 -05001650 :term:`BB_INVALIDCONF` is set in the datastore by the event
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001651 handler, the configuration is reparsed and a new event triggered,
1652 allowing the metadata to update configuration.
1653
1654- ``bb.event.HeartbeatEvent()``: Fires at regular time intervals of one
1655 second. You can configure the interval time using the
1656 ``BB_HEARTBEAT_EVENT`` variable. The event's "time" attribute is the
1657 ``time.time()`` value when the event is triggered. This event is
1658 useful for activities such as system state monitoring.
1659
1660- ``bb.event.ParseStarted()``: Fired when BitBake is about to start
1661 parsing recipes. This event's "total" attribute represents the number
1662 of recipes BitBake plans to parse.
1663
1664- ``bb.event.ParseProgress()``: Fired as parsing progresses. This
1665 event's "current" attribute is the number of recipes parsed as well
1666 as the "total" attribute.
1667
1668- ``bb.event.ParseCompleted()``: Fired when parsing is complete. This
1669 event's "cached", "parsed", "skipped", "virtuals", "masked", and
1670 "errors" attributes provide statistics for the parsing results.
1671
1672- ``bb.event.BuildStarted()``: Fired when a new build starts. BitBake
1673 fires multiple "BuildStarted" events (one per configuration) when
1674 multiple configuration (multiconfig) is enabled.
1675
1676- ``bb.build.TaskStarted()``: Fired when a task starts. This event's
1677 "taskfile" attribute points to the recipe from which the task
1678 originates. The "taskname" attribute, which is the task's name,
1679 includes the ``do_`` prefix, and the "logfile" attribute point to
1680 where the task's output is stored. Finally, the "time" attribute is
1681 the task's execution start time.
1682
1683- ``bb.build.TaskInvalid()``: Fired if BitBake tries to execute a task
1684 that does not exist.
1685
1686- ``bb.build.TaskFailedSilent()``: Fired for setscene tasks that fail
1687 and should not be presented to the user verbosely.
1688
1689- ``bb.build.TaskFailed()``: Fired for normal tasks that fail.
1690
1691- ``bb.build.TaskSucceeded()``: Fired when a task successfully
1692 completes.
1693
1694- ``bb.event.BuildCompleted()``: Fired when a build finishes.
1695
1696- ``bb.cooker.CookerExit()``: Fired when the BitBake server/cooker
1697 shuts down. This event is usually only seen by the UIs as a sign they
1698 should also shutdown.
1699
1700This next list of example events occur based on specific requests to the
1701server. These events are often used to communicate larger pieces of
1702information from the BitBake server to other parts of BitBake such as
1703user interfaces:
1704
1705- ``bb.event.TreeDataPreparationStarted()``
1706- ``bb.event.TreeDataPreparationProgress()``
1707- ``bb.event.TreeDataPreparationCompleted()``
1708- ``bb.event.DepTreeGenerated()``
1709- ``bb.event.CoreBaseFilesFound()``
1710- ``bb.event.ConfigFilePathFound()``
1711- ``bb.event.FilesMatchingFound()``
1712- ``bb.event.ConfigFilesFound()``
1713- ``bb.event.TargetsTreeGenerated()``
1714
1715.. _variants-class-extension-mechanism:
1716
Andrew Geissler9aee5002022-03-30 16:27:02 +00001717Variants --- Class Extension Mechanism
1718======================================
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001719
Andrew Geissler09036742021-06-25 14:25:14 -05001720BitBake supports multiple incarnations of a recipe file via the
1721:term:`BBCLASSEXTEND` variable.
1722
1723The :term:`BBCLASSEXTEND` variable is a space separated list of classes used
1724to "extend" the recipe for each variant. Here is an example that results in a
1725second incarnation of the current recipe being available. This second
1726incarnation will have the "native" class inherited. ::
1727
1728 BBCLASSEXTEND = "native"
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001729
1730.. note::
1731
1732 The mechanism for this class extension is extremely specific to the
1733 implementation. Usually, the recipe's :term:`PROVIDES` , :term:`PN` , and
1734 :term:`DEPENDS` variables would need to be modified by the extension
1735 class. For specific examples, see the OE-Core native , nativesdk , and
1736 multilib classes.
1737
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001738Dependencies
1739============
1740
1741To allow for efficient parallel processing, BitBake handles dependencies
1742at the task level. Dependencies can exist both between tasks within a
1743single recipe and between tasks in different recipes. Following are
1744examples of each:
1745
1746- For tasks within a single recipe, a recipe's ``do_configure`` task
1747 might need to complete before its ``do_compile`` task can run.
1748
1749- For tasks in different recipes, one recipe's ``do_configure`` task
1750 might require another recipe's ``do_populate_sysroot`` task to finish
1751 first such that the libraries and headers provided by the other
1752 recipe are available.
1753
1754This section describes several ways to declare dependencies. Remember,
1755even though dependencies are declared in different ways, they are all
1756simply dependencies between tasks.
1757
1758.. _dependencies-internal-to-the-bb-file:
1759
1760Dependencies Internal to the ``.bb`` File
1761-----------------------------------------
1762
1763BitBake uses the ``addtask`` directive to manage dependencies that are
1764internal to a given recipe file. You can use the ``addtask`` directive
1765to indicate when a task is dependent on other tasks or when other tasks
Andrew Geisslerc926e172021-05-07 16:11:35 -05001766depend on that recipe. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001767
1768 addtask printdate after do_fetch before do_build
1769
1770In this example, the ``do_printdate`` task
1771depends on the completion of the ``do_fetch`` task, and the ``do_build``
1772task depends on the completion of the ``do_printdate`` task.
1773
1774.. note::
1775
1776 For a task to run, it must be a direct or indirect dependency of some
1777 other task that is scheduled to run.
1778
1779 For illustration, here are some examples:
1780
1781 - The directive ``addtask mytask before do_configure`` causes
1782 ``do_mytask`` to run before ``do_configure`` runs. Be aware that
1783 ``do_mytask`` still only runs if its :ref:`input
1784 checksum <bitbake-user-manual/bitbake-user-manual-execution:checksums (signatures)>` has changed since the last time it was
1785 run. Changes to the input checksum of ``do_mytask`` also
1786 indirectly cause ``do_configure`` to run.
1787
1788 - The directive ``addtask mytask after do_configure`` by itself
1789 never causes ``do_mytask`` to run. ``do_mytask`` can still be run
Andrew Geisslerc926e172021-05-07 16:11:35 -05001790 manually as follows::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001791
1792 $ bitbake recipe -c mytask
1793
1794 Declaring ``do_mytask`` as a dependency of some other task that is
1795 scheduled to run also causes it to run. Regardless, the task runs after
1796 ``do_configure``.
1797
1798Build Dependencies
1799------------------
1800
1801BitBake uses the :term:`DEPENDS` variable to manage
1802build time dependencies. The ``[deptask]`` varflag for tasks signifies
Patrick Williams213cb262021-08-07 19:21:33 -05001803the task of each item listed in :term:`DEPENDS` that must complete before
Andrew Geisslerc926e172021-05-07 16:11:35 -05001804that task can be executed. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001805
1806 do_configure[deptask] = "do_populate_sysroot"
1807
1808In this example, the ``do_populate_sysroot`` task
Patrick Williams213cb262021-08-07 19:21:33 -05001809of each item in :term:`DEPENDS` must complete before ``do_configure`` can
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001810execute.
1811
1812Runtime Dependencies
1813--------------------
1814
1815BitBake uses the :term:`PACKAGES`, :term:`RDEPENDS`, and :term:`RRECOMMENDS`
1816variables to manage runtime dependencies.
1817
Patrick Williams213cb262021-08-07 19:21:33 -05001818The :term:`PACKAGES` variable lists runtime packages. Each of those packages
1819can have :term:`RDEPENDS` and :term:`RRECOMMENDS` runtime dependencies. The
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001820``[rdeptask]`` flag for tasks is used to signify the task of each item
1821runtime dependency which must have completed before that task can be
1822executed. ::
1823
1824 do_package_qa[rdeptask] = "do_packagedata"
1825
1826In the previous
Patrick Williams213cb262021-08-07 19:21:33 -05001827example, the ``do_packagedata`` task of each item in :term:`RDEPENDS` must
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001828have completed before ``do_package_qa`` can execute.
Patrick Williams213cb262021-08-07 19:21:33 -05001829Although :term:`RDEPENDS` contains entries from the
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001830runtime dependency namespace, BitBake knows how to map them back
1831to the build-time dependency namespace, in which the tasks are defined.
1832
1833Recursive Dependencies
1834----------------------
1835
1836BitBake uses the ``[recrdeptask]`` flag to manage recursive task
1837dependencies. BitBake looks through the build-time and runtime
1838dependencies of the current recipe, looks through the task's inter-task
1839dependencies, and then adds dependencies for the listed task. Once
1840BitBake has accomplished this, it recursively works through the
1841dependencies of those tasks. Iterative passes continue until all
1842dependencies are discovered and added.
1843
1844The ``[recrdeptask]`` flag is most commonly used in high-level recipes
1845that need to wait for some task to finish "globally". For example,
Andrew Geisslerc926e172021-05-07 16:11:35 -05001846``image.bbclass`` has the following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001847
1848 do_rootfs[recrdeptask] += "do_packagedata"
1849
1850This statement says that the ``do_packagedata`` task of
1851the current recipe and all recipes reachable (by way of dependencies)
1852from the image recipe must run before the ``do_rootfs`` task can run.
1853
1854BitBake allows a task to recursively depend on itself by
Andrew Geisslerc926e172021-05-07 16:11:35 -05001855referencing itself in the task list::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001856
1857 do_a[recrdeptask] = "do_a do_b"
1858
1859In the same way as before, this means that the ``do_a``
1860and ``do_b`` tasks of the current recipe and all
1861recipes reachable (by way of dependencies) from the recipe
1862must run before the ``do_a`` task can run. In this
1863case BitBake will ignore the current recipe's ``do_a``
1864task circular dependency on itself.
1865
1866Inter-Task Dependencies
1867-----------------------
1868
1869BitBake uses the ``[depends]`` flag in a more generic form to manage
1870inter-task dependencies. This more generic form allows for
1871inter-dependency checks for specific tasks rather than checks for the
Patrick Williams213cb262021-08-07 19:21:33 -05001872data in :term:`DEPENDS`. Here is an example::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001873
1874 do_patch[depends] = "quilt-native:do_populate_sysroot"
1875
1876In this example, the ``do_populate_sysroot`` task of the target ``quilt-native``
1877must have completed before the ``do_patch`` task can execute.
1878
1879The ``[rdepends]`` flag works in a similar way but takes targets in the
1880runtime namespace instead of the build-time dependency namespace.
1881
1882Functions You Can Call From Within Python
1883=========================================
1884
1885BitBake provides many functions you can call from within Python
1886functions. This section lists the most commonly used functions, and
1887mentions where to find others.
1888
1889Functions for Accessing Datastore Variables
1890-------------------------------------------
1891
1892It is often necessary to access variables in the BitBake datastore using
1893Python functions. The BitBake datastore has an API that allows you this
1894access. Here is a list of available operations:
1895
1896.. list-table::
1897 :widths: auto
1898 :header-rows: 1
1899
1900 * - *Operation*
1901 - *Description*
1902 * - ``d.getVar("X", expand)``
1903 - Returns the value of variable "X". Using "expand=True" expands the
1904 value. Returns "None" if the variable "X" does not exist.
1905 * - ``d.setVar("X", "value")``
1906 - Sets the variable "X" to "value"
1907 * - ``d.appendVar("X", "value")``
1908 - Adds "value" to the end of the variable "X". Acts like ``d.setVar("X",
1909 "value")`` if the variable "X" does not exist.
1910 * - ``d.prependVar("X", "value")``
1911 - Adds "value" to the start of the variable "X". Acts like
1912 ``d.setVar("X","value")`` if the variable "X" does not exist.
1913 * - ``d.delVar("X")``
1914 - Deletes the variable "X" from the datastore. Does nothing if the variable
1915 "X" does not exist.
1916 * - ``d.renameVar("X", "Y")``
1917 - Renames the variable "X" to "Y". Does nothing if the variable "X" does
1918 not exist.
1919 * - ``d.getVarFlag("X", flag, expand)``
1920 - Returns the value of variable "X". Using "expand=True" expands the
1921 value. Returns "None" if either the variable "X" or the named flag does
1922 not exist.
1923 * - ``d.setVarFlag("X", flag, "value")``
1924 - Sets the named flag for variable "X" to "value".
1925 * - ``d.appendVarFlag("X", flag, "value")``
1926 - Appends "value" to the named flag on the variable "X". Acts like
1927 ``d.setVarFlag("X", flag, "value")`` if the named flag does not exist.
1928 * - ``d.prependVarFlag("X", flag, "value")``
1929 - Prepends "value" to the named flag on the variable "X". Acts like
1930 ``d.setVarFlag("X", flag, "value")`` if the named flag does not exist.
1931 * - ``d.delVarFlag("X", flag)``
1932 - Deletes the named flag on the variable "X" from the datastore.
1933 * - ``d.setVarFlags("X", flagsdict)``
1934 - Sets the flags specified in the ``flagsdict()``
1935 parameter. ``setVarFlags`` does not clear previous flags. Think of this
1936 operation as ``addVarFlags``.
1937 * - ``d.getVarFlags("X")``
1938 - Returns a ``flagsdict`` of the flags for the variable "X". Returns "None"
1939 if the variable "X" does not exist.
1940 * - ``d.delVarFlags("X")``
1941 - Deletes all the flags for the variable "X". Does nothing if the variable
1942 "X" does not exist.
1943 * - ``d.expand(expression)``
1944 - Expands variable references in the specified string
1945 expression. References to variables that do not exist are left as is. For
1946 example, ``d.expand("foo ${X}")`` expands to the literal string "foo
1947 ${X}" if the variable "X" does not exist.
1948
1949Other Functions
1950---------------
1951
1952You can find many other functions that can be called from Python by
1953looking at the source code of the ``bb`` module, which is in
1954``bitbake/lib/bb``. For example, ``bitbake/lib/bb/utils.py`` includes
1955the commonly used functions ``bb.utils.contains()`` and
1956``bb.utils.mkdirhier()``, which come with docstrings.
1957
Andrew Geissler87f5cff2022-09-30 13:13:31 -05001958Testing and Debugging BitBake Python code
1959-----------------------------------------
1960
1961The OpenEmbedded build system implements a convenient ``pydevshell`` target which
1962you can use to access the BitBake datastore and experiment with your own Python
1963code. See :yocto_docs:`Using a Python Development Shell
Andrew Geissler517393d2023-01-13 08:55:19 -06001964</dev-manual/python-development-shell.html#using-a-python-development-shell>` in the Yocto
Andrew Geissler87f5cff2022-09-30 13:13:31 -05001965Project manual for details.
1966
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001967Task Checksums and Setscene
1968===========================
1969
1970BitBake uses checksums (or signatures) along with the setscene to
1971determine if a task needs to be run. This section describes the process.
1972To help understand how BitBake does this, the section assumes an
1973OpenEmbedded metadata-based example.
1974
1975These checksums are stored in :term:`STAMP`. You can
Andrew Geisslerc926e172021-05-07 16:11:35 -05001976examine the checksums using the following BitBake command::
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001977
1978 $ bitbake-dumpsigs
1979
1980This command returns the signature data in a readable
1981format that allows you to examine the inputs used when the OpenEmbedded
1982build system generates signatures. For example, using
1983``bitbake-dumpsigs`` allows you to examine the ``do_compile`` task's
Andrew Geisslerf0343792020-11-18 10:42:21 -06001984"sigdata" for a C application (e.g. ``bash``). Running the command also
1985reveals that the "CC" variable is part of the inputs that are hashed.
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001986Any changes to this variable would invalidate the stamp and cause the
1987``do_compile`` task to run.
1988
1989The following list describes related variables:
1990
1991- :term:`BB_HASHCHECK_FUNCTION`:
1992 Specifies the name of the function to call during the "setscene" part
1993 of the task's execution in order to validate the list of task hashes.
1994
1995- :term:`BB_SETSCENE_DEPVALID`:
1996 Specifies a function BitBake calls that determines whether BitBake
1997 requires a setscene dependency to be met.
1998
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001999- :term:`BB_TASKHASH`: Within an executing task,
2000 this variable holds the hash of the task as returned by the currently
2001 enabled signature generator.
2002
2003- :term:`STAMP`: The base path to create stamp files.
2004
2005- :term:`STAMPCLEAN`: Again, the base path to
2006 create stamp files but can use wildcards for matching a range of
2007 files for clean operations.
2008
2009Wildcard Support in Variables
2010=============================
2011
2012Support for wildcard use in variables varies depending on the context in
Andrew Geissler5199d832021-09-24 16:47:35 -05002013which it is used. For example, some variables and filenames allow
Andrew Geisslerc9f78652020-09-18 14:11:35 -05002014limited use of wildcards through the "``%``" and "``*``" characters.
2015Other variables or names support Python's
2016`glob <https://docs.python.org/3/library/glob.html>`_ syntax,
2017`fnmatch <https://docs.python.org/3/library/fnmatch.html#module-fnmatch>`_
2018syntax, or
2019`Regular Expression (re) <https://docs.python.org/3/library/re.html>`_
2020syntax.
2021
2022For variables that have wildcard suport, the documentation describes
2023which form of wildcard, its use, and its limitations.