blob: d91f437d3e83d364aea488bf4287135d3c47a91c [file] [log] [blame]
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
3
4<chapter id="bitbake-user-manual-metadata">
5 <title>Syntax and Operators</title>
6
7 <para>
8 Bitbake files have their own syntax.
9 The syntax has similarities to several
10 other languages but also has some unique features.
11 This section describes the available syntax and operators
12 as well as provides examples.
13 </para>
14
15 <section id='basic-syntax'>
16 <title>Basic Syntax</title>
17
18 <para>
19 This section provides some basic syntax examples.
20 </para>
21
22 <section id='basic-variable-setting'>
23 <title>Basic Variable Setting</title>
24
25 <para>
26 The following example sets <filename>VARIABLE</filename> to
27 "value".
28 This assignment occurs immediately as the statement is parsed.
29 It is a "hard" assignment.
30 <literallayout class='monospaced'>
31 VARIABLE = "value"
32 </literallayout>
33 As expected, if you include leading or trailing spaces as part of
34 an assignment, the spaces are retained:
35 <literallayout class='monospaced'>
36 VARIABLE = " value"
37 VARIABLE = "value "
38 </literallayout>
39 Setting <filename>VARIABLE</filename> to "" sets it to an empty string,
40 while setting the variable to " " sets it to a blank space
41 (i.e. these are not the same values).
42 <literallayout class='monospaced'>
43 VARIABLE = ""
44 VARIABLE = " "
45 </literallayout>
46 </para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -060047
48 <para>
49 You can use single quotes instead of double quotes
50 when setting a variable's value.
51 Doing so allows you to use values that contain the double
52 quote character:
53 <literallayout class='monospaced'>
54 VARIABLE = 'I have a " in my value'
55 </literallayout>
56 <note>
57 Unlike in Bourne shells, single quotes work identically
58 to double quotes in all other ways.
59 They do not suppress variable expansions.
60 </note>
61 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -050062 </section>
63
Brad Bishop6e60e8b2018-02-01 10:27:11 -050064 <section id='line-joining'>
65 <title>Line Joining</title>
66
67 <para>
68 Outside of
69 <link linkend='functions'>functions</link>, BitBake joins
70 any line ending in a backslash character ("\")
71 with the following line before parsing statements.
72 The most common use for the "\" character is to split variable
73 assignments over multiple lines, as in the following example:
74 <literallayout class='monospaced'>
75 FOO = "bar \
76 baz \
77 qaz"
78 </literallayout>
79 Both the "\" character and the newline character
80 that follow it are removed when joining lines.
81 Thus, no newline characters end up in the value of
82 <filename>FOO</filename>.
83 </para>
84
85 <para>
86 Consider this additional example where the two
87 assignments both assign "barbaz" to
88 <filename>FOO</filename>:
89 <literallayout class='monospaced'>
90 FOO = "barbaz"
91
92 FOO = "bar\
93 baz"
94 </literallayout>
95 <note>
96 BitBake does not interpret escape sequences like
97 "\n" in variable values.
98 For these to have an effect, the value must be passed
99 to some utility that interprets escape sequences,
100 such as <filename>printf</filename> or
101 <filename>echo -n</filename>.
102 </note>
103 </para>
104 </section>
105
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500106 <section id='variable-expansion'>
107 <title>Variable Expansion</title>
108
109 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600110 Variables can reference the contents of other variables
111 using a syntax that is similar to variable expansion in
112 Bourne shells.
113 The following assignments
114 result in A containing "aval" and B evaluating to "preavalpost".
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500115 <literallayout class='monospaced'>
116 A = "aval"
117 B = "pre${A}post"
118 </literallayout>
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600119 <note>
120 Unlike in Bourne shells, the curly braces are mandatory:
121 Only <filename>${FOO}</filename> and not
122 <filename>$FOO</filename> is recognized as an expansion of
123 <filename>FOO</filename>.
124 </note>
125 The "=" operator does not immediately expand variable
126 references in the right-hand side.
127 Instead, expansion is deferred until the variable assigned to
128 is actually used.
129 The result depends on the current values of the referenced
130 variables.
131 The following example should clarify this behavior:
132 <literallayout class='monospaced'>
133 A = "${B} baz"
134 B = "${C} bar"
135 C = "foo"
136 *At this point, ${A} equals "foo bar baz"*
137 C = "qux"
138 *At this point, ${A} equals "qux bar baz"*
139 B = "norf"
140 *At this point, ${A} equals "norf baz"*
141 </literallayout>
142 Contrast this behavior with the
143 <link linkend='immediate-variable-expansion'>immediate variable expansion</link>
144 operator (i.e. ":=").
145 </para>
146
147 <para>
148 If the variable expansion syntax is used on a variable that
149 does not exist, the string is kept as is.
150 For example, given the following assignment,
151 <filename>BAR</filename> expands to the literal string
152 "${FOO}" as long as <filename>FOO</filename> does not exist.
153 <literallayout class='monospaced'>
154 BAR = "${FOO}"
155 </literallayout>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500156 </para>
157 </section>
158
159 <section id='setting-a-default-value'>
160 <title>Setting a default value (?=)</title>
161
162 <para>
163 You can use the "?=" operator to achieve a "softer" assignment
164 for a variable.
165 This type of assignment allows you to define a variable if it
166 is undefined when the statement is parsed, but to leave the
167 value alone if the variable has a value.
168 Here is an example:
169 <literallayout class='monospaced'>
170 A ?= "aval"
171 </literallayout>
172 If <filename>A</filename> is set at the time this statement is parsed,
173 the variable retains its value.
174 However, if <filename>A</filename> is not set,
175 the variable is set to "aval".
176 <note>
177 This assignment is immediate.
178 Consequently, if multiple "?=" assignments
179 to a single variable exist, the first of those ends up getting
180 used.
181 </note>
182 </para>
183 </section>
184
185 <section id='setting-a-weak-default-value'>
186 <title>Setting a weak default value (??=)</title>
187
188 <para>
189 It is possible to use a "weaker" assignment than in the
190 previous section by using the "??=" operator.
191 This assignment behaves identical to "?=" except that the
192 assignment is made at the end of the parsing process rather
193 than immediately.
194 Consequently, when multiple "??=" assignments exist, the last
195 one is used.
196 Also, any "=" or "?=" assignment will override the value set with
197 "??=".
198 Here is an example:
199 <literallayout class='monospaced'>
200 A ??= "somevalue"
201 A ??= "someothervalue"
202 </literallayout>
203 If <filename>A</filename> is set before the above statements are parsed,
204 the variable retains its value.
205 If <filename>A</filename> is not set,
206 the variable is set to "someothervalue".
207 </para>
208
209 <para>
210 Again, this assignment is a "lazy" or "weak" assignment
211 because it does not occur until the end
212 of the parsing process.
213 </para>
214 </section>
215
216 <section id='immediate-variable-expansion'>
217 <title>Immediate variable expansion (:=)</title>
218
219 <para>
220 The ":=" operator results in a variable's
221 contents being expanded immediately,
222 rather than when the variable is actually used:
223 <literallayout class='monospaced'>
224 T = "123"
225 A := "${B} ${A} test ${T}"
226 T = "456"
227 B = "${T} bval"
228 C = "cval"
229 C := "${C}append"
230 </literallayout>
231 In this example, <filename>A</filename> contains
232 "test 123" because <filename>${B}</filename> and
233 <filename>${A}</filename> at the time of parsing are undefined,
234 which leaves "test 123".
235 And, the variable <filename>C</filename>
236 contains "cvalappend" since <filename>${C}</filename> immediately
237 expands to "cval".
238 </para>
239 </section>
240
241 <section id='appending-and-prepending'>
242 <title>Appending (+=) and prepending (=+) With Spaces</title>
243
244 <para>
245 Appending and prepending values is common and can be accomplished
246 using the "+=" and "=+" operators.
247 These operators insert a space between the current
248 value and prepended or appended value.
249 </para>
250
251 <para>
252 These operators take immediate effect during parsing.
253 Here are some examples:
254 <literallayout class='monospaced'>
255 B = "bval"
256 B += "additionaldata"
257 C = "cval"
258 C =+ "test"
259 </literallayout>
260 The variable <filename>B</filename> contains
261 "bval additionaldata" and <filename>C</filename>
262 contains "test cval".
263 </para>
264 </section>
265
266 <section id='appending-and-prepending-without-spaces'>
267 <title>Appending (.=) and Prepending (=.) Without Spaces</title>
268
269 <para>
270 If you want to append or prepend values without an
271 inserted space, use the ".=" and "=." operators.
272 </para>
273
274 <para>
275 These operators take immediate effect during parsing.
276 Here are some examples:
277 <literallayout class='monospaced'>
278 B = "bval"
279 B .= "additionaldata"
280 C = "cval"
281 C =. "test"
282 </literallayout>
283 The variable <filename>B</filename> contains
284 "bvaladditionaldata" and
285 <filename>C</filename> contains "testcval".
286 </para>
287 </section>
288
289 <section id='appending-and-prepending-override-style-syntax'>
290 <title>Appending and Prepending (Override Style Syntax)</title>
291
292 <para>
293 You can also append and prepend a variable's value
294 using an override style syntax.
295 When you use this syntax, no spaces are inserted.
296 </para>
297
298 <para>
299 These operators differ from the ":=", ".=", "=.", "+=", and "=+"
300 operators in that their effects are deferred
301 until after parsing completes rather than being immediately
302 applied.
303 Here are some examples:
304 <literallayout class='monospaced'>
305 B = "bval"
306 B_append = " additional data"
307 C = "cval"
308 C_prepend = "additional data "
309 D = "dval"
310 D_append = "additional data"
311 </literallayout>
312 The variable <filename>B</filename> becomes
313 "bval additional data" and <filename>C</filename> becomes
314 "additional data cval".
315 The variable <filename>D</filename> becomes
316 "dvaladditional data".
317 <note>
318 You must control all spacing when you use the
319 override syntax.
320 </note>
321 </para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600322
323 <para>
324 It is also possible to append and prepend to shell
325 functions and BitBake-style Python functions.
326 See the
327 "<link linkend='shell-functions'>Shell Functions</link>" and
328 "<link linkend='bitbake-style-python-functions'>BitBake-Style Python Functions</link>
329 sections for examples.
330 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500331 </section>
332
333 <section id='removing-override-style-syntax'>
334 <title>Removal (Override Style Syntax)</title>
335
336 <para>
337 You can remove values from lists using the removal
338 override style syntax.
339 Specifying a value for removal causes all occurrences of that
340 value to be removed from the variable.
341 </para>
342
343 <para>
344 When you use this syntax, BitBake expects one or more strings.
Brad Bishop1a4b7ee2018-12-16 17:11:34 -0800345 Surrounding spaces and spacing are preserved.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500346 Here is an example:
347 <literallayout class='monospaced'>
348 FOO = "123 456 789 123456 123 456 123 456"
349 FOO_remove = "123"
350 FOO_remove = "456"
351 FOO2 = "abc def ghi abcdef abc def abc def"
352 FOO2_remove = "abc def"
353 </literallayout>
354 The variable <filename>FOO</filename> becomes
Brad Bishop1a4b7ee2018-12-16 17:11:34 -0800355 "&nbsp;&nbsp;789 123456&nbsp;&nbsp;&nbsp;&nbsp;"
356 and <filename>FOO2</filename> becomes
357 "&nbsp;&nbsp;ghi abcdef&nbsp;&nbsp;&nbsp;&nbsp;".
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500358 </para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600359
360 <para>
361 Like "_append" and "_prepend", "_remove"
362 is deferred until after parsing completes.
363 </para>
364 </section>
365
366 <section id='override-style-operation-advantages'>
367 <title>Override Style Operation Advantages</title>
368
369 <para>
370 An advantage of the override style operations
371 "_append", "_prepend", and "_remove" as compared to the
372 "+=" and "=+" operators is that the override style
373 operators provide guaranteed operations.
374 For example, consider a class <filename>foo.bbclass</filename>
375 that needs to add the value "val" to the variable
376 <filename>FOO</filename>, and a recipe that uses
377 <filename>foo.bbclass</filename> as follows:
378 <literallayout class='monospaced'>
379 inherit foo
380
381 FOO = "initial"
382 </literallayout>
383 If <filename>foo.bbclass</filename> uses the "+=" operator,
384 as follows, then the final value of <filename>FOO</filename>
385 will be "initial", which is not what is desired:
386 <literallayout class='monospaced'>
387 FOO += "val"
388 </literallayout>
389 If, on the other hand, <filename>foo.bbclass</filename>
390 uses the "_append" operator, then the final value of
391 <filename>FOO</filename> will be "initial val", as intended:
392 <literallayout class='monospaced'>
393 FOO_append = " val"
394 </literallayout>
395 <note>
396 It is never necessary to use "+=" together with "_append".
397 The following sequence of assignments appends "barbaz" to
398 <filename>FOO</filename>:
399 <literallayout class='monospaced'>
400 FOO_append = "bar"
401 FOO_append = "baz"
402 </literallayout>
403 The only effect of changing the second assignment in the
404 previous example to use "+=" would be to add a space before
405 "baz" in the appended value (due to how the "+=" operator
406 works).
407 </note>
408 Another advantage of the override style operations is that
409 you can combine them with other overrides as described in the
410 "<link linkend='conditional-syntax-overrides'>Conditional Syntax (Overrides)</link>"
411 section.
412 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500413 </section>
414
415 <section id='variable-flag-syntax'>
416 <title>Variable Flag Syntax</title>
417
418 <para>
419 Variable flags are BitBake's implementation of variable properties
420 or attributes.
421 It is a way of tagging extra information onto a variable.
422 You can find more out about variable flags in general in the
423 "<link linkend='variable-flags'>Variable Flags</link>"
424 section.
425 </para>
426
427 <para>
428 You can define, append, and prepend values to variable flags.
429 All the standard syntax operations previously mentioned work
430 for variable flags except for override style syntax
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600431 (i.e. "_prepend", "_append", and "_remove").
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500432 </para>
433
434 <para>
435 Here are some examples showing how to set variable flags:
436 <literallayout class='monospaced'>
437 FOO[a] = "abc"
438 FOO[b] = "123"
439 FOO[a] += "456"
440 </literallayout>
441 The variable <filename>FOO</filename> has two flags:
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600442 <filename>[a]</filename> and <filename>[b]</filename>.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500443 The flags are immediately set to "abc" and "123", respectively.
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600444 The <filename>[a]</filename> flag becomes "abc 456".
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500445 </para>
446
447 <para>
448 No need exists to pre-define variable flags.
449 You can simply start using them.
450 One extremely common application
451 is to attach some brief documentation to a BitBake variable as
452 follows:
453 <literallayout class='monospaced'>
454 CACHE[doc] = "The directory holding the cache of the metadata."
455 </literallayout>
456 </para>
457 </section>
458
459 <section id='inline-python-variable-expansion'>
460 <title>Inline Python Variable Expansion</title>
461
462 <para>
463 You can use inline Python variable expansion to
464 set variables.
465 Here is an example:
466 <literallayout class='monospaced'>
467 DATE = "${@time.strftime('%Y%m%d',time.gmtime())}"
468 </literallayout>
469 This example results in the <filename>DATE</filename>
470 variable being set to the current date.
471 </para>
472
473 <para>
474 Probably the most common use of this feature is to extract
475 the value of variables from BitBake's internal data dictionary,
476 <filename>d</filename>.
477 The following lines select the values of a package name
478 and its version number, respectively:
479 <literallayout class='monospaced'>
480 PN = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'}"
481 PV = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[1] or '1.0'}"
482 </literallayout>
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600483 <note>
484 Inline Python expressions work just like variable expansions
485 insofar as the "=" and ":=" operators are concerned.
486 Given the following assignment, <filename>foo()</filename>
487 is called each time <filename>FOO</filename> is expanded:
488 <literallayout class='monospaced'>
489 FOO = "${@foo()}"
490 </literallayout>
491 Contrast this with the following immediate assignment, where
492 <filename>foo()</filename> is only called once, while the
493 assignment is parsed:
494 <literallayout class='monospaced'>
495 FOO := "${@foo()}"
496 </literallayout>
497 </note>
498 For a different way to set variables with Python code during
499 parsing, see the
500 "<link linkend='anonymous-python-functions'>Anonymous Python Functions</link>"
501 section.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500502 </para>
503 </section>
504
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600505 <section id='unsetting-variables'>
Brad Bishopb1114e52019-02-13 07:56:10 -0500506 <title>Unsetting variables</title>
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600507
508 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500509 It is possible to completely remove a variable or a variable flag
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600510 from BitBake's internal data dictionary by using the "unset" keyword.
511 Here is an example:
512 <literallayout class='monospaced'>
513 unset DATE
514 unset do_fetch[noexec]
515 </literallayout>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500516 These two statements remove the <filename>DATE</filename> and the
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600517 <filename>do_fetch[noexec]</filename> flag.
518 </para>
519
520 </section>
521
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500522 <section id='providing-pathnames'>
523 <title>Providing Pathnames</title>
524
525 <para>
526 When specifying pathnames for use with BitBake,
527 do not use the tilde ("~") character as a shortcut
528 for your home directory.
529 Doing so might cause BitBake to not recognize the
530 path since BitBake does not expand this character in
531 the same way a shell would.
532 </para>
533
534 <para>
535 Instead, provide a fuller path as the following
536 example illustrates:
537 <literallayout class='monospaced'>
538 BBLAYERS ?= " \
539 /home/scott-lenovo/LayerA \
540 "
541 </literallayout>
542 </para>
543 </section>
544 </section>
545
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600546 <section id='exporting-variables-to-the-environment'>
547 <title>Exporting Variables to the Environment</title>
548
549 <para>
550 You can export variables to the environment of running
551 tasks by using the <filename>export</filename> keyword.
552 For example, in the following example, the
553 <filename>do_foo</filename> task prints "value from
554 the environment" when run:
555 <literallayout class='monospaced'>
556 export ENV_VARIABLE
557 ENV_VARIABLE = "value from the environment"
558
559 do_foo() {
560 bbplain "$ENV_VARIABLE"
561 }
562 </literallayout>
563 <note>
564 BitBake does not expand <filename>$ENV_VARIABLE</filename>
565 in this case because it lacks the obligatory
566 <filename>{}</filename>.
567 Rather, <filename>$ENV_VARIABLE</filename> is expanded
568 by the shell.
569 </note>
570 It does not matter whether
571 <filename>export ENV_VARIABLE</filename> appears before or
572 after assignments to <filename>ENV_VARIABLE</filename>.
573 </para>
574
575 <para>
576 It is also possible to combine <filename>export</filename>
577 with setting a value for the variable.
578 Here is an example:
579 <literallayout class='monospaced'>
580 export ENV_VARIABLE = "<replaceable>variable-value</replaceable>"
581 </literallayout>
582 In the output of <filename>bitbake -e</filename>, variables
583 that are exported to the environment are preceded by "export".
584 </para>
585
586 <para>
587 Among the variables commonly exported to the environment
588 are <filename>CC</filename> and <filename>CFLAGS</filename>,
589 which are picked up by many build systems.
590 </para>
591 </section>
592
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500593 <section id='conditional-syntax-overrides'>
594 <title>Conditional Syntax (Overrides)</title>
595
596 <para>
597 BitBake uses
598 <link linkend='var-OVERRIDES'><filename>OVERRIDES</filename></link>
599 to control what variables are overridden after BitBake
600 parses recipes and configuration files.
601 This section describes how you can use
602 <filename>OVERRIDES</filename> as conditional metadata,
603 talks about key expansion in relationship to
604 <filename>OVERRIDES</filename>, and provides some examples
605 to help with understanding.
606 </para>
607
608 <section id='conditional-metadata'>
609 <title>Conditional Metadata</title>
610
611 <para>
612 You can use <filename>OVERRIDES</filename> to conditionally select
613 a specific version of a variable and to conditionally
614 append or prepend the value of a variable.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500615 <note>
616 Overrides can only use lower-case characters.
617 Additionally, underscores are not permitted in override names
618 as they are used to separate overrides from each other and
619 from the variable name.
620 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500621 <itemizedlist>
622 <listitem><para><emphasis>Selecting a Variable:</emphasis>
623 The <filename>OVERRIDES</filename> variable is
624 a colon-character-separated list that contains items
625 for which you want to satisfy conditions.
626 Thus, if you have a variable that is conditional on “arm”, and “arm”
627 is in <filename>OVERRIDES</filename>, then the “arm”-specific
628 version of the variable is used rather than the non-conditional
629 version.
630 Here is an example:
631 <literallayout class='monospaced'>
632 OVERRIDES = "architecture:os:machine"
633 TEST = "default"
634 TEST_os = "osspecific"
635 TEST_nooverride = "othercondvalue"
636 </literallayout>
637 In this example, the <filename>OVERRIDES</filename>
638 variable lists three overrides:
639 "architecture", "os", and "machine".
640 The variable <filename>TEST</filename> by itself has a default
641 value of "default".
642 You select the os-specific version of the <filename>TEST</filename>
643 variable by appending the "os" override to the variable
644 (i.e.<filename>TEST_os</filename>).
645 </para>
646
647 <para>
648 To better understand this, consider a practical example
649 that assumes an OpenEmbedded metadata-based Linux
650 kernel recipe file.
651 The following lines from the recipe file first set
652 the kernel branch variable <filename>KBRANCH</filename>
653 to a default value, then conditionally override that
654 value based on the architecture of the build:
655 <literallayout class='monospaced'>
656 KBRANCH = "standard/base"
657 KBRANCH_qemuarm = "standard/arm-versatile-926ejs"
658 KBRANCH_qemumips = "standard/mti-malta32"
659 KBRANCH_qemuppc = "standard/qemuppc"
660 KBRANCH_qemux86 = "standard/common-pc/base"
661 KBRANCH_qemux86-64 = "standard/common-pc-64/base"
662 KBRANCH_qemumips64 = "standard/mti-malta64"
663 </literallayout>
664 </para></listitem>
665 <listitem><para><emphasis>Appending and Prepending:</emphasis>
666 BitBake also supports append and prepend operations to
667 variable values based on whether a specific item is
668 listed in <filename>OVERRIDES</filename>.
669 Here is an example:
670 <literallayout class='monospaced'>
671 DEPENDS = "glibc ncurses"
672 OVERRIDES = "machine:local"
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500673 DEPENDS_append_machine = " libmad"
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500674 </literallayout>
675 In this example, <filename>DEPENDS</filename> becomes
676 "glibc ncurses libmad".
677 </para>
678
679 <para>
680 Again, using an OpenEmbedded metadata-based
681 kernel recipe file as an example, the
682 following lines will conditionally append to the
683 <filename>KERNEL_FEATURES</filename> variable based
684 on the architecture:
685 <literallayout class='monospaced'>
686 KERNEL_FEATURES_append = " ${KERNEL_EXTRA_FEATURES}"
687 KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
688 KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc"
689 </literallayout>
690 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600691 <listitem><para><emphasis>Setting a Variable for a Single Task:</emphasis>
692 BitBake supports setting a variable just for the
693 duration of a single task.
694 Here is an example:
695 <literallayout class='monospaced'>
696 FOO_task-configure = "val 1"
697 FOO_task-compile = "val 2"
698 </literallayout>
699 In the previous example, <filename>FOO</filename>
700 has the value "val 1" while the
701 <filename>do_configure</filename> task is executed,
702 and the value "val 2" while the
703 <filename>do_compile</filename> task is executed.
704 </para>
705
706 <para>Internally, this is implemented by prepending
707 the task (e.g. "task-compile:") to the value of
708 <link linkend='var-OVERRIDES'><filename>OVERRIDES</filename></link>
709 for the local datastore of the <filename>do_compile</filename>
710 task.</para>
711
712 <para>You can also use this syntax with other combinations
713 (e.g. "<filename>_prepend</filename>") as shown in the
714 following example:
715 <literallayout class='monospaced'>
716 EXTRA_OEMAKE_prepend_task-compile = "${PARALLEL_MAKE} "
717 </literallayout>
718 </para></listitem>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500719 </itemizedlist>
720 </para>
721 </section>
722
723 <section id='key-expansion'>
724 <title>Key Expansion</title>
725
726 <para>
727 Key expansion happens when the BitBake datastore is finalized
728 just before BitBake expands overrides.
729 To better understand this, consider the following example:
730 <literallayout class='monospaced'>
731 A${B} = "X"
732 B = "2"
733 A2 = "Y"
734 </literallayout>
735 In this case, after all the parsing is complete, and
736 before any overrides are handled, BitBake expands
737 <filename>${B}</filename> into "2".
738 This expansion causes <filename>A2</filename>, which was
739 set to "Y" before the expansion, to become "X".
740 </para>
741 </section>
742
743 <section id='variable-interaction-worked-examples'>
744 <title>Examples</title>
745
746 <para>
747 Despite the previous explanations that show the different forms of
748 variable definitions, it can be hard to work
749 out exactly what happens when variable operators, conditional
750 overrides, and unconditional overrides are combined.
751 This section presents some common scenarios along
752 with explanations for variable interactions that
753 typically confuse users.
754 </para>
755
756 <para>
757 There is often confusion concerning the order in which
758 overrides and various "append" operators take effect.
759 Recall that an append or prepend operation using "_append"
760 and "_prepend" does not result in an immediate assignment
761 as would "+=", ".=", "=+", or "=.".
762 Consider the following example:
763 <literallayout class='monospaced'>
764 OVERRIDES = "foo"
765 A = "Z"
766 A_foo_append = "X"
767 </literallayout>
768 For this case, <filename>A</filename> is
769 unconditionally set to "Z" and "X" is
770 unconditionally and immediately appended to the variable
771 <filename>A_foo</filename>.
772 Because overrides have not been applied yet,
773 <filename>A_foo</filename> is set to "X" due to the append
774 and <filename>A</filename> simply equals "Z".
775 </para>
776
777 <para>
778 Applying overrides, however, changes things.
779 Since "foo" is listed in <filename>OVERRIDES</filename>,
780 the conditional variable <filename>A</filename> is replaced
781 with the "foo" version, which is equal to "X".
782 So effectively, <filename>A_foo</filename> replaces <filename>A</filename>.
783 </para>
784
785 <para>
786 This next example changes the order of the override and
787 the append:
788 <literallayout class='monospaced'>
789 OVERRIDES = "foo"
790 A = "Z"
791 A_append_foo = "X"
792 </literallayout>
793 For this case, before overrides are handled,
794 <filename>A</filename> is set to "Z" and <filename>A_append_foo</filename>
795 is set to "X".
796 Once the override for "foo" is applied, however,
797 <filename>A</filename> gets appended with "X".
798 Consequently, <filename>A</filename> becomes "ZX".
799 Notice that spaces are not appended.
800 </para>
801
802 <para>
803 This next example has the order of the appends and overrides reversed
804 back as in the first example:
805 <literallayout class='monospaced'>
806 OVERRIDES = "foo"
807 A = "Y"
808 A_foo_append = "Z"
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600809 A_foo_append = "X"
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500810 </literallayout>
811 For this case, before any overrides are resolved,
812 <filename>A</filename> is set to "Y" using an immediate assignment.
813 After this immediate assignment, <filename>A_foo</filename> is set
814 to "Z", and then further appended with
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600815 "X" leaving the variable set to "ZX".
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500816 Finally, applying the override for "foo" results in the conditional
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600817 variable <filename>A</filename> becoming "ZX" (i.e.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500818 <filename>A</filename> is replaced with <filename>A_foo</filename>).
819 </para>
820
821 <para>
822 This final example mixes in some varying operators:
823 <literallayout class='monospaced'>
824 A = "1"
825 A_append = "2"
826 A_append = "3"
827 A += "4"
828 A .= "5"
829 </literallayout>
830 For this case, the type of append operators are affecting the
831 order of assignments as BitBake passes through the code
832 multiple times.
833 Initially, <filename>A</filename> is set to "1 45" because
834 of the three statements that use immediate operators.
835 After these assignments are made, BitBake applies the
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600836 "_append" operations.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500837 Those operations result in <filename>A</filename> becoming "1 4523".
838 </para>
839 </section>
840 </section>
841
842 <section id='sharing-functionality'>
843 <title>Sharing Functionality</title>
844
845 <para>
846 BitBake allows for metadata sharing through include files
847 (<filename>.inc</filename>) and class files
848 (<filename>.bbclass</filename>).
849 For example, suppose you have a piece of common functionality
850 such as a task definition that you want to share between
851 more than one recipe.
852 In this case, creating a <filename>.bbclass</filename>
853 file that contains the common functionality and then using
854 the <filename>inherit</filename> directive in your recipes to
855 inherit the class would be a common way to share the task.
856 </para>
857
858 <para>
859 This section presents the mechanisms BitBake provides to
860 allow you to share functionality between recipes.
861 Specifically, the mechanisms include <filename>include</filename>,
862 <filename>inherit</filename>, <filename>INHERIT</filename>, and
863 <filename>require</filename> directives.
864 </para>
865
866 <section id='locating-include-and-class-files'>
867 <title>Locating Include and Class Files</title>
868
869 <para>
870 BitBake uses the
871 <link linkend='var-BBPATH'><filename>BBPATH</filename></link>
872 variable to locate needed include and class files.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500873 Additionally, BitBake searches the current directory for
874 <filename>include</filename> and <filename>require</filename>
875 directives.
876 <note>
877 The <filename>BBPATH</filename> variable is analogous to
878 the environment variable <filename>PATH</filename>.
879 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500880 </para>
881
882 <para>
883 In order for include and class files to be found by BitBake,
884 they need to be located in a "classes" subdirectory that can
885 be found in <filename>BBPATH</filename>.
886 </para>
887 </section>
888
889 <section id='inherit-directive'>
890 <title><filename>inherit</filename> Directive</title>
891
892 <para>
893 When writing a recipe or class file, you can use the
894 <filename>inherit</filename> directive to inherit the
895 functionality of a class (<filename>.bbclass</filename>).
896 BitBake only supports this directive when used within recipe
897 and class files (i.e. <filename>.bb</filename> and
898 <filename>.bbclass</filename>).
899 </para>
900
901 <para>
902 The <filename>inherit</filename> directive is a rudimentary
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500903 means of specifying functionality contained in class files
904 that your recipes require.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500905 For example, you can easily abstract out the tasks involved in
906 building a package that uses Autoconf and Automake and put
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500907 those tasks into a class file and then have your recipe
908 inherit that class file.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500909 </para>
910
911 <para>
912 As an example, your recipes could use the following directive
913 to inherit an <filename>autotools.bbclass</filename> file.
914 The class file would contain common functionality for using
915 Autotools that could be shared across recipes:
916 <literallayout class='monospaced'>
917 inherit autotools
918 </literallayout>
919 In this case, BitBake would search for the directory
920 <filename>classes/autotools.bbclass</filename>
921 in <filename>BBPATH</filename>.
922 <note>
923 You can override any values and functions of the
924 inherited class within your recipe by doing so
925 after the "inherit" statement.
926 </note>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500927 If you want to use the directive to inherit
928 multiple classes, separate them with spaces.
929 The following example shows how to inherit both the
930 <filename>buildhistory</filename> and <filename>rm_work</filename>
931 classes:
932 <literallayout class='monospaced'>
933 inherit buildhistory rm_work
934 </literallayout>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500935 </para>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500936
937 <para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500938 An advantage with the inherit directive as compared to both
939 the
940 <link linkend='include-directive'>include</link> and
941 <link linkend='require-inclusion'>require</link> directives
942 is that you can inherit class files conditionally.
943 You can accomplish this by using a variable expression
944 after the <filename>inherit</filename> statement.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500945 Here is an example:
946 <literallayout class='monospaced'>
947 inherit ${VARNAME}
948 </literallayout>
949 If <filename>VARNAME</filename> is going to be set, it needs
950 to be set before the <filename>inherit</filename> statement
951 is parsed.
952 One way to achieve a conditional inherit in this case is to use
953 overrides:
954 <literallayout class='monospaced'>
955 VARIABLE = ""
956 VARIABLE_someoverride = "myclass"
957 </literallayout>
958 </para>
959
960 <para>
961 Another method is by using anonymous Python.
962 Here is an example:
963 <literallayout class='monospaced'>
964 python () {
965 if condition == value:
966 d.setVar('VARIABLE', 'myclass')
967 else:
968 d.setVar('VARIABLE', '')
969 }
970 </literallayout>
971 </para>
972
973 <para>
974 Alternatively, you could use an in-line Python expression
975 in the following form:
976 <literallayout class='monospaced'>
977 inherit ${@'classname' if condition else ''}
978 inherit ${@functionname(params)}
979 </literallayout>
980 In all cases, if the expression evaluates to an empty
981 string, the statement does not trigger a syntax error
982 because it becomes a no-op.
983 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500984 </section>
985
986 <section id='include-directive'>
987 <title><filename>include</filename> Directive</title>
988
989 <para>
990 BitBake understands the <filename>include</filename>
991 directive.
992 This directive causes BitBake to parse whatever file you specify,
993 and to insert that file at that location.
994 The directive is much like its equivalent in Make except
995 that if the path specified on the include line is a relative
996 path, BitBake locates the first file it can find
997 within <filename>BBPATH</filename>.
998 </para>
999
1000 <para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -05001001 The include directive is a more generic method of including
1002 functionality as compared to the
1003 <link linkend='inherit-directive'>inherit</link> directive,
1004 which is restricted to class (i.e. <filename>.bbclass</filename>)
1005 files.
1006 The include directive is applicable for any other kind of
1007 shared or encapsulated functionality or configuration that
1008 does not suit a <filename>.bbclass</filename> file.
1009 </para>
1010
1011 <para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001012 As an example, suppose you needed a recipe to include some
1013 self-test definitions:
1014 <literallayout class='monospaced'>
1015 include test_defs.inc
1016 </literallayout>
1017 <note>
1018 The <filename>include</filename> directive does not
1019 produce an error when the file cannot be found.
1020 Consequently, it is recommended that if the file you
1021 are including is expected to exist, you should use
1022 <link linkend='require-inclusion'><filename>require</filename></link>
1023 instead of <filename>include</filename>.
1024 Doing so makes sure that an error is produced if the
1025 file cannot be found.
1026 </note>
1027 </para>
1028 </section>
1029
1030 <section id='require-inclusion'>
1031 <title><filename>require</filename> Directive</title>
1032
1033 <para>
1034 BitBake understands the <filename>require</filename>
1035 directive.
1036 This directive behaves just like the
1037 <filename>include</filename> directive with the exception that
1038 BitBake raises a parsing error if the file to be included cannot
1039 be found.
1040 Thus, any file you require is inserted into the file that is
1041 being parsed at the location of the directive.
1042 </para>
1043
1044 <para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -05001045 The require directive, like the include directive previously
1046 described, is a more generic method of including
1047 functionality as compared to the
1048 <link linkend='inherit-directive'>inherit</link> directive,
1049 which is restricted to class (i.e. <filename>.bbclass</filename>)
1050 files.
1051 The require directive is applicable for any other kind of
1052 shared or encapsulated functionality or configuration that
1053 does not suit a <filename>.bbclass</filename> file.
1054 </para>
1055
1056 <para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001057 Similar to how BitBake handles
1058 <link linkend='include-directive'><filename>include</filename></link>,
1059 if the path specified
1060 on the require line is a relative path, BitBake locates
1061 the first file it can find within <filename>BBPATH</filename>.
1062 </para>
1063
1064 <para>
1065 As an example, suppose you have two versions of a recipe
1066 (e.g. <filename>foo_1.2.2.bb</filename> and
1067 <filename>foo_2.0.0.bb</filename>) where
1068 each version contains some identical functionality that could be
1069 shared.
1070 You could create an include file named <filename>foo.inc</filename>
1071 that contains the common definitions needed to build "foo".
1072 You need to be sure <filename>foo.inc</filename> is located in the
1073 same directory as your two recipe files as well.
1074 Once these conditions are set up, you can share the functionality
1075 using a <filename>require</filename> directive from within each
1076 recipe:
1077 <literallayout class='monospaced'>
1078 require foo.inc
1079 </literallayout>
1080 </para>
1081 </section>
1082
1083 <section id='inherit-configuration-directive'>
1084 <title><filename>INHERIT</filename> Configuration Directive</title>
1085
1086 <para>
1087 When creating a configuration file (<filename>.conf</filename>),
Brad Bishopd7bf8c12018-02-25 22:55:05 -05001088 you can use the
1089 <link linkend='var-INHERIT'><filename>INHERIT</filename></link>
1090 configuration directive to inherit a class.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001091 BitBake only supports this directive when used within
1092 a configuration file.
1093 </para>
1094
1095 <para>
1096 As an example, suppose you needed to inherit a class
1097 file called <filename>abc.bbclass</filename> from a
1098 configuration file as follows:
1099 <literallayout class='monospaced'>
1100 INHERIT += "abc"
1101 </literallayout>
1102 This configuration directive causes the named
1103 class to be inherited at the point of the directive
1104 during parsing.
1105 As with the <filename>inherit</filename> directive, the
1106 <filename>.bbclass</filename> file must be located in a
1107 "classes" subdirectory in one of the directories specified
1108 in <filename>BBPATH</filename>.
1109 <note>
1110 Because <filename>.conf</filename> files are parsed
1111 first during BitBake's execution, using
1112 <filename>INHERIT</filename> to inherit a class effectively
1113 inherits the class globally (i.e. for all recipes).
1114 </note>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001115 If you want to use the directive to inherit
1116 multiple classes, you can provide them on the same line in the
1117 <filename>local.conf</filename> file.
1118 Use spaces to separate the classes.
1119 The following example shows how to inherit both the
1120 <filename>autotools</filename> and <filename>pkgconfig</filename>
1121 classes:
1122 <literallayout class='monospaced'>
Brad Bishopd7bf8c12018-02-25 22:55:05 -05001123 INHERIT += "autotools pkgconfig"
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001124 </literallayout>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001125 </para>
1126 </section>
1127 </section>
1128
1129 <section id='functions'>
1130 <title>Functions</title>
1131
1132 <para>
1133 As with most languages, functions are the building blocks that
1134 are used to build up operations into tasks.
1135 BitBake supports these types of functions:
1136 <itemizedlist>
1137 <listitem><para><emphasis>Shell Functions:</emphasis>
1138 Functions written in shell script and executed either
1139 directly as functions, tasks, or both.
1140 They can also be called by other shell functions.
1141 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001142 <listitem><para><emphasis>BitBake-Style Python Functions:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001143 Functions written in Python and executed by BitBake or other
1144 Python functions using <filename>bb.build.exec_func()</filename>.
1145 </para></listitem>
1146 <listitem><para><emphasis>Python Functions:</emphasis>
1147 Functions written in Python and executed by Python.
1148 </para></listitem>
1149 <listitem><para><emphasis>Anonymous Python Functions:</emphasis>
1150 Python functions executed automatically during
1151 parsing.
1152 </para></listitem>
1153 </itemizedlist>
1154 Regardless of the type of function, you can only
1155 define them in class (<filename>.bbclass</filename>)
1156 and recipe (<filename>.bb</filename> or <filename>.inc</filename>)
1157 files.
1158 </para>
1159
1160 <section id='shell-functions'>
1161 <title>Shell Functions</title>
1162
1163 <para>
1164 Functions written in shell script and executed either
1165 directly as functions, tasks, or both.
1166 They can also be called by other shell functions.
1167 Here is an example shell function definition:
1168 <literallayout class='monospaced'>
1169 some_function () {
1170 echo "Hello World"
1171 }
1172 </literallayout>
1173 When you create these types of functions in your recipe
1174 or class files, you need to follow the shell programming
1175 rules.
1176 The scripts are executed by <filename>/bin/sh</filename>,
1177 which may not be a bash shell but might be something
1178 such as <filename>dash</filename>.
1179 You should not use Bash-specific script (bashisms).
1180 </para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001181
1182 <para>
1183 Overrides and override-style operators like
1184 <filename>_append</filename> and
1185 <filename>_prepend</filename> can also be applied to
1186 shell functions.
1187 Most commonly, this application would be used in a
1188 <filename>.bbappend</filename> file to modify functions in
1189 the main recipe.
1190 It can also be used to modify functions inherited from
1191 classes.
1192 </para>
1193
1194 <para>
1195 As an example, consider the following:
1196 <literallayout class='monospaced'>
1197 do_foo() {
1198 bbplain first
1199 fn
1200 }
1201
1202 fn_prepend() {
1203 bbplain second
1204 }
1205
1206 fn() {
1207 bbplain third
1208 }
1209
1210 do_foo_append() {
1211 bbplain fourth
1212 }
1213 </literallayout>
1214 Running <filename>do_foo</filename>
1215 prints the following:
1216 <literallayout class='monospaced'>
1217 recipename do_foo: first
1218 recipename do_foo: second
1219 recipename do_foo: third
1220 recipename do_foo: fourth
1221 </literallayout>
1222 <note>
1223 Overrides and override-style operators can
1224 be applied to any shell function, not just
1225 <link linkend='tasks'>tasks</link>.
1226 </note>
1227 You can use the <filename>bitbake -e</filename>&nbsp;<replaceable>recipename</replaceable>
1228 command to view the final assembled function
1229 after all overrides have been applied.
1230 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001231 </section>
1232
1233 <section id='bitbake-style-python-functions'>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001234 <title>BitBake-Style Python Functions</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001235
1236 <para>
1237 These functions are written in Python and executed by
1238 BitBake or other Python functions using
1239 <filename>bb.build.exec_func()</filename>.
1240 </para>
1241
1242 <para>
1243 An example BitBake function is:
1244 <literallayout class='monospaced'>
1245 python some_python_function () {
1246 d.setVar("TEXT", "Hello World")
Brad Bishop6e60e8b2018-02-01 10:27:11 -05001247 print d.getVar("TEXT")
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001248 }
1249 </literallayout>
1250 Because the Python "bb" and "os" modules are already
1251 imported, you do not need to import these modules.
1252 Also in these types of functions, the datastore ("d")
1253 is a global variable and is always automatically
1254 available.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001255 <note>
1256 Variable expressions (e.g. <filename>${X}</filename>)
1257 are no longer expanded within Python functions.
1258 This behavior is intentional in order to allow you
1259 to freely set variable values to expandable expressions
1260 without having them expanded prematurely.
1261 If you do wish to expand a variable within a Python
Brad Bishop6e60e8b2018-02-01 10:27:11 -05001262 function, use <filename>d.getVar("X")</filename>.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001263 Or, for more complicated expressions, use
1264 <filename>d.expand()</filename>.
1265 </note>
1266 </para>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001267
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001268 <para>
1269 Similar to shell functions, you can also apply overrides
1270 and override-style operators to BitBake-style Python
1271 functions.
1272 </para>
1273
1274 <para>
1275 As an example, consider the following:
1276 <literallayout class='monospaced'>
1277 python do_foo_prepend() {
1278 bb.plain("first")
1279 }
1280
1281 python do_foo() {
1282 bb.plain("second")
1283 }
1284
1285 python do_foo_append() {
1286 bb.plain("third")
1287 }
1288 </literallayout>
1289 Running <filename>do_foo</filename> prints
1290 the following:
1291 <literallayout class='monospaced'>
1292 recipename do_foo: first
1293 recipename do_foo: second
1294 recipename do_foo: third
1295 </literallayout>
1296 You can use the <filename>bitbake -e</filename>&nbsp;<replaceable>recipename</replaceable>
1297 command to view the final assembled function
1298 after all overrides have been applied.
1299 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001300 </section>
1301
1302 <section id='python-functions'>
1303 <title>Python Functions</title>
1304
1305 <para>
1306 These functions are written in Python and are executed by
1307 other Python code.
1308 Examples of Python functions are utility functions
1309 that you intend to call from in-line Python or
1310 from within other Python functions.
1311 Here is an example:
1312 <literallayout class='monospaced'>
1313 def get_depends(d):
Brad Bishop6e60e8b2018-02-01 10:27:11 -05001314 if d.getVar('SOMECONDITION'):
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001315 return "dependencywithcond"
1316 else:
1317 return "dependency"
1318 SOMECONDITION = "1"
1319 DEPENDS = "${@get_depends(d)}"
1320 </literallayout>
1321 This would result in <filename>DEPENDS</filename>
1322 containing <filename>dependencywithcond</filename>.
1323 </para>
1324
1325 <para>
1326 Here are some things to know about Python functions:
1327 <itemizedlist>
1328 <listitem><para>Python functions can take parameters.
1329 </para></listitem>
1330 <listitem><para>The BitBake datastore is not
1331 automatically available.
1332 Consequently, you must pass it in as a
1333 parameter to the function.
1334 </para></listitem>
1335 <listitem><para>The "bb" and "os" Python modules are
1336 automatically available.
1337 You do not need to import them.
1338 </para></listitem>
1339 </itemizedlist>
1340 </para>
1341 </section>
1342
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001343 <section id='bitbake-style-python-functions-versus-python-functions'>
1344 <title>Bitbake-Style Python Functions Versus Python Functions</title>
1345
1346 <para>
1347 Following are some important differences between
1348 BitBake-style Python functions and regular Python
1349 functions defined with "def":
1350 <itemizedlist>
1351 <listitem><para>
1352 Only BitBake-style Python functions can be
1353 <link linkend='tasks'>tasks</link>.
1354 </para></listitem>
1355 <listitem><para>
1356 Overrides and override-style operators can only
1357 be applied to BitBake-style Python functions.
1358 </para></listitem>
1359 <listitem><para>
1360 Only regular Python functions can take arguments
1361 and return values.
1362 </para></listitem>
1363 <listitem><para>
1364 <link linkend='variable-flags'>Variable flags</link>
1365 such as <filename>[dirs]</filename>,
1366 <filename>[cleandirs]</filename>, and
1367 <filename>[lockfiles]</filename> can be used
1368 on BitBake-style Python functions, but not on
1369 regular Python functions.
1370 </para></listitem>
1371 <listitem><para>
1372 BitBake-style Python functions generate a separate
1373 <filename>${</filename><link linkend='var-T'><filename>T</filename></link><filename>}/run.</filename><replaceable>function-name</replaceable><filename>.</filename><replaceable>pid</replaceable>
1374 script that is executed to run the function, and also
1375 generate a log file in
1376 <filename>${T}/log.</filename><replaceable>function-name</replaceable><filename>.</filename><replaceable>pid</replaceable>
1377 if they are executed as tasks.</para>
1378
1379 <para>
1380 Regular Python functions execute "inline" and do not
1381 generate any files in <filename>${T}</filename>.
1382 </para></listitem>
1383 <listitem><para>
1384 Regular Python functions are called with the usual
1385 Python syntax.
1386 BitBake-style Python functions are usually tasks and
1387 are called directly by BitBake, but can also be called
1388 manually from Python code by using the
1389 <filename>bb.build.exec_func()</filename> function.
1390 Here is an example:
1391 <literallayout class='monospaced'>
1392 bb.build.exec_func("my_bitbake_style_function", d)
1393 </literallayout>
1394 <note>
1395 <filename>bb.build.exec_func()</filename> can also
1396 be used to run shell functions from Python code.
1397 If you want to run a shell function before a Python
1398 function within the same task, then you can use a
1399 parent helper Python function that starts by running
1400 the shell function with
1401 <filename>bb.build.exec_func()</filename> and then
1402 runs the Python code.
1403 </note></para>
1404
1405 <para>To detect errors from functions executed with
1406 <filename>bb.build.exec_func()</filename>, you
1407 can catch the <filename>bb.build.FuncFailed</filename>
1408 exception.
1409 <note>
1410 Functions in metadata (recipes and classes) should
1411 not themselves raise
1412 <filename>bb.build.FuncFailed</filename>.
1413 Rather, <filename>bb.build.FuncFailed</filename>
1414 should be viewed as a general indicator that the
1415 called function failed by raising an exception.
1416 For example, an exception raised by
1417 <filename>bb.fatal()</filename> will be caught inside
1418 <filename>bb.build.exec_func()</filename>, and a
1419 <filename>bb.build.FuncFailed</filename> will be raised
1420 in response.
1421 </note>
1422 </para></listitem>
1423 </itemizedlist>
1424 </para>
1425
1426 <para>
1427 Due to their simplicity, you should prefer regular Python functions
1428 over BitBake-style Python functions unless you need a feature specific
1429 to BitBake-style Python functions.
1430 Regular Python functions in metadata are a more recent invention than
1431 BitBake-style Python functions, and older code tends to use
1432 <filename>bb.build.exec_func()</filename> more often.
1433 </para>
1434 </section>
1435
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001436 <section id='anonymous-python-functions'>
1437 <title>Anonymous Python Functions</title>
1438
1439 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001440 Sometimes it is useful to set variables or perform
1441 other operations programmatically during parsing.
1442 To do this, you can define special Python functions,
1443 called anonymous Python functions, that run at the
1444 end of parsing.
1445 For example, the following conditionally sets a variable
1446 based on the value of another variable:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001447 <literallayout class='monospaced'>
1448 python () {
Brad Bishop6e60e8b2018-02-01 10:27:11 -05001449 if d.getVar('SOMEVAR') == 'value':
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001450 d.setVar('ANOTHERVAR', 'value2')
1451 }
1452 </literallayout>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001453 An equivalent way to mark a function as an anonymous
1454 function is to give it the name "__anonymous", rather
1455 than no name.
1456 </para>
1457
1458 <para>
1459 Anonymous Python functions always run at the end
1460 of parsing, regardless of where they are defined.
1461 If a recipe contains many anonymous functions, they
1462 run in the same order as they are defined within the
1463 recipe.
1464 As an example, consider the following snippet:
1465 <literallayout class='monospaced'>
1466 python () {
1467 d.setVar('FOO', 'foo 2')
1468 }
1469
1470 FOO = "foo 1"
1471
1472 python () {
1473 d.appendVar('BAR', ' bar 2')
1474 }
1475
1476 BAR = "bar 1"
1477 </literallayout>
1478 The previous example is conceptually equivalent to the
1479 following snippet:
1480 <literallayout class='monospaced'>
1481 FOO = "foo 1"
1482 BAR = "bar 1"
1483 FOO = "foo 2"
1484 BAR += "bar 2"
1485 </literallayout>
1486 <filename>FOO</filename> ends up with the value "foo 2",
1487 and <filename>BAR</filename> with the value "bar 1 bar 2".
1488 Just as in the second snippet, the values set for the
1489 variables within the anonymous functions become available
1490 to tasks, which always run after parsing.
1491 </para>
1492
1493 <para>
1494 Overrides and override-style operators such as
1495 "<filename>_append</filename>" are applied before
1496 anonymous functions run.
1497 In the following example, <filename>FOO</filename> ends
1498 up with the value "foo from anonymous":
1499 <literallayout class='monospaced'>
1500 FOO = "foo"
1501 FOO_append = " from outside"
1502
1503 python () {
1504 d.setVar("FOO", "foo from anonymous")
1505 }
1506 </literallayout>
1507 For methods you can use with anonymous Python functions,
1508 see the
1509 "<link linkend='functions-you-can-call-from-within-python'>Functions You Can Call From Within Python</link>"
1510 section.
1511 For a different method to run Python code during parsing,
1512 see the
1513 "<link linkend='inline-python-variable-expansion'>Inline Python Variable Expansion</link>"
1514 section.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001515 </para>
1516 </section>
1517
1518 <section id='flexible-inheritance-for-class-functions'>
1519 <title>Flexible Inheritance for Class Functions</title>
1520
1521 <para>
1522 Through coding techniques and the use of
1523 <filename>EXPORT_FUNCTIONS</filename>, BitBake supports
1524 exporting a function from a class such that the
1525 class function appears as the default implementation
1526 of the function, but can still be called if a recipe
1527 inheriting the class needs to define its own version of
1528 the function.
1529 </para>
1530
1531 <para>
1532 To understand the benefits of this feature, consider
1533 the basic scenario where a class defines a task function
1534 and your recipe inherits the class.
1535 In this basic scenario, your recipe inherits the task
1536 function as defined in the class.
1537 If desired, your recipe can add to the start and end of the
1538 function by using the "_prepend" or "_append" operations
1539 respectively, or it can redefine the function completely.
1540 However, if it redefines the function, there is
1541 no means for it to call the class version of the function.
1542 <filename>EXPORT_FUNCTIONS</filename> provides a mechanism
1543 that enables the recipe's version of the function to call
1544 the original version of the function.
1545 </para>
1546
1547 <para>
1548 To make use of this technique, you need the following
1549 things in place:
1550 <itemizedlist>
1551 <listitem><para>
1552 The class needs to define the function as follows:
1553 <literallayout class='monospaced'>
1554 <replaceable>classname</replaceable><filename>_</filename><replaceable>functionname</replaceable>
1555 </literallayout>
1556 For example, if you have a class file
1557 <filename>bar.bbclass</filename> and a function named
1558 <filename>do_foo</filename>, the class must define the function
1559 as follows:
1560 <literallayout class='monospaced'>
1561 bar_do_foo
1562 </literallayout>
1563 </para></listitem>
1564 <listitem><para>
1565 The class needs to contain the <filename>EXPORT_FUNCTIONS</filename>
1566 statement as follows:
1567 <literallayout class='monospaced'>
1568 EXPORT_FUNCTIONS <replaceable>functionname</replaceable>
1569 </literallayout>
1570 For example, continuing with the same example, the
1571 statement in the <filename>bar.bbclass</filename> would be
1572 as follows:
1573 <literallayout class='monospaced'>
1574 EXPORT_FUNCTIONS do_foo
1575 </literallayout>
1576 </para></listitem>
1577 <listitem><para>
1578 You need to call the function appropriately from within your
1579 recipe.
1580 Continuing with the same example, if your recipe
1581 needs to call the class version of the function,
1582 it should call <filename>bar_do_foo</filename>.
1583 Assuming <filename>do_foo</filename> was a shell function
1584 and <filename>EXPORT_FUNCTIONS</filename> was used as above,
1585 the recipe's function could conditionally call the
1586 class version of the function as follows:
1587 <literallayout class='monospaced'>
1588 do_foo() {
1589 if [ somecondition ] ; then
1590 bar_do_foo
1591 else
1592 # Do something else
1593 fi
1594 }
1595 </literallayout>
1596 To call your modified version of the function as defined
1597 in your recipe, call it as <filename>do_foo</filename>.
1598 </para></listitem>
1599 </itemizedlist>
1600 With these conditions met, your single recipe
1601 can freely choose between the original function
1602 as defined in the class file and the modified function in your recipe.
1603 If you do not set up these conditions, you are limited to using one function
1604 or the other.
1605 </para>
1606 </section>
1607 </section>
1608
1609 <section id='tasks'>
1610 <title>Tasks</title>
1611
1612 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001613 Tasks are BitBake execution units that make up the
1614 steps that BitBake can run for a given recipe.
1615 Tasks are only supported in recipes and classes
1616 (i.e. in <filename>.bb</filename> files and files
1617 included or inherited from <filename>.bb</filename>
1618 files).
1619 By convention, tasks have names that start with "do_".
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001620 </para>
1621
1622 <section id='promoting-a-function-to-a-task'>
1623 <title>Promoting a Function to a Task</title>
1624
1625 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001626 Tasks are either
1627 <link linkend='shell-functions'>shell functions</link> or
1628 <link linkend='bitbake-style-python-functions'>BitBake-style Python functions</link>
1629 that have been promoted to tasks by using the
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001630 <filename>addtask</filename> command.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001631 The <filename>addtask</filename> command can also
1632 optionally describe dependencies between the
1633 task and other tasks.
1634 Here is an example that shows how to define a task
1635 and declare some dependencies:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001636 <literallayout class='monospaced'>
1637 python do_printdate () {
1638 import time
1639 print time.strftime('%Y%m%d', time.gmtime())
1640 }
1641 addtask printdate after do_fetch before do_build
1642 </literallayout>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001643 The first argument to <filename>addtask</filename>
1644 is the name of the function to promote to
1645 a task.
1646 If the name does not start with "do_", "do_" is
1647 implicitly added, which enforces the convention that
1648 all task names start with "do_".
1649 </para>
1650
1651 <para>
1652 In the previous example, the
1653 <filename>do_printdate</filename> task becomes a
1654 dependency of the <filename>do_build</filename>
1655 task, which is the default task (i.e. the task run by
1656 the <filename>bitbake</filename> command unless
1657 another task is specified explicitly).
1658 Additionally, the <filename>do_printdate</filename>
1659 task becomes dependent upon the
1660 <filename>do_fetch</filename> task.
1661 Running the <filename>do_build</filename> task
1662 results in the <filename>do_printdate</filename>
1663 task running first.
1664 <note>
1665 If you try out the previous example, you might see that
1666 the <filename>do_printdate</filename> task is only run
1667 the first time you build the recipe with
1668 the <filename>bitbake</filename> command.
1669 This is because BitBake considers the task "up-to-date"
1670 after that initial run.
1671 If you want to force the task to always be rerun for
1672 experimentation purposes, you can make BitBake always
1673 consider the task "out-of-date" by using the
1674 <filename>[</filename><link linkend='variable-flags'><filename>nostamp</filename></link><filename>]</filename>
1675 variable flag, as follows:
1676 <literallayout class='monospaced'>
1677 do_printdate[nostamp] = "1"
1678 </literallayout>
1679 You can also explicitly run the task and provide the
1680 <filename>-f</filename> option as follows:
1681 <literallayout class='monospaced'>
1682 $ bitbake <replaceable>recipe</replaceable> -c printdate -f
1683 </literallayout>
1684 When manually selecting a task to run with the
1685 <filename>bitbake</filename>&nbsp;<replaceable>recipe</replaceable>&nbsp;<filename>-c</filename>&nbsp;<replaceable>task</replaceable>
1686 command, you can omit the "do_" prefix as part of the
1687 task name.
1688 </note>
1689 </para>
1690
1691 <para>
1692 You might wonder about the practical effects of using
1693 <filename>addtask</filename> without specifying any
1694 dependencies as is done in the following example:
1695 <literallayout class='monospaced'>
1696 addtask printdate
1697 </literallayout>
1698 In this example, assuming dependencies have not been
1699 added through some other means, the only way to run
1700 the task is by explicitly selecting it with
1701 <filename>bitbake</filename>&nbsp;<replaceable>recipe</replaceable>&nbsp;<filename>-c printdate</filename>.
1702 You can use the
1703 <filename>do_listtasks</filename> task to list all tasks
1704 defined in a recipe as shown in the following example:
1705 <literallayout class='monospaced'>
1706 $ bitbake <replaceable>recipe</replaceable> -c listtasks
1707 </literallayout>
1708 For more information on task dependencies, see the
1709 "<link linkend='dependencies'>Dependencies</link>"
1710 section.
1711 </para>
1712
1713 <para>
1714 See the
1715 "<link linkend='variable-flags'>Variable Flags</link>"
1716 section for information on variable flags you can use with
1717 tasks.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001718 </para>
1719 </section>
1720
1721 <section id='deleting-a-task'>
1722 <title>Deleting a Task</title>
1723
1724 <para>
1725 As well as being able to add tasks, you can delete them.
1726 Simply use the <filename>deltask</filename> command to
1727 delete a task.
1728 For example, to delete the example task used in the previous
1729 sections, you would use:
1730 <literallayout class='monospaced'>
1731 deltask printdate
1732 </literallayout>
1733 If you delete a task using the <filename>deltask</filename>
1734 command and the task has dependencies, the dependencies are
1735 not reconnected.
1736 For example, suppose you have three tasks named
1737 <filename>do_a</filename>, <filename>do_b</filename>, and
1738 <filename>do_c</filename>.
1739 Furthermore, <filename>do_c</filename> is dependent on
1740 <filename>do_b</filename>, which in turn is dependent on
1741 <filename>do_a</filename>.
1742 Given this scenario, if you use <filename>deltask</filename>
1743 to delete <filename>do_b</filename>, the implicit dependency
1744 relationship between <filename>do_c</filename> and
1745 <filename>do_a</filename> through <filename>do_b</filename>
1746 no longer exists, and <filename>do_c</filename> dependencies
1747 are not updated to include <filename>do_a</filename>.
1748 Thus, <filename>do_c</filename> is free to run before
1749 <filename>do_a</filename>.
1750 </para>
1751
1752 <para>
1753 If you want dependencies such as these to remain intact, use
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001754 the <filename>[noexec]</filename> varflag to disable the task
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001755 instead of using the <filename>deltask</filename> command to
1756 delete it:
1757 <literallayout class='monospaced'>
1758 do_b[noexec] = "1"
1759 </literallayout>
1760 </para>
1761 </section>
1762
1763 <section id='passing-information-into-the-build-task-environment'>
1764 <title>Passing Information Into the Build Task Environment</title>
1765
1766 <para>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001767 When running a task, BitBake tightly controls the shell execution
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001768 environment of the build tasks to make
1769 sure unwanted contamination from the build machine cannot
1770 influence the build.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001771 <note>
1772 By default, BitBake cleans the environment to include only those
1773 things exported or listed in its whitelist to ensure that the build
1774 environment is reproducible and consistent.
1775 You can prevent this "cleaning" by setting the
1776 <link linkend='var-BB_PRESERVE_ENV'><filename>BB_PRESERVE_ENV</filename></link>
1777 variable.
1778 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001779 Consequently, if you do want something to get passed into the
1780 build task environment, you must take these two steps:
1781 <orderedlist>
1782 <listitem><para>
1783 Tell BitBake to load what you want from the environment
1784 into the datastore.
1785 You can do so through the
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001786 <link linkend='var-BB_ENV_WHITELIST'><filename>BB_ENV_WHITELIST</filename></link>
1787 and
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001788 <link linkend='var-BB_ENV_EXTRAWHITE'><filename>BB_ENV_EXTRAWHITE</filename></link>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001789 variables.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001790 For example, assume you want to prevent the build system from
1791 accessing your <filename>$HOME/.ccache</filename>
1792 directory.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001793 The following command "whitelists" the environment variable
1794 <filename>CCACHE_DIR</filename> causing BitBack to allow that
1795 variable into the datastore:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001796 <literallayout class='monospaced'>
1797 export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE CCACHE_DIR"
1798 </literallayout></para></listitem>
1799 <listitem><para>
1800 Tell BitBake to export what you have loaded into the
1801 datastore to the task environment of every running task.
1802 Loading something from the environment into the datastore
1803 (previous step) only makes it available in the datastore.
1804 To export it to the task environment of every running task,
1805 use a command similar to the following in your local configuration
1806 file <filename>local.conf</filename> or your
1807 distribution configuration file:
1808 <literallayout class='monospaced'>
1809 export CCACHE_DIR
1810 </literallayout>
1811 <note>
1812 A side effect of the previous steps is that BitBake
1813 records the variable as a dependency of the build process
1814 in things like the setscene checksums.
1815 If doing so results in unnecessary rebuilds of tasks, you can
1816 whitelist the variable so that the setscene code
1817 ignores the dependency when it creates checksums.
1818 </note></para></listitem>
1819 </orderedlist>
1820 </para>
1821
1822 <para>
1823 Sometimes, it is useful to be able to obtain information
1824 from the original execution environment.
1825 Bitbake saves a copy of the original environment into
1826 a special variable named
1827 <link linkend='var-BB_ORIGENV'><filename>BB_ORIGENV</filename></link>.
1828 </para>
1829
1830 <para>
1831 The <filename>BB_ORIGENV</filename> variable returns a datastore
1832 object that can be queried using the standard datastore operators
1833 such as <filename>getVar(, False)</filename>.
1834 The datastore object is useful, for example, to find the original
1835 <filename>DISPLAY</filename> variable.
1836 Here is an example:
1837 <literallayout class='monospaced'>
1838 origenv = d.getVar("BB_ORIGENV", False)
1839 bar = origenv.getVar("BAR", False)
1840 </literallayout>
1841 The previous example returns <filename>BAR</filename> from the original
1842 execution environment.
1843 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001844 </section>
1845 </section>
1846
1847 <section id='variable-flags'>
1848 <title>Variable Flags</title>
1849
1850 <para>
1851 Variable flags (varflags) help control a task's functionality
1852 and dependencies.
1853 BitBake reads and writes varflags to the datastore using the following
1854 command forms:
1855 <literallayout class='monospaced'>
1856 <replaceable>variable</replaceable> = d.getVarFlags("<replaceable>variable</replaceable>")
1857 self.d.setVarFlags("FOO", {"func": True})
1858 </literallayout>
1859 </para>
1860
1861 <para>
1862 When working with varflags, the same syntax, with the exception of
1863 overrides, applies.
1864 In other words, you can set, append, and prepend varflags just like
1865 variables.
1866 See the
1867 "<link linkend='variable-flag-syntax'>Variable Flag Syntax</link>"
1868 section for details.
1869 </para>
1870
1871 <para>
1872 BitBake has a defined set of varflags available for recipes and
1873 classes.
1874 Tasks support a number of these flags which control various
1875 functionality of the task:
1876 <itemizedlist>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001877 <listitem><para><emphasis><filename>[cleandirs]</filename>:</emphasis>
1878 Empty directories that should be created before the
1879 task runs.
1880 Directories that already exist are removed and recreated
1881 to empty them.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001882 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001883 <listitem><para><emphasis><filename>[depends]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001884 Controls inter-task dependencies.
1885 See the
1886 <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link>
1887 variable and the
1888 "<link linkend='inter-task-dependencies'>Inter-Task Dependencies</link>"
1889 section for more information.
1890 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001891 <listitem><para><emphasis><filename>[deptask]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001892 Controls task build-time dependencies.
1893 See the
1894 <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link>
1895 variable and the
1896 "<link linkend='build-dependencies'>Build Dependencies</link>"
1897 section for more information.
1898 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001899 <listitem><para><emphasis><filename>[dirs]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001900 Directories that should be created before the task runs.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001901 Directories that already exist are left as is.
1902 The last directory listed is used as the
1903 current working directory for the task.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001904 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001905 <listitem><para><emphasis><filename>[lockfiles]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001906 Specifies one or more lockfiles to lock while the task
1907 executes.
1908 Only one task may hold a lockfile, and any task that
1909 attempts to lock an already locked file will block until
1910 the lock is released.
1911 You can use this variable flag to accomplish mutual
1912 exclusion.
1913 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001914 <listitem><para><emphasis><filename>[noexec]</filename>:</emphasis>
1915 When set to "1", marks the task as being empty, with
1916 no execution required.
1917 You can use the <filename>[noexec]</filename> flag to set up
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001918 tasks as dependency placeholders, or to disable tasks defined
1919 elsewhere that are not needed in a particular recipe.
1920 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001921 <listitem><para><emphasis><filename>[nostamp]</filename>:</emphasis>
1922 When set to "1", tells BitBake to not generate a stamp
1923 file for a task, which implies the task should always
1924 be executed.
1925 <note><title>Caution</title>
1926 Any task that depends (possibly indirectly) on a
1927 <filename>[nostamp]</filename> task will always be
1928 executed as well.
1929 This can cause unnecessary rebuilding if you are
1930 not careful.
1931 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001932 </para></listitem>
Brad Bishop1a4b7ee2018-12-16 17:11:34 -08001933 <listitem><para><emphasis><filename>[number_threads]</filename>:</emphasis>
1934 Limits tasks to a specific number of simultaneous threads
1935 during execution.
1936 This varflag is useful when your build host has a large number
1937 of cores but certain tasks need to be rate-limited due to various
1938 kinds of resource constraints (e.g. to avoid network throttling).
1939 <filename>number_threads</filename> works similarly to the
1940 <link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename></link>
1941 variable but is task-specific.</para>
1942
1943 <para>Set the value globally.
1944 For example, the following makes sure the
1945 <filename>do_fetch</filename> task uses no more than two
1946 simultaneous execution threads:
1947 <literallayout class='monospaced'>
1948 do_fetch[number_threads] = "2"
1949 </literallayout>
1950 <note><title>Warnings</title>
1951 <itemizedlist>
1952 <listitem><para>
1953 Setting the varflag in individual recipes rather
1954 than globally can result in unpredictable behavior.
1955 </para></listitem>
1956 <listitem><para>
1957 Setting the varflag to a value greater than the
1958 value used in the <filename>BB_NUMBER_THREADS</filename>
1959 variable causes <filename>number_threads</filename>
1960 to have no effect.
1961 </para></listitem>
1962 </itemizedlist>
1963 </note>
1964 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001965 <listitem><para><emphasis><filename>[postfuncs]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001966 List of functions to call after the completion of the task.
1967 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001968 <listitem><para><emphasis><filename>[prefuncs]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001969 List of functions to call before the task executes.
1970 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001971 <listitem><para><emphasis><filename>[rdepends]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001972 Controls inter-task runtime dependencies.
1973 See the
1974 <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link>
1975 variable, the
1976 <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link>
1977 variable, and the
1978 "<link linkend='inter-task-dependencies'>Inter-Task Dependencies</link>"
1979 section for more information.
1980 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001981 <listitem><para><emphasis><filename>[rdeptask]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001982 Controls task runtime dependencies.
1983 See the
1984 <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link>
1985 variable, the
1986 <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link>
1987 variable, and the
1988 "<link linkend='runtime-dependencies'>Runtime Dependencies</link>"
1989 section for more information.
1990 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001991 <listitem><para><emphasis><filename>[recideptask]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001992 When set in conjunction with
1993 <filename>recrdeptask</filename>, specifies a task that
1994 should be inspected for additional dependencies.
1995 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001996 <listitem><para><emphasis><filename>[recrdeptask]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001997 Controls task recursive runtime dependencies.
1998 See the
1999 <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link>
2000 variable, the
2001 <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link>
2002 variable, and the
2003 "<link linkend='recursive-dependencies'>Recursive Dependencies</link>"
2004 section for more information.
2005 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002006 <listitem><para><emphasis><filename>[stamp-extra-info]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002007 Extra stamp information to append to the task's stamp.
2008 As an example, OpenEmbedded uses this flag to allow
2009 machine-specific tasks.
2010 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002011 <listitem><para><emphasis><filename>[umask]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002012 The umask to run the task under.
2013 </para></listitem>
2014 </itemizedlist>
2015 </para>
2016
2017 <para>
2018 Several varflags are useful for controlling how signatures are
2019 calculated for variables.
2020 For more information on this process, see the
2021 "<link linkend='checksums'>Checksums (Signatures)</link>"
2022 section.
2023 <itemizedlist>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002024 <listitem><para><emphasis><filename>[vardeps]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002025 Specifies a space-separated list of additional
2026 variables to add to a variable's dependencies
2027 for the purposes of calculating its signature.
2028 Adding variables to this list is useful, for example, when
2029 a function refers to a variable in a manner that
2030 does not allow BitBake to automatically determine
2031 that the variable is referred to.
2032 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002033 <listitem><para><emphasis><filename>[vardepsexclude]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002034 Specifies a space-separated list of variables
2035 that should be excluded from a variable's dependencies
2036 for the purposes of calculating its signature.
2037 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002038 <listitem><para><emphasis><filename>[vardepvalue]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002039 If set, instructs BitBake to ignore the actual
2040 value of the variable and instead use the specified
2041 value when calculating the variable's signature.
2042 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002043 <listitem><para><emphasis><filename>[vardepvalueexclude]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002044 Specifies a pipe-separated list of strings to exclude
2045 from the variable's value when calculating the
2046 variable's signature.
2047 </para></listitem>
2048 </itemizedlist>
2049 </para>
2050 </section>
2051
2052 <section id='events'>
2053 <title>Events</title>
2054
2055 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002056 BitBake allows installation of event handlers within recipe
2057 and class files.
2058 Events are triggered at certain points during operation, such
2059 as the beginning of operation against a given recipe
2060 (i.e. <filename>*.bb</filename>), the start of a given task,
2061 a task failure, a task success, and so forth.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002062 The intent is to make it easy to do things like email
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002063 notification on build failures.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002064 </para>
2065
2066 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002067 Following is an example event handler that prints the name
2068 of the event and the content of the
2069 <filename>FILE</filename> variable:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002070 <literallayout class='monospaced'>
2071 addhandler myclass_eventhandler
2072 python myclass_eventhandler() {
2073 from bb.event import getName
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002074 print("The name of the Event is %s" % getName(e))
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002075 print("The file we run for is %s" % d.getVar('FILE'))
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002076 }
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002077 myclass_eventhandler[eventmask] = "bb.event.BuildStarted bb.event.BuildCompleted"
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002078 </literallayout>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002079 In the previous example, an eventmask has been set so that
2080 the handler only sees the "BuildStarted" and "BuildCompleted"
2081 events.
2082 This event handler gets called every time an event matching
2083 the eventmask is triggered.
2084 A global variable "e" is defined, which represents the current
2085 event.
2086 With the <filename>getName(e)</filename> method, you can get
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002087 the name of the triggered event.
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002088 The global datastore is available as "d".
2089 In legacy code, you might see "e.data" used to get the datastore.
2090 However, realize that "e.data" is deprecated and you should use
2091 "d" going forward.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002092 </para>
2093
2094 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002095 The context of the datastore is appropriate to the event
2096 in question.
2097 For example, "BuildStarted" and "BuildCompleted" events run
2098 before any tasks are executed so would be in the global
2099 configuration datastore namespace.
2100 No recipe-specific metadata exists in that namespace.
Brad Bishopd7bf8c12018-02-25 22:55:05 -05002101 The "BuildStarted" and "BuildCompleted" events also run in
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002102 the main cooker/server process rather than any worker context.
2103 Thus, any changes made to the datastore would be seen by other
2104 cooker/server events within the current build but not seen
2105 outside of that build or in any worker context.
2106 Task events run in the actual tasks in question consequently
2107 have recipe-specific and task-specific contents.
2108 These events run in the worker context and are discarded at
2109 the end of task execution.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002110 </para>
2111
2112 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002113 During a standard build, the following common events might
2114 occur.
2115 The following events are the most common kinds of events that
2116 most metadata might have an interest in viewing:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002117 <itemizedlist>
2118 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002119 <filename>bb.event.ConfigParsed()</filename>:
2120 Fired when the base configuration; which consists of
2121 <filename>bitbake.conf</filename>,
2122 <filename>base.bbclass</filename> and any global
2123 <filename>INHERIT</filename> statements; has been parsed.
2124 You can see multiple such events when each of the
2125 workers parse the base configuration or if the server
2126 changes configuration and reparses.
2127 Any given datastore only has one such event executed
2128 against it, however.
2129 If
2130 <link linkende='var-BB_INVALIDCONF'><filename>BB_INVALIDCONF</filename></link>
2131 is set in the datastore by the event handler, the
2132 configuration is reparsed and a new event triggered,
2133 allowing the metadata to update configuration.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002134 </para></listitem>
2135 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002136 <filename>bb.event.HeartbeatEvent()</filename>:
2137 Fires at regular time intervals of one second.
2138 You can configure the interval time using the
2139 <filename>BB_HEARTBEAT_EVENT</filename> variable.
2140 The event's "time" attribute is the
2141 <filename>time.time()</filename> value when the
2142 event is triggered.
2143 This event is useful for activities such as
2144 system state monitoring.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002145 </para></listitem>
2146 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002147 <filename>bb.event.ParseStarted()</filename>:
2148 Fired when BitBake is about to start parsing recipes.
2149 This event's "total" attribute represents the number of
2150 recipes BitBake plans to parse.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002151 </para></listitem>
2152 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002153 <filename>bb.event.ParseProgress()</filename>:
2154 Fired as parsing progresses.
2155 This event's "current" attribute is the number of
2156 recipes parsed as well as the "total" attribute.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002157 </para></listitem>
2158 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002159 <filename>bb.event.ParseCompleted()</filename>:
2160 Fired when parsing is complete.
2161 This event's "cached", "parsed", "skipped", "virtuals",
2162 "masked", and "errors" attributes provide statistics
2163 for the parsing results.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002164 </para></listitem>
2165 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002166 <filename>bb.event.BuildStarted()</filename>:
2167 Fired when a new build starts.
Brad Bishop316dfdd2018-06-25 12:45:53 -04002168 BitBake fires multiple "BuildStarted" events (one per configuration)
2169 when multiple configuration (multiconfig) is enabled.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002170 </para></listitem>
2171 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002172 <filename>bb.build.TaskStarted()</filename>:
2173 Fired when a task starts.
2174 This event's "taskfile" attribute points to the recipe
2175 from which the task originates.
2176 The "taskname" attribute, which is the task's name,
2177 includes the <filename>do_</filename> prefix, and the
2178 "logfile" attribute point to where the task's output is
2179 stored.
2180 Finally, the "time" attribute is the task's execution start
2181 time.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002182 </para></listitem>
2183 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002184 <filename>bb.build.TaskInvalid()</filename>:
2185 Fired if BitBake tries to execute a task that does not exist.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002186 </para></listitem>
2187 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002188 <filename>bb.build.TaskFailedSilent()</filename>:
2189 Fired for setscene tasks that fail and should not be
2190 presented to the user verbosely.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002191 </para></listitem>
2192 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002193 <filename>bb.build.TaskFailed()</filename>:
2194 Fired for normal tasks that fail.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002195 </para></listitem>
2196 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002197 <filename>bb.build.TaskSucceeded()</filename>:
2198 Fired when a task successfully completes.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002199 </para></listitem>
2200 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002201 <filename>bb.event.BuildCompleted()</filename>:
2202 Fired when a build finishes.
2203 </para></listitem>
2204 <listitem><para>
2205 <filename>bb.cooker.CookerExit()</filename>:
2206 Fired when the BitBake server/cooker shuts down.
2207 This event is usually only seen by the UIs as a
2208 sign they should also shutdown.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002209 </para></listitem>
2210 </itemizedlist>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002211 </para>
2212
2213 <para>
2214 This next list of example events occur based on specific
2215 requests to the server.
2216 These events are often used to communicate larger pieces of
2217 information from the BitBake server to other parts of
2218 BitBake such as user interfaces:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002219 <itemizedlist>
2220 <listitem><para>
2221 <filename>bb.event.TreeDataPreparationStarted()</filename>
2222 </para></listitem>
2223 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002224 <filename>bb.event.TreeDataPreparationProgress()</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002225 </para></listitem>
2226 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002227 <filename>bb.event.TreeDataPreparationCompleted()</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002228 </para></listitem>
2229 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002230 <filename>bb.event.DepTreeGenerated()</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002231 </para></listitem>
2232 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002233 <filename>bb.event.CoreBaseFilesFound()</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002234 </para></listitem>
2235 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002236 <filename>bb.event.ConfigFilePathFound()</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002237 </para></listitem>
2238 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002239 <filename>bb.event.FilesMatchingFound()</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002240 </para></listitem>
2241 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002242 <filename>bb.event.ConfigFilesFound()</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002243 </para></listitem>
2244 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002245 <filename>bb.event.TargetsTreeGenerated()</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002246 </para></listitem>
2247 </itemizedlist>
2248 </para>
2249 </section>
2250
2251 <section id='variants-class-extension-mechanism'>
2252 <title>Variants - Class Extension Mechanism</title>
2253
2254 <para>
2255 BitBake supports two features that facilitate creating
2256 from a single recipe file multiple incarnations of that
2257 recipe file where all incarnations are buildable.
2258 These features are enabled through the
2259 <link linkend='var-BBCLASSEXTEND'><filename>BBCLASSEXTEND</filename></link>
2260 and
2261 <link linkend='var-BBVERSIONS'><filename>BBVERSIONS</filename></link>
2262 variables.
2263 <note>
2264 The mechanism for this class extension is extremely
2265 specific to the implementation.
2266 Usually, the recipe's
2267 <link linkend='var-PROVIDES'><filename>PROVIDES</filename></link>,
2268 <link linkend='var-PN'><filename>PN</filename></link>, and
2269 <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link>
2270 variables would need to be modified by the extension class.
2271 For specific examples, see the OE-Core
2272 <filename>native</filename>, <filename>nativesdk</filename>,
2273 and <filename>multilib</filename> classes.
2274 </note>
2275 <itemizedlist>
2276 <listitem><para><emphasis><filename>BBCLASSEXTEND</filename>:</emphasis>
2277 This variable is a space separated list of classes used to "extend" the
2278 recipe for each variant.
2279 Here is an example that results in a second incarnation of the current
2280 recipe being available.
2281 This second incarnation will have the "native" class inherited.
2282 <literallayout class='monospaced'>
2283 BBCLASSEXTEND = "native"
2284 </literallayout></para></listitem>
2285 <listitem><para><emphasis><filename>BBVERSIONS</filename>:</emphasis>
2286 This variable allows a single recipe to build multiple versions of a
2287 project from a single recipe file.
2288 You can also specify conditional metadata
2289 (using the
2290 <link linkend='var-OVERRIDES'><filename>OVERRIDES</filename></link>
2291 mechanism) for a single version, or an optionally named range of versions.
2292 Here is an example:
2293 <literallayout class='monospaced'>
2294 BBVERSIONS = "1.0 2.0 git"
2295 SRC_URI_git = "git://someurl/somepath.git"
2296
2297 BBVERSIONS = "1.0.[0-6]:1.0.0+ \ 1.0.[7-9]:1.0.7+"
2298 SRC_URI_append_1.0.7+ = "file://some_patch_which_the_new_versions_need.patch;patch=1"
2299 </literallayout>
2300 The name of the range defaults to the original version of the
2301 recipe.
2302 For example, in OpenEmbedded, the recipe file
2303 <filename>foo_1.0.0+.bb</filename> creates a default name range
2304 of <filename>1.0.0+</filename>.
2305 This is useful because the range name is not only placed
2306 into overrides, but it is also made available for the metadata to use
2307 in the variable that defines the base recipe versions for use in
2308 <filename>file://</filename> search paths
2309 (<link linkend='var-FILESPATH'><filename>FILESPATH</filename></link>).
2310 </para></listitem>
2311 </itemizedlist>
2312 </para>
2313 </section>
2314
2315 <section id='dependencies'>
2316 <title>Dependencies</title>
2317
2318 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002319 To allow for efficient parallel processing, BitBake handles
2320 dependencies at the task level.
2321 Dependencies can exist both between tasks within a single recipe
2322 and between tasks in different recipes.
2323 Following are examples of each:
2324 <itemizedlist>
2325 <listitem><para>For tasks within a single recipe, a
2326 recipe's <filename>do_configure</filename>
2327 task might need to complete before its
2328 <filename>do_compile</filename> task can run.
2329 </para></listitem>
2330 <listitem><para>For tasks in different recipes, one
2331 recipe's <filename>do_configure</filename>
2332 task might require another recipe's
2333 <filename>do_populate_sysroot</filename>
2334 task to finish first such that the libraries and headers
2335 provided by the other recipe are available.
2336 </para></listitem>
2337 </itemizedlist>
2338 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002339
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002340 <para>
2341 This section describes several ways to declare dependencies.
2342 Remember, even though dependencies are declared in different ways, they
2343 are all simply dependencies between tasks.
2344 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002345
2346 <section id='dependencies-internal-to-the-bb-file'>
2347 <title>Dependencies Internal to the <filename>.bb</filename> File</title>
2348
2349 <para>
2350 BitBake uses the <filename>addtask</filename> directive
2351 to manage dependencies that are internal to a given recipe
2352 file.
2353 You can use the <filename>addtask</filename> directive to
2354 indicate when a task is dependent on other tasks or when
2355 other tasks depend on that recipe.
2356 Here is an example:
2357 <literallayout class='monospaced'>
2358 addtask printdate after do_fetch before do_build
2359 </literallayout>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002360 In this example, the <filename>do_printdate</filename>
2361 task depends on the completion of the
2362 <filename>do_fetch</filename> task, and the
2363 <filename>do_build</filename> task depends on the
2364 completion of the <filename>do_printdate</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002365 task.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002366 <note><para>
2367 For a task to run, it must be a direct or indirect
2368 dependency of some other task that is scheduled to
2369 run.</para>
2370
2371 <para>For illustration, here are some examples:
2372 <itemizedlist>
2373 <listitem><para>
2374 The directive
2375 <filename>addtask mytask before do_configure</filename>
2376 causes <filename>do_mytask</filename> to run before
2377 <filename>do_configure</filename> runs.
2378 Be aware that <filename>do_mytask</filename> still only
2379 runs if its <link linkend='checksums'>input checksum</link>
2380 has changed since the last time it was run.
2381 Changes to the input checksum of
2382 <filename>do_mytask</filename> also indirectly cause
2383 <filename>do_configure</filename> to run.
2384 </para></listitem>
2385 <listitem><para>
2386 The directive
2387 <filename>addtask mytask after do_configure</filename>
2388 by itself never causes <filename>do_mytask</filename>
2389 to run.
2390 <filename>do_mytask</filename> can still be run manually
2391 as follows:
2392 <literallayout class='monospaced'>
2393 $ bitbake <replaceable>recipe</replaceable> -c mytask
2394 </literallayout>
2395 Declaring <filename>do_mytask</filename> as a dependency
2396 of some other task that is scheduled to run also causes
2397 it to run.
2398 Regardless, the task runs after
2399 <filename>do_configure</filename>.
2400 </para></listitem>
2401 </itemizedlist></para>
2402 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002403 </para>
2404 </section>
2405
2406 <section id='build-dependencies'>
2407 <title>Build Dependencies</title>
2408
2409 <para>
2410 BitBake uses the
2411 <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link>
2412 variable to manage build time dependencies.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002413 The <filename>[deptask]</filename> varflag for tasks
2414 signifies the task of each
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002415 item listed in <filename>DEPENDS</filename> that must
2416 complete before that task can be executed.
2417 Here is an example:
2418 <literallayout class='monospaced'>
2419 do_configure[deptask] = "do_populate_sysroot"
2420 </literallayout>
2421 In this example, the <filename>do_populate_sysroot</filename>
2422 task of each item in <filename>DEPENDS</filename> must complete before
2423 <filename>do_configure</filename> can execute.
2424 </para>
2425 </section>
2426
2427 <section id='runtime-dependencies'>
2428 <title>Runtime Dependencies</title>
2429
2430 <para>
2431 BitBake uses the
2432 <link linkend='var-PACKAGES'><filename>PACKAGES</filename></link>,
2433 <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link>, and
2434 <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link>
2435 variables to manage runtime dependencies.
2436 </para>
2437
2438 <para>
2439 The <filename>PACKAGES</filename> variable lists runtime
2440 packages.
2441 Each of those packages can have <filename>RDEPENDS</filename> and
2442 <filename>RRECOMMENDS</filename> runtime dependencies.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002443 The <filename>[rdeptask]</filename> flag for tasks is used to
2444 signify the task of each
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002445 item runtime dependency which must have completed before that
2446 task can be executed.
2447 <literallayout class='monospaced'>
2448 do_package_qa[rdeptask] = "do_packagedata"
2449 </literallayout>
2450 In the previous example, the <filename>do_packagedata</filename>
2451 task of each item in <filename>RDEPENDS</filename> must have
2452 completed before <filename>do_package_qa</filename> can execute.
2453 </para>
2454 </section>
2455
2456 <section id='recursive-dependencies'>
2457 <title>Recursive Dependencies</title>
2458
2459 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002460 BitBake uses the <filename>[recrdeptask]</filename> flag to manage
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002461 recursive task dependencies.
2462 BitBake looks through the build-time and runtime
2463 dependencies of the current recipe, looks through
2464 the task's inter-task
2465 dependencies, and then adds dependencies for the
2466 listed task.
2467 Once BitBake has accomplished this, it recursively works through
2468 the dependencies of those tasks.
2469 Iterative passes continue until all dependencies are discovered
2470 and added.
2471 </para>
2472
2473 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002474 The <filename>[recrdeptask]</filename> flag is most commonly
2475 used in high-level
2476 recipes that need to wait for some task to finish "globally".
2477 For example, <filename>image.bbclass</filename> has the following:
2478 <literallayout class='monospaced'>
2479 do_rootfs[recrdeptask] += "do_packagedata"
2480 </literallayout>
2481 This statement says that the <filename>do_packagedata</filename>
2482 task of the current recipe and all recipes reachable
2483 (by way of dependencies) from the
2484 image recipe must run before the <filename>do_rootfs</filename>
2485 task can run.
2486 </para>
2487
2488 <para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002489 You might want to not only have BitBake look for
2490 dependencies of those tasks, but also have BitBake look
2491 for build-time and runtime dependencies of the dependent
2492 tasks as well.
2493 If that is the case, you need to reference the task name
2494 itself in the task list:
2495 <literallayout class='monospaced'>
2496 do_a[recrdeptask] = "do_a do_b"
2497 </literallayout>
2498 </para>
2499 </section>
2500
2501 <section id='inter-task-dependencies'>
2502 <title>Inter-Task Dependencies</title>
2503
2504 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002505 BitBake uses the <filename>[depends]</filename>
2506 flag in a more generic form
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002507 to manage inter-task dependencies.
2508 This more generic form allows for inter-dependency
2509 checks for specific tasks rather than checks for
2510 the data in <filename>DEPENDS</filename>.
2511 Here is an example:
2512 <literallayout class='monospaced'>
2513 do_patch[depends] = "quilt-native:do_populate_sysroot"
2514 </literallayout>
2515 In this example, the <filename>do_populate_sysroot</filename>
2516 task of the target <filename>quilt-native</filename>
2517 must have completed before the
2518 <filename>do_patch</filename> task can execute.
2519 </para>
2520
2521 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002522 The <filename>[rdepends]</filename> flag works in a similar
2523 way but takes targets
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002524 in the runtime namespace instead of the build-time dependency
2525 namespace.
2526 </para>
2527 </section>
2528 </section>
2529
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002530 <section id='functions-you-can-call-from-within-python'>
2531 <title>Functions You Can Call From Within Python</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002532
2533 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002534 BitBake provides many functions you can call from
2535 within Python functions.
2536 This section lists the most commonly used functions,
2537 and mentions where to find others.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002538 </para>
2539
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002540 <section id='functions-for-accessing-datastore-variables'>
2541 <title>Functions for Accessing Datastore Variables</title>
2542
2543 <para>
2544 It is often necessary to access variables in the
2545 BitBake datastore using Python functions.
2546 The Bitbake datastore has an API that allows you this
2547 access.
2548 Here is a list of available operations:
2549 </para>
2550
2551 <para>
2552 <informaltable frame='none'>
2553 <tgroup cols='2' align='left' colsep='1' rowsep='1'>
2554 <colspec colname='c1' colwidth='1*'/>
2555 <colspec colname='c2' colwidth='1*'/>
2556 <thead>
2557 <row>
2558 <entry align="left"><emphasis>Operation</emphasis></entry>
2559 <entry align="left"><emphasis>Description</emphasis></entry>
2560 </row>
2561 </thead>
2562 <tbody>
2563 <row>
2564 <entry align="left"><filename>d.getVar("X", expand)</filename></entry>
2565 <entry align="left">Returns the value of variable "X".
2566 Using "expand=True" expands the value.
2567 Returns "None" if the variable "X" does not exist.</entry>
2568 </row>
2569 <row>
2570 <entry align="left"><filename>d.setVar("X", "value")</filename></entry>
2571 <entry align="left">Sets the variable "X" to "value".</entry>
2572 </row>
2573 <row>
2574 <entry align="left"><filename>d.appendVar("X", "value")</filename></entry>
2575 <entry align="left">Adds "value" to the end of the variable "X".
2576 Acts like <filename>d.setVar("X", "value")</filename>
2577 if the variable "X" does not exist.</entry>
2578 </row>
2579 <row>
2580 <entry align="left"><filename>d.prependVar("X", "value")</filename></entry>
2581 <entry align="left">Adds "value" to the start of the variable "X".
2582 Acts like <filename>d.setVar("X", "value")</filename>
2583 if the variable "X" does not exist.</entry>
2584 </row>
2585 <row>
2586 <entry align="left"><filename>d.delVar("X")</filename></entry>
2587 <entry align="left">Deletes the variable "X" from the datastore.
2588 Does nothing if the variable "X" does not exist.</entry>
2589 </row>
2590 <row>
2591 <entry align="left"><filename>d.renameVar("X", "Y")</filename></entry>
2592 <entry align="left">Renames the variable "X" to "Y".
2593 Does nothing if the variable "X" does not exist.</entry>
2594 </row>
2595 <row>
2596 <entry align="left"><filename>d.getVarFlag("X", flag, expand)</filename></entry>
2597 <entry align="left">Returns the value of variable "X".
2598 Using "expand=True" expands the value.
2599 Returns "None" if either the variable "X" or the named flag
2600 does not exist.</entry>
2601 </row>
2602 <row>
2603 <entry align="left"><filename>d.setVarFlag("X", flag, "value")</filename></entry>
2604 <entry align="left">Sets the named flag for variable "X" to "value".</entry>
2605 </row>
2606 <row>
2607 <entry align="left"><filename>d.appendVarFlag("X", flag, "value")</filename></entry>
2608 <entry align="left">Appends "value" to the named flag on the
2609 variable "X".
2610 Acts like <filename>d.setVarFlag("X", flag, "value")</filename>
2611 if the named flag does not exist.</entry>
2612 </row>
2613 <row>
2614 <entry align="left"><filename>d.prependVarFlag("X", flag, "value")</filename></entry>
2615 <entry align="left">Prepends "value" to the named flag on
2616 the variable "X".
2617 Acts like <filename>d.setVarFlag("X", flag, "value")</filename>
2618 if the named flag does not exist.</entry>
2619 </row>
2620 <row>
2621 <entry align="left"><filename>d.delVarFlag("X", flag)</filename></entry>
2622 <entry align="left">Deletes the named flag on the variable
2623 "X" from the datastore.</entry>
2624 </row>
2625 <row>
2626 <entry align="left"><filename>d.setVarFlags("X", flagsdict)</filename></entry>
2627 <entry align="left">Sets the flags specified in
2628 the <filename>flagsdict()</filename> parameter.
2629 <filename>setVarFlags</filename> does not clear previous flags.
2630 Think of this operation as <filename>addVarFlags</filename>.</entry>
2631 </row>
2632 <row>
2633 <entry align="left"><filename>d.getVarFlags("X")</filename></entry>
2634 <entry align="left">Returns a <filename>flagsdict</filename>
2635 of the flags for the variable "X".
2636 Returns "None" if the variable "X" does not exist.</entry>
2637 </row>
2638 <row>
2639 <entry align="left"><filename>d.delVarFlags("X")</filename></entry>
2640 <entry align="left">Deletes all the flags for the variable "X".
2641 Does nothing if the variable "X" does not exist.</entry>
2642 </row>
2643 <row>
2644 <entry align="left"><filename>d.expand(expression)</filename></entry>
2645 <entry align="left">Expands variable references in the specified
2646 string expression.
2647 References to variables that do not exist are left as is.
2648 For example, <filename>d.expand("foo ${X}")</filename>
2649 expands to the literal string "foo ${X}" if the
2650 variable "X" does not exist.</entry>
2651 </row>
2652 </tbody>
2653 </tgroup>
2654 </informaltable>
2655 </para>
2656 </section>
2657
2658 <section id='other-functions'>
2659 <title>Other Functions</title>
2660
2661 <para>
2662 You can find many other functions that can be called
2663 from Python by looking at the source code of the
2664 <filename>bb</filename> module, which is in
2665 <filename>bitbake/lib/bb</filename>.
2666 For example,
2667 <filename>bitbake/lib/bb/utils.py</filename> includes
2668 the commonly used functions
2669 <filename>bb.utils.contains()</filename> and
2670 <filename>bb.utils.mkdirhier()</filename>, which come
2671 with docstrings.
2672 </para>
2673 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002674 </section>
2675
2676 <section id='task-checksums-and-setscene'>
2677 <title>Task Checksums and Setscene</title>
2678
2679 <para>
2680 BitBake uses checksums (or signatures) along with the setscene
2681 to determine if a task needs to be run.
2682 This section describes the process.
2683 To help understand how BitBake does this, the section assumes an
2684 OpenEmbedded metadata-based example.
2685 </para>
2686
2687 <para>
Brad Bishop316dfdd2018-06-25 12:45:53 -04002688 These checksums are stored in
2689 <link linkend='var-STAMP'><filename>STAMP</filename></link>.
2690 You can examine the checksums using the following BitBake command:
2691 <literallayout class='monospaced'>
2692 $ bitbake-dumpsigs
2693 </literallayout>
2694 This command returns the signature data in a readable format
2695 that allows you to examine the inputs used when the
2696 OpenEmbedded build system generates signatures.
2697 For example, using <filename>bitbake-dumpsigs</filename>
2698 allows you to examine the <filename>do_compile</filename>
2699 task's “sigdata” for a C application (e.g.
2700 <filename>bash</filename>).
2701 Running the command also reveals that the “CC” variable is part of
2702 the inputs that are hashed.
2703 Any changes to this variable would invalidate the stamp and
2704 cause the <filename>do_compile</filename> task to run.
2705 </para>
2706
2707 <para>
2708 The following list describes related variables:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002709 <itemizedlist>
Brad Bishop316dfdd2018-06-25 12:45:53 -04002710 <listitem><para>
2711 <link linkend='var-BB_HASHCHECK_FUNCTION'><filename>BB_HASHCHECK_FUNCTION</filename></link>:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002712 Specifies the name of the function to call during
2713 the "setscene" part of the task's execution in order
2714 to validate the list of task hashes.
2715 </para></listitem>
Brad Bishop316dfdd2018-06-25 12:45:53 -04002716 <listitem><para>
2717 <link linkend='var-BB_SETSCENE_DEPVALID'><filename>BB_SETSCENE_DEPVALID</filename></link>:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002718 Specifies a function BitBake calls that determines
2719 whether BitBake requires a setscene dependency to
2720 be met.
2721 </para></listitem>
Brad Bishop316dfdd2018-06-25 12:45:53 -04002722 <listitem><para>
2723 <link linkend='var-BB_SETSCENE_VERIFY_FUNCTION2'><filename>BB_SETSCENE_VERIFY_FUNCTION2</filename></link>:
2724 Specifies a function to call that verifies the list of
2725 planned task execution before the main task execution
2726 happens.
2727 </para></listitem>
2728 <listitem><para>
2729 <link linkend='var-BB_STAMP_POLICY'><filename>BB_STAMP_POLICY</filename></link>:
2730 Defines the mode for comparing timestamps of stamp files.
2731 </para></listitem>
2732 <listitem><para>
2733 <link linkend='var-BB_STAMP_WHITELIST'><filename>BB_STAMP_WHITELIST</filename></link>:
2734 Lists stamp files that are looked at when the stamp policy
2735 is "whitelist".
2736 </para></listitem>
2737 <listitem><para>
2738 <link linkend='var-BB_TASKHASH'><filename>BB_TASKHASH</filename></link>:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002739 Within an executing task, this variable holds the hash
2740 of the task as returned by the currently enabled
2741 signature generator.
2742 </para></listitem>
Brad Bishop316dfdd2018-06-25 12:45:53 -04002743 <listitem><para>
2744 <link linkend='var-STAMP'><filename>STAMP</filename></link>:
2745 The base path to create stamp files.
2746 </para></listitem>
2747 <listitem><para>
2748 <link linkend='var-STAMPCLEAN'><filename>STAMPCLEAN</filename></link>:
2749 Again, the base path to create stamp files but can use wildcards
2750 for matching a range of files for clean operations.
2751 </para></listitem>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002752 </itemizedlist>
2753 </para>
2754 </section>
Brad Bishop1a4b7ee2018-12-16 17:11:34 -08002755
2756 <section id='wildcard-support-in-variables'>
2757 <title>Wildcard Support in Variables</title>
2758
2759 <para>
2760 Support for wildcard use in variables varies depending on the
2761 context in which it is used.
2762 For example, some variables and file names allow limited use of
2763 wildcards through the "<filename>%</filename>" and
2764 "<filename>*</filename>" characters.
2765 Other variables or names support Python's
2766 <ulink url='https://docs.python.org/3/library/glob.html'><filename>glob</filename></ulink>
2767 syntax,
2768 <ulink url='https://docs.python.org/3/library/fnmatch.html#module-fnmatch'><filename>fnmatch</filename></ulink>
2769 syntax, or
2770 <ulink url='https://docs.python.org/3/library/re.html#re'><filename>Regular Expression (re)</filename></ulink>
2771 syntax.
2772 </para>
2773
2774 <para>
2775 For variables that have wildcard suport, the
2776 documentation describes which form of wildcard, its
2777 use, and its limitations.
2778 </para>
2779 </section>
2780
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002781</chapter>