blob: b4fc64e75352b61e8771679debbfa1c69bf70ea4 [file] [log] [blame]
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
3
4<chapter id="bitbake-user-manual-metadata">
5 <title>Syntax and Operators</title>
6
7 <para>
8 Bitbake files have their own syntax.
9 The syntax has similarities to several
10 other languages but also has some unique features.
11 This section describes the available syntax and operators
12 as well as provides examples.
13 </para>
14
15 <section id='basic-syntax'>
16 <title>Basic Syntax</title>
17
18 <para>
19 This section provides some basic syntax examples.
20 </para>
21
22 <section id='basic-variable-setting'>
23 <title>Basic Variable Setting</title>
24
25 <para>
26 The following example sets <filename>VARIABLE</filename> to
27 "value".
28 This assignment occurs immediately as the statement is parsed.
29 It is a "hard" assignment.
30 <literallayout class='monospaced'>
31 VARIABLE = "value"
32 </literallayout>
33 As expected, if you include leading or trailing spaces as part of
34 an assignment, the spaces are retained:
35 <literallayout class='monospaced'>
36 VARIABLE = " value"
37 VARIABLE = "value "
38 </literallayout>
39 Setting <filename>VARIABLE</filename> to "" sets it to an empty string,
40 while setting the variable to " " sets it to a blank space
41 (i.e. these are not the same values).
42 <literallayout class='monospaced'>
43 VARIABLE = ""
44 VARIABLE = " "
45 </literallayout>
46 </para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -060047
48 <para>
49 You can use single quotes instead of double quotes
50 when setting a variable's value.
51 Doing so allows you to use values that contain the double
52 quote character:
53 <literallayout class='monospaced'>
54 VARIABLE = 'I have a " in my value'
55 </literallayout>
56 <note>
57 Unlike in Bourne shells, single quotes work identically
58 to double quotes in all other ways.
59 They do not suppress variable expansions.
60 </note>
61 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -050062 </section>
63
Brad Bishop6e60e8b2018-02-01 10:27:11 -050064 <section id='line-joining'>
65 <title>Line Joining</title>
66
67 <para>
68 Outside of
69 <link linkend='functions'>functions</link>, BitBake joins
70 any line ending in a backslash character ("\")
71 with the following line before parsing statements.
72 The most common use for the "\" character is to split variable
73 assignments over multiple lines, as in the following example:
74 <literallayout class='monospaced'>
75 FOO = "bar \
76 baz \
77 qaz"
78 </literallayout>
79 Both the "\" character and the newline character
80 that follow it are removed when joining lines.
81 Thus, no newline characters end up in the value of
82 <filename>FOO</filename>.
83 </para>
84
85 <para>
86 Consider this additional example where the two
87 assignments both assign "barbaz" to
88 <filename>FOO</filename>:
89 <literallayout class='monospaced'>
90 FOO = "barbaz"
91
92 FOO = "bar\
93 baz"
94 </literallayout>
95 <note>
96 BitBake does not interpret escape sequences like
97 "\n" in variable values.
98 For these to have an effect, the value must be passed
99 to some utility that interprets escape sequences,
100 such as <filename>printf</filename> or
101 <filename>echo -n</filename>.
102 </note>
103 </para>
104 </section>
105
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500106 <section id='variable-expansion'>
107 <title>Variable Expansion</title>
108
109 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600110 Variables can reference the contents of other variables
111 using a syntax that is similar to variable expansion in
112 Bourne shells.
113 The following assignments
114 result in A containing "aval" and B evaluating to "preavalpost".
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500115 <literallayout class='monospaced'>
116 A = "aval"
117 B = "pre${A}post"
118 </literallayout>
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600119 <note>
120 Unlike in Bourne shells, the curly braces are mandatory:
121 Only <filename>${FOO}</filename> and not
122 <filename>$FOO</filename> is recognized as an expansion of
123 <filename>FOO</filename>.
124 </note>
125 The "=" operator does not immediately expand variable
126 references in the right-hand side.
127 Instead, expansion is deferred until the variable assigned to
128 is actually used.
129 The result depends on the current values of the referenced
130 variables.
131 The following example should clarify this behavior:
132 <literallayout class='monospaced'>
133 A = "${B} baz"
134 B = "${C} bar"
135 C = "foo"
136 *At this point, ${A} equals "foo bar baz"*
137 C = "qux"
138 *At this point, ${A} equals "qux bar baz"*
139 B = "norf"
140 *At this point, ${A} equals "norf baz"*
141 </literallayout>
142 Contrast this behavior with the
143 <link linkend='immediate-variable-expansion'>immediate variable expansion</link>
144 operator (i.e. ":=").
145 </para>
146
147 <para>
148 If the variable expansion syntax is used on a variable that
149 does not exist, the string is kept as is.
150 For example, given the following assignment,
151 <filename>BAR</filename> expands to the literal string
152 "${FOO}" as long as <filename>FOO</filename> does not exist.
153 <literallayout class='monospaced'>
154 BAR = "${FOO}"
155 </literallayout>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500156 </para>
157 </section>
158
159 <section id='setting-a-default-value'>
160 <title>Setting a default value (?=)</title>
161
162 <para>
163 You can use the "?=" operator to achieve a "softer" assignment
164 for a variable.
165 This type of assignment allows you to define a variable if it
166 is undefined when the statement is parsed, but to leave the
167 value alone if the variable has a value.
168 Here is an example:
169 <literallayout class='monospaced'>
170 A ?= "aval"
171 </literallayout>
172 If <filename>A</filename> is set at the time this statement is parsed,
173 the variable retains its value.
174 However, if <filename>A</filename> is not set,
175 the variable is set to "aval".
176 <note>
177 This assignment is immediate.
178 Consequently, if multiple "?=" assignments
179 to a single variable exist, the first of those ends up getting
180 used.
181 </note>
182 </para>
183 </section>
184
185 <section id='setting-a-weak-default-value'>
186 <title>Setting a weak default value (??=)</title>
187
188 <para>
189 It is possible to use a "weaker" assignment than in the
190 previous section by using the "??=" operator.
191 This assignment behaves identical to "?=" except that the
192 assignment is made at the end of the parsing process rather
193 than immediately.
194 Consequently, when multiple "??=" assignments exist, the last
195 one is used.
196 Also, any "=" or "?=" assignment will override the value set with
197 "??=".
198 Here is an example:
199 <literallayout class='monospaced'>
200 A ??= "somevalue"
201 A ??= "someothervalue"
202 </literallayout>
203 If <filename>A</filename> is set before the above statements are parsed,
204 the variable retains its value.
205 If <filename>A</filename> is not set,
206 the variable is set to "someothervalue".
207 </para>
208
209 <para>
210 Again, this assignment is a "lazy" or "weak" assignment
211 because it does not occur until the end
212 of the parsing process.
213 </para>
214 </section>
215
216 <section id='immediate-variable-expansion'>
217 <title>Immediate variable expansion (:=)</title>
218
219 <para>
220 The ":=" operator results in a variable's
221 contents being expanded immediately,
222 rather than when the variable is actually used:
223 <literallayout class='monospaced'>
224 T = "123"
225 A := "${B} ${A} test ${T}"
226 T = "456"
227 B = "${T} bval"
228 C = "cval"
229 C := "${C}append"
230 </literallayout>
231 In this example, <filename>A</filename> contains
232 "test 123" because <filename>${B}</filename> and
233 <filename>${A}</filename> at the time of parsing are undefined,
234 which leaves "test 123".
235 And, the variable <filename>C</filename>
236 contains "cvalappend" since <filename>${C}</filename> immediately
237 expands to "cval".
238 </para>
239 </section>
240
241 <section id='appending-and-prepending'>
242 <title>Appending (+=) and prepending (=+) With Spaces</title>
243
244 <para>
245 Appending and prepending values is common and can be accomplished
246 using the "+=" and "=+" operators.
247 These operators insert a space between the current
248 value and prepended or appended value.
249 </para>
250
251 <para>
252 These operators take immediate effect during parsing.
253 Here are some examples:
254 <literallayout class='monospaced'>
255 B = "bval"
256 B += "additionaldata"
257 C = "cval"
258 C =+ "test"
259 </literallayout>
260 The variable <filename>B</filename> contains
261 "bval additionaldata" and <filename>C</filename>
262 contains "test cval".
263 </para>
264 </section>
265
266 <section id='appending-and-prepending-without-spaces'>
267 <title>Appending (.=) and Prepending (=.) Without Spaces</title>
268
269 <para>
270 If you want to append or prepend values without an
271 inserted space, use the ".=" and "=." operators.
272 </para>
273
274 <para>
275 These operators take immediate effect during parsing.
276 Here are some examples:
277 <literallayout class='monospaced'>
278 B = "bval"
279 B .= "additionaldata"
280 C = "cval"
281 C =. "test"
282 </literallayout>
283 The variable <filename>B</filename> contains
284 "bvaladditionaldata" and
285 <filename>C</filename> contains "testcval".
286 </para>
287 </section>
288
289 <section id='appending-and-prepending-override-style-syntax'>
290 <title>Appending and Prepending (Override Style Syntax)</title>
291
292 <para>
293 You can also append and prepend a variable's value
294 using an override style syntax.
295 When you use this syntax, no spaces are inserted.
296 </para>
297
298 <para>
299 These operators differ from the ":=", ".=", "=.", "+=", and "=+"
300 operators in that their effects are deferred
301 until after parsing completes rather than being immediately
302 applied.
303 Here are some examples:
304 <literallayout class='monospaced'>
305 B = "bval"
306 B_append = " additional data"
307 C = "cval"
308 C_prepend = "additional data "
309 D = "dval"
310 D_append = "additional data"
311 </literallayout>
312 The variable <filename>B</filename> becomes
313 "bval additional data" and <filename>C</filename> becomes
314 "additional data cval".
315 The variable <filename>D</filename> becomes
316 "dvaladditional data".
317 <note>
318 You must control all spacing when you use the
319 override syntax.
320 </note>
321 </para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600322
323 <para>
324 It is also possible to append and prepend to shell
325 functions and BitBake-style Python functions.
326 See the
327 "<link linkend='shell-functions'>Shell Functions</link>" and
328 "<link linkend='bitbake-style-python-functions'>BitBake-Style Python Functions</link>
329 sections for examples.
330 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500331 </section>
332
333 <section id='removing-override-style-syntax'>
334 <title>Removal (Override Style Syntax)</title>
335
336 <para>
337 You can remove values from lists using the removal
338 override style syntax.
339 Specifying a value for removal causes all occurrences of that
340 value to be removed from the variable.
341 </para>
342
343 <para>
344 When you use this syntax, BitBake expects one or more strings.
345 Surrounding spaces are removed as well.
346 Here is an example:
347 <literallayout class='monospaced'>
348 FOO = "123 456 789 123456 123 456 123 456"
349 FOO_remove = "123"
350 FOO_remove = "456"
351 FOO2 = "abc def ghi abcdef abc def abc def"
352 FOO2_remove = "abc def"
353 </literallayout>
354 The variable <filename>FOO</filename> becomes
355 "789 123456" and <filename>FOO2</filename> becomes
356 "ghi abcdef".
357 </para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600358
359 <para>
360 Like "_append" and "_prepend", "_remove"
361 is deferred until after parsing completes.
362 </para>
363 </section>
364
365 <section id='override-style-operation-advantages'>
366 <title>Override Style Operation Advantages</title>
367
368 <para>
369 An advantage of the override style operations
370 "_append", "_prepend", and "_remove" as compared to the
371 "+=" and "=+" operators is that the override style
372 operators provide guaranteed operations.
373 For example, consider a class <filename>foo.bbclass</filename>
374 that needs to add the value "val" to the variable
375 <filename>FOO</filename>, and a recipe that uses
376 <filename>foo.bbclass</filename> as follows:
377 <literallayout class='monospaced'>
378 inherit foo
379
380 FOO = "initial"
381 </literallayout>
382 If <filename>foo.bbclass</filename> uses the "+=" operator,
383 as follows, then the final value of <filename>FOO</filename>
384 will be "initial", which is not what is desired:
385 <literallayout class='monospaced'>
386 FOO += "val"
387 </literallayout>
388 If, on the other hand, <filename>foo.bbclass</filename>
389 uses the "_append" operator, then the final value of
390 <filename>FOO</filename> will be "initial val", as intended:
391 <literallayout class='monospaced'>
392 FOO_append = " val"
393 </literallayout>
394 <note>
395 It is never necessary to use "+=" together with "_append".
396 The following sequence of assignments appends "barbaz" to
397 <filename>FOO</filename>:
398 <literallayout class='monospaced'>
399 FOO_append = "bar"
400 FOO_append = "baz"
401 </literallayout>
402 The only effect of changing the second assignment in the
403 previous example to use "+=" would be to add a space before
404 "baz" in the appended value (due to how the "+=" operator
405 works).
406 </note>
407 Another advantage of the override style operations is that
408 you can combine them with other overrides as described in the
409 "<link linkend='conditional-syntax-overrides'>Conditional Syntax (Overrides)</link>"
410 section.
411 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500412 </section>
413
414 <section id='variable-flag-syntax'>
415 <title>Variable Flag Syntax</title>
416
417 <para>
418 Variable flags are BitBake's implementation of variable properties
419 or attributes.
420 It is a way of tagging extra information onto a variable.
421 You can find more out about variable flags in general in the
422 "<link linkend='variable-flags'>Variable Flags</link>"
423 section.
424 </para>
425
426 <para>
427 You can define, append, and prepend values to variable flags.
428 All the standard syntax operations previously mentioned work
429 for variable flags except for override style syntax
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600430 (i.e. "_prepend", "_append", and "_remove").
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500431 </para>
432
433 <para>
434 Here are some examples showing how to set variable flags:
435 <literallayout class='monospaced'>
436 FOO[a] = "abc"
437 FOO[b] = "123"
438 FOO[a] += "456"
439 </literallayout>
440 The variable <filename>FOO</filename> has two flags:
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600441 <filename>[a]</filename> and <filename>[b]</filename>.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500442 The flags are immediately set to "abc" and "123", respectively.
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600443 The <filename>[a]</filename> flag becomes "abc 456".
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500444 </para>
445
446 <para>
447 No need exists to pre-define variable flags.
448 You can simply start using them.
449 One extremely common application
450 is to attach some brief documentation to a BitBake variable as
451 follows:
452 <literallayout class='monospaced'>
453 CACHE[doc] = "The directory holding the cache of the metadata."
454 </literallayout>
455 </para>
456 </section>
457
458 <section id='inline-python-variable-expansion'>
459 <title>Inline Python Variable Expansion</title>
460
461 <para>
462 You can use inline Python variable expansion to
463 set variables.
464 Here is an example:
465 <literallayout class='monospaced'>
466 DATE = "${@time.strftime('%Y%m%d',time.gmtime())}"
467 </literallayout>
468 This example results in the <filename>DATE</filename>
469 variable being set to the current date.
470 </para>
471
472 <para>
473 Probably the most common use of this feature is to extract
474 the value of variables from BitBake's internal data dictionary,
475 <filename>d</filename>.
476 The following lines select the values of a package name
477 and its version number, respectively:
478 <literallayout class='monospaced'>
479 PN = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'}"
480 PV = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[1] or '1.0'}"
481 </literallayout>
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600482 <note>
483 Inline Python expressions work just like variable expansions
484 insofar as the "=" and ":=" operators are concerned.
485 Given the following assignment, <filename>foo()</filename>
486 is called each time <filename>FOO</filename> is expanded:
487 <literallayout class='monospaced'>
488 FOO = "${@foo()}"
489 </literallayout>
490 Contrast this with the following immediate assignment, where
491 <filename>foo()</filename> is only called once, while the
492 assignment is parsed:
493 <literallayout class='monospaced'>
494 FOO := "${@foo()}"
495 </literallayout>
496 </note>
497 For a different way to set variables with Python code during
498 parsing, see the
499 "<link linkend='anonymous-python-functions'>Anonymous Python Functions</link>"
500 section.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500501 </para>
502 </section>
503
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600504 <section id='unsetting-variables'>
505 <title>Unseting variables</title>
506
507 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500508 It is possible to completely remove a variable or a variable flag
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600509 from BitBake's internal data dictionary by using the "unset" keyword.
510 Here is an example:
511 <literallayout class='monospaced'>
512 unset DATE
513 unset do_fetch[noexec]
514 </literallayout>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500515 These two statements remove the <filename>DATE</filename> and the
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600516 <filename>do_fetch[noexec]</filename> flag.
517 </para>
518
519 </section>
520
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500521 <section id='providing-pathnames'>
522 <title>Providing Pathnames</title>
523
524 <para>
525 When specifying pathnames for use with BitBake,
526 do not use the tilde ("~") character as a shortcut
527 for your home directory.
528 Doing so might cause BitBake to not recognize the
529 path since BitBake does not expand this character in
530 the same way a shell would.
531 </para>
532
533 <para>
534 Instead, provide a fuller path as the following
535 example illustrates:
536 <literallayout class='monospaced'>
537 BBLAYERS ?= " \
538 /home/scott-lenovo/LayerA \
539 "
540 </literallayout>
541 </para>
542 </section>
543 </section>
544
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600545 <section id='exporting-variables-to-the-environment'>
546 <title>Exporting Variables to the Environment</title>
547
548 <para>
549 You can export variables to the environment of running
550 tasks by using the <filename>export</filename> keyword.
551 For example, in the following example, the
552 <filename>do_foo</filename> task prints "value from
553 the environment" when run:
554 <literallayout class='monospaced'>
555 export ENV_VARIABLE
556 ENV_VARIABLE = "value from the environment"
557
558 do_foo() {
559 bbplain "$ENV_VARIABLE"
560 }
561 </literallayout>
562 <note>
563 BitBake does not expand <filename>$ENV_VARIABLE</filename>
564 in this case because it lacks the obligatory
565 <filename>{}</filename>.
566 Rather, <filename>$ENV_VARIABLE</filename> is expanded
567 by the shell.
568 </note>
569 It does not matter whether
570 <filename>export ENV_VARIABLE</filename> appears before or
571 after assignments to <filename>ENV_VARIABLE</filename>.
572 </para>
573
574 <para>
575 It is also possible to combine <filename>export</filename>
576 with setting a value for the variable.
577 Here is an example:
578 <literallayout class='monospaced'>
579 export ENV_VARIABLE = "<replaceable>variable-value</replaceable>"
580 </literallayout>
581 In the output of <filename>bitbake -e</filename>, variables
582 that are exported to the environment are preceded by "export".
583 </para>
584
585 <para>
586 Among the variables commonly exported to the environment
587 are <filename>CC</filename> and <filename>CFLAGS</filename>,
588 which are picked up by many build systems.
589 </para>
590 </section>
591
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500592 <section id='conditional-syntax-overrides'>
593 <title>Conditional Syntax (Overrides)</title>
594
595 <para>
596 BitBake uses
597 <link linkend='var-OVERRIDES'><filename>OVERRIDES</filename></link>
598 to control what variables are overridden after BitBake
599 parses recipes and configuration files.
600 This section describes how you can use
601 <filename>OVERRIDES</filename> as conditional metadata,
602 talks about key expansion in relationship to
603 <filename>OVERRIDES</filename>, and provides some examples
604 to help with understanding.
605 </para>
606
607 <section id='conditional-metadata'>
608 <title>Conditional Metadata</title>
609
610 <para>
611 You can use <filename>OVERRIDES</filename> to conditionally select
612 a specific version of a variable and to conditionally
613 append or prepend the value of a variable.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500614 <note>
615 Overrides can only use lower-case characters.
616 Additionally, underscores are not permitted in override names
617 as they are used to separate overrides from each other and
618 from the variable name.
619 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500620 <itemizedlist>
621 <listitem><para><emphasis>Selecting a Variable:</emphasis>
622 The <filename>OVERRIDES</filename> variable is
623 a colon-character-separated list that contains items
624 for which you want to satisfy conditions.
625 Thus, if you have a variable that is conditional on “arm”, and “arm”
626 is in <filename>OVERRIDES</filename>, then the “arm”-specific
627 version of the variable is used rather than the non-conditional
628 version.
629 Here is an example:
630 <literallayout class='monospaced'>
631 OVERRIDES = "architecture:os:machine"
632 TEST = "default"
633 TEST_os = "osspecific"
634 TEST_nooverride = "othercondvalue"
635 </literallayout>
636 In this example, the <filename>OVERRIDES</filename>
637 variable lists three overrides:
638 "architecture", "os", and "machine".
639 The variable <filename>TEST</filename> by itself has a default
640 value of "default".
641 You select the os-specific version of the <filename>TEST</filename>
642 variable by appending the "os" override to the variable
643 (i.e.<filename>TEST_os</filename>).
644 </para>
645
646 <para>
647 To better understand this, consider a practical example
648 that assumes an OpenEmbedded metadata-based Linux
649 kernel recipe file.
650 The following lines from the recipe file first set
651 the kernel branch variable <filename>KBRANCH</filename>
652 to a default value, then conditionally override that
653 value based on the architecture of the build:
654 <literallayout class='monospaced'>
655 KBRANCH = "standard/base"
656 KBRANCH_qemuarm = "standard/arm-versatile-926ejs"
657 KBRANCH_qemumips = "standard/mti-malta32"
658 KBRANCH_qemuppc = "standard/qemuppc"
659 KBRANCH_qemux86 = "standard/common-pc/base"
660 KBRANCH_qemux86-64 = "standard/common-pc-64/base"
661 KBRANCH_qemumips64 = "standard/mti-malta64"
662 </literallayout>
663 </para></listitem>
664 <listitem><para><emphasis>Appending and Prepending:</emphasis>
665 BitBake also supports append and prepend operations to
666 variable values based on whether a specific item is
667 listed in <filename>OVERRIDES</filename>.
668 Here is an example:
669 <literallayout class='monospaced'>
670 DEPENDS = "glibc ncurses"
671 OVERRIDES = "machine:local"
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500672 DEPENDS_append_machine = " libmad"
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500673 </literallayout>
674 In this example, <filename>DEPENDS</filename> becomes
675 "glibc ncurses libmad".
676 </para>
677
678 <para>
679 Again, using an OpenEmbedded metadata-based
680 kernel recipe file as an example, the
681 following lines will conditionally append to the
682 <filename>KERNEL_FEATURES</filename> variable based
683 on the architecture:
684 <literallayout class='monospaced'>
685 KERNEL_FEATURES_append = " ${KERNEL_EXTRA_FEATURES}"
686 KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
687 KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc"
688 </literallayout>
689 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600690 <listitem><para><emphasis>Setting a Variable for a Single Task:</emphasis>
691 BitBake supports setting a variable just for the
692 duration of a single task.
693 Here is an example:
694 <literallayout class='monospaced'>
695 FOO_task-configure = "val 1"
696 FOO_task-compile = "val 2"
697 </literallayout>
698 In the previous example, <filename>FOO</filename>
699 has the value "val 1" while the
700 <filename>do_configure</filename> task is executed,
701 and the value "val 2" while the
702 <filename>do_compile</filename> task is executed.
703 </para>
704
705 <para>Internally, this is implemented by prepending
706 the task (e.g. "task-compile:") to the value of
707 <link linkend='var-OVERRIDES'><filename>OVERRIDES</filename></link>
708 for the local datastore of the <filename>do_compile</filename>
709 task.</para>
710
711 <para>You can also use this syntax with other combinations
712 (e.g. "<filename>_prepend</filename>") as shown in the
713 following example:
714 <literallayout class='monospaced'>
715 EXTRA_OEMAKE_prepend_task-compile = "${PARALLEL_MAKE} "
716 </literallayout>
717 </para></listitem>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500718 </itemizedlist>
719 </para>
720 </section>
721
722 <section id='key-expansion'>
723 <title>Key Expansion</title>
724
725 <para>
726 Key expansion happens when the BitBake datastore is finalized
727 just before BitBake expands overrides.
728 To better understand this, consider the following example:
729 <literallayout class='monospaced'>
730 A${B} = "X"
731 B = "2"
732 A2 = "Y"
733 </literallayout>
734 In this case, after all the parsing is complete, and
735 before any overrides are handled, BitBake expands
736 <filename>${B}</filename> into "2".
737 This expansion causes <filename>A2</filename>, which was
738 set to "Y" before the expansion, to become "X".
739 </para>
740 </section>
741
742 <section id='variable-interaction-worked-examples'>
743 <title>Examples</title>
744
745 <para>
746 Despite the previous explanations that show the different forms of
747 variable definitions, it can be hard to work
748 out exactly what happens when variable operators, conditional
749 overrides, and unconditional overrides are combined.
750 This section presents some common scenarios along
751 with explanations for variable interactions that
752 typically confuse users.
753 </para>
754
755 <para>
756 There is often confusion concerning the order in which
757 overrides and various "append" operators take effect.
758 Recall that an append or prepend operation using "_append"
759 and "_prepend" does not result in an immediate assignment
760 as would "+=", ".=", "=+", or "=.".
761 Consider the following example:
762 <literallayout class='monospaced'>
763 OVERRIDES = "foo"
764 A = "Z"
765 A_foo_append = "X"
766 </literallayout>
767 For this case, <filename>A</filename> is
768 unconditionally set to "Z" and "X" is
769 unconditionally and immediately appended to the variable
770 <filename>A_foo</filename>.
771 Because overrides have not been applied yet,
772 <filename>A_foo</filename> is set to "X" due to the append
773 and <filename>A</filename> simply equals "Z".
774 </para>
775
776 <para>
777 Applying overrides, however, changes things.
778 Since "foo" is listed in <filename>OVERRIDES</filename>,
779 the conditional variable <filename>A</filename> is replaced
780 with the "foo" version, which is equal to "X".
781 So effectively, <filename>A_foo</filename> replaces <filename>A</filename>.
782 </para>
783
784 <para>
785 This next example changes the order of the override and
786 the append:
787 <literallayout class='monospaced'>
788 OVERRIDES = "foo"
789 A = "Z"
790 A_append_foo = "X"
791 </literallayout>
792 For this case, before overrides are handled,
793 <filename>A</filename> is set to "Z" and <filename>A_append_foo</filename>
794 is set to "X".
795 Once the override for "foo" is applied, however,
796 <filename>A</filename> gets appended with "X".
797 Consequently, <filename>A</filename> becomes "ZX".
798 Notice that spaces are not appended.
799 </para>
800
801 <para>
802 This next example has the order of the appends and overrides reversed
803 back as in the first example:
804 <literallayout class='monospaced'>
805 OVERRIDES = "foo"
806 A = "Y"
807 A_foo_append = "Z"
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600808 A_foo_append = "X"
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500809 </literallayout>
810 For this case, before any overrides are resolved,
811 <filename>A</filename> is set to "Y" using an immediate assignment.
812 After this immediate assignment, <filename>A_foo</filename> is set
813 to "Z", and then further appended with
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600814 "X" leaving the variable set to "ZX".
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500815 Finally, applying the override for "foo" results in the conditional
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600816 variable <filename>A</filename> becoming "ZX" (i.e.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500817 <filename>A</filename> is replaced with <filename>A_foo</filename>).
818 </para>
819
820 <para>
821 This final example mixes in some varying operators:
822 <literallayout class='monospaced'>
823 A = "1"
824 A_append = "2"
825 A_append = "3"
826 A += "4"
827 A .= "5"
828 </literallayout>
829 For this case, the type of append operators are affecting the
830 order of assignments as BitBake passes through the code
831 multiple times.
832 Initially, <filename>A</filename> is set to "1 45" because
833 of the three statements that use immediate operators.
834 After these assignments are made, BitBake applies the
Patrick Williamsc0f7c042017-02-23 20:41:17 -0600835 "_append" operations.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500836 Those operations result in <filename>A</filename> becoming "1 4523".
837 </para>
838 </section>
839 </section>
840
841 <section id='sharing-functionality'>
842 <title>Sharing Functionality</title>
843
844 <para>
845 BitBake allows for metadata sharing through include files
846 (<filename>.inc</filename>) and class files
847 (<filename>.bbclass</filename>).
848 For example, suppose you have a piece of common functionality
849 such as a task definition that you want to share between
850 more than one recipe.
851 In this case, creating a <filename>.bbclass</filename>
852 file that contains the common functionality and then using
853 the <filename>inherit</filename> directive in your recipes to
854 inherit the class would be a common way to share the task.
855 </para>
856
857 <para>
858 This section presents the mechanisms BitBake provides to
859 allow you to share functionality between recipes.
860 Specifically, the mechanisms include <filename>include</filename>,
861 <filename>inherit</filename>, <filename>INHERIT</filename>, and
862 <filename>require</filename> directives.
863 </para>
864
865 <section id='locating-include-and-class-files'>
866 <title>Locating Include and Class Files</title>
867
868 <para>
869 BitBake uses the
870 <link linkend='var-BBPATH'><filename>BBPATH</filename></link>
871 variable to locate needed include and class files.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500872 Additionally, BitBake searches the current directory for
873 <filename>include</filename> and <filename>require</filename>
874 directives.
875 <note>
876 The <filename>BBPATH</filename> variable is analogous to
877 the environment variable <filename>PATH</filename>.
878 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500879 </para>
880
881 <para>
882 In order for include and class files to be found by BitBake,
883 they need to be located in a "classes" subdirectory that can
884 be found in <filename>BBPATH</filename>.
885 </para>
886 </section>
887
888 <section id='inherit-directive'>
889 <title><filename>inherit</filename> Directive</title>
890
891 <para>
892 When writing a recipe or class file, you can use the
893 <filename>inherit</filename> directive to inherit the
894 functionality of a class (<filename>.bbclass</filename>).
895 BitBake only supports this directive when used within recipe
896 and class files (i.e. <filename>.bb</filename> and
897 <filename>.bbclass</filename>).
898 </para>
899
900 <para>
901 The <filename>inherit</filename> directive is a rudimentary
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500902 means of specifying functionality contained in class files
903 that your recipes require.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500904 For example, you can easily abstract out the tasks involved in
905 building a package that uses Autoconf and Automake and put
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500906 those tasks into a class file and then have your recipe
907 inherit that class file.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500908 </para>
909
910 <para>
911 As an example, your recipes could use the following directive
912 to inherit an <filename>autotools.bbclass</filename> file.
913 The class file would contain common functionality for using
914 Autotools that could be shared across recipes:
915 <literallayout class='monospaced'>
916 inherit autotools
917 </literallayout>
918 In this case, BitBake would search for the directory
919 <filename>classes/autotools.bbclass</filename>
920 in <filename>BBPATH</filename>.
921 <note>
922 You can override any values and functions of the
923 inherited class within your recipe by doing so
924 after the "inherit" statement.
925 </note>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500926 If you want to use the directive to inherit
927 multiple classes, separate them with spaces.
928 The following example shows how to inherit both the
929 <filename>buildhistory</filename> and <filename>rm_work</filename>
930 classes:
931 <literallayout class='monospaced'>
932 inherit buildhistory rm_work
933 </literallayout>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500934 </para>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500935
936 <para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500937 An advantage with the inherit directive as compared to both
938 the
939 <link linkend='include-directive'>include</link> and
940 <link linkend='require-inclusion'>require</link> directives
941 is that you can inherit class files conditionally.
942 You can accomplish this by using a variable expression
943 after the <filename>inherit</filename> statement.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500944 Here is an example:
945 <literallayout class='monospaced'>
946 inherit ${VARNAME}
947 </literallayout>
948 If <filename>VARNAME</filename> is going to be set, it needs
949 to be set before the <filename>inherit</filename> statement
950 is parsed.
951 One way to achieve a conditional inherit in this case is to use
952 overrides:
953 <literallayout class='monospaced'>
954 VARIABLE = ""
955 VARIABLE_someoverride = "myclass"
956 </literallayout>
957 </para>
958
959 <para>
960 Another method is by using anonymous Python.
961 Here is an example:
962 <literallayout class='monospaced'>
963 python () {
964 if condition == value:
965 d.setVar('VARIABLE', 'myclass')
966 else:
967 d.setVar('VARIABLE', '')
968 }
969 </literallayout>
970 </para>
971
972 <para>
973 Alternatively, you could use an in-line Python expression
974 in the following form:
975 <literallayout class='monospaced'>
976 inherit ${@'classname' if condition else ''}
977 inherit ${@functionname(params)}
978 </literallayout>
979 In all cases, if the expression evaluates to an empty
980 string, the statement does not trigger a syntax error
981 because it becomes a no-op.
982 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500983 </section>
984
985 <section id='include-directive'>
986 <title><filename>include</filename> Directive</title>
987
988 <para>
989 BitBake understands the <filename>include</filename>
990 directive.
991 This directive causes BitBake to parse whatever file you specify,
992 and to insert that file at that location.
993 The directive is much like its equivalent in Make except
994 that if the path specified on the include line is a relative
995 path, BitBake locates the first file it can find
996 within <filename>BBPATH</filename>.
997 </para>
998
999 <para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -05001000 The include directive is a more generic method of including
1001 functionality as compared to the
1002 <link linkend='inherit-directive'>inherit</link> directive,
1003 which is restricted to class (i.e. <filename>.bbclass</filename>)
1004 files.
1005 The include directive is applicable for any other kind of
1006 shared or encapsulated functionality or configuration that
1007 does not suit a <filename>.bbclass</filename> file.
1008 </para>
1009
1010 <para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001011 As an example, suppose you needed a recipe to include some
1012 self-test definitions:
1013 <literallayout class='monospaced'>
1014 include test_defs.inc
1015 </literallayout>
1016 <note>
1017 The <filename>include</filename> directive does not
1018 produce an error when the file cannot be found.
1019 Consequently, it is recommended that if the file you
1020 are including is expected to exist, you should use
1021 <link linkend='require-inclusion'><filename>require</filename></link>
1022 instead of <filename>include</filename>.
1023 Doing so makes sure that an error is produced if the
1024 file cannot be found.
1025 </note>
1026 </para>
1027 </section>
1028
1029 <section id='require-inclusion'>
1030 <title><filename>require</filename> Directive</title>
1031
1032 <para>
1033 BitBake understands the <filename>require</filename>
1034 directive.
1035 This directive behaves just like the
1036 <filename>include</filename> directive with the exception that
1037 BitBake raises a parsing error if the file to be included cannot
1038 be found.
1039 Thus, any file you require is inserted into the file that is
1040 being parsed at the location of the directive.
1041 </para>
1042
1043 <para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -05001044 The require directive, like the include directive previously
1045 described, is a more generic method of including
1046 functionality as compared to the
1047 <link linkend='inherit-directive'>inherit</link> directive,
1048 which is restricted to class (i.e. <filename>.bbclass</filename>)
1049 files.
1050 The require directive is applicable for any other kind of
1051 shared or encapsulated functionality or configuration that
1052 does not suit a <filename>.bbclass</filename> file.
1053 </para>
1054
1055 <para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001056 Similar to how BitBake handles
1057 <link linkend='include-directive'><filename>include</filename></link>,
1058 if the path specified
1059 on the require line is a relative path, BitBake locates
1060 the first file it can find within <filename>BBPATH</filename>.
1061 </para>
1062
1063 <para>
1064 As an example, suppose you have two versions of a recipe
1065 (e.g. <filename>foo_1.2.2.bb</filename> and
1066 <filename>foo_2.0.0.bb</filename>) where
1067 each version contains some identical functionality that could be
1068 shared.
1069 You could create an include file named <filename>foo.inc</filename>
1070 that contains the common definitions needed to build "foo".
1071 You need to be sure <filename>foo.inc</filename> is located in the
1072 same directory as your two recipe files as well.
1073 Once these conditions are set up, you can share the functionality
1074 using a <filename>require</filename> directive from within each
1075 recipe:
1076 <literallayout class='monospaced'>
1077 require foo.inc
1078 </literallayout>
1079 </para>
1080 </section>
1081
1082 <section id='inherit-configuration-directive'>
1083 <title><filename>INHERIT</filename> Configuration Directive</title>
1084
1085 <para>
1086 When creating a configuration file (<filename>.conf</filename>),
Brad Bishopd7bf8c12018-02-25 22:55:05 -05001087 you can use the
1088 <link linkend='var-INHERIT'><filename>INHERIT</filename></link>
1089 configuration directive to inherit a class.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001090 BitBake only supports this directive when used within
1091 a configuration file.
1092 </para>
1093
1094 <para>
1095 As an example, suppose you needed to inherit a class
1096 file called <filename>abc.bbclass</filename> from a
1097 configuration file as follows:
1098 <literallayout class='monospaced'>
1099 INHERIT += "abc"
1100 </literallayout>
1101 This configuration directive causes the named
1102 class to be inherited at the point of the directive
1103 during parsing.
1104 As with the <filename>inherit</filename> directive, the
1105 <filename>.bbclass</filename> file must be located in a
1106 "classes" subdirectory in one of the directories specified
1107 in <filename>BBPATH</filename>.
1108 <note>
1109 Because <filename>.conf</filename> files are parsed
1110 first during BitBake's execution, using
1111 <filename>INHERIT</filename> to inherit a class effectively
1112 inherits the class globally (i.e. for all recipes).
1113 </note>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001114 If you want to use the directive to inherit
1115 multiple classes, you can provide them on the same line in the
1116 <filename>local.conf</filename> file.
1117 Use spaces to separate the classes.
1118 The following example shows how to inherit both the
1119 <filename>autotools</filename> and <filename>pkgconfig</filename>
1120 classes:
1121 <literallayout class='monospaced'>
Brad Bishopd7bf8c12018-02-25 22:55:05 -05001122 INHERIT += "autotools pkgconfig"
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001123 </literallayout>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001124 </para>
1125 </section>
1126 </section>
1127
1128 <section id='functions'>
1129 <title>Functions</title>
1130
1131 <para>
1132 As with most languages, functions are the building blocks that
1133 are used to build up operations into tasks.
1134 BitBake supports these types of functions:
1135 <itemizedlist>
1136 <listitem><para><emphasis>Shell Functions:</emphasis>
1137 Functions written in shell script and executed either
1138 directly as functions, tasks, or both.
1139 They can also be called by other shell functions.
1140 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001141 <listitem><para><emphasis>BitBake-Style Python Functions:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001142 Functions written in Python and executed by BitBake or other
1143 Python functions using <filename>bb.build.exec_func()</filename>.
1144 </para></listitem>
1145 <listitem><para><emphasis>Python Functions:</emphasis>
1146 Functions written in Python and executed by Python.
1147 </para></listitem>
1148 <listitem><para><emphasis>Anonymous Python Functions:</emphasis>
1149 Python functions executed automatically during
1150 parsing.
1151 </para></listitem>
1152 </itemizedlist>
1153 Regardless of the type of function, you can only
1154 define them in class (<filename>.bbclass</filename>)
1155 and recipe (<filename>.bb</filename> or <filename>.inc</filename>)
1156 files.
1157 </para>
1158
1159 <section id='shell-functions'>
1160 <title>Shell Functions</title>
1161
1162 <para>
1163 Functions written in shell script and executed either
1164 directly as functions, tasks, or both.
1165 They can also be called by other shell functions.
1166 Here is an example shell function definition:
1167 <literallayout class='monospaced'>
1168 some_function () {
1169 echo "Hello World"
1170 }
1171 </literallayout>
1172 When you create these types of functions in your recipe
1173 or class files, you need to follow the shell programming
1174 rules.
1175 The scripts are executed by <filename>/bin/sh</filename>,
1176 which may not be a bash shell but might be something
1177 such as <filename>dash</filename>.
1178 You should not use Bash-specific script (bashisms).
1179 </para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001180
1181 <para>
1182 Overrides and override-style operators like
1183 <filename>_append</filename> and
1184 <filename>_prepend</filename> can also be applied to
1185 shell functions.
1186 Most commonly, this application would be used in a
1187 <filename>.bbappend</filename> file to modify functions in
1188 the main recipe.
1189 It can also be used to modify functions inherited from
1190 classes.
1191 </para>
1192
1193 <para>
1194 As an example, consider the following:
1195 <literallayout class='monospaced'>
1196 do_foo() {
1197 bbplain first
1198 fn
1199 }
1200
1201 fn_prepend() {
1202 bbplain second
1203 }
1204
1205 fn() {
1206 bbplain third
1207 }
1208
1209 do_foo_append() {
1210 bbplain fourth
1211 }
1212 </literallayout>
1213 Running <filename>do_foo</filename>
1214 prints the following:
1215 <literallayout class='monospaced'>
1216 recipename do_foo: first
1217 recipename do_foo: second
1218 recipename do_foo: third
1219 recipename do_foo: fourth
1220 </literallayout>
1221 <note>
1222 Overrides and override-style operators can
1223 be applied to any shell function, not just
1224 <link linkend='tasks'>tasks</link>.
1225 </note>
1226 You can use the <filename>bitbake -e</filename>&nbsp;<replaceable>recipename</replaceable>
1227 command to view the final assembled function
1228 after all overrides have been applied.
1229 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001230 </section>
1231
1232 <section id='bitbake-style-python-functions'>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001233 <title>BitBake-Style Python Functions</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001234
1235 <para>
1236 These functions are written in Python and executed by
1237 BitBake or other Python functions using
1238 <filename>bb.build.exec_func()</filename>.
1239 </para>
1240
1241 <para>
1242 An example BitBake function is:
1243 <literallayout class='monospaced'>
1244 python some_python_function () {
1245 d.setVar("TEXT", "Hello World")
Brad Bishop6e60e8b2018-02-01 10:27:11 -05001246 print d.getVar("TEXT")
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001247 }
1248 </literallayout>
1249 Because the Python "bb" and "os" modules are already
1250 imported, you do not need to import these modules.
1251 Also in these types of functions, the datastore ("d")
1252 is a global variable and is always automatically
1253 available.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001254 <note>
1255 Variable expressions (e.g. <filename>${X}</filename>)
1256 are no longer expanded within Python functions.
1257 This behavior is intentional in order to allow you
1258 to freely set variable values to expandable expressions
1259 without having them expanded prematurely.
1260 If you do wish to expand a variable within a Python
Brad Bishop6e60e8b2018-02-01 10:27:11 -05001261 function, use <filename>d.getVar("X")</filename>.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001262 Or, for more complicated expressions, use
1263 <filename>d.expand()</filename>.
1264 </note>
1265 </para>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001266
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001267 <para>
1268 Similar to shell functions, you can also apply overrides
1269 and override-style operators to BitBake-style Python
1270 functions.
1271 </para>
1272
1273 <para>
1274 As an example, consider the following:
1275 <literallayout class='monospaced'>
1276 python do_foo_prepend() {
1277 bb.plain("first")
1278 }
1279
1280 python do_foo() {
1281 bb.plain("second")
1282 }
1283
1284 python do_foo_append() {
1285 bb.plain("third")
1286 }
1287 </literallayout>
1288 Running <filename>do_foo</filename> prints
1289 the following:
1290 <literallayout class='monospaced'>
1291 recipename do_foo: first
1292 recipename do_foo: second
1293 recipename do_foo: third
1294 </literallayout>
1295 You can use the <filename>bitbake -e</filename>&nbsp;<replaceable>recipename</replaceable>
1296 command to view the final assembled function
1297 after all overrides have been applied.
1298 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001299 </section>
1300
1301 <section id='python-functions'>
1302 <title>Python Functions</title>
1303
1304 <para>
1305 These functions are written in Python and are executed by
1306 other Python code.
1307 Examples of Python functions are utility functions
1308 that you intend to call from in-line Python or
1309 from within other Python functions.
1310 Here is an example:
1311 <literallayout class='monospaced'>
1312 def get_depends(d):
Brad Bishop6e60e8b2018-02-01 10:27:11 -05001313 if d.getVar('SOMECONDITION'):
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001314 return "dependencywithcond"
1315 else:
1316 return "dependency"
1317 SOMECONDITION = "1"
1318 DEPENDS = "${@get_depends(d)}"
1319 </literallayout>
1320 This would result in <filename>DEPENDS</filename>
1321 containing <filename>dependencywithcond</filename>.
1322 </para>
1323
1324 <para>
1325 Here are some things to know about Python functions:
1326 <itemizedlist>
1327 <listitem><para>Python functions can take parameters.
1328 </para></listitem>
1329 <listitem><para>The BitBake datastore is not
1330 automatically available.
1331 Consequently, you must pass it in as a
1332 parameter to the function.
1333 </para></listitem>
1334 <listitem><para>The "bb" and "os" Python modules are
1335 automatically available.
1336 You do not need to import them.
1337 </para></listitem>
1338 </itemizedlist>
1339 </para>
1340 </section>
1341
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001342 <section id='bitbake-style-python-functions-versus-python-functions'>
1343 <title>Bitbake-Style Python Functions Versus Python Functions</title>
1344
1345 <para>
1346 Following are some important differences between
1347 BitBake-style Python functions and regular Python
1348 functions defined with "def":
1349 <itemizedlist>
1350 <listitem><para>
1351 Only BitBake-style Python functions can be
1352 <link linkend='tasks'>tasks</link>.
1353 </para></listitem>
1354 <listitem><para>
1355 Overrides and override-style operators can only
1356 be applied to BitBake-style Python functions.
1357 </para></listitem>
1358 <listitem><para>
1359 Only regular Python functions can take arguments
1360 and return values.
1361 </para></listitem>
1362 <listitem><para>
1363 <link linkend='variable-flags'>Variable flags</link>
1364 such as <filename>[dirs]</filename>,
1365 <filename>[cleandirs]</filename>, and
1366 <filename>[lockfiles]</filename> can be used
1367 on BitBake-style Python functions, but not on
1368 regular Python functions.
1369 </para></listitem>
1370 <listitem><para>
1371 BitBake-style Python functions generate a separate
1372 <filename>${</filename><link linkend='var-T'><filename>T</filename></link><filename>}/run.</filename><replaceable>function-name</replaceable><filename>.</filename><replaceable>pid</replaceable>
1373 script that is executed to run the function, and also
1374 generate a log file in
1375 <filename>${T}/log.</filename><replaceable>function-name</replaceable><filename>.</filename><replaceable>pid</replaceable>
1376 if they are executed as tasks.</para>
1377
1378 <para>
1379 Regular Python functions execute "inline" and do not
1380 generate any files in <filename>${T}</filename>.
1381 </para></listitem>
1382 <listitem><para>
1383 Regular Python functions are called with the usual
1384 Python syntax.
1385 BitBake-style Python functions are usually tasks and
1386 are called directly by BitBake, but can also be called
1387 manually from Python code by using the
1388 <filename>bb.build.exec_func()</filename> function.
1389 Here is an example:
1390 <literallayout class='monospaced'>
1391 bb.build.exec_func("my_bitbake_style_function", d)
1392 </literallayout>
1393 <note>
1394 <filename>bb.build.exec_func()</filename> can also
1395 be used to run shell functions from Python code.
1396 If you want to run a shell function before a Python
1397 function within the same task, then you can use a
1398 parent helper Python function that starts by running
1399 the shell function with
1400 <filename>bb.build.exec_func()</filename> and then
1401 runs the Python code.
1402 </note></para>
1403
1404 <para>To detect errors from functions executed with
1405 <filename>bb.build.exec_func()</filename>, you
1406 can catch the <filename>bb.build.FuncFailed</filename>
1407 exception.
1408 <note>
1409 Functions in metadata (recipes and classes) should
1410 not themselves raise
1411 <filename>bb.build.FuncFailed</filename>.
1412 Rather, <filename>bb.build.FuncFailed</filename>
1413 should be viewed as a general indicator that the
1414 called function failed by raising an exception.
1415 For example, an exception raised by
1416 <filename>bb.fatal()</filename> will be caught inside
1417 <filename>bb.build.exec_func()</filename>, and a
1418 <filename>bb.build.FuncFailed</filename> will be raised
1419 in response.
1420 </note>
1421 </para></listitem>
1422 </itemizedlist>
1423 </para>
1424
1425 <para>
1426 Due to their simplicity, you should prefer regular Python functions
1427 over BitBake-style Python functions unless you need a feature specific
1428 to BitBake-style Python functions.
1429 Regular Python functions in metadata are a more recent invention than
1430 BitBake-style Python functions, and older code tends to use
1431 <filename>bb.build.exec_func()</filename> more often.
1432 </para>
1433 </section>
1434
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001435 <section id='anonymous-python-functions'>
1436 <title>Anonymous Python Functions</title>
1437
1438 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001439 Sometimes it is useful to set variables or perform
1440 other operations programmatically during parsing.
1441 To do this, you can define special Python functions,
1442 called anonymous Python functions, that run at the
1443 end of parsing.
1444 For example, the following conditionally sets a variable
1445 based on the value of another variable:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001446 <literallayout class='monospaced'>
1447 python () {
Brad Bishop6e60e8b2018-02-01 10:27:11 -05001448 if d.getVar('SOMEVAR') == 'value':
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001449 d.setVar('ANOTHERVAR', 'value2')
1450 }
1451 </literallayout>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001452 An equivalent way to mark a function as an anonymous
1453 function is to give it the name "__anonymous", rather
1454 than no name.
1455 </para>
1456
1457 <para>
1458 Anonymous Python functions always run at the end
1459 of parsing, regardless of where they are defined.
1460 If a recipe contains many anonymous functions, they
1461 run in the same order as they are defined within the
1462 recipe.
1463 As an example, consider the following snippet:
1464 <literallayout class='monospaced'>
1465 python () {
1466 d.setVar('FOO', 'foo 2')
1467 }
1468
1469 FOO = "foo 1"
1470
1471 python () {
1472 d.appendVar('BAR', ' bar 2')
1473 }
1474
1475 BAR = "bar 1"
1476 </literallayout>
1477 The previous example is conceptually equivalent to the
1478 following snippet:
1479 <literallayout class='monospaced'>
1480 FOO = "foo 1"
1481 BAR = "bar 1"
1482 FOO = "foo 2"
1483 BAR += "bar 2"
1484 </literallayout>
1485 <filename>FOO</filename> ends up with the value "foo 2",
1486 and <filename>BAR</filename> with the value "bar 1 bar 2".
1487 Just as in the second snippet, the values set for the
1488 variables within the anonymous functions become available
1489 to tasks, which always run after parsing.
1490 </para>
1491
1492 <para>
1493 Overrides and override-style operators such as
1494 "<filename>_append</filename>" are applied before
1495 anonymous functions run.
1496 In the following example, <filename>FOO</filename> ends
1497 up with the value "foo from anonymous":
1498 <literallayout class='monospaced'>
1499 FOO = "foo"
1500 FOO_append = " from outside"
1501
1502 python () {
1503 d.setVar("FOO", "foo from anonymous")
1504 }
1505 </literallayout>
1506 For methods you can use with anonymous Python functions,
1507 see the
1508 "<link linkend='functions-you-can-call-from-within-python'>Functions You Can Call From Within Python</link>"
1509 section.
1510 For a different method to run Python code during parsing,
1511 see the
1512 "<link linkend='inline-python-variable-expansion'>Inline Python Variable Expansion</link>"
1513 section.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001514 </para>
1515 </section>
1516
1517 <section id='flexible-inheritance-for-class-functions'>
1518 <title>Flexible Inheritance for Class Functions</title>
1519
1520 <para>
1521 Through coding techniques and the use of
1522 <filename>EXPORT_FUNCTIONS</filename>, BitBake supports
1523 exporting a function from a class such that the
1524 class function appears as the default implementation
1525 of the function, but can still be called if a recipe
1526 inheriting the class needs to define its own version of
1527 the function.
1528 </para>
1529
1530 <para>
1531 To understand the benefits of this feature, consider
1532 the basic scenario where a class defines a task function
1533 and your recipe inherits the class.
1534 In this basic scenario, your recipe inherits the task
1535 function as defined in the class.
1536 If desired, your recipe can add to the start and end of the
1537 function by using the "_prepend" or "_append" operations
1538 respectively, or it can redefine the function completely.
1539 However, if it redefines the function, there is
1540 no means for it to call the class version of the function.
1541 <filename>EXPORT_FUNCTIONS</filename> provides a mechanism
1542 that enables the recipe's version of the function to call
1543 the original version of the function.
1544 </para>
1545
1546 <para>
1547 To make use of this technique, you need the following
1548 things in place:
1549 <itemizedlist>
1550 <listitem><para>
1551 The class needs to define the function as follows:
1552 <literallayout class='monospaced'>
1553 <replaceable>classname</replaceable><filename>_</filename><replaceable>functionname</replaceable>
1554 </literallayout>
1555 For example, if you have a class file
1556 <filename>bar.bbclass</filename> and a function named
1557 <filename>do_foo</filename>, the class must define the function
1558 as follows:
1559 <literallayout class='monospaced'>
1560 bar_do_foo
1561 </literallayout>
1562 </para></listitem>
1563 <listitem><para>
1564 The class needs to contain the <filename>EXPORT_FUNCTIONS</filename>
1565 statement as follows:
1566 <literallayout class='monospaced'>
1567 EXPORT_FUNCTIONS <replaceable>functionname</replaceable>
1568 </literallayout>
1569 For example, continuing with the same example, the
1570 statement in the <filename>bar.bbclass</filename> would be
1571 as follows:
1572 <literallayout class='monospaced'>
1573 EXPORT_FUNCTIONS do_foo
1574 </literallayout>
1575 </para></listitem>
1576 <listitem><para>
1577 You need to call the function appropriately from within your
1578 recipe.
1579 Continuing with the same example, if your recipe
1580 needs to call the class version of the function,
1581 it should call <filename>bar_do_foo</filename>.
1582 Assuming <filename>do_foo</filename> was a shell function
1583 and <filename>EXPORT_FUNCTIONS</filename> was used as above,
1584 the recipe's function could conditionally call the
1585 class version of the function as follows:
1586 <literallayout class='monospaced'>
1587 do_foo() {
1588 if [ somecondition ] ; then
1589 bar_do_foo
1590 else
1591 # Do something else
1592 fi
1593 }
1594 </literallayout>
1595 To call your modified version of the function as defined
1596 in your recipe, call it as <filename>do_foo</filename>.
1597 </para></listitem>
1598 </itemizedlist>
1599 With these conditions met, your single recipe
1600 can freely choose between the original function
1601 as defined in the class file and the modified function in your recipe.
1602 If you do not set up these conditions, you are limited to using one function
1603 or the other.
1604 </para>
1605 </section>
1606 </section>
1607
1608 <section id='tasks'>
1609 <title>Tasks</title>
1610
1611 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001612 Tasks are BitBake execution units that make up the
1613 steps that BitBake can run for a given recipe.
1614 Tasks are only supported in recipes and classes
1615 (i.e. in <filename>.bb</filename> files and files
1616 included or inherited from <filename>.bb</filename>
1617 files).
1618 By convention, tasks have names that start with "do_".
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001619 </para>
1620
1621 <section id='promoting-a-function-to-a-task'>
1622 <title>Promoting a Function to a Task</title>
1623
1624 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001625 Tasks are either
1626 <link linkend='shell-functions'>shell functions</link> or
1627 <link linkend='bitbake-style-python-functions'>BitBake-style Python functions</link>
1628 that have been promoted to tasks by using the
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001629 <filename>addtask</filename> command.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001630 The <filename>addtask</filename> command can also
1631 optionally describe dependencies between the
1632 task and other tasks.
1633 Here is an example that shows how to define a task
1634 and declare some dependencies:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001635 <literallayout class='monospaced'>
1636 python do_printdate () {
1637 import time
1638 print time.strftime('%Y%m%d', time.gmtime())
1639 }
1640 addtask printdate after do_fetch before do_build
1641 </literallayout>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001642 The first argument to <filename>addtask</filename>
1643 is the name of the function to promote to
1644 a task.
1645 If the name does not start with "do_", "do_" is
1646 implicitly added, which enforces the convention that
1647 all task names start with "do_".
1648 </para>
1649
1650 <para>
1651 In the previous example, the
1652 <filename>do_printdate</filename> task becomes a
1653 dependency of the <filename>do_build</filename>
1654 task, which is the default task (i.e. the task run by
1655 the <filename>bitbake</filename> command unless
1656 another task is specified explicitly).
1657 Additionally, the <filename>do_printdate</filename>
1658 task becomes dependent upon the
1659 <filename>do_fetch</filename> task.
1660 Running the <filename>do_build</filename> task
1661 results in the <filename>do_printdate</filename>
1662 task running first.
1663 <note>
1664 If you try out the previous example, you might see that
1665 the <filename>do_printdate</filename> task is only run
1666 the first time you build the recipe with
1667 the <filename>bitbake</filename> command.
1668 This is because BitBake considers the task "up-to-date"
1669 after that initial run.
1670 If you want to force the task to always be rerun for
1671 experimentation purposes, you can make BitBake always
1672 consider the task "out-of-date" by using the
1673 <filename>[</filename><link linkend='variable-flags'><filename>nostamp</filename></link><filename>]</filename>
1674 variable flag, as follows:
1675 <literallayout class='monospaced'>
1676 do_printdate[nostamp] = "1"
1677 </literallayout>
1678 You can also explicitly run the task and provide the
1679 <filename>-f</filename> option as follows:
1680 <literallayout class='monospaced'>
1681 $ bitbake <replaceable>recipe</replaceable> -c printdate -f
1682 </literallayout>
1683 When manually selecting a task to run with the
1684 <filename>bitbake</filename>&nbsp;<replaceable>recipe</replaceable>&nbsp;<filename>-c</filename>&nbsp;<replaceable>task</replaceable>
1685 command, you can omit the "do_" prefix as part of the
1686 task name.
1687 </note>
1688 </para>
1689
1690 <para>
1691 You might wonder about the practical effects of using
1692 <filename>addtask</filename> without specifying any
1693 dependencies as is done in the following example:
1694 <literallayout class='monospaced'>
1695 addtask printdate
1696 </literallayout>
1697 In this example, assuming dependencies have not been
1698 added through some other means, the only way to run
1699 the task is by explicitly selecting it with
1700 <filename>bitbake</filename>&nbsp;<replaceable>recipe</replaceable>&nbsp;<filename>-c printdate</filename>.
1701 You can use the
1702 <filename>do_listtasks</filename> task to list all tasks
1703 defined in a recipe as shown in the following example:
1704 <literallayout class='monospaced'>
1705 $ bitbake <replaceable>recipe</replaceable> -c listtasks
1706 </literallayout>
1707 For more information on task dependencies, see the
1708 "<link linkend='dependencies'>Dependencies</link>"
1709 section.
1710 </para>
1711
1712 <para>
1713 See the
1714 "<link linkend='variable-flags'>Variable Flags</link>"
1715 section for information on variable flags you can use with
1716 tasks.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001717 </para>
1718 </section>
1719
1720 <section id='deleting-a-task'>
1721 <title>Deleting a Task</title>
1722
1723 <para>
1724 As well as being able to add tasks, you can delete them.
1725 Simply use the <filename>deltask</filename> command to
1726 delete a task.
1727 For example, to delete the example task used in the previous
1728 sections, you would use:
1729 <literallayout class='monospaced'>
1730 deltask printdate
1731 </literallayout>
1732 If you delete a task using the <filename>deltask</filename>
1733 command and the task has dependencies, the dependencies are
1734 not reconnected.
1735 For example, suppose you have three tasks named
1736 <filename>do_a</filename>, <filename>do_b</filename>, and
1737 <filename>do_c</filename>.
1738 Furthermore, <filename>do_c</filename> is dependent on
1739 <filename>do_b</filename>, which in turn is dependent on
1740 <filename>do_a</filename>.
1741 Given this scenario, if you use <filename>deltask</filename>
1742 to delete <filename>do_b</filename>, the implicit dependency
1743 relationship between <filename>do_c</filename> and
1744 <filename>do_a</filename> through <filename>do_b</filename>
1745 no longer exists, and <filename>do_c</filename> dependencies
1746 are not updated to include <filename>do_a</filename>.
1747 Thus, <filename>do_c</filename> is free to run before
1748 <filename>do_a</filename>.
1749 </para>
1750
1751 <para>
1752 If you want dependencies such as these to remain intact, use
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001753 the <filename>[noexec]</filename> varflag to disable the task
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001754 instead of using the <filename>deltask</filename> command to
1755 delete it:
1756 <literallayout class='monospaced'>
1757 do_b[noexec] = "1"
1758 </literallayout>
1759 </para>
1760 </section>
1761
1762 <section id='passing-information-into-the-build-task-environment'>
1763 <title>Passing Information Into the Build Task Environment</title>
1764
1765 <para>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001766 When running a task, BitBake tightly controls the shell execution
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001767 environment of the build tasks to make
1768 sure unwanted contamination from the build machine cannot
1769 influence the build.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001770 <note>
1771 By default, BitBake cleans the environment to include only those
1772 things exported or listed in its whitelist to ensure that the build
1773 environment is reproducible and consistent.
1774 You can prevent this "cleaning" by setting the
1775 <link linkend='var-BB_PRESERVE_ENV'><filename>BB_PRESERVE_ENV</filename></link>
1776 variable.
1777 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001778 Consequently, if you do want something to get passed into the
1779 build task environment, you must take these two steps:
1780 <orderedlist>
1781 <listitem><para>
1782 Tell BitBake to load what you want from the environment
1783 into the datastore.
1784 You can do so through the
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001785 <link linkend='var-BB_ENV_WHITELIST'><filename>BB_ENV_WHITELIST</filename></link>
1786 and
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001787 <link linkend='var-BB_ENV_EXTRAWHITE'><filename>BB_ENV_EXTRAWHITE</filename></link>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001788 variables.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001789 For example, assume you want to prevent the build system from
1790 accessing your <filename>$HOME/.ccache</filename>
1791 directory.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001792 The following command "whitelists" the environment variable
1793 <filename>CCACHE_DIR</filename> causing BitBack to allow that
1794 variable into the datastore:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001795 <literallayout class='monospaced'>
1796 export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE CCACHE_DIR"
1797 </literallayout></para></listitem>
1798 <listitem><para>
1799 Tell BitBake to export what you have loaded into the
1800 datastore to the task environment of every running task.
1801 Loading something from the environment into the datastore
1802 (previous step) only makes it available in the datastore.
1803 To export it to the task environment of every running task,
1804 use a command similar to the following in your local configuration
1805 file <filename>local.conf</filename> or your
1806 distribution configuration file:
1807 <literallayout class='monospaced'>
1808 export CCACHE_DIR
1809 </literallayout>
1810 <note>
1811 A side effect of the previous steps is that BitBake
1812 records the variable as a dependency of the build process
1813 in things like the setscene checksums.
1814 If doing so results in unnecessary rebuilds of tasks, you can
1815 whitelist the variable so that the setscene code
1816 ignores the dependency when it creates checksums.
1817 </note></para></listitem>
1818 </orderedlist>
1819 </para>
1820
1821 <para>
1822 Sometimes, it is useful to be able to obtain information
1823 from the original execution environment.
1824 Bitbake saves a copy of the original environment into
1825 a special variable named
1826 <link linkend='var-BB_ORIGENV'><filename>BB_ORIGENV</filename></link>.
1827 </para>
1828
1829 <para>
1830 The <filename>BB_ORIGENV</filename> variable returns a datastore
1831 object that can be queried using the standard datastore operators
1832 such as <filename>getVar(, False)</filename>.
1833 The datastore object is useful, for example, to find the original
1834 <filename>DISPLAY</filename> variable.
1835 Here is an example:
1836 <literallayout class='monospaced'>
1837 origenv = d.getVar("BB_ORIGENV", False)
1838 bar = origenv.getVar("BAR", False)
1839 </literallayout>
1840 The previous example returns <filename>BAR</filename> from the original
1841 execution environment.
1842 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001843 </section>
1844 </section>
1845
1846 <section id='variable-flags'>
1847 <title>Variable Flags</title>
1848
1849 <para>
1850 Variable flags (varflags) help control a task's functionality
1851 and dependencies.
1852 BitBake reads and writes varflags to the datastore using the following
1853 command forms:
1854 <literallayout class='monospaced'>
1855 <replaceable>variable</replaceable> = d.getVarFlags("<replaceable>variable</replaceable>")
1856 self.d.setVarFlags("FOO", {"func": True})
1857 </literallayout>
1858 </para>
1859
1860 <para>
1861 When working with varflags, the same syntax, with the exception of
1862 overrides, applies.
1863 In other words, you can set, append, and prepend varflags just like
1864 variables.
1865 See the
1866 "<link linkend='variable-flag-syntax'>Variable Flag Syntax</link>"
1867 section for details.
1868 </para>
1869
1870 <para>
1871 BitBake has a defined set of varflags available for recipes and
1872 classes.
1873 Tasks support a number of these flags which control various
1874 functionality of the task:
1875 <itemizedlist>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001876 <listitem><para><emphasis><filename>[cleandirs]</filename>:</emphasis>
1877 Empty directories that should be created before the
1878 task runs.
1879 Directories that already exist are removed and recreated
1880 to empty them.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001881 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001882 <listitem><para><emphasis><filename>[depends]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001883 Controls inter-task dependencies.
1884 See the
1885 <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link>
1886 variable and the
1887 "<link linkend='inter-task-dependencies'>Inter-Task Dependencies</link>"
1888 section for more information.
1889 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001890 <listitem><para><emphasis><filename>[deptask]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001891 Controls task build-time dependencies.
1892 See the
1893 <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link>
1894 variable and the
1895 "<link linkend='build-dependencies'>Build Dependencies</link>"
1896 section for more information.
1897 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001898 <listitem><para><emphasis><filename>[dirs]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001899 Directories that should be created before the task runs.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001900 Directories that already exist are left as is.
1901 The last directory listed is used as the
1902 current working directory for the task.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001903 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001904 <listitem><para><emphasis><filename>[lockfiles]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001905 Specifies one or more lockfiles to lock while the task
1906 executes.
1907 Only one task may hold a lockfile, and any task that
1908 attempts to lock an already locked file will block until
1909 the lock is released.
1910 You can use this variable flag to accomplish mutual
1911 exclusion.
1912 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001913 <listitem><para><emphasis><filename>[noexec]</filename>:</emphasis>
1914 When set to "1", marks the task as being empty, with
1915 no execution required.
1916 You can use the <filename>[noexec]</filename> flag to set up
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001917 tasks as dependency placeholders, or to disable tasks defined
1918 elsewhere that are not needed in a particular recipe.
1919 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001920 <listitem><para><emphasis><filename>[nostamp]</filename>:</emphasis>
1921 When set to "1", tells BitBake to not generate a stamp
1922 file for a task, which implies the task should always
1923 be executed.
1924 <note><title>Caution</title>
1925 Any task that depends (possibly indirectly) on a
1926 <filename>[nostamp]</filename> task will always be
1927 executed as well.
1928 This can cause unnecessary rebuilding if you are
1929 not careful.
1930 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001931 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001932 <listitem><para><emphasis><filename>[postfuncs]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001933 List of functions to call after the completion of the task.
1934 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001935 <listitem><para><emphasis><filename>[prefuncs]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001936 List of functions to call before the task executes.
1937 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001938 <listitem><para><emphasis><filename>[rdepends]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001939 Controls inter-task runtime dependencies.
1940 See the
1941 <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link>
1942 variable, the
1943 <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link>
1944 variable, and the
1945 "<link linkend='inter-task-dependencies'>Inter-Task Dependencies</link>"
1946 section for more information.
1947 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001948 <listitem><para><emphasis><filename>[rdeptask]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001949 Controls task runtime dependencies.
1950 See the
1951 <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link>
1952 variable, the
1953 <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link>
1954 variable, and the
1955 "<link linkend='runtime-dependencies'>Runtime Dependencies</link>"
1956 section for more information.
1957 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001958 <listitem><para><emphasis><filename>[recideptask]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001959 When set in conjunction with
1960 <filename>recrdeptask</filename>, specifies a task that
1961 should be inspected for additional dependencies.
1962 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001963 <listitem><para><emphasis><filename>[recrdeptask]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001964 Controls task recursive runtime dependencies.
1965 See the
1966 <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link>
1967 variable, the
1968 <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link>
1969 variable, and the
1970 "<link linkend='recursive-dependencies'>Recursive Dependencies</link>"
1971 section for more information.
1972 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001973 <listitem><para><emphasis><filename>[stamp-extra-info]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001974 Extra stamp information to append to the task's stamp.
1975 As an example, OpenEmbedded uses this flag to allow
1976 machine-specific tasks.
1977 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001978 <listitem><para><emphasis><filename>[umask]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001979 The umask to run the task under.
1980 </para></listitem>
1981 </itemizedlist>
1982 </para>
1983
1984 <para>
1985 Several varflags are useful for controlling how signatures are
1986 calculated for variables.
1987 For more information on this process, see the
1988 "<link linkend='checksums'>Checksums (Signatures)</link>"
1989 section.
1990 <itemizedlist>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001991 <listitem><para><emphasis><filename>[vardeps]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001992 Specifies a space-separated list of additional
1993 variables to add to a variable's dependencies
1994 for the purposes of calculating its signature.
1995 Adding variables to this list is useful, for example, when
1996 a function refers to a variable in a manner that
1997 does not allow BitBake to automatically determine
1998 that the variable is referred to.
1999 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002000 <listitem><para><emphasis><filename>[vardepsexclude]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002001 Specifies a space-separated list of variables
2002 that should be excluded from a variable's dependencies
2003 for the purposes of calculating its signature.
2004 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002005 <listitem><para><emphasis><filename>[vardepvalue]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002006 If set, instructs BitBake to ignore the actual
2007 value of the variable and instead use the specified
2008 value when calculating the variable's signature.
2009 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002010 <listitem><para><emphasis><filename>[vardepvalueexclude]</filename>:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002011 Specifies a pipe-separated list of strings to exclude
2012 from the variable's value when calculating the
2013 variable's signature.
2014 </para></listitem>
2015 </itemizedlist>
2016 </para>
2017 </section>
2018
2019 <section id='events'>
2020 <title>Events</title>
2021
2022 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002023 BitBake allows installation of event handlers within recipe
2024 and class files.
2025 Events are triggered at certain points during operation, such
2026 as the beginning of operation against a given recipe
2027 (i.e. <filename>*.bb</filename>), the start of a given task,
2028 a task failure, a task success, and so forth.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002029 The intent is to make it easy to do things like email
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002030 notification on build failures.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002031 </para>
2032
2033 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002034 Following is an example event handler that prints the name
2035 of the event and the content of the
2036 <filename>FILE</filename> variable:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002037 <literallayout class='monospaced'>
2038 addhandler myclass_eventhandler
2039 python myclass_eventhandler() {
2040 from bb.event import getName
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002041 print("The name of the Event is %s" % getName(e))
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002042 print("The file we run for is %s" % d.getVar('FILE'))
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002043 }
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002044 myclass_eventhandler[eventmask] = "bb.event.BuildStarted bb.event.BuildCompleted"
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002045 </literallayout>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002046 In the previous example, an eventmask has been set so that
2047 the handler only sees the "BuildStarted" and "BuildCompleted"
2048 events.
2049 This event handler gets called every time an event matching
2050 the eventmask is triggered.
2051 A global variable "e" is defined, which represents the current
2052 event.
2053 With the <filename>getName(e)</filename> method, you can get
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002054 the name of the triggered event.
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002055 The global datastore is available as "d".
2056 In legacy code, you might see "e.data" used to get the datastore.
2057 However, realize that "e.data" is deprecated and you should use
2058 "d" going forward.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002059 </para>
2060
2061 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002062 The context of the datastore is appropriate to the event
2063 in question.
2064 For example, "BuildStarted" and "BuildCompleted" events run
2065 before any tasks are executed so would be in the global
2066 configuration datastore namespace.
2067 No recipe-specific metadata exists in that namespace.
Brad Bishopd7bf8c12018-02-25 22:55:05 -05002068 The "BuildStarted" and "BuildCompleted" events also run in
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002069 the main cooker/server process rather than any worker context.
2070 Thus, any changes made to the datastore would be seen by other
2071 cooker/server events within the current build but not seen
2072 outside of that build or in any worker context.
2073 Task events run in the actual tasks in question consequently
2074 have recipe-specific and task-specific contents.
2075 These events run in the worker context and are discarded at
2076 the end of task execution.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002077 </para>
2078
2079 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002080 During a standard build, the following common events might
2081 occur.
2082 The following events are the most common kinds of events that
2083 most metadata might have an interest in viewing:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002084 <itemizedlist>
2085 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002086 <filename>bb.event.ConfigParsed()</filename>:
2087 Fired when the base configuration; which consists of
2088 <filename>bitbake.conf</filename>,
2089 <filename>base.bbclass</filename> and any global
2090 <filename>INHERIT</filename> statements; has been parsed.
2091 You can see multiple such events when each of the
2092 workers parse the base configuration or if the server
2093 changes configuration and reparses.
2094 Any given datastore only has one such event executed
2095 against it, however.
2096 If
2097 <link linkende='var-BB_INVALIDCONF'><filename>BB_INVALIDCONF</filename></link>
2098 is set in the datastore by the event handler, the
2099 configuration is reparsed and a new event triggered,
2100 allowing the metadata to update configuration.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002101 </para></listitem>
2102 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002103 <filename>bb.event.HeartbeatEvent()</filename>:
2104 Fires at regular time intervals of one second.
2105 You can configure the interval time using the
2106 <filename>BB_HEARTBEAT_EVENT</filename> variable.
2107 The event's "time" attribute is the
2108 <filename>time.time()</filename> value when the
2109 event is triggered.
2110 This event is useful for activities such as
2111 system state monitoring.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002112 </para></listitem>
2113 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002114 <filename>bb.event.ParseStarted()</filename>:
2115 Fired when BitBake is about to start parsing recipes.
2116 This event's "total" attribute represents the number of
2117 recipes BitBake plans to parse.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002118 </para></listitem>
2119 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002120 <filename>bb.event.ParseProgress()</filename>:
2121 Fired as parsing progresses.
2122 This event's "current" attribute is the number of
2123 recipes parsed as well as the "total" attribute.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002124 </para></listitem>
2125 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002126 <filename>bb.event.ParseCompleted()</filename>:
2127 Fired when parsing is complete.
2128 This event's "cached", "parsed", "skipped", "virtuals",
2129 "masked", and "errors" attributes provide statistics
2130 for the parsing results.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002131 </para></listitem>
2132 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002133 <filename>bb.event.BuildStarted()</filename>:
2134 Fired when a new build starts.
Brad Bishop316dfdd2018-06-25 12:45:53 -04002135 BitBake fires multiple "BuildStarted" events (one per configuration)
2136 when multiple configuration (multiconfig) is enabled.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002137 </para></listitem>
2138 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002139 <filename>bb.build.TaskStarted()</filename>:
2140 Fired when a task starts.
2141 This event's "taskfile" attribute points to the recipe
2142 from which the task originates.
2143 The "taskname" attribute, which is the task's name,
2144 includes the <filename>do_</filename> prefix, and the
2145 "logfile" attribute point to where the task's output is
2146 stored.
2147 Finally, the "time" attribute is the task's execution start
2148 time.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002149 </para></listitem>
2150 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002151 <filename>bb.build.TaskInvalid()</filename>:
2152 Fired if BitBake tries to execute a task that does not exist.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002153 </para></listitem>
2154 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002155 <filename>bb.build.TaskFailedSilent()</filename>:
2156 Fired for setscene tasks that fail and should not be
2157 presented to the user verbosely.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002158 </para></listitem>
2159 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002160 <filename>bb.build.TaskFailed()</filename>:
2161 Fired for normal tasks that fail.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002162 </para></listitem>
2163 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002164 <filename>bb.build.TaskSucceeded()</filename>:
2165 Fired when a task successfully completes.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002166 </para></listitem>
2167 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002168 <filename>bb.event.BuildCompleted()</filename>:
2169 Fired when a build finishes.
2170 </para></listitem>
2171 <listitem><para>
2172 <filename>bb.cooker.CookerExit()</filename>:
2173 Fired when the BitBake server/cooker shuts down.
2174 This event is usually only seen by the UIs as a
2175 sign they should also shutdown.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002176 </para></listitem>
2177 </itemizedlist>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002178 </para>
2179
2180 <para>
2181 This next list of example events occur based on specific
2182 requests to the server.
2183 These events are often used to communicate larger pieces of
2184 information from the BitBake server to other parts of
2185 BitBake such as user interfaces:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002186 <itemizedlist>
2187 <listitem><para>
2188 <filename>bb.event.TreeDataPreparationStarted()</filename>
2189 </para></listitem>
2190 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002191 <filename>bb.event.TreeDataPreparationProgress()</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002192 </para></listitem>
2193 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002194 <filename>bb.event.TreeDataPreparationCompleted()</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002195 </para></listitem>
2196 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002197 <filename>bb.event.DepTreeGenerated()</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002198 </para></listitem>
2199 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002200 <filename>bb.event.CoreBaseFilesFound()</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002201 </para></listitem>
2202 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002203 <filename>bb.event.ConfigFilePathFound()</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002204 </para></listitem>
2205 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002206 <filename>bb.event.FilesMatchingFound()</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002207 </para></listitem>
2208 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002209 <filename>bb.event.ConfigFilesFound()</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002210 </para></listitem>
2211 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002212 <filename>bb.event.TargetsTreeGenerated()</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002213 </para></listitem>
2214 </itemizedlist>
2215 </para>
2216 </section>
2217
2218 <section id='variants-class-extension-mechanism'>
2219 <title>Variants - Class Extension Mechanism</title>
2220
2221 <para>
2222 BitBake supports two features that facilitate creating
2223 from a single recipe file multiple incarnations of that
2224 recipe file where all incarnations are buildable.
2225 These features are enabled through the
2226 <link linkend='var-BBCLASSEXTEND'><filename>BBCLASSEXTEND</filename></link>
2227 and
2228 <link linkend='var-BBVERSIONS'><filename>BBVERSIONS</filename></link>
2229 variables.
2230 <note>
2231 The mechanism for this class extension is extremely
2232 specific to the implementation.
2233 Usually, the recipe's
2234 <link linkend='var-PROVIDES'><filename>PROVIDES</filename></link>,
2235 <link linkend='var-PN'><filename>PN</filename></link>, and
2236 <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link>
2237 variables would need to be modified by the extension class.
2238 For specific examples, see the OE-Core
2239 <filename>native</filename>, <filename>nativesdk</filename>,
2240 and <filename>multilib</filename> classes.
2241 </note>
2242 <itemizedlist>
2243 <listitem><para><emphasis><filename>BBCLASSEXTEND</filename>:</emphasis>
2244 This variable is a space separated list of classes used to "extend" the
2245 recipe for each variant.
2246 Here is an example that results in a second incarnation of the current
2247 recipe being available.
2248 This second incarnation will have the "native" class inherited.
2249 <literallayout class='monospaced'>
2250 BBCLASSEXTEND = "native"
2251 </literallayout></para></listitem>
2252 <listitem><para><emphasis><filename>BBVERSIONS</filename>:</emphasis>
2253 This variable allows a single recipe to build multiple versions of a
2254 project from a single recipe file.
2255 You can also specify conditional metadata
2256 (using the
2257 <link linkend='var-OVERRIDES'><filename>OVERRIDES</filename></link>
2258 mechanism) for a single version, or an optionally named range of versions.
2259 Here is an example:
2260 <literallayout class='monospaced'>
2261 BBVERSIONS = "1.0 2.0 git"
2262 SRC_URI_git = "git://someurl/somepath.git"
2263
2264 BBVERSIONS = "1.0.[0-6]:1.0.0+ \ 1.0.[7-9]:1.0.7+"
2265 SRC_URI_append_1.0.7+ = "file://some_patch_which_the_new_versions_need.patch;patch=1"
2266 </literallayout>
2267 The name of the range defaults to the original version of the
2268 recipe.
2269 For example, in OpenEmbedded, the recipe file
2270 <filename>foo_1.0.0+.bb</filename> creates a default name range
2271 of <filename>1.0.0+</filename>.
2272 This is useful because the range name is not only placed
2273 into overrides, but it is also made available for the metadata to use
2274 in the variable that defines the base recipe versions for use in
2275 <filename>file://</filename> search paths
2276 (<link linkend='var-FILESPATH'><filename>FILESPATH</filename></link>).
2277 </para></listitem>
2278 </itemizedlist>
2279 </para>
2280 </section>
2281
2282 <section id='dependencies'>
2283 <title>Dependencies</title>
2284
2285 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002286 To allow for efficient parallel processing, BitBake handles
2287 dependencies at the task level.
2288 Dependencies can exist both between tasks within a single recipe
2289 and between tasks in different recipes.
2290 Following are examples of each:
2291 <itemizedlist>
2292 <listitem><para>For tasks within a single recipe, a
2293 recipe's <filename>do_configure</filename>
2294 task might need to complete before its
2295 <filename>do_compile</filename> task can run.
2296 </para></listitem>
2297 <listitem><para>For tasks in different recipes, one
2298 recipe's <filename>do_configure</filename>
2299 task might require another recipe's
2300 <filename>do_populate_sysroot</filename>
2301 task to finish first such that the libraries and headers
2302 provided by the other recipe are available.
2303 </para></listitem>
2304 </itemizedlist>
2305 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002306
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002307 <para>
2308 This section describes several ways to declare dependencies.
2309 Remember, even though dependencies are declared in different ways, they
2310 are all simply dependencies between tasks.
2311 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002312
2313 <section id='dependencies-internal-to-the-bb-file'>
2314 <title>Dependencies Internal to the <filename>.bb</filename> File</title>
2315
2316 <para>
2317 BitBake uses the <filename>addtask</filename> directive
2318 to manage dependencies that are internal to a given recipe
2319 file.
2320 You can use the <filename>addtask</filename> directive to
2321 indicate when a task is dependent on other tasks or when
2322 other tasks depend on that recipe.
2323 Here is an example:
2324 <literallayout class='monospaced'>
2325 addtask printdate after do_fetch before do_build
2326 </literallayout>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002327 In this example, the <filename>do_printdate</filename>
2328 task depends on the completion of the
2329 <filename>do_fetch</filename> task, and the
2330 <filename>do_build</filename> task depends on the
2331 completion of the <filename>do_printdate</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002332 task.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002333 <note><para>
2334 For a task to run, it must be a direct or indirect
2335 dependency of some other task that is scheduled to
2336 run.</para>
2337
2338 <para>For illustration, here are some examples:
2339 <itemizedlist>
2340 <listitem><para>
2341 The directive
2342 <filename>addtask mytask before do_configure</filename>
2343 causes <filename>do_mytask</filename> to run before
2344 <filename>do_configure</filename> runs.
2345 Be aware that <filename>do_mytask</filename> still only
2346 runs if its <link linkend='checksums'>input checksum</link>
2347 has changed since the last time it was run.
2348 Changes to the input checksum of
2349 <filename>do_mytask</filename> also indirectly cause
2350 <filename>do_configure</filename> to run.
2351 </para></listitem>
2352 <listitem><para>
2353 The directive
2354 <filename>addtask mytask after do_configure</filename>
2355 by itself never causes <filename>do_mytask</filename>
2356 to run.
2357 <filename>do_mytask</filename> can still be run manually
2358 as follows:
2359 <literallayout class='monospaced'>
2360 $ bitbake <replaceable>recipe</replaceable> -c mytask
2361 </literallayout>
2362 Declaring <filename>do_mytask</filename> as a dependency
2363 of some other task that is scheduled to run also causes
2364 it to run.
2365 Regardless, the task runs after
2366 <filename>do_configure</filename>.
2367 </para></listitem>
2368 </itemizedlist></para>
2369 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002370 </para>
2371 </section>
2372
2373 <section id='build-dependencies'>
2374 <title>Build Dependencies</title>
2375
2376 <para>
2377 BitBake uses the
2378 <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link>
2379 variable to manage build time dependencies.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002380 The <filename>[deptask]</filename> varflag for tasks
2381 signifies the task of each
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002382 item listed in <filename>DEPENDS</filename> that must
2383 complete before that task can be executed.
2384 Here is an example:
2385 <literallayout class='monospaced'>
2386 do_configure[deptask] = "do_populate_sysroot"
2387 </literallayout>
2388 In this example, the <filename>do_populate_sysroot</filename>
2389 task of each item in <filename>DEPENDS</filename> must complete before
2390 <filename>do_configure</filename> can execute.
2391 </para>
2392 </section>
2393
2394 <section id='runtime-dependencies'>
2395 <title>Runtime Dependencies</title>
2396
2397 <para>
2398 BitBake uses the
2399 <link linkend='var-PACKAGES'><filename>PACKAGES</filename></link>,
2400 <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link>, and
2401 <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link>
2402 variables to manage runtime dependencies.
2403 </para>
2404
2405 <para>
2406 The <filename>PACKAGES</filename> variable lists runtime
2407 packages.
2408 Each of those packages can have <filename>RDEPENDS</filename> and
2409 <filename>RRECOMMENDS</filename> runtime dependencies.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002410 The <filename>[rdeptask]</filename> flag for tasks is used to
2411 signify the task of each
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002412 item runtime dependency which must have completed before that
2413 task can be executed.
2414 <literallayout class='monospaced'>
2415 do_package_qa[rdeptask] = "do_packagedata"
2416 </literallayout>
2417 In the previous example, the <filename>do_packagedata</filename>
2418 task of each item in <filename>RDEPENDS</filename> must have
2419 completed before <filename>do_package_qa</filename> can execute.
2420 </para>
2421 </section>
2422
2423 <section id='recursive-dependencies'>
2424 <title>Recursive Dependencies</title>
2425
2426 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002427 BitBake uses the <filename>[recrdeptask]</filename> flag to manage
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002428 recursive task dependencies.
2429 BitBake looks through the build-time and runtime
2430 dependencies of the current recipe, looks through
2431 the task's inter-task
2432 dependencies, and then adds dependencies for the
2433 listed task.
2434 Once BitBake has accomplished this, it recursively works through
2435 the dependencies of those tasks.
2436 Iterative passes continue until all dependencies are discovered
2437 and added.
2438 </para>
2439
2440 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002441 The <filename>[recrdeptask]</filename> flag is most commonly
2442 used in high-level
2443 recipes that need to wait for some task to finish "globally".
2444 For example, <filename>image.bbclass</filename> has the following:
2445 <literallayout class='monospaced'>
2446 do_rootfs[recrdeptask] += "do_packagedata"
2447 </literallayout>
2448 This statement says that the <filename>do_packagedata</filename>
2449 task of the current recipe and all recipes reachable
2450 (by way of dependencies) from the
2451 image recipe must run before the <filename>do_rootfs</filename>
2452 task can run.
2453 </para>
2454
2455 <para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002456 You might want to not only have BitBake look for
2457 dependencies of those tasks, but also have BitBake look
2458 for build-time and runtime dependencies of the dependent
2459 tasks as well.
2460 If that is the case, you need to reference the task name
2461 itself in the task list:
2462 <literallayout class='monospaced'>
2463 do_a[recrdeptask] = "do_a do_b"
2464 </literallayout>
2465 </para>
2466 </section>
2467
2468 <section id='inter-task-dependencies'>
2469 <title>Inter-Task Dependencies</title>
2470
2471 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002472 BitBake uses the <filename>[depends]</filename>
2473 flag in a more generic form
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002474 to manage inter-task dependencies.
2475 This more generic form allows for inter-dependency
2476 checks for specific tasks rather than checks for
2477 the data in <filename>DEPENDS</filename>.
2478 Here is an example:
2479 <literallayout class='monospaced'>
2480 do_patch[depends] = "quilt-native:do_populate_sysroot"
2481 </literallayout>
2482 In this example, the <filename>do_populate_sysroot</filename>
2483 task of the target <filename>quilt-native</filename>
2484 must have completed before the
2485 <filename>do_patch</filename> task can execute.
2486 </para>
2487
2488 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002489 The <filename>[rdepends]</filename> flag works in a similar
2490 way but takes targets
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002491 in the runtime namespace instead of the build-time dependency
2492 namespace.
2493 </para>
2494 </section>
2495 </section>
2496
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002497 <section id='functions-you-can-call-from-within-python'>
2498 <title>Functions You Can Call From Within Python</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002499
2500 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002501 BitBake provides many functions you can call from
2502 within Python functions.
2503 This section lists the most commonly used functions,
2504 and mentions where to find others.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002505 </para>
2506
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002507 <section id='functions-for-accessing-datastore-variables'>
2508 <title>Functions for Accessing Datastore Variables</title>
2509
2510 <para>
2511 It is often necessary to access variables in the
2512 BitBake datastore using Python functions.
2513 The Bitbake datastore has an API that allows you this
2514 access.
2515 Here is a list of available operations:
2516 </para>
2517
2518 <para>
2519 <informaltable frame='none'>
2520 <tgroup cols='2' align='left' colsep='1' rowsep='1'>
2521 <colspec colname='c1' colwidth='1*'/>
2522 <colspec colname='c2' colwidth='1*'/>
2523 <thead>
2524 <row>
2525 <entry align="left"><emphasis>Operation</emphasis></entry>
2526 <entry align="left"><emphasis>Description</emphasis></entry>
2527 </row>
2528 </thead>
2529 <tbody>
2530 <row>
2531 <entry align="left"><filename>d.getVar("X", expand)</filename></entry>
2532 <entry align="left">Returns the value of variable "X".
2533 Using "expand=True" expands the value.
2534 Returns "None" if the variable "X" does not exist.</entry>
2535 </row>
2536 <row>
2537 <entry align="left"><filename>d.setVar("X", "value")</filename></entry>
2538 <entry align="left">Sets the variable "X" to "value".</entry>
2539 </row>
2540 <row>
2541 <entry align="left"><filename>d.appendVar("X", "value")</filename></entry>
2542 <entry align="left">Adds "value" to the end of the variable "X".
2543 Acts like <filename>d.setVar("X", "value")</filename>
2544 if the variable "X" does not exist.</entry>
2545 </row>
2546 <row>
2547 <entry align="left"><filename>d.prependVar("X", "value")</filename></entry>
2548 <entry align="left">Adds "value" to the start of the variable "X".
2549 Acts like <filename>d.setVar("X", "value")</filename>
2550 if the variable "X" does not exist.</entry>
2551 </row>
2552 <row>
2553 <entry align="left"><filename>d.delVar("X")</filename></entry>
2554 <entry align="left">Deletes the variable "X" from the datastore.
2555 Does nothing if the variable "X" does not exist.</entry>
2556 </row>
2557 <row>
2558 <entry align="left"><filename>d.renameVar("X", "Y")</filename></entry>
2559 <entry align="left">Renames the variable "X" to "Y".
2560 Does nothing if the variable "X" does not exist.</entry>
2561 </row>
2562 <row>
2563 <entry align="left"><filename>d.getVarFlag("X", flag, expand)</filename></entry>
2564 <entry align="left">Returns the value of variable "X".
2565 Using "expand=True" expands the value.
2566 Returns "None" if either the variable "X" or the named flag
2567 does not exist.</entry>
2568 </row>
2569 <row>
2570 <entry align="left"><filename>d.setVarFlag("X", flag, "value")</filename></entry>
2571 <entry align="left">Sets the named flag for variable "X" to "value".</entry>
2572 </row>
2573 <row>
2574 <entry align="left"><filename>d.appendVarFlag("X", flag, "value")</filename></entry>
2575 <entry align="left">Appends "value" to the named flag on the
2576 variable "X".
2577 Acts like <filename>d.setVarFlag("X", flag, "value")</filename>
2578 if the named flag does not exist.</entry>
2579 </row>
2580 <row>
2581 <entry align="left"><filename>d.prependVarFlag("X", flag, "value")</filename></entry>
2582 <entry align="left">Prepends "value" to the named flag on
2583 the variable "X".
2584 Acts like <filename>d.setVarFlag("X", flag, "value")</filename>
2585 if the named flag does not exist.</entry>
2586 </row>
2587 <row>
2588 <entry align="left"><filename>d.delVarFlag("X", flag)</filename></entry>
2589 <entry align="left">Deletes the named flag on the variable
2590 "X" from the datastore.</entry>
2591 </row>
2592 <row>
2593 <entry align="left"><filename>d.setVarFlags("X", flagsdict)</filename></entry>
2594 <entry align="left">Sets the flags specified in
2595 the <filename>flagsdict()</filename> parameter.
2596 <filename>setVarFlags</filename> does not clear previous flags.
2597 Think of this operation as <filename>addVarFlags</filename>.</entry>
2598 </row>
2599 <row>
2600 <entry align="left"><filename>d.getVarFlags("X")</filename></entry>
2601 <entry align="left">Returns a <filename>flagsdict</filename>
2602 of the flags for the variable "X".
2603 Returns "None" if the variable "X" does not exist.</entry>
2604 </row>
2605 <row>
2606 <entry align="left"><filename>d.delVarFlags("X")</filename></entry>
2607 <entry align="left">Deletes all the flags for the variable "X".
2608 Does nothing if the variable "X" does not exist.</entry>
2609 </row>
2610 <row>
2611 <entry align="left"><filename>d.expand(expression)</filename></entry>
2612 <entry align="left">Expands variable references in the specified
2613 string expression.
2614 References to variables that do not exist are left as is.
2615 For example, <filename>d.expand("foo ${X}")</filename>
2616 expands to the literal string "foo ${X}" if the
2617 variable "X" does not exist.</entry>
2618 </row>
2619 </tbody>
2620 </tgroup>
2621 </informaltable>
2622 </para>
2623 </section>
2624
2625 <section id='other-functions'>
2626 <title>Other Functions</title>
2627
2628 <para>
2629 You can find many other functions that can be called
2630 from Python by looking at the source code of the
2631 <filename>bb</filename> module, which is in
2632 <filename>bitbake/lib/bb</filename>.
2633 For example,
2634 <filename>bitbake/lib/bb/utils.py</filename> includes
2635 the commonly used functions
2636 <filename>bb.utils.contains()</filename> and
2637 <filename>bb.utils.mkdirhier()</filename>, which come
2638 with docstrings.
2639 </para>
2640 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002641 </section>
2642
2643 <section id='task-checksums-and-setscene'>
2644 <title>Task Checksums and Setscene</title>
2645
2646 <para>
2647 BitBake uses checksums (or signatures) along with the setscene
2648 to determine if a task needs to be run.
2649 This section describes the process.
2650 To help understand how BitBake does this, the section assumes an
2651 OpenEmbedded metadata-based example.
2652 </para>
2653
2654 <para>
Brad Bishop316dfdd2018-06-25 12:45:53 -04002655 These checksums are stored in
2656 <link linkend='var-STAMP'><filename>STAMP</filename></link>.
2657 You can examine the checksums using the following BitBake command:
2658 <literallayout class='monospaced'>
2659 $ bitbake-dumpsigs
2660 </literallayout>
2661 This command returns the signature data in a readable format
2662 that allows you to examine the inputs used when the
2663 OpenEmbedded build system generates signatures.
2664 For example, using <filename>bitbake-dumpsigs</filename>
2665 allows you to examine the <filename>do_compile</filename>
2666 task's “sigdata” for a C application (e.g.
2667 <filename>bash</filename>).
2668 Running the command also reveals that the “CC” variable is part of
2669 the inputs that are hashed.
2670 Any changes to this variable would invalidate the stamp and
2671 cause the <filename>do_compile</filename> task to run.
2672 </para>
2673
2674 <para>
2675 The following list describes related variables:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002676 <itemizedlist>
Brad Bishop316dfdd2018-06-25 12:45:53 -04002677 <listitem><para>
2678 <link linkend='var-BB_HASHCHECK_FUNCTION'><filename>BB_HASHCHECK_FUNCTION</filename></link>:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002679 Specifies the name of the function to call during
2680 the "setscene" part of the task's execution in order
2681 to validate the list of task hashes.
2682 </para></listitem>
Brad Bishop316dfdd2018-06-25 12:45:53 -04002683 <listitem><para>
2684 <link linkend='var-BB_SETSCENE_DEPVALID'><filename>BB_SETSCENE_DEPVALID</filename></link>:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002685 Specifies a function BitBake calls that determines
2686 whether BitBake requires a setscene dependency to
2687 be met.
2688 </para></listitem>
Brad Bishop316dfdd2018-06-25 12:45:53 -04002689 <listitem><para>
2690 <link linkend='var-BB_SETSCENE_VERIFY_FUNCTION2'><filename>BB_SETSCENE_VERIFY_FUNCTION2</filename></link>:
2691 Specifies a function to call that verifies the list of
2692 planned task execution before the main task execution
2693 happens.
2694 </para></listitem>
2695 <listitem><para>
2696 <link linkend='var-BB_STAMP_POLICY'><filename>BB_STAMP_POLICY</filename></link>:
2697 Defines the mode for comparing timestamps of stamp files.
2698 </para></listitem>
2699 <listitem><para>
2700 <link linkend='var-BB_STAMP_WHITELIST'><filename>BB_STAMP_WHITELIST</filename></link>:
2701 Lists stamp files that are looked at when the stamp policy
2702 is "whitelist".
2703 </para></listitem>
2704 <listitem><para>
2705 <link linkend='var-BB_TASKHASH'><filename>BB_TASKHASH</filename></link>:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002706 Within an executing task, this variable holds the hash
2707 of the task as returned by the currently enabled
2708 signature generator.
2709 </para></listitem>
Brad Bishop316dfdd2018-06-25 12:45:53 -04002710 <listitem><para>
2711 <link linkend='var-STAMP'><filename>STAMP</filename></link>:
2712 The base path to create stamp files.
2713 </para></listitem>
2714 <listitem><para>
2715 <link linkend='var-STAMPCLEAN'><filename>STAMPCLEAN</filename></link>:
2716 Again, the base path to create stamp files but can use wildcards
2717 for matching a range of files for clean operations.
2718 </para></listitem>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002719 </itemizedlist>
2720 </para>
2721 </section>
2722</chapter>