blob: 6329cd6e458964fdbcf448ae2c8c5cb7491af0a0 [file] [log] [blame]
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
3
4<chapter id="bitbake-user-manual-metadata">
5 <title>Syntax and Operators</title>
6
7 <para>
8 Bitbake files have their own syntax.
9 The syntax has similarities to several
10 other languages but also has some unique features.
11 This section describes the available syntax and operators
12 as well as provides examples.
13 </para>
14
15 <section id='basic-syntax'>
16 <title>Basic Syntax</title>
17
18 <para>
19 This section provides some basic syntax examples.
20 </para>
21
22 <section id='basic-variable-setting'>
23 <title>Basic Variable Setting</title>
24
25 <para>
26 The following example sets <filename>VARIABLE</filename> to
27 "value".
28 This assignment occurs immediately as the statement is parsed.
29 It is a "hard" assignment.
30 <literallayout class='monospaced'>
31 VARIABLE = "value"
32 </literallayout>
33 As expected, if you include leading or trailing spaces as part of
34 an assignment, the spaces are retained:
35 <literallayout class='monospaced'>
36 VARIABLE = " value"
37 VARIABLE = "value "
38 </literallayout>
39 Setting <filename>VARIABLE</filename> to "" sets it to an empty string,
40 while setting the variable to " " sets it to a blank space
41 (i.e. these are not the same values).
42 <literallayout class='monospaced'>
43 VARIABLE = ""
44 VARIABLE = " "
45 </literallayout>
46 </para>
47 </section>
48
49 <section id='variable-expansion'>
50 <title>Variable Expansion</title>
51
52 <para>
53 BitBake supports variables referencing one another's
54 contents using a syntax that is similar to shell scripting.
55 Following is an example that results in <filename>A</filename>
56 containing "aval" and <filename>B</filename> evaluating to
57 "preavalpost" based on that current value of
58 <filename>A</filename>.
59 <literallayout class='monospaced'>
60 A = "aval"
61 B = "pre${A}post"
62 </literallayout>
63 You should realize that whenever <filename>B</filename> is
64 referenced, its evaluation will depend on the state of
65 <filename>A</filename> at that time.
66 Thus, later evaluations of <filename>B</filename> in the
67 previous example could result in different values
68 depending on the value of <filename>A</filename>.
69 </para>
70 </section>
71
72 <section id='setting-a-default-value'>
73 <title>Setting a default value (?=)</title>
74
75 <para>
76 You can use the "?=" operator to achieve a "softer" assignment
77 for a variable.
78 This type of assignment allows you to define a variable if it
79 is undefined when the statement is parsed, but to leave the
80 value alone if the variable has a value.
81 Here is an example:
82 <literallayout class='monospaced'>
83 A ?= "aval"
84 </literallayout>
85 If <filename>A</filename> is set at the time this statement is parsed,
86 the variable retains its value.
87 However, if <filename>A</filename> is not set,
88 the variable is set to "aval".
89 <note>
90 This assignment is immediate.
91 Consequently, if multiple "?=" assignments
92 to a single variable exist, the first of those ends up getting
93 used.
94 </note>
95 </para>
96 </section>
97
98 <section id='setting-a-weak-default-value'>
99 <title>Setting a weak default value (??=)</title>
100
101 <para>
102 It is possible to use a "weaker" assignment than in the
103 previous section by using the "??=" operator.
104 This assignment behaves identical to "?=" except that the
105 assignment is made at the end of the parsing process rather
106 than immediately.
107 Consequently, when multiple "??=" assignments exist, the last
108 one is used.
109 Also, any "=" or "?=" assignment will override the value set with
110 "??=".
111 Here is an example:
112 <literallayout class='monospaced'>
113 A ??= "somevalue"
114 A ??= "someothervalue"
115 </literallayout>
116 If <filename>A</filename> is set before the above statements are parsed,
117 the variable retains its value.
118 If <filename>A</filename> is not set,
119 the variable is set to "someothervalue".
120 </para>
121
122 <para>
123 Again, this assignment is a "lazy" or "weak" assignment
124 because it does not occur until the end
125 of the parsing process.
126 </para>
127 </section>
128
129 <section id='immediate-variable-expansion'>
130 <title>Immediate variable expansion (:=)</title>
131
132 <para>
133 The ":=" operator results in a variable's
134 contents being expanded immediately,
135 rather than when the variable is actually used:
136 <literallayout class='monospaced'>
137 T = "123"
138 A := "${B} ${A} test ${T}"
139 T = "456"
140 B = "${T} bval"
141 C = "cval"
142 C := "${C}append"
143 </literallayout>
144 In this example, <filename>A</filename> contains
145 "test 123" because <filename>${B}</filename> and
146 <filename>${A}</filename> at the time of parsing are undefined,
147 which leaves "test 123".
148 And, the variable <filename>C</filename>
149 contains "cvalappend" since <filename>${C}</filename> immediately
150 expands to "cval".
151 </para>
152 </section>
153
154 <section id='appending-and-prepending'>
155 <title>Appending (+=) and prepending (=+) With Spaces</title>
156
157 <para>
158 Appending and prepending values is common and can be accomplished
159 using the "+=" and "=+" operators.
160 These operators insert a space between the current
161 value and prepended or appended value.
162 </para>
163
164 <para>
165 These operators take immediate effect during parsing.
166 Here are some examples:
167 <literallayout class='monospaced'>
168 B = "bval"
169 B += "additionaldata"
170 C = "cval"
171 C =+ "test"
172 </literallayout>
173 The variable <filename>B</filename> contains
174 "bval additionaldata" and <filename>C</filename>
175 contains "test cval".
176 </para>
177 </section>
178
179 <section id='appending-and-prepending-without-spaces'>
180 <title>Appending (.=) and Prepending (=.) Without Spaces</title>
181
182 <para>
183 If you want to append or prepend values without an
184 inserted space, use the ".=" and "=." operators.
185 </para>
186
187 <para>
188 These operators take immediate effect during parsing.
189 Here are some examples:
190 <literallayout class='monospaced'>
191 B = "bval"
192 B .= "additionaldata"
193 C = "cval"
194 C =. "test"
195 </literallayout>
196 The variable <filename>B</filename> contains
197 "bvaladditionaldata" and
198 <filename>C</filename> contains "testcval".
199 </para>
200 </section>
201
202 <section id='appending-and-prepending-override-style-syntax'>
203 <title>Appending and Prepending (Override Style Syntax)</title>
204
205 <para>
206 You can also append and prepend a variable's value
207 using an override style syntax.
208 When you use this syntax, no spaces are inserted.
209 </para>
210
211 <para>
212 These operators differ from the ":=", ".=", "=.", "+=", and "=+"
213 operators in that their effects are deferred
214 until after parsing completes rather than being immediately
215 applied.
216 Here are some examples:
217 <literallayout class='monospaced'>
218 B = "bval"
219 B_append = " additional data"
220 C = "cval"
221 C_prepend = "additional data "
222 D = "dval"
223 D_append = "additional data"
224 </literallayout>
225 The variable <filename>B</filename> becomes
226 "bval additional data" and <filename>C</filename> becomes
227 "additional data cval".
228 The variable <filename>D</filename> becomes
229 "dvaladditional data".
230 <note>
231 You must control all spacing when you use the
232 override syntax.
233 </note>
234 </para>
235 </section>
236
237 <section id='removing-override-style-syntax'>
238 <title>Removal (Override Style Syntax)</title>
239
240 <para>
241 You can remove values from lists using the removal
242 override style syntax.
243 Specifying a value for removal causes all occurrences of that
244 value to be removed from the variable.
245 </para>
246
247 <para>
248 When you use this syntax, BitBake expects one or more strings.
249 Surrounding spaces are removed as well.
250 Here is an example:
251 <literallayout class='monospaced'>
252 FOO = "123 456 789 123456 123 456 123 456"
253 FOO_remove = "123"
254 FOO_remove = "456"
255 FOO2 = "abc def ghi abcdef abc def abc def"
256 FOO2_remove = "abc def"
257 </literallayout>
258 The variable <filename>FOO</filename> becomes
259 "789 123456" and <filename>FOO2</filename> becomes
260 "ghi abcdef".
261 </para>
262 </section>
263
264 <section id='variable-flag-syntax'>
265 <title>Variable Flag Syntax</title>
266
267 <para>
268 Variable flags are BitBake's implementation of variable properties
269 or attributes.
270 It is a way of tagging extra information onto a variable.
271 You can find more out about variable flags in general in the
272 "<link linkend='variable-flags'>Variable Flags</link>"
273 section.
274 </para>
275
276 <para>
277 You can define, append, and prepend values to variable flags.
278 All the standard syntax operations previously mentioned work
279 for variable flags except for override style syntax
280 (i.e. <filename>_prepend</filename>, <filename>_append</filename>,
281 and <filename>_remove</filename>).
282 </para>
283
284 <para>
285 Here are some examples showing how to set variable flags:
286 <literallayout class='monospaced'>
287 FOO[a] = "abc"
288 FOO[b] = "123"
289 FOO[a] += "456"
290 </literallayout>
291 The variable <filename>FOO</filename> has two flags:
292 <filename>a</filename> and <filename>b</filename>.
293 The flags are immediately set to "abc" and "123", respectively.
294 The <filename>a</filename> flag becomes "abc 456".
295 </para>
296
297 <para>
298 No need exists to pre-define variable flags.
299 You can simply start using them.
300 One extremely common application
301 is to attach some brief documentation to a BitBake variable as
302 follows:
303 <literallayout class='monospaced'>
304 CACHE[doc] = "The directory holding the cache of the metadata."
305 </literallayout>
306 </para>
307 </section>
308
309 <section id='inline-python-variable-expansion'>
310 <title>Inline Python Variable Expansion</title>
311
312 <para>
313 You can use inline Python variable expansion to
314 set variables.
315 Here is an example:
316 <literallayout class='monospaced'>
317 DATE = "${@time.strftime('%Y%m%d',time.gmtime())}"
318 </literallayout>
319 This example results in the <filename>DATE</filename>
320 variable being set to the current date.
321 </para>
322
323 <para>
324 Probably the most common use of this feature is to extract
325 the value of variables from BitBake's internal data dictionary,
326 <filename>d</filename>.
327 The following lines select the values of a package name
328 and its version number, respectively:
329 <literallayout class='monospaced'>
330 PN = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'}"
331 PV = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[1] or '1.0'}"
332 </literallayout>
333 </para>
334 </section>
335
336 <section id='providing-pathnames'>
337 <title>Providing Pathnames</title>
338
339 <para>
340 When specifying pathnames for use with BitBake,
341 do not use the tilde ("~") character as a shortcut
342 for your home directory.
343 Doing so might cause BitBake to not recognize the
344 path since BitBake does not expand this character in
345 the same way a shell would.
346 </para>
347
348 <para>
349 Instead, provide a fuller path as the following
350 example illustrates:
351 <literallayout class='monospaced'>
352 BBLAYERS ?= " \
353 /home/scott-lenovo/LayerA \
354 "
355 </literallayout>
356 </para>
357 </section>
358 </section>
359
360 <section id='conditional-syntax-overrides'>
361 <title>Conditional Syntax (Overrides)</title>
362
363 <para>
364 BitBake uses
365 <link linkend='var-OVERRIDES'><filename>OVERRIDES</filename></link>
366 to control what variables are overridden after BitBake
367 parses recipes and configuration files.
368 This section describes how you can use
369 <filename>OVERRIDES</filename> as conditional metadata,
370 talks about key expansion in relationship to
371 <filename>OVERRIDES</filename>, and provides some examples
372 to help with understanding.
373 </para>
374
375 <section id='conditional-metadata'>
376 <title>Conditional Metadata</title>
377
378 <para>
379 You can use <filename>OVERRIDES</filename> to conditionally select
380 a specific version of a variable and to conditionally
381 append or prepend the value of a variable.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500382 <note>
383 Overrides can only use lower-case characters.
384 Additionally, underscores are not permitted in override names
385 as they are used to separate overrides from each other and
386 from the variable name.
387 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500388 <itemizedlist>
389 <listitem><para><emphasis>Selecting a Variable:</emphasis>
390 The <filename>OVERRIDES</filename> variable is
391 a colon-character-separated list that contains items
392 for which you want to satisfy conditions.
393 Thus, if you have a variable that is conditional on “arm”, and “arm”
394 is in <filename>OVERRIDES</filename>, then the “arm”-specific
395 version of the variable is used rather than the non-conditional
396 version.
397 Here is an example:
398 <literallayout class='monospaced'>
399 OVERRIDES = "architecture:os:machine"
400 TEST = "default"
401 TEST_os = "osspecific"
402 TEST_nooverride = "othercondvalue"
403 </literallayout>
404 In this example, the <filename>OVERRIDES</filename>
405 variable lists three overrides:
406 "architecture", "os", and "machine".
407 The variable <filename>TEST</filename> by itself has a default
408 value of "default".
409 You select the os-specific version of the <filename>TEST</filename>
410 variable by appending the "os" override to the variable
411 (i.e.<filename>TEST_os</filename>).
412 </para>
413
414 <para>
415 To better understand this, consider a practical example
416 that assumes an OpenEmbedded metadata-based Linux
417 kernel recipe file.
418 The following lines from the recipe file first set
419 the kernel branch variable <filename>KBRANCH</filename>
420 to a default value, then conditionally override that
421 value based on the architecture of the build:
422 <literallayout class='monospaced'>
423 KBRANCH = "standard/base"
424 KBRANCH_qemuarm = "standard/arm-versatile-926ejs"
425 KBRANCH_qemumips = "standard/mti-malta32"
426 KBRANCH_qemuppc = "standard/qemuppc"
427 KBRANCH_qemux86 = "standard/common-pc/base"
428 KBRANCH_qemux86-64 = "standard/common-pc-64/base"
429 KBRANCH_qemumips64 = "standard/mti-malta64"
430 </literallayout>
431 </para></listitem>
432 <listitem><para><emphasis>Appending and Prepending:</emphasis>
433 BitBake also supports append and prepend operations to
434 variable values based on whether a specific item is
435 listed in <filename>OVERRIDES</filename>.
436 Here is an example:
437 <literallayout class='monospaced'>
438 DEPENDS = "glibc ncurses"
439 OVERRIDES = "machine:local"
440 DEPENDS_append_machine = "libmad"
441 </literallayout>
442 In this example, <filename>DEPENDS</filename> becomes
443 "glibc ncurses libmad".
444 </para>
445
446 <para>
447 Again, using an OpenEmbedded metadata-based
448 kernel recipe file as an example, the
449 following lines will conditionally append to the
450 <filename>KERNEL_FEATURES</filename> variable based
451 on the architecture:
452 <literallayout class='monospaced'>
453 KERNEL_FEATURES_append = " ${KERNEL_EXTRA_FEATURES}"
454 KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
455 KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc"
456 </literallayout>
457 </para></listitem>
458 </itemizedlist>
459 </para>
460 </section>
461
462 <section id='key-expansion'>
463 <title>Key Expansion</title>
464
465 <para>
466 Key expansion happens when the BitBake datastore is finalized
467 just before BitBake expands overrides.
468 To better understand this, consider the following example:
469 <literallayout class='monospaced'>
470 A${B} = "X"
471 B = "2"
472 A2 = "Y"
473 </literallayout>
474 In this case, after all the parsing is complete, and
475 before any overrides are handled, BitBake expands
476 <filename>${B}</filename> into "2".
477 This expansion causes <filename>A2</filename>, which was
478 set to "Y" before the expansion, to become "X".
479 </para>
480 </section>
481
482 <section id='variable-interaction-worked-examples'>
483 <title>Examples</title>
484
485 <para>
486 Despite the previous explanations that show the different forms of
487 variable definitions, it can be hard to work
488 out exactly what happens when variable operators, conditional
489 overrides, and unconditional overrides are combined.
490 This section presents some common scenarios along
491 with explanations for variable interactions that
492 typically confuse users.
493 </para>
494
495 <para>
496 There is often confusion concerning the order in which
497 overrides and various "append" operators take effect.
498 Recall that an append or prepend operation using "_append"
499 and "_prepend" does not result in an immediate assignment
500 as would "+=", ".=", "=+", or "=.".
501 Consider the following example:
502 <literallayout class='monospaced'>
503 OVERRIDES = "foo"
504 A = "Z"
505 A_foo_append = "X"
506 </literallayout>
507 For this case, <filename>A</filename> is
508 unconditionally set to "Z" and "X" is
509 unconditionally and immediately appended to the variable
510 <filename>A_foo</filename>.
511 Because overrides have not been applied yet,
512 <filename>A_foo</filename> is set to "X" due to the append
513 and <filename>A</filename> simply equals "Z".
514 </para>
515
516 <para>
517 Applying overrides, however, changes things.
518 Since "foo" is listed in <filename>OVERRIDES</filename>,
519 the conditional variable <filename>A</filename> is replaced
520 with the "foo" version, which is equal to "X".
521 So effectively, <filename>A_foo</filename> replaces <filename>A</filename>.
522 </para>
523
524 <para>
525 This next example changes the order of the override and
526 the append:
527 <literallayout class='monospaced'>
528 OVERRIDES = "foo"
529 A = "Z"
530 A_append_foo = "X"
531 </literallayout>
532 For this case, before overrides are handled,
533 <filename>A</filename> is set to "Z" and <filename>A_append_foo</filename>
534 is set to "X".
535 Once the override for "foo" is applied, however,
536 <filename>A</filename> gets appended with "X".
537 Consequently, <filename>A</filename> becomes "ZX".
538 Notice that spaces are not appended.
539 </para>
540
541 <para>
542 This next example has the order of the appends and overrides reversed
543 back as in the first example:
544 <literallayout class='monospaced'>
545 OVERRIDES = "foo"
546 A = "Y"
547 A_foo_append = "Z"
548 A_foo_append += "X"
549 </literallayout>
550 For this case, before any overrides are resolved,
551 <filename>A</filename> is set to "Y" using an immediate assignment.
552 After this immediate assignment, <filename>A_foo</filename> is set
553 to "Z", and then further appended with
554 "X" leaving the variable set to "Z X".
555 Finally, applying the override for "foo" results in the conditional
556 variable <filename>A</filename> becoming "Z X" (i.e.
557 <filename>A</filename> is replaced with <filename>A_foo</filename>).
558 </para>
559
560 <para>
561 This final example mixes in some varying operators:
562 <literallayout class='monospaced'>
563 A = "1"
564 A_append = "2"
565 A_append = "3"
566 A += "4"
567 A .= "5"
568 </literallayout>
569 For this case, the type of append operators are affecting the
570 order of assignments as BitBake passes through the code
571 multiple times.
572 Initially, <filename>A</filename> is set to "1 45" because
573 of the three statements that use immediate operators.
574 After these assignments are made, BitBake applies the
575 <filename>_append</filename> operations.
576 Those operations result in <filename>A</filename> becoming "1 4523".
577 </para>
578 </section>
579 </section>
580
581 <section id='sharing-functionality'>
582 <title>Sharing Functionality</title>
583
584 <para>
585 BitBake allows for metadata sharing through include files
586 (<filename>.inc</filename>) and class files
587 (<filename>.bbclass</filename>).
588 For example, suppose you have a piece of common functionality
589 such as a task definition that you want to share between
590 more than one recipe.
591 In this case, creating a <filename>.bbclass</filename>
592 file that contains the common functionality and then using
593 the <filename>inherit</filename> directive in your recipes to
594 inherit the class would be a common way to share the task.
595 </para>
596
597 <para>
598 This section presents the mechanisms BitBake provides to
599 allow you to share functionality between recipes.
600 Specifically, the mechanisms include <filename>include</filename>,
601 <filename>inherit</filename>, <filename>INHERIT</filename>, and
602 <filename>require</filename> directives.
603 </para>
604
605 <section id='locating-include-and-class-files'>
606 <title>Locating Include and Class Files</title>
607
608 <para>
609 BitBake uses the
610 <link linkend='var-BBPATH'><filename>BBPATH</filename></link>
611 variable to locate needed include and class files.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500612 Additionally, BitBake searches the current directory for
613 <filename>include</filename> and <filename>require</filename>
614 directives.
615 <note>
616 The <filename>BBPATH</filename> variable is analogous to
617 the environment variable <filename>PATH</filename>.
618 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500619 </para>
620
621 <para>
622 In order for include and class files to be found by BitBake,
623 they need to be located in a "classes" subdirectory that can
624 be found in <filename>BBPATH</filename>.
625 </para>
626 </section>
627
628 <section id='inherit-directive'>
629 <title><filename>inherit</filename> Directive</title>
630
631 <para>
632 When writing a recipe or class file, you can use the
633 <filename>inherit</filename> directive to inherit the
634 functionality of a class (<filename>.bbclass</filename>).
635 BitBake only supports this directive when used within recipe
636 and class files (i.e. <filename>.bb</filename> and
637 <filename>.bbclass</filename>).
638 </para>
639
640 <para>
641 The <filename>inherit</filename> directive is a rudimentary
642 means of specifying what classes of functionality your
643 recipes require.
644 For example, you can easily abstract out the tasks involved in
645 building a package that uses Autoconf and Automake and put
646 those tasks into a class file that can be used by your recipe.
647 </para>
648
649 <para>
650 As an example, your recipes could use the following directive
651 to inherit an <filename>autotools.bbclass</filename> file.
652 The class file would contain common functionality for using
653 Autotools that could be shared across recipes:
654 <literallayout class='monospaced'>
655 inherit autotools
656 </literallayout>
657 In this case, BitBake would search for the directory
658 <filename>classes/autotools.bbclass</filename>
659 in <filename>BBPATH</filename>.
660 <note>
661 You can override any values and functions of the
662 inherited class within your recipe by doing so
663 after the "inherit" statement.
664 </note>
665 </para>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500666
667 <para>
668 If necessary, it is possible to inherit a class
669 conditionally by using
670 a variable expression after the <filename>inherit</filename>
671 statement.
672 Here is an example:
673 <literallayout class='monospaced'>
674 inherit ${VARNAME}
675 </literallayout>
676 If <filename>VARNAME</filename> is going to be set, it needs
677 to be set before the <filename>inherit</filename> statement
678 is parsed.
679 One way to achieve a conditional inherit in this case is to use
680 overrides:
681 <literallayout class='monospaced'>
682 VARIABLE = ""
683 VARIABLE_someoverride = "myclass"
684 </literallayout>
685 </para>
686
687 <para>
688 Another method is by using anonymous Python.
689 Here is an example:
690 <literallayout class='monospaced'>
691 python () {
692 if condition == value:
693 d.setVar('VARIABLE', 'myclass')
694 else:
695 d.setVar('VARIABLE', '')
696 }
697 </literallayout>
698 </para>
699
700 <para>
701 Alternatively, you could use an in-line Python expression
702 in the following form:
703 <literallayout class='monospaced'>
704 inherit ${@'classname' if condition else ''}
705 inherit ${@functionname(params)}
706 </literallayout>
707 In all cases, if the expression evaluates to an empty
708 string, the statement does not trigger a syntax error
709 because it becomes a no-op.
710 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500711 </section>
712
713 <section id='include-directive'>
714 <title><filename>include</filename> Directive</title>
715
716 <para>
717 BitBake understands the <filename>include</filename>
718 directive.
719 This directive causes BitBake to parse whatever file you specify,
720 and to insert that file at that location.
721 The directive is much like its equivalent in Make except
722 that if the path specified on the include line is a relative
723 path, BitBake locates the first file it can find
724 within <filename>BBPATH</filename>.
725 </para>
726
727 <para>
728 As an example, suppose you needed a recipe to include some
729 self-test definitions:
730 <literallayout class='monospaced'>
731 include test_defs.inc
732 </literallayout>
733 <note>
734 The <filename>include</filename> directive does not
735 produce an error when the file cannot be found.
736 Consequently, it is recommended that if the file you
737 are including is expected to exist, you should use
738 <link linkend='require-inclusion'><filename>require</filename></link>
739 instead of <filename>include</filename>.
740 Doing so makes sure that an error is produced if the
741 file cannot be found.
742 </note>
743 </para>
744 </section>
745
746 <section id='require-inclusion'>
747 <title><filename>require</filename> Directive</title>
748
749 <para>
750 BitBake understands the <filename>require</filename>
751 directive.
752 This directive behaves just like the
753 <filename>include</filename> directive with the exception that
754 BitBake raises a parsing error if the file to be included cannot
755 be found.
756 Thus, any file you require is inserted into the file that is
757 being parsed at the location of the directive.
758 </para>
759
760 <para>
761 Similar to how BitBake handles
762 <link linkend='include-directive'><filename>include</filename></link>,
763 if the path specified
764 on the require line is a relative path, BitBake locates
765 the first file it can find within <filename>BBPATH</filename>.
766 </para>
767
768 <para>
769 As an example, suppose you have two versions of a recipe
770 (e.g. <filename>foo_1.2.2.bb</filename> and
771 <filename>foo_2.0.0.bb</filename>) where
772 each version contains some identical functionality that could be
773 shared.
774 You could create an include file named <filename>foo.inc</filename>
775 that contains the common definitions needed to build "foo".
776 You need to be sure <filename>foo.inc</filename> is located in the
777 same directory as your two recipe files as well.
778 Once these conditions are set up, you can share the functionality
779 using a <filename>require</filename> directive from within each
780 recipe:
781 <literallayout class='monospaced'>
782 require foo.inc
783 </literallayout>
784 </para>
785 </section>
786
787 <section id='inherit-configuration-directive'>
788 <title><filename>INHERIT</filename> Configuration Directive</title>
789
790 <para>
791 When creating a configuration file (<filename>.conf</filename>),
792 you can use the <filename>INHERIT</filename> directive to
793 inherit a class.
794 BitBake only supports this directive when used within
795 a configuration file.
796 </para>
797
798 <para>
799 As an example, suppose you needed to inherit a class
800 file called <filename>abc.bbclass</filename> from a
801 configuration file as follows:
802 <literallayout class='monospaced'>
803 INHERIT += "abc"
804 </literallayout>
805 This configuration directive causes the named
806 class to be inherited at the point of the directive
807 during parsing.
808 As with the <filename>inherit</filename> directive, the
809 <filename>.bbclass</filename> file must be located in a
810 "classes" subdirectory in one of the directories specified
811 in <filename>BBPATH</filename>.
812 <note>
813 Because <filename>.conf</filename> files are parsed
814 first during BitBake's execution, using
815 <filename>INHERIT</filename> to inherit a class effectively
816 inherits the class globally (i.e. for all recipes).
817 </note>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500818 If you want to use the directive to inherit
819 multiple classes, you can provide them on the same line in the
820 <filename>local.conf</filename> file.
821 Use spaces to separate the classes.
822 The following example shows how to inherit both the
823 <filename>autotools</filename> and <filename>pkgconfig</filename>
824 classes:
825 <literallayout class='monospaced'>
826 inherit autotools pkgconfig
827 </literallayout>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500828 </para>
829 </section>
830 </section>
831
832 <section id='functions'>
833 <title>Functions</title>
834
835 <para>
836 As with most languages, functions are the building blocks that
837 are used to build up operations into tasks.
838 BitBake supports these types of functions:
839 <itemizedlist>
840 <listitem><para><emphasis>Shell Functions:</emphasis>
841 Functions written in shell script and executed either
842 directly as functions, tasks, or both.
843 They can also be called by other shell functions.
844 </para></listitem>
845 <listitem><para><emphasis>BitBake Style Python Functions:</emphasis>
846 Functions written in Python and executed by BitBake or other
847 Python functions using <filename>bb.build.exec_func()</filename>.
848 </para></listitem>
849 <listitem><para><emphasis>Python Functions:</emphasis>
850 Functions written in Python and executed by Python.
851 </para></listitem>
852 <listitem><para><emphasis>Anonymous Python Functions:</emphasis>
853 Python functions executed automatically during
854 parsing.
855 </para></listitem>
856 </itemizedlist>
857 Regardless of the type of function, you can only
858 define them in class (<filename>.bbclass</filename>)
859 and recipe (<filename>.bb</filename> or <filename>.inc</filename>)
860 files.
861 </para>
862
863 <section id='shell-functions'>
864 <title>Shell Functions</title>
865
866 <para>
867 Functions written in shell script and executed either
868 directly as functions, tasks, or both.
869 They can also be called by other shell functions.
870 Here is an example shell function definition:
871 <literallayout class='monospaced'>
872 some_function () {
873 echo "Hello World"
874 }
875 </literallayout>
876 When you create these types of functions in your recipe
877 or class files, you need to follow the shell programming
878 rules.
879 The scripts are executed by <filename>/bin/sh</filename>,
880 which may not be a bash shell but might be something
881 such as <filename>dash</filename>.
882 You should not use Bash-specific script (bashisms).
883 </para>
884 </section>
885
886 <section id='bitbake-style-python-functions'>
887 <title>BitBake Style Python Functions</title>
888
889 <para>
890 These functions are written in Python and executed by
891 BitBake or other Python functions using
892 <filename>bb.build.exec_func()</filename>.
893 </para>
894
895 <para>
896 An example BitBake function is:
897 <literallayout class='monospaced'>
898 python some_python_function () {
899 d.setVar("TEXT", "Hello World")
900 print d.getVar("TEXT", True)
901 }
902 </literallayout>
903 Because the Python "bb" and "os" modules are already
904 imported, you do not need to import these modules.
905 Also in these types of functions, the datastore ("d")
906 is a global variable and is always automatically
907 available.
908 </para>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500909
910 <note>
911 Variable expressions (e.g. <filename>${X}</filename>) are no
912 longer expanded within Python functions.
913 This behavior is intentional in order to allow you to freely
914 set variable values to expandable expressions without having
915 them expanded prematurely.
916 If you do wish to expand a variable within a Python function,
917 use <filename>d.getVar("X", True)</filename>.
918 Or, for more complicated expressions, use
919 <filename>d.expand()</filename>.
920 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500921 </section>
922
923 <section id='python-functions'>
924 <title>Python Functions</title>
925
926 <para>
927 These functions are written in Python and are executed by
928 other Python code.
929 Examples of Python functions are utility functions
930 that you intend to call from in-line Python or
931 from within other Python functions.
932 Here is an example:
933 <literallayout class='monospaced'>
934 def get_depends(d):
935 if d.getVar('SOMECONDITION', True):
936 return "dependencywithcond"
937 else:
938 return "dependency"
939 SOMECONDITION = "1"
940 DEPENDS = "${@get_depends(d)}"
941 </literallayout>
942 This would result in <filename>DEPENDS</filename>
943 containing <filename>dependencywithcond</filename>.
944 </para>
945
946 <para>
947 Here are some things to know about Python functions:
948 <itemizedlist>
949 <listitem><para>Python functions can take parameters.
950 </para></listitem>
951 <listitem><para>The BitBake datastore is not
952 automatically available.
953 Consequently, you must pass it in as a
954 parameter to the function.
955 </para></listitem>
956 <listitem><para>The "bb" and "os" Python modules are
957 automatically available.
958 You do not need to import them.
959 </para></listitem>
960 </itemizedlist>
961 </para>
962 </section>
963
964 <section id='anonymous-python-functions'>
965 <title>Anonymous Python Functions</title>
966
967 <para>
968 Sometimes it is useful to run some code during
969 parsing to set variables or to perform other operations
970 programmatically.
971 To do this, you can define an anonymous Python function.
972 Here is an example that conditionally sets a
973 variable based on the value of another variable:
974 <literallayout class='monospaced'>
975 python __anonymous () {
976 if d.getVar('SOMEVAR', True) == 'value':
977 d.setVar('ANOTHERVAR', 'value2')
978 }
979 </literallayout>
980 The "__anonymous" function name is optional, so the
981 following example is functionally equivalent to the above:
982 <literallayout class='monospaced'>
983 python () {
984 if d.getVar('SOMEVAR', True) == 'value':
985 d.setVar('ANOTHERVAR', 'value2')
986 }
987 </literallayout>
988 Because unlike other Python functions anonymous
989 Python functions are executed during parsing, the
990 "d" variable within an anonymous Python function represents
991 the datastore for the entire recipe.
992 Consequently, you can set variable values here and
993 those values can be picked up by other functions.
994 </para>
995 </section>
996
997 <section id='flexible-inheritance-for-class-functions'>
998 <title>Flexible Inheritance for Class Functions</title>
999
1000 <para>
1001 Through coding techniques and the use of
1002 <filename>EXPORT_FUNCTIONS</filename>, BitBake supports
1003 exporting a function from a class such that the
1004 class function appears as the default implementation
1005 of the function, but can still be called if a recipe
1006 inheriting the class needs to define its own version of
1007 the function.
1008 </para>
1009
1010 <para>
1011 To understand the benefits of this feature, consider
1012 the basic scenario where a class defines a task function
1013 and your recipe inherits the class.
1014 In this basic scenario, your recipe inherits the task
1015 function as defined in the class.
1016 If desired, your recipe can add to the start and end of the
1017 function by using the "_prepend" or "_append" operations
1018 respectively, or it can redefine the function completely.
1019 However, if it redefines the function, there is
1020 no means for it to call the class version of the function.
1021 <filename>EXPORT_FUNCTIONS</filename> provides a mechanism
1022 that enables the recipe's version of the function to call
1023 the original version of the function.
1024 </para>
1025
1026 <para>
1027 To make use of this technique, you need the following
1028 things in place:
1029 <itemizedlist>
1030 <listitem><para>
1031 The class needs to define the function as follows:
1032 <literallayout class='monospaced'>
1033 <replaceable>classname</replaceable><filename>_</filename><replaceable>functionname</replaceable>
1034 </literallayout>
1035 For example, if you have a class file
1036 <filename>bar.bbclass</filename> and a function named
1037 <filename>do_foo</filename>, the class must define the function
1038 as follows:
1039 <literallayout class='monospaced'>
1040 bar_do_foo
1041 </literallayout>
1042 </para></listitem>
1043 <listitem><para>
1044 The class needs to contain the <filename>EXPORT_FUNCTIONS</filename>
1045 statement as follows:
1046 <literallayout class='monospaced'>
1047 EXPORT_FUNCTIONS <replaceable>functionname</replaceable>
1048 </literallayout>
1049 For example, continuing with the same example, the
1050 statement in the <filename>bar.bbclass</filename> would be
1051 as follows:
1052 <literallayout class='monospaced'>
1053 EXPORT_FUNCTIONS do_foo
1054 </literallayout>
1055 </para></listitem>
1056 <listitem><para>
1057 You need to call the function appropriately from within your
1058 recipe.
1059 Continuing with the same example, if your recipe
1060 needs to call the class version of the function,
1061 it should call <filename>bar_do_foo</filename>.
1062 Assuming <filename>do_foo</filename> was a shell function
1063 and <filename>EXPORT_FUNCTIONS</filename> was used as above,
1064 the recipe's function could conditionally call the
1065 class version of the function as follows:
1066 <literallayout class='monospaced'>
1067 do_foo() {
1068 if [ somecondition ] ; then
1069 bar_do_foo
1070 else
1071 # Do something else
1072 fi
1073 }
1074 </literallayout>
1075 To call your modified version of the function as defined
1076 in your recipe, call it as <filename>do_foo</filename>.
1077 </para></listitem>
1078 </itemizedlist>
1079 With these conditions met, your single recipe
1080 can freely choose between the original function
1081 as defined in the class file and the modified function in your recipe.
1082 If you do not set up these conditions, you are limited to using one function
1083 or the other.
1084 </para>
1085 </section>
1086 </section>
1087
1088 <section id='tasks'>
1089 <title>Tasks</title>
1090
1091 <para>
1092 Tasks are BitBake execution units that originate as
1093 functions and make up the steps that BitBake needs to run
1094 for given recipe.
1095 Tasks are only supported in recipe (<filename>.bb</filename>
1096 or <filename>.inc</filename>) and class
1097 (<filename>.bbclass</filename>) files.
1098 By convention, task names begin with the string "do_".
1099 </para>
1100
1101 <para>
1102 Here is an example of a task that prints out the date:
1103 <literallayout class='monospaced'>
1104 python do_printdate () {
1105 import time
1106 print time.strftime('%Y%m%d', time.gmtime())
1107 }
1108 addtask printdate after do_fetch before do_build
1109 </literallayout>
1110 </para>
1111
1112 <section id='promoting-a-function-to-a-task'>
1113 <title>Promoting a Function to a Task</title>
1114
1115 <para>
1116 Any function can be promoted to a task by applying the
1117 <filename>addtask</filename> command.
1118 The <filename>addtask</filename> command also describes
1119 inter-task dependencies.
1120 Here is the function from the previous section but with the
1121 <filename>addtask</filename> command promoting it to a task
1122 and defining some dependencies:
1123 <literallayout class='monospaced'>
1124 python do_printdate () {
1125 import time
1126 print time.strftime('%Y%m%d', time.gmtime())
1127 }
1128 addtask printdate after do_fetch before do_build
1129 </literallayout>
1130 In the example, the function is defined and then promoted
1131 as a task.
1132 The <filename>do_printdate</filename> task becomes a dependency of
1133 the <filename>do_build</filename> task, which is the default
1134 task.
1135 And, the <filename>do_printdate</filename> task is dependent upon
1136 the <filename>do_fetch</filename> task.
1137 Execution of the <filename>do_build</filename> task results
1138 in the <filename>do_printdate</filename> task running first.
1139 </para>
1140 </section>
1141
1142 <section id='deleting-a-task'>
1143 <title>Deleting a Task</title>
1144
1145 <para>
1146 As well as being able to add tasks, you can delete them.
1147 Simply use the <filename>deltask</filename> command to
1148 delete a task.
1149 For example, to delete the example task used in the previous
1150 sections, you would use:
1151 <literallayout class='monospaced'>
1152 deltask printdate
1153 </literallayout>
1154 If you delete a task using the <filename>deltask</filename>
1155 command and the task has dependencies, the dependencies are
1156 not reconnected.
1157 For example, suppose you have three tasks named
1158 <filename>do_a</filename>, <filename>do_b</filename>, and
1159 <filename>do_c</filename>.
1160 Furthermore, <filename>do_c</filename> is dependent on
1161 <filename>do_b</filename>, which in turn is dependent on
1162 <filename>do_a</filename>.
1163 Given this scenario, if you use <filename>deltask</filename>
1164 to delete <filename>do_b</filename>, the implicit dependency
1165 relationship between <filename>do_c</filename> and
1166 <filename>do_a</filename> through <filename>do_b</filename>
1167 no longer exists, and <filename>do_c</filename> dependencies
1168 are not updated to include <filename>do_a</filename>.
1169 Thus, <filename>do_c</filename> is free to run before
1170 <filename>do_a</filename>.
1171 </para>
1172
1173 <para>
1174 If you want dependencies such as these to remain intact, use
1175 the <filename>noexec</filename> varflag to disable the task
1176 instead of using the <filename>deltask</filename> command to
1177 delete it:
1178 <literallayout class='monospaced'>
1179 do_b[noexec] = "1"
1180 </literallayout>
1181 </para>
1182 </section>
1183
1184 <section id='passing-information-into-the-build-task-environment'>
1185 <title>Passing Information Into the Build Task Environment</title>
1186
1187 <para>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001188 When running a task, BitBake tightly controls the shell execution
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001189 environment of the build tasks to make
1190 sure unwanted contamination from the build machine cannot
1191 influence the build.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001192 <note>
1193 By default, BitBake cleans the environment to include only those
1194 things exported or listed in its whitelist to ensure that the build
1195 environment is reproducible and consistent.
1196 You can prevent this "cleaning" by setting the
1197 <link linkend='var-BB_PRESERVE_ENV'><filename>BB_PRESERVE_ENV</filename></link>
1198 variable.
1199 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001200 Consequently, if you do want something to get passed into the
1201 build task environment, you must take these two steps:
1202 <orderedlist>
1203 <listitem><para>
1204 Tell BitBake to load what you want from the environment
1205 into the datastore.
1206 You can do so through the
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001207 <link linkend='var-BB_ENV_WHITELIST'><filename>BB_ENV_WHITELIST</filename></link>
1208 and
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001209 <link linkend='var-BB_ENV_EXTRAWHITE'><filename>BB_ENV_EXTRAWHITE</filename></link>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001210 variables.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001211 For example, assume you want to prevent the build system from
1212 accessing your <filename>$HOME/.ccache</filename>
1213 directory.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001214 The following command "whitelists" the environment variable
1215 <filename>CCACHE_DIR</filename> causing BitBack to allow that
1216 variable into the datastore:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001217 <literallayout class='monospaced'>
1218 export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE CCACHE_DIR"
1219 </literallayout></para></listitem>
1220 <listitem><para>
1221 Tell BitBake to export what you have loaded into the
1222 datastore to the task environment of every running task.
1223 Loading something from the environment into the datastore
1224 (previous step) only makes it available in the datastore.
1225 To export it to the task environment of every running task,
1226 use a command similar to the following in your local configuration
1227 file <filename>local.conf</filename> or your
1228 distribution configuration file:
1229 <literallayout class='monospaced'>
1230 export CCACHE_DIR
1231 </literallayout>
1232 <note>
1233 A side effect of the previous steps is that BitBake
1234 records the variable as a dependency of the build process
1235 in things like the setscene checksums.
1236 If doing so results in unnecessary rebuilds of tasks, you can
1237 whitelist the variable so that the setscene code
1238 ignores the dependency when it creates checksums.
1239 </note></para></listitem>
1240 </orderedlist>
1241 </para>
1242
1243 <para>
1244 Sometimes, it is useful to be able to obtain information
1245 from the original execution environment.
1246 Bitbake saves a copy of the original environment into
1247 a special variable named
1248 <link linkend='var-BB_ORIGENV'><filename>BB_ORIGENV</filename></link>.
1249 </para>
1250
1251 <para>
1252 The <filename>BB_ORIGENV</filename> variable returns a datastore
1253 object that can be queried using the standard datastore operators
1254 such as <filename>getVar(, False)</filename>.
1255 The datastore object is useful, for example, to find the original
1256 <filename>DISPLAY</filename> variable.
1257 Here is an example:
1258 <literallayout class='monospaced'>
1259 origenv = d.getVar("BB_ORIGENV", False)
1260 bar = origenv.getVar("BAR", False)
1261 </literallayout>
1262 The previous example returns <filename>BAR</filename> from the original
1263 execution environment.
1264 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001265 </section>
1266 </section>
1267
1268 <section id='variable-flags'>
1269 <title>Variable Flags</title>
1270
1271 <para>
1272 Variable flags (varflags) help control a task's functionality
1273 and dependencies.
1274 BitBake reads and writes varflags to the datastore using the following
1275 command forms:
1276 <literallayout class='monospaced'>
1277 <replaceable>variable</replaceable> = d.getVarFlags("<replaceable>variable</replaceable>")
1278 self.d.setVarFlags("FOO", {"func": True})
1279 </literallayout>
1280 </para>
1281
1282 <para>
1283 When working with varflags, the same syntax, with the exception of
1284 overrides, applies.
1285 In other words, you can set, append, and prepend varflags just like
1286 variables.
1287 See the
1288 "<link linkend='variable-flag-syntax'>Variable Flag Syntax</link>"
1289 section for details.
1290 </para>
1291
1292 <para>
1293 BitBake has a defined set of varflags available for recipes and
1294 classes.
1295 Tasks support a number of these flags which control various
1296 functionality of the task:
1297 <itemizedlist>
1298 <listitem><para><emphasis>cleandirs:</emphasis>
1299 Empty directories that should created before the task runs.
1300 </para></listitem>
1301 <listitem><para><emphasis>depends:</emphasis>
1302 Controls inter-task dependencies.
1303 See the
1304 <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link>
1305 variable and the
1306 "<link linkend='inter-task-dependencies'>Inter-Task Dependencies</link>"
1307 section for more information.
1308 </para></listitem>
1309 <listitem><para><emphasis>deptask:</emphasis>
1310 Controls task build-time dependencies.
1311 See the
1312 <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link>
1313 variable and the
1314 "<link linkend='build-dependencies'>Build Dependencies</link>"
1315 section for more information.
1316 </para></listitem>
1317 <listitem><para><emphasis>dirs:</emphasis>
1318 Directories that should be created before the task runs.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001319 The last directory listed will be used as the work directory
1320 for the task.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001321 </para></listitem>
1322 <listitem><para><emphasis>lockfiles:</emphasis>
1323 Specifies one or more lockfiles to lock while the task
1324 executes.
1325 Only one task may hold a lockfile, and any task that
1326 attempts to lock an already locked file will block until
1327 the lock is released.
1328 You can use this variable flag to accomplish mutual
1329 exclusion.
1330 </para></listitem>
1331 <listitem><para><emphasis>noexec:</emphasis>
1332 Marks the tasks as being empty and no execution required.
1333 The <filename>noexec</filename> flag can be used to set up
1334 tasks as dependency placeholders, or to disable tasks defined
1335 elsewhere that are not needed in a particular recipe.
1336 </para></listitem>
1337 <listitem><para><emphasis>nostamp:</emphasis>
1338 Tells BitBake to not generate a stamp file for a task,
1339 which implies the task should always be executed.
1340 </para></listitem>
1341 <listitem><para><emphasis>postfuncs:</emphasis>
1342 List of functions to call after the completion of the task.
1343 </para></listitem>
1344 <listitem><para><emphasis>prefuncs:</emphasis>
1345 List of functions to call before the task executes.
1346 </para></listitem>
1347 <listitem><para><emphasis>rdepends:</emphasis>
1348 Controls inter-task runtime dependencies.
1349 See the
1350 <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link>
1351 variable, the
1352 <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link>
1353 variable, and the
1354 "<link linkend='inter-task-dependencies'>Inter-Task Dependencies</link>"
1355 section for more information.
1356 </para></listitem>
1357 <listitem><para><emphasis>rdeptask:</emphasis>
1358 Controls task runtime dependencies.
1359 See the
1360 <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link>
1361 variable, the
1362 <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link>
1363 variable, and the
1364 "<link linkend='runtime-dependencies'>Runtime Dependencies</link>"
1365 section for more information.
1366 </para></listitem>
1367 <listitem><para><emphasis>recideptask:</emphasis>
1368 When set in conjunction with
1369 <filename>recrdeptask</filename>, specifies a task that
1370 should be inspected for additional dependencies.
1371 </para></listitem>
1372 <listitem><para><emphasis>recrdeptask:</emphasis>
1373 Controls task recursive runtime dependencies.
1374 See the
1375 <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link>
1376 variable, the
1377 <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link>
1378 variable, and the
1379 "<link linkend='recursive-dependencies'>Recursive Dependencies</link>"
1380 section for more information.
1381 </para></listitem>
1382 <listitem><para><emphasis>stamp-extra-info:</emphasis>
1383 Extra stamp information to append to the task's stamp.
1384 As an example, OpenEmbedded uses this flag to allow
1385 machine-specific tasks.
1386 </para></listitem>
1387 <listitem><para><emphasis>umask:</emphasis>
1388 The umask to run the task under.
1389 </para></listitem>
1390 </itemizedlist>
1391 </para>
1392
1393 <para>
1394 Several varflags are useful for controlling how signatures are
1395 calculated for variables.
1396 For more information on this process, see the
1397 "<link linkend='checksums'>Checksums (Signatures)</link>"
1398 section.
1399 <itemizedlist>
1400 <listitem><para><emphasis>vardeps:</emphasis>
1401 Specifies a space-separated list of additional
1402 variables to add to a variable's dependencies
1403 for the purposes of calculating its signature.
1404 Adding variables to this list is useful, for example, when
1405 a function refers to a variable in a manner that
1406 does not allow BitBake to automatically determine
1407 that the variable is referred to.
1408 </para></listitem>
1409 <listitem><para><emphasis>vardepsexclude:</emphasis>
1410 Specifies a space-separated list of variables
1411 that should be excluded from a variable's dependencies
1412 for the purposes of calculating its signature.
1413 </para></listitem>
1414 <listitem><para><emphasis>vardepvalue:</emphasis>
1415 If set, instructs BitBake to ignore the actual
1416 value of the variable and instead use the specified
1417 value when calculating the variable's signature.
1418 </para></listitem>
1419 <listitem><para><emphasis>vardepvalueexclude:</emphasis>
1420 Specifies a pipe-separated list of strings to exclude
1421 from the variable's value when calculating the
1422 variable's signature.
1423 </para></listitem>
1424 </itemizedlist>
1425 </para>
1426 </section>
1427
1428 <section id='events'>
1429 <title>Events</title>
1430
1431 <para>
1432 BitBake allows installation of event handlers within
1433 recipe and class files.
1434 Events are triggered at certain points during operation,
1435 such as the beginning of an operation against a given recipe
1436 (<filename>*.bb</filename> file), the start of a given task,
1437 task failure, task success, and so forth.
1438 The intent is to make it easy to do things like email
1439 notification on build failure.
1440 </para>
1441
1442 <para>
1443 Following is an example event handler that
1444 prints the name of the event and the content of
1445 the <filename>FILE</filename> variable:
1446 <literallayout class='monospaced'>
1447 addhandler myclass_eventhandler
1448 python myclass_eventhandler() {
1449 from bb.event import getName
1450 from bb import data
1451 print("The name of the Event is %s" % getName(e))
1452 print("The file we run for is %s" % data.getVar('FILE', e.data, True))
1453 }
1454 </literallayout>
1455 This event handler gets called every time an event is
1456 triggered.
1457 A global variable "<filename>e</filename>" is defined and
1458 "<filename>e.data</filename>" contains an instance of
1459 "<filename>bb.data</filename>".
1460 With the <filename>getName(e)</filename> method, one can get
1461 the name of the triggered event.
1462 </para>
1463
1464 <para>
1465 Because you probably are only interested in a subset of events,
1466 you would likely use the <filename>[eventmask]</filename> flag
1467 for your event handler to be sure that only certain events
1468 trigger the handler.
1469 Given the previous example, suppose you only wanted the
1470 <filename>bb.build.TaskFailed</filename> event to trigger that
1471 event handler.
1472 Use the flag as follows:
1473 <literallayout class='monospaced'>
1474 addhandler myclass_eventhandler
1475 myclass_eventhandler[eventmask] = "bb.build.TaskFailed"
1476 python myclass_eventhandler() {
1477 from bb.event import getName
1478 from bb import data
1479 print("The name of the Event is %s" % getName(e))
1480 print("The file we run for is %s" % data.getVar('FILE', e.data, True))
1481 }
1482 </literallayout>
1483 </para>
1484
1485 <para>
1486 During a standard build, the following common events might occur:
1487 <itemizedlist>
1488 <listitem><para>
1489 <filename>bb.event.ConfigParsed()</filename>
1490 </para></listitem>
1491 <listitem><para>
1492 <filename>bb.event.ParseStarted()</filename>
1493 </para></listitem>
1494 <listitem><para>
1495 <filename>bb.event.ParseProgress()</filename>
1496 </para></listitem>
1497 <listitem><para>
1498 <filename>bb.event.ParseCompleted()</filename>
1499 </para></listitem>
1500 <listitem><para>
1501 <filename>bb.event.BuildStarted()</filename>
1502 </para></listitem>
1503 <listitem><para>
1504 <filename>bb.build.TaskStarted()</filename>
1505 </para></listitem>
1506 <listitem><para>
1507 <filename>bb.build.TaskInvalid()</filename>
1508 </para></listitem>
1509 <listitem><para>
1510 <filename>bb.build.TaskFailedSilent()</filename>
1511 </para></listitem>
1512 <listitem><para>
1513 <filename>bb.build.TaskFailed()</filename>
1514 </para></listitem>
1515 <listitem><para>
1516 <filename>bb.build.TaskSucceeded()</filename>
1517 </para></listitem>
1518 <listitem><para>
1519 <filename>bb.event.BuildCompleted()</filename>
1520 </para></listitem>
1521 <listitem><para>
1522 <filename>bb.cooker.CookerExit()</filename>
1523 </para></listitem>
1524 </itemizedlist>
1525 Here is a list of other events that occur based on specific requests
1526 to the server:
1527 <itemizedlist>
1528 <listitem><para>
1529 <filename>bb.event.TreeDataPreparationStarted()</filename>
1530 </para></listitem>
1531 <listitem><para>
1532 <filename>bb.event.TreeDataPreparationProgress</filename>
1533 </para></listitem>
1534 <listitem><para>
1535 <filename>bb.event.TreeDataPreparationCompleted</filename>
1536 </para></listitem>
1537 <listitem><para>
1538 <filename>bb.event.DepTreeGenerated</filename>
1539 </para></listitem>
1540 <listitem><para>
1541 <filename>bb.event.CoreBaseFilesFound</filename>
1542 </para></listitem>
1543 <listitem><para>
1544 <filename>bb.event.ConfigFilePathFound</filename>
1545 </para></listitem>
1546 <listitem><para>
1547 <filename>bb.event.FilesMatchingFound</filename>
1548 </para></listitem>
1549 <listitem><para>
1550 <filename>bb.event.ConfigFilesFound</filename>
1551 </para></listitem>
1552 <listitem><para>
1553 <filename>bb.event.TargetsTreeGenerated</filename>
1554 </para></listitem>
1555 </itemizedlist>
1556 </para>
1557 </section>
1558
1559 <section id='variants-class-extension-mechanism'>
1560 <title>Variants - Class Extension Mechanism</title>
1561
1562 <para>
1563 BitBake supports two features that facilitate creating
1564 from a single recipe file multiple incarnations of that
1565 recipe file where all incarnations are buildable.
1566 These features are enabled through the
1567 <link linkend='var-BBCLASSEXTEND'><filename>BBCLASSEXTEND</filename></link>
1568 and
1569 <link linkend='var-BBVERSIONS'><filename>BBVERSIONS</filename></link>
1570 variables.
1571 <note>
1572 The mechanism for this class extension is extremely
1573 specific to the implementation.
1574 Usually, the recipe's
1575 <link linkend='var-PROVIDES'><filename>PROVIDES</filename></link>,
1576 <link linkend='var-PN'><filename>PN</filename></link>, and
1577 <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link>
1578 variables would need to be modified by the extension class.
1579 For specific examples, see the OE-Core
1580 <filename>native</filename>, <filename>nativesdk</filename>,
1581 and <filename>multilib</filename> classes.
1582 </note>
1583 <itemizedlist>
1584 <listitem><para><emphasis><filename>BBCLASSEXTEND</filename>:</emphasis>
1585 This variable is a space separated list of classes used to "extend" the
1586 recipe for each variant.
1587 Here is an example that results in a second incarnation of the current
1588 recipe being available.
1589 This second incarnation will have the "native" class inherited.
1590 <literallayout class='monospaced'>
1591 BBCLASSEXTEND = "native"
1592 </literallayout></para></listitem>
1593 <listitem><para><emphasis><filename>BBVERSIONS</filename>:</emphasis>
1594 This variable allows a single recipe to build multiple versions of a
1595 project from a single recipe file.
1596 You can also specify conditional metadata
1597 (using the
1598 <link linkend='var-OVERRIDES'><filename>OVERRIDES</filename></link>
1599 mechanism) for a single version, or an optionally named range of versions.
1600 Here is an example:
1601 <literallayout class='monospaced'>
1602 BBVERSIONS = "1.0 2.0 git"
1603 SRC_URI_git = "git://someurl/somepath.git"
1604
1605 BBVERSIONS = "1.0.[0-6]:1.0.0+ \ 1.0.[7-9]:1.0.7+"
1606 SRC_URI_append_1.0.7+ = "file://some_patch_which_the_new_versions_need.patch;patch=1"
1607 </literallayout>
1608 The name of the range defaults to the original version of the
1609 recipe.
1610 For example, in OpenEmbedded, the recipe file
1611 <filename>foo_1.0.0+.bb</filename> creates a default name range
1612 of <filename>1.0.0+</filename>.
1613 This is useful because the range name is not only placed
1614 into overrides, but it is also made available for the metadata to use
1615 in the variable that defines the base recipe versions for use in
1616 <filename>file://</filename> search paths
1617 (<link linkend='var-FILESPATH'><filename>FILESPATH</filename></link>).
1618 </para></listitem>
1619 </itemizedlist>
1620 </para>
1621 </section>
1622
1623 <section id='dependencies'>
1624 <title>Dependencies</title>
1625
1626 <para>
1627 To allow for efficient operation given multiple processes
1628 executing in parallel, BitBake handles dependencies at
1629 the task level.
1630 BitBake supports a robust method to handle these dependencies.
1631 </para>
1632
1633 <para>
1634 This section describes several types of dependency mechanisms.
1635 </para>
1636
1637 <section id='dependencies-internal-to-the-bb-file'>
1638 <title>Dependencies Internal to the <filename>.bb</filename> File</title>
1639
1640 <para>
1641 BitBake uses the <filename>addtask</filename> directive
1642 to manage dependencies that are internal to a given recipe
1643 file.
1644 You can use the <filename>addtask</filename> directive to
1645 indicate when a task is dependent on other tasks or when
1646 other tasks depend on that recipe.
1647 Here is an example:
1648 <literallayout class='monospaced'>
1649 addtask printdate after do_fetch before do_build
1650 </literallayout>
1651 In this example, the <filename>printdate</filename> task is
1652 depends on the completion of the <filename>do_fetch</filename>
1653 task.
1654 And, the <filename>do_build</filename> depends on the completion
1655 of the <filename>printdate</filename> task.
1656 </para>
1657 </section>
1658
1659 <section id='build-dependencies'>
1660 <title>Build Dependencies</title>
1661
1662 <para>
1663 BitBake uses the
1664 <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link>
1665 variable to manage build time dependencies.
1666 The "deptask" varflag for tasks signifies the task of each
1667 item listed in <filename>DEPENDS</filename> that must
1668 complete before that task can be executed.
1669 Here is an example:
1670 <literallayout class='monospaced'>
1671 do_configure[deptask] = "do_populate_sysroot"
1672 </literallayout>
1673 In this example, the <filename>do_populate_sysroot</filename>
1674 task of each item in <filename>DEPENDS</filename> must complete before
1675 <filename>do_configure</filename> can execute.
1676 </para>
1677 </section>
1678
1679 <section id='runtime-dependencies'>
1680 <title>Runtime Dependencies</title>
1681
1682 <para>
1683 BitBake uses the
1684 <link linkend='var-PACKAGES'><filename>PACKAGES</filename></link>,
1685 <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link>, and
1686 <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link>
1687 variables to manage runtime dependencies.
1688 </para>
1689
1690 <para>
1691 The <filename>PACKAGES</filename> variable lists runtime
1692 packages.
1693 Each of those packages can have <filename>RDEPENDS</filename> and
1694 <filename>RRECOMMENDS</filename> runtime dependencies.
1695 The "rdeptask" flag for tasks is used to signify the task of each
1696 item runtime dependency which must have completed before that
1697 task can be executed.
1698 <literallayout class='monospaced'>
1699 do_package_qa[rdeptask] = "do_packagedata"
1700 </literallayout>
1701 In the previous example, the <filename>do_packagedata</filename>
1702 task of each item in <filename>RDEPENDS</filename> must have
1703 completed before <filename>do_package_qa</filename> can execute.
1704 </para>
1705 </section>
1706
1707 <section id='recursive-dependencies'>
1708 <title>Recursive Dependencies</title>
1709
1710 <para>
1711 BitBake uses the "recrdeptask" flag to manage
1712 recursive task dependencies.
1713 BitBake looks through the build-time and runtime
1714 dependencies of the current recipe, looks through
1715 the task's inter-task
1716 dependencies, and then adds dependencies for the
1717 listed task.
1718 Once BitBake has accomplished this, it recursively works through
1719 the dependencies of those tasks.
1720 Iterative passes continue until all dependencies are discovered
1721 and added.
1722 </para>
1723
1724 <para>
1725 You might want to not only have BitBake look for
1726 dependencies of those tasks, but also have BitBake look
1727 for build-time and runtime dependencies of the dependent
1728 tasks as well.
1729 If that is the case, you need to reference the task name
1730 itself in the task list:
1731 <literallayout class='monospaced'>
1732 do_a[recrdeptask] = "do_a do_b"
1733 </literallayout>
1734 </para>
1735 </section>
1736
1737 <section id='inter-task-dependencies'>
1738 <title>Inter-Task Dependencies</title>
1739
1740 <para>
1741 BitBake uses the "depends" flag in a more generic form
1742 to manage inter-task dependencies.
1743 This more generic form allows for inter-dependency
1744 checks for specific tasks rather than checks for
1745 the data in <filename>DEPENDS</filename>.
1746 Here is an example:
1747 <literallayout class='monospaced'>
1748 do_patch[depends] = "quilt-native:do_populate_sysroot"
1749 </literallayout>
1750 In this example, the <filename>do_populate_sysroot</filename>
1751 task of the target <filename>quilt-native</filename>
1752 must have completed before the
1753 <filename>do_patch</filename> task can execute.
1754 </para>
1755
1756 <para>
1757 The "rdepends" flag works in a similar way but takes targets
1758 in the runtime namespace instead of the build-time dependency
1759 namespace.
1760 </para>
1761 </section>
1762 </section>
1763
1764 <section id='accessing-datastore-variables-using-python'>
1765 <title>Accessing Datastore Variables Using Python</title>
1766
1767 <para>
1768 It is often necessary to access variables in the
1769 BitBake datastore using Python functions.
1770 The Bitbake datastore has an API that allows you this
1771 access.
1772 Here is a list of available operations:
1773 </para>
1774
1775 <para>
1776 <informaltable frame='none'>
1777 <tgroup cols='2' align='left' colsep='1' rowsep='1'>
1778 <colspec colname='c1' colwidth='1*'/>
1779 <colspec colname='c2' colwidth='1*'/>
1780 <thead>
1781 <row>
1782 <entry align="left"><emphasis>Operation</emphasis></entry>
1783 <entry align="left"><emphasis>Description</emphasis></entry>
1784 </row>
1785 </thead>
1786 <tbody>
1787 <row>
1788 <entry align="left"><filename>d.getVar("X", expand=False)</filename></entry>
1789 <entry align="left">Returns the value of variable "X".
1790 Using "expand=True" expands the value.</entry>
1791 </row>
1792 <row>
1793 <entry align="left"><filename>d.setVar("X", "value")</filename></entry>
1794 <entry align="left">Sets the variable "X" to "value".</entry>
1795 </row>
1796 <row>
1797 <entry align="left"><filename>d.appendVar("X", "value")</filename></entry>
1798 <entry align="left">Adds "value" to the end of the variable "X".</entry>
1799 </row>
1800 <row>
1801 <entry align="left"><filename>d.prependVar("X", "value")</filename></entry>
1802 <entry align="left">Adds "value" to the start of the variable "X".</entry>
1803 </row>
1804 <row>
1805 <entry align="left"><filename>d.delVar("X")</filename></entry>
1806 <entry align="left">Deletes the variable "X" from the datastore.</entry>
1807 </row>
1808 <row>
1809 <entry align="left"><filename>d.renameVar("X", "Y")</filename></entry>
1810 <entry align="left">Renames the variable "X" to "Y".</entry>
1811 </row>
1812 <row>
1813 <entry align="left"><filename>d.getVarFlag("X", flag, expand=False)</filename></entry>
1814 <entry align="left">Gets then named flag from the variable "X".
1815 Using "expand=True" expands the named flag.</entry>
1816 </row>
1817 <row>
1818 <entry align="left"><filename>d.setVarFlag("X", flag, "value")</filename></entry>
1819 <entry align="left">Sets the named flag for variable "X" to "value".</entry>
1820 </row>
1821 <row>
1822 <entry align="left"><filename>d.appendVarFlag("X", flag, "value")</filename></entry>
1823 <entry align="left">Appends "value" to the named flag on the
1824 variable "X".</entry>
1825 </row>
1826 <row>
1827 <entry align="left"><filename>d.prependVarFlag("X", flag, "value")</filename></entry>
1828 <entry align="left">Prepends "value" to the named flag on
1829 the variable "X".</entry>
1830 </row>
1831 <row>
1832 <entry align="left"><filename>d.delVarFlag("X", flag)</filename></entry>
1833 <entry align="left">Deletes the named flag on the variable
1834 "X" from the datastore.</entry>
1835 </row>
1836 <row>
1837 <entry align="left"><filename>d.setVarFlags("X", flagsdict)</filename></entry>
1838 <entry align="left">Sets the flags specified in
1839 the <filename>flagsdict()</filename> parameter.
1840 <filename>setVarFlags</filename> does not clear previous flags.
1841 Think of this operation as <filename>addVarFlags</filename>.</entry>
1842 </row>
1843 <row>
1844 <entry align="left"><filename>d.getVarFlags("X")</filename></entry>
1845 <entry align="left">Returns a <filename>flagsdict</filename> of the flags for
1846 the variable "X".</entry>
1847 </row>
1848 <row>
1849 <entry align="left"><filename>d.delVarFlags("X")</filename></entry>
1850 <entry align="left">Deletes all the flags for the variable "X".</entry>
1851 </row>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001852 <row>
1853 <entry align="left"><filename>d.expand(expression)</filename></entry>
1854 <entry align="left">Expands variable references in the specified string expression.</entry>
1855 </row>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001856 </tbody>
1857 </tgroup>
1858 </informaltable>
1859 </para>
1860 </section>
1861
1862 <section id='task-checksums-and-setscene'>
1863 <title>Task Checksums and Setscene</title>
1864
1865 <para>
1866 BitBake uses checksums (or signatures) along with the setscene
1867 to determine if a task needs to be run.
1868 This section describes the process.
1869 To help understand how BitBake does this, the section assumes an
1870 OpenEmbedded metadata-based example.
1871 </para>
1872
1873 <para>
1874 This list is a place holder of content existed from previous work
1875 on the manual.
1876 Some or all of it probably needs integrated into the subsections
1877 that make up this section.
1878 For now, I have just provided a short glossary-like description
1879 for each variable.
1880 Ultimately, this list goes away.
1881 <itemizedlist>
1882 <listitem><para><filename>STAMP</filename>:
1883 The base path to create stamp files.</para></listitem>
1884 <listitem><para><filename>STAMPCLEAN</filename>
1885 Again, the base path to create stamp files but can use wildcards
1886 for matching a range of files for clean operations.
1887 </para></listitem>
1888 <listitem><para><filename>BB_STAMP_WHITELIST</filename>
1889 Lists stamp files that are looked at when the stamp policy
1890 is "whitelist".
1891 </para></listitem>
1892 <listitem><para><filename>BB_STAMP_POLICY</filename>
1893 Defines the mode for comparing timestamps of stamp files.
1894 </para></listitem>
1895 <listitem><para><filename>BB_HASHCHECK_FUNCTION</filename>
1896 Specifies the name of the function to call during
1897 the "setscene" part of the task's execution in order
1898 to validate the list of task hashes.
1899 </para></listitem>
1900 <listitem><para><filename>BB_SETSCENE_VERIFY_FUNCTION</filename>
1901 Specifies a function to call that verifies the list of
1902 planned task execution before the main task execution
1903 happens.
1904 </para></listitem>
1905 <listitem><para><filename>BB_SETSCENE_DEPVALID</filename>
1906 Specifies a function BitBake calls that determines
1907 whether BitBake requires a setscene dependency to
1908 be met.
1909 </para></listitem>
1910 <listitem><para><filename>BB_TASKHASH</filename>
1911 Within an executing task, this variable holds the hash
1912 of the task as returned by the currently enabled
1913 signature generator.
1914 </para></listitem>
1915 </itemizedlist>
1916 </para>
1917 </section>
1918</chapter>