blob: 598f8775db221846e502f809bbe9ed0d751c6e1a [file] [log] [blame]
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4
5<chapter id='extendpoky'>
6
7<title>Common Tasks</title>
8 <para>
9 This chapter describes fundamental procedures such as creating layers,
10 adding new software packages, extending or customizing images,
11 porting work to new hardware (adding a new machine), and so forth.
12 You will find that the procedures documented here occur often in the
13 development cycle using the Yocto Project.
14 </para>
15
16 <section id="understanding-and-creating-layers">
17 <title>Understanding and Creating Layers</title>
18
19 <para>
20 The OpenEmbedded build system supports organizing
21 <link linkend='metadata'>Metadata</link> into multiple layers.
22 Layers allow you to isolate different types of customizations from
23 each other.
24 You might find it tempting to keep everything in one layer when
25 working on a single project.
26 However, the more modular your Metadata, the easier
27 it is to cope with future changes.
28 </para>
29
30 <para>
31 To illustrate how layers are used to keep things modular, consider
32 machine customizations.
33 These types of customizations typically reside in a special layer,
34 rather than a general layer, called a Board Support Package (BSP)
35 Layer.
36 Furthermore, the machine customizations should be isolated from
37 recipes and Metadata that support a new GUI environment,
38 for example.
39 This situation gives you a couple of layers: one for the machine
40 configurations, and one for the GUI environment.
41 It is important to understand, however, that the BSP layer can
42 still make machine-specific additions to recipes within the GUI
43 environment layer without polluting the GUI layer itself
44 with those machine-specific changes.
45 You can accomplish this through a recipe that is a BitBake append
46 (<filename>.bbappend</filename>) file, which is described later
47 in this section.
Brad Bishop6e60e8b2018-02-01 10:27:11 -050048 <note>
49 For general information on BSP layer structure, see the
50 <ulink url='&YOCTO_DOCS_BSP_URL;#bsp'>Board Support Packages (BSP) - Developer's Guide</ulink>.
51 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -050052 </para>
53
54 <para>
55 </para>
56
57 <section id='yocto-project-layers'>
58 <title>Layers</title>
59
60 <para>
61 The <link linkend='source-directory'>Source Directory</link>
62 contains both general layers and BSP
63 layers right out of the box.
64 You can easily identify layers that ship with a
65 Yocto Project release in the Source Directory by their
66 folder names.
67 Folders that represent layers typically have names that begin with
68 the string <filename>meta-</filename>.
69 <note>
70 It is not a requirement that a layer name begin with the
71 prefix <filename>meta-</filename>, but it is a commonly
72 accepted standard in the Yocto Project community.
73 </note>
74 For example, when you set up the Source Directory structure,
75 you will see several layers:
76 <filename>meta</filename>,
77 <filename>meta-skeleton</filename>,
78 <filename>meta-selftest</filename>,
Patrick Williamsd8c66bc2016-06-20 12:57:21 -050079 <filename>meta-poky</filename>, and
Patrick Williamsc124f4f2015-09-15 14:41:29 -050080 <filename>meta-yocto-bsp</filename>.
81 Each of these folders represents a distinct layer.
82 </para>
83
84 <para>
85 As another example, if you set up a local copy of the
86 <filename>meta-intel</filename> Git repository
87 and then explore the folder of that general layer,
88 you will discover many Intel-specific BSP layers inside.
89 For more information on BSP layers, see the
90 "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>"
91 section in the Yocto Project Board Support Package (BSP)
92 Developer's Guide.
93 </para>
94 </section>
95
96 <section id='creating-your-own-layer'>
97 <title>Creating Your Own Layer</title>
98
99 <para>
100 It is very easy to create your own layers to use with the
101 OpenEmbedded build system.
102 The Yocto Project ships with scripts that speed up creating
103 general layers and BSP layers.
104 This section describes the steps you perform by hand to create
105 a layer so that you can better understand them.
106 For information about the layer-creation scripts, see the
107 "<ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-yocto-bsp-script'>Creating a New BSP Layer Using the yocto-bsp Script</ulink>"
108 section in the Yocto Project Board Support Package (BSP)
109 Developer's Guide and the
110 "<link linkend='creating-a-general-layer-using-the-yocto-layer-script'>Creating a General Layer Using the yocto-layer Script</link>"
111 section further down in this manual.
112 </para>
113
114 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500115 Follow these general steps to create your layer without the aid of a script:
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500116 <orderedlist>
117 <listitem><para><emphasis>Check Existing Layers:</emphasis>
118 Before creating a new layer, you should be sure someone
119 has not already created a layer containing the Metadata
120 you need.
121 You can see the
122 <ulink url='http://layers.openembedded.org/layerindex/layers/'><filename>OpenEmbedded Metadata Index</filename></ulink>
123 for a list of layers from the OpenEmbedded community
124 that can be used in the Yocto Project.
125 </para></listitem>
126 <listitem><para><emphasis>Create a Directory:</emphasis>
127 Create the directory for your layer.
128 While not strictly required, prepend the name of the
129 folder with the string <filename>meta-</filename>.
130 For example:
131 <literallayout class='monospaced'>
132 meta-mylayer
133 meta-GUI_xyz
134 meta-mymachine
135 </literallayout>
136 </para></listitem>
137 <listitem><para><emphasis>Create a Layer Configuration
138 File:</emphasis>
139 Inside your new layer folder, you need to create a
140 <filename>conf/layer.conf</filename> file.
141 It is easiest to take an existing layer configuration
142 file and copy that to your layer's
143 <filename>conf</filename> directory and then modify the
144 file as needed.</para>
145 <para>The
146 <filename>meta-yocto-bsp/conf/layer.conf</filename> file
147 demonstrates the required syntax:
148 <literallayout class='monospaced'>
149 # We have a conf and classes directory, add to BBPATH
150 BBPATH .= ":${LAYERDIR}"
151
152 # We have recipes-* directories, add to BBFILES
153 BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
154 ${LAYERDIR}/recipes-*/*/*.bbappend"
155
156 BBFILE_COLLECTIONS += "yoctobsp"
157 BBFILE_PATTERN_yoctobsp = "^${LAYERDIR}/"
158 BBFILE_PRIORITY_yoctobsp = "5"
159 LAYERVERSION_yoctobsp = "3"
160 </literallayout></para>
161 <para>Here is an explanation of the example:
162 <itemizedlist>
163 <listitem><para>The configuration and
164 classes directory is appended to
165 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>.
166 <note>
167 All non-distro layers, which include all BSP
168 layers, are expected to append the layer
169 directory to the
170 <filename>BBPATH</filename>.
171 On the other hand, distro layers, such as
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500172 <filename>meta-poky</filename>, can choose
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500173 to enforce their own precedence over
174 <filename>BBPATH</filename>.
175 For an example of that syntax, see the
176 <filename>layer.conf</filename> file for
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500177 the <filename>meta-poky</filename> layer.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500178 </note></para></listitem>
179 <listitem><para>The recipes for the layers are
180 appended to
181 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILES'>BBFILES</ulink></filename>.
182 </para></listitem>
183 <listitem><para>The
184 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_COLLECTIONS'>BBFILE_COLLECTIONS</ulink></filename>
185 variable is then appended with the layer name.
186 </para></listitem>
187 <listitem><para>The
188 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PATTERN'>BBFILE_PATTERN</ulink></filename>
189 variable is set to a regular expression and is
190 used to match files from
191 <filename>BBFILES</filename> into a particular
192 layer.
193 In this case,
194 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERDIR'>LAYERDIR</ulink></filename>
195 is used to make <filename>BBFILE_PATTERN</filename> match within the
196 layer's path.</para></listitem>
197 <listitem><para>The
198 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY'>BBFILE_PRIORITY</ulink></filename>
199 variable then assigns a priority to the layer.
200 Applying priorities is useful in situations
201 where the same recipe might appear in multiple
202 layers and allows you to choose the layer
203 that takes precedence.</para></listitem>
204 <listitem><para>The
205 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERVERSION'>LAYERVERSION</ulink></filename>
206 variable optionally specifies the version of a
207 layer as a single number.</para></listitem>
208 </itemizedlist></para>
209 <para>Note the use of the
210 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERDIR'>LAYERDIR</ulink></filename>
211 variable, which expands to the directory of the current
212 layer.</para>
213 <para>Through the use of the <filename>BBPATH</filename>
214 variable, BitBake locates class files
215 (<filename>.bbclass</filename>),
216 configuration files, and files that are included
217 with <filename>include</filename> and
218 <filename>require</filename> statements.
219 For these cases, BitBake uses the first file that
220 matches the name found in <filename>BBPATH</filename>.
221 This is similar to the way the <filename>PATH</filename>
222 variable is used for binaries.
223 It is recommended, therefore, that you use unique
224 class and configuration
225 filenames in your custom layer.</para></listitem>
226 <listitem><para><emphasis>Add Content:</emphasis> Depending
227 on the type of layer, add the content.
228 If the layer adds support for a machine, add the machine
229 configuration in a <filename>conf/machine/</filename>
230 file within the layer.
231 If the layer adds distro policy, add the distro
232 configuration in a <filename>conf/distro/</filename>
233 file within the layer.
234 If the layer introduces new recipes, put the recipes
235 you need in <filename>recipes-*</filename>
236 subdirectories within the layer.
237 <note>In order to be compliant with the Yocto Project,
238 a layer must contain a
239 <ulink url='&YOCTO_DOCS_BSP_URL;#bsp-filelayout-readme'>README file.</ulink>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500240 </note>
241 </para></listitem>
242 <listitem><para>
243 <emphasis>Optionally Test for Compatibility:</emphasis>
244 If you want permission to use the Yocto Project
245 Compatibility logo with your layer or application that
246 uses your layer, perform the steps to apply for
247 compatibility.
248 See the
249 "<link linkend='making-sure-your-layer-is-compatible-with-yocto-project'>Making Sure Your Layer is Compatible With Yocto Project</link>"
250 section for more information.
251 </para></listitem>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500252 </orderedlist>
253 </para>
254 </section>
255
256 <section id='best-practices-to-follow-when-creating-layers'>
257 <title>Best Practices to Follow When Creating Layers</title>
258
259 <para>
260 To create layers that are easier to maintain and that will
261 not impact builds for other machines, you should consider the
262 information in the following sections.
263 </para>
264
265 <section id='avoid-overlaying-entire-recipes'>
266 <title>Avoid "Overlaying" Entire Recipes</title>
267
268 <para>
269 Avoid "overlaying" entire recipes from other layers in your
270 configuration.
271 In other words, do not copy an entire recipe into your
272 layer and then modify it.
273 Rather, use an append file (<filename>.bbappend</filename>)
274 to override
275 only those parts of the original recipe you need to modify.
276 </para>
277 </section>
278
279 <section id='avoid-duplicating-include-files'>
280 <title>Avoid Duplicating Include Files</title>
281
282 <para>
283 Avoid duplicating include files.
284 Use append files (<filename>.bbappend</filename>)
285 for each recipe
286 that uses an include file.
287 Or, if you are introducing a new recipe that requires
288 the included file, use the path relative to the original
289 layer directory to refer to the file.
290 For example, use
291 <filename>require recipes-core/</filename><replaceable>package</replaceable><filename>/</filename><replaceable>file</replaceable><filename>.inc</filename>
292 instead of <filename>require </filename><replaceable>file</replaceable><filename>.inc</filename>.
293 If you're finding you have to overlay the include file,
294 it could indicate a deficiency in the include file in
295 the layer to which it originally belongs.
296 If this is the case, you should try to address that
297 deficiency instead of overlaying the include file.
298 For example, you could address this by getting the
299 maintainer of the include file to add a variable or
300 variables to make it easy to override the parts needing
301 to be overridden.
302 </para>
303 </section>
304
305 <section id='structure-your-layers'>
306 <title>Structure Your Layers</title>
307
308 <para>
309 Proper use of overrides within append files and placement
310 of machine-specific files within your layer can ensure that
311 a build is not using the wrong Metadata and negatively
312 impacting a build for a different machine.
313 Following are some examples:
314 <itemizedlist>
315 <listitem><para><emphasis>Modifying Variables to Support
316 a Different Machine:</emphasis>
317 Suppose you have a layer named
318 <filename>meta-one</filename> that adds support
319 for building machine "one".
320 To do so, you use an append file named
321 <filename>base-files.bbappend</filename> and
322 create a dependency on "foo" by altering the
323 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
324 variable:
325 <literallayout class='monospaced'>
326 DEPENDS = "foo"
327 </literallayout>
328 The dependency is created during any build that
329 includes the layer
330 <filename>meta-one</filename>.
331 However, you might not want this dependency
332 for all machines.
333 For example, suppose you are building for
334 machine "two" but your
335 <filename>bblayers.conf</filename> file has the
336 <filename>meta-one</filename> layer included.
337 During the build, the
338 <filename>base-files</filename> for machine
339 "two" will also have the dependency on
340 <filename>foo</filename>.</para>
341 <para>To make sure your changes apply only when
342 building machine "one", use a machine override
343 with the <filename>DEPENDS</filename> statement:
344 <literallayout class='monospaced'>
345 DEPENDS_one = "foo"
346 </literallayout>
347 You should follow the same strategy when using
348 <filename>_append</filename> and
349 <filename>_prepend</filename> operations:
350 <literallayout class='monospaced'>
351 DEPENDS_append_one = " foo"
352 DEPENDS_prepend_one = "foo "
353 </literallayout>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500354 As an actual example, here's a line from the recipe
355 for gnutls, which adds dependencies on
356 "argp-standalone" when building with the musl C
357 library:
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500358 <literallayout class='monospaced'>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500359 DEPENDS_append_libc-musl = " argp-standalone"
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500360 </literallayout>
361 <note>
362 Avoiding "+=" and "=+" and using
363 machine-specific
364 <filename>_append</filename>
365 and <filename>_prepend</filename> operations
366 is recommended as well.
367 </note></para></listitem>
368 <listitem><para><emphasis>Place Machine-Specific Files
369 in Machine-Specific Locations:</emphasis>
370 When you have a base recipe, such as
371 <filename>base-files.bb</filename>, that
372 contains a
373 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
374 statement to a file, you can use an append file
375 to cause the build to use your own version of
376 the file.
377 For example, an append file in your layer at
378 <filename>meta-one/recipes-core/base-files/base-files.bbappend</filename>
379 could extend
380 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink>
381 using
382 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink>
383 as follows:
384 <literallayout class='monospaced'>
385 FILESEXTRAPATHS_prepend := "${THISDIR}/${BPN}:"
386 </literallayout>
387 The build for machine "one" will pick up your
388 machine-specific file as long as you have the
389 file in
390 <filename>meta-one/recipes-core/base-files/base-files/</filename>.
391 However, if you are building for a different
392 machine and the
393 <filename>bblayers.conf</filename> file includes
394 the <filename>meta-one</filename> layer and
395 the location of your machine-specific file is
396 the first location where that file is found
397 according to <filename>FILESPATH</filename>,
398 builds for all machines will also use that
399 machine-specific file.</para>
400 <para>You can make sure that a machine-specific
401 file is used for a particular machine by putting
402 the file in a subdirectory specific to the
403 machine.
404 For example, rather than placing the file in
405 <filename>meta-one/recipes-core/base-files/base-files/</filename>
406 as shown above, put it in
407 <filename>meta-one/recipes-core/base-files/base-files/one/</filename>.
408 Not only does this make sure the file is used
409 only when building for machine "one", but the
410 build process locates the file more quickly.</para>
411 <para>In summary, you need to place all files
412 referenced from <filename>SRC_URI</filename>
413 in a machine-specific subdirectory within the
414 layer in order to restrict those files to
415 machine-specific builds.</para></listitem>
416 </itemizedlist>
417 </para>
418 </section>
419
420 <section id='other-recommendations'>
421 <title>Other Recommendations</title>
422
423 <para>
424 We also recommend the following:
425 <itemizedlist>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500426 <listitem><para>If you want permission to use the
427 Yocto Project Compatibility logo with your layer
428 or application that uses your layer, perform the
429 steps to apply for compatibility.
430 See the
431 "<link linkend='making-sure-your-layer-is-compatible-with-yocto-project'>Making Sure Your Layer is Compatible With Yocto Project</link>"
432 section for more information.
433 </para></listitem>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500434 <listitem><para>Store custom layers in a Git repository
435 that uses the
436 <filename>meta-<replaceable>layer_name</replaceable></filename> format.
437 </para></listitem>
438 <listitem><para>Clone the repository alongside other
439 <filename>meta</filename> directories in the
440 <link linkend='source-directory'>Source Directory</link>.
441 </para></listitem>
442 </itemizedlist>
443 Following these recommendations keeps your Source Directory and
444 its configuration entirely inside the Yocto Project's core
445 base.
446 </para>
447 </section>
448 </section>
449
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500450 <section id='making-sure-your-layer-is-compatible-with-yocto-project'>
451 <title>Making Sure Your Layer is Compatible With Yocto Project</title>
452
453 <para>
454 When you create a layer used with the Yocto Project, it is
455 advantageous to make sure that the layer interacts well with
456 existing Yocto Project layers (i.e. the layer is compatible
457 with the Yocto Project).
458 Ensuring compatibility makes the layer easy to be consumed
459 by others in the Yocto Project community and allows you
460 permission to use the Yocto Project Compatibility logo.
461 </para>
462
463 <para>
464 Version 1.0 of the Yocto Project Compatibility Program has
465 been in existence for a number of releases.
466 This version of the program consists of the layer application
467 process that requests permission to use the Yocto Project
468 Compatibility logo for your layer and application.
469 You can find version 1.0 of the form at
470 <ulink url='https://www.yoctoproject.org/webform/yocto-project-compatible-registration'></ulink>.
471 To be granted permission to use the logo, you need to be able
472 to answer "Yes" to the questions or have an acceptable
473 explanation for any questions answered "No".
474 </para>
475
476 <para>
477 A second version (2.0) of the Yocto Project Compatibility
478 Program is currently under development.
479 Included as part of version 2.0 (and currently available) is
480 the <filename>yocto-compat-layer.py</filename> script.
481 When run against a layer, this script tests the layer against
482 tighter constraints based on experiences of how layers have
483 worked in the real world and where pitfalls have been found.
484 </para>
485
486 <para>
487 Part of the 2.0 version of the program that is not currently
488 available but is in development is an updated compatibility
489 application form.
490 This updated form, among other questions, specifically
491 asks if your layer has passed the test using the
492 <filename>yocto-compat-layer.py</filename> script.
493 <note><title>Tip</title>
494 Even though the updated application form is currently
495 unavailable for version 2.0 of the Yocto Project
496 Compatibility Program, the
497 <filename>yocto-compat-layer.py</filename> script is
498 available in OE-Core.
499 You can use the script to assess the status of your
500 layers in advance of the 2.0 release of the program.
501 </note>
502 </para>
503
504 <para>
505 The remainder of this section presents information on the
506 version 1.0 registration form and on the
507 <filename>yocto-compat-layer.py</filename> script.
508 </para>
509
510 <section id='yocto-project-compatibility-program-application'>
511 <title>Yocto Project Compatibility Program Application</title>
512
513 <para>
514 Use the 1.0 version of the form to apply for your
515 layer's compatibility approval.
516 Upon successful application, you can use the Yocto
517 Project Compatibility logo with your layer and the
518 application that uses your layer.
519 </para>
520
521 <para>
522 To access the form, use this link:
523 <ulink url='https://www.yoctoproject.org/webform/yocto-project-compatible-registration'></ulink>.
524 Follow the instructions on the form to complete your
525 application.
526 </para>
527
528 <para>
529 The application consists of the following sections:
530 <itemizedlist>
531 <listitem><para>
532 <emphasis>Contact Information:</emphasis>
533 Provide your contact information as the fields
534 require.
535 Along with your information, provide the
536 released versions of the Yocto Project for which
537 your layer is compatible.
538 </para></listitem>
539 <listitem><para>
540 <emphasis>Acceptance Criteria:</emphasis>
541 Provide "Yes" or "No" answers for each of the
542 items in the checklist.
543 Space exists at the bottom of the form for any
544 explanations for items for which you answered "No".
545 </para></listitem>
546 <listitem><para>
547 <emphasis>Recommendations:</emphasis>
548 Provide answers for the questions regarding Linux
549 kernel use and build success.
550 </para></listitem>
551 </itemizedlist>
552 </para>
553 </section>
554
555 <section id='yocto-compat-layer-py-script'>
556 <title><filename>yocto-compat-layer.py</filename> Script</title>
557
558 <para>
559 The <filename>yocto-compat-layer.py</filename> script,
560 which is currently available, provides you a way to
561 assess how compatible your layer is with the Yocto
562 Project.
563 You should run this script prior to using the form to
564 apply for compatibility as described in the previous
565 section.
566 <note>
567 Because the script is part of the 2.0 release of the
568 Yocto Project Compatibility Program, you are not
569 required to successfully run your layer against it
570 in order to be granted compatibility status.
571 However, it is a good idea as it promotes
572 well-behaved layers and gives you an idea of where your
573 layer stands regarding compatibility.
574 </note>
575 </para>
576
577 <para>
578 The script divides tests into three areas: COMMON, BSD,
579 and DISTRO.
580 For example, given a distribution layer (DISTRO), the
581 layer must pass both the COMMON and DISTRO related tests.
582 Furthermore, if your layer is a BSP layer, the layer must
583 pass the COMMON and BSP set of tests.
584 </para>
585
586 <para>
587 To execute the script, enter the following commands from
588 your build directory:
589 <literallayout class='monospaced'>
590 $ source oe-init-build-env
591 $ yocto-compat-layer.py <replaceable>your_layer_directory</replaceable>
592 </literallayout>
593 Be sure to provide the actual directory for your layer
594 as part of the command.
595 </para>
596
597 <para>
598 Entering the command causes the script to determine the
599 type of layer and then to execute a set of specific
600 tests against the layer.
601 The following list overviews the test:
602 <itemizedlist>
603 <listitem><para>
604 <filename>common.test_readme</filename>:
605 Tests if a <filename>README</filename> file
606 exists in the layer and the file is not empty.
607 </para></listitem>
608 <listitem><para>
609 <filename>common.test_parse</filename>:
610 Tests to make sure that BitBake can parse the
611 files without error (i.e.
612 <filename>bitbake -p</filename>).
613 </para></listitem>
614 <listitem><para>
615 <filename>common.test_show_environment</filename>:
616 Tests that the global or per-recipe environment
617 is in order without errors (i.e.
618 <filename>bitbake -e</filename>).
619 </para></listitem>
620 <listitem><para>
621 <filename>common.test_signatures</filename>:
622 Tests to be sure that BSP and DISTRO layers do not
623 come with recipes that change signatures.
624 </para></listitem>
625 <listitem><para>
626 <filename>bsp.test_bsp_defines_machines</filename>:
627 Tests if a BSP layer has machine configurations.
628 </para></listitem>
629 <listitem><para>
630 <filename>bsp.test_bsp_no_set_machine</filename>:
631 Tests to ensure a BSP layer does not set the
632 machine when the layer is added.
633 </para></listitem>
634 <listitem><para>
635 <filename>distro.test_distro_defines_distros</filename>:
636 Tests if a DISTRO layer has distro configurations.
637 </para></listitem>
638 <listitem><para>
639 <filename>distro.test_distro_no_set_distro</filename>:
640 Tests to ensure a DISTRO layer does not set the
641 distribution when the layer is added.
642 </para></listitem>
643 </itemizedlist>
644 </para>
645 </section>
646 </section>
647
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500648 <section id='enabling-your-layer'>
649 <title>Enabling Your Layer</title>
650
651 <para>
652 Before the OpenEmbedded build system can use your new layer,
653 you need to enable it.
654 To enable your layer, simply add your layer's path to the
655 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBLAYERS'>BBLAYERS</ulink></filename>
656 variable in your <filename>conf/bblayers.conf</filename> file,
657 which is found in the
658 <link linkend='build-directory'>Build Directory</link>.
659 The following example shows how to enable a layer named
660 <filename>meta-mylayer</filename>:
661 <literallayout class='monospaced'>
662 LCONF_VERSION = "6"
663
664 BBPATH = "${TOPDIR}"
665 BBFILES ?= ""
666
667 BBLAYERS ?= " \
668 $HOME/poky/meta \
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500669 $HOME/poky/meta-poky \
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500670 $HOME/poky/meta-yocto-bsp \
671 $HOME/poky/meta-mylayer \
672 "
673 </literallayout>
674 </para>
675
676 <para>
677 BitBake parses each <filename>conf/layer.conf</filename> file
678 as specified in the <filename>BBLAYERS</filename> variable
679 within the <filename>conf/bblayers.conf</filename> file.
680 During the processing of each
681 <filename>conf/layer.conf</filename> file, BitBake adds the
682 recipes, classes and configurations contained within the
683 particular layer to the source directory.
684 </para>
685 </section>
686
687 <section id='using-bbappend-files'>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500688 <title>Using .bbappend Files in Your Layer</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500689
690 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500691 A recipe that appends Metadata to another recipe is called a
692 BitBake append file.
693 A BitBake append file uses the <filename>.bbappend</filename>
694 file type suffix, while the corresponding recipe to which
695 Metadata is being appended uses the <filename>.bb</filename>
696 file type suffix.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500697 </para>
698
699 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500700 You can use a <filename>.bbappend</filename> file in your
701 layer to make additions or changes to the content of another
702 layer's recipe without having to copy the other layer's
703 recipe into your layer.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500704 Your <filename>.bbappend</filename> file resides in your layer,
705 while the main <filename>.bb</filename> recipe file to
706 which you are appending Metadata resides in a different layer.
707 </para>
708
709 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500710 Being able to append information to an existing recipe not only
711 avoids duplication, but also automatically applies recipe
712 changes from a different layer into your layer.
713 If you were copying recipes, you would have to manually merge
714 changes as they occur.
715 </para>
716
717 <para>
718 When you create an append file, you must use the same root
719 name as the corresponding recipe file.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500720 For example, the append file
721 <filename>someapp_&DISTRO;.bbappend</filename> must apply to
722 <filename>someapp_&DISTRO;.bb</filename>.
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500723 This means the original recipe and append file names are
724 version number-specific.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500725 If the corresponding recipe is renamed to update to a newer
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500726 version, you must also rename and possibly update
727 the corresponding <filename>.bbappend</filename> as well.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500728 During the build process, BitBake displays an error on starting
729 if it detects a <filename>.bbappend</filename> file that does
730 not have a corresponding recipe with a matching name.
731 See the
732 <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_DANGLINGAPPENDS_WARNONLY'><filename>BB_DANGLINGAPPENDS_WARNONLY</filename></ulink>
733 variable for information on how to handle this error.
734 </para>
735
736 <para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500737 As an example, consider the main formfactor recipe and a
738 corresponding formfactor append file both from the
739 <link linkend='source-directory'>Source Directory</link>.
740 Here is the main formfactor recipe, which is named
741 <filename>formfactor_0.0.bb</filename> and located in the
742 "meta" layer at
743 <filename>meta/recipes-bsp/formfactor</filename>:
744 <literallayout class='monospaced'>
745 SUMMARY = "Device formfactor information"
746 SECTION = "base"
747 LICENSE = "MIT"
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500748 LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500749 PR = "r45"
750
751 SRC_URI = "file://config file://machconfig"
752 S = "${WORKDIR}"
753
754 PACKAGE_ARCH = "${MACHINE_ARCH}"
755 INHIBIT_DEFAULT_DEPS = "1"
756
757 do_install() {
758 # Install file only if it has contents
759 install -d ${D}${sysconfdir}/formfactor/
760 install -m 0644 ${S}/config ${D}${sysconfdir}/formfactor/
761 if [ -s "${S}/machconfig" ]; then
762 install -m 0644 ${S}/machconfig ${D}${sysconfdir}/formfactor/
763 fi
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500764 } </literallayout>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500765 In the main recipe, note the
766 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
767 variable, which tells the OpenEmbedded build system where to
768 find files during the build.
769 </para>
770
771 <para>
772 Following is the append file, which is named
773 <filename>formfactor_0.0.bbappend</filename> and is from the
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500774 Raspberry Pi BSP Layer named
775 <filename>meta-raspberrypi</filename>.
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500776 The file is in the layer at
777 <filename>recipes-bsp/formfactor</filename>:
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500778 <literallayout class='monospaced'>
779 FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
780 </literallayout>
781 </para>
782
783 <para>
784 By default, the build system uses the
785 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink>
786 variable to locate files.
787 This append file extends the locations by setting the
788 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink>
789 variable.
790 Setting this variable in the <filename>.bbappend</filename>
791 file is the most reliable and recommended method for adding
792 directories to the search path used by the build system
793 to find files.
794 </para>
795
796 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500797 The statement in this example extends the directories to
798 include
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500799 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-THISDIR'><filename>THISDIR</filename></ulink><filename>}/${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename>,
800 which resolves to a directory named
801 <filename>formfactor</filename> in the same directory
802 in which the append file resides (i.e.
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500803 <filename>meta-raspberrypi/recipes-bsp/formfactor</filename>.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500804 This implies that you must have the supporting directory
805 structure set up that will contain any files or patches you
806 will be including from the layer.
807 </para>
808
809 <para>
810 Using the immediate expansion assignment operator
Brad Bishop6e60e8b2018-02-01 10:27:11 -0500811 <filename>:=</filename> is important because of the reference
812 to <filename>THISDIR</filename>.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500813 The trailing colon character is important as it ensures that
814 items in the list remain colon-separated.
815 <note>
816 <para>
817 BitBake automatically defines the
818 <filename>THISDIR</filename> variable.
819 You should never set this variable yourself.
820 Using "_prepend" as part of the
821 <filename>FILESEXTRAPATHS</filename> ensures your path
822 will be searched prior to other paths in the final
823 list.
824 </para>
825
826 <para>
827 Also, not all append files add extra files.
828 Many append files simply exist to add build options
829 (e.g. <filename>systemd</filename>).
830 For these cases, your append file would not even
831 use the <filename>FILESEXTRAPATHS</filename> statement.
832 </para>
833 </note>
834 </para>
835 </section>
836
837 <section id='prioritizing-your-layer'>
838 <title>Prioritizing Your Layer</title>
839
840 <para>
841 Each layer is assigned a priority value.
842 Priority values control which layer takes precedence if there
843 are recipe files with the same name in multiple layers.
844 For these cases, the recipe file from the layer with a higher
845 priority number takes precedence.
846 Priority values also affect the order in which multiple
847 <filename>.bbappend</filename> files for the same recipe are
848 applied.
849 You can either specify the priority manually, or allow the
850 build system to calculate it based on the layer's dependencies.
851 </para>
852
853 <para>
854 To specify the layer's priority manually, use the
855 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY'><filename>BBFILE_PRIORITY</filename></ulink>
856 variable.
857 For example:
858 <literallayout class='monospaced'>
859 BBFILE_PRIORITY_mylayer = "1"
860 </literallayout>
861 </para>
862
863 <note>
864 <para>It is possible for a recipe with a lower version number
865 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>
866 in a layer that has a higher priority to take precedence.</para>
867 <para>Also, the layer priority does not currently affect the
868 precedence order of <filename>.conf</filename>
869 or <filename>.bbclass</filename> files.
870 Future versions of BitBake might address this.</para>
871 </note>
872 </section>
873
874 <section id='managing-layers'>
875 <title>Managing Layers</title>
876
877 <para>
878 You can use the BitBake layer management tool to provide a view
879 into the structure of recipes across a multi-layer project.
880 Being able to generate output that reports on configured layers
881 with their paths and priorities and on
882 <filename>.bbappend</filename> files and their applicable
883 recipes can help to reveal potential problems.
884 </para>
885
886 <para>
887 Use the following form when running the layer management tool.
888 <literallayout class='monospaced'>
889 $ bitbake-layers <replaceable>command</replaceable> [<replaceable>arguments</replaceable>]
890 </literallayout>
891 The following list describes the available commands:
892 <itemizedlist>
893 <listitem><para><filename><emphasis>help:</emphasis></filename>
894 Displays general help or help on a specified command.
895 </para></listitem>
896 <listitem><para><filename><emphasis>show-layers:</emphasis></filename>
897 Shows the current configured layers.
898 </para></listitem>
899 <listitem><para><filename><emphasis>show-recipes:</emphasis></filename>
900 Lists available recipes and the layers that provide them.
901 </para></listitem>
902 <listitem><para><filename><emphasis>show-overlayed:</emphasis></filename>
903 Lists overlayed recipes.
904 A recipe is overlayed when a recipe with the same name
905 exists in another layer that has a higher layer
906 priority.
907 </para></listitem>
908 <listitem><para><filename><emphasis>show-appends:</emphasis></filename>
909 Lists <filename>.bbappend</filename> files and the
910 recipe files to which they apply.
911 </para></listitem>
912 <listitem><para><filename><emphasis>show-cross-depends:</emphasis></filename>
913 Lists dependency relationships between recipes that
914 cross layer boundaries.
915 </para></listitem>
916 <listitem><para><filename><emphasis>add-layer:</emphasis></filename>
917 Adds a layer to <filename>bblayers.conf</filename>.
918 </para></listitem>
919 <listitem><para><filename><emphasis>remove-layer:</emphasis></filename>
920 Removes a layer from <filename>bblayers.conf</filename>
921 </para></listitem>
922 <listitem><para><filename><emphasis>flatten:</emphasis></filename>
923 Flattens the layer configuration into a separate output
924 directory.
925 Flattening your layer configuration builds a "flattened"
926 directory that contains the contents of all layers,
927 with any overlayed recipes removed and any
928 <filename>.bbappend</filename> files appended to the
929 corresponding recipes.
930 You might have to perform some manual cleanup of the
931 flattened layer as follows:
932 <itemizedlist>
933 <listitem><para>Non-recipe files (such as patches)
934 are overwritten.
935 The flatten command shows a warning for these
936 files.
937 </para></listitem>
938 <listitem><para>Anything beyond the normal layer
939 setup has been added to the
940 <filename>layer.conf</filename> file.
941 Only the lowest priority layer's
942 <filename>layer.conf</filename> is used.
943 </para></listitem>
944 <listitem><para>Overridden and appended items from
945 <filename>.bbappend</filename> files need to be
946 cleaned up.
947 The contents of each
948 <filename>.bbappend</filename> end up in the
949 flattened recipe.
950 However, if there are appended or changed
951 variable values, you need to tidy these up
952 yourself.
953 Consider the following example.
954 Here, the <filename>bitbake-layers</filename>
955 command adds the line
956 <filename>#### bbappended ...</filename> so that
957 you know where the following lines originate:
958 <literallayout class='monospaced'>
959 ...
960 DESCRIPTION = "A useful utility"
961 ...
962 EXTRA_OECONF = "--enable-something"
963 ...
964
965 #### bbappended from meta-anotherlayer ####
966
967 DESCRIPTION = "Customized utility"
968 EXTRA_OECONF += "--enable-somethingelse"
969 </literallayout>
970 Ideally, you would tidy up these utilities as
971 follows:
972 <literallayout class='monospaced'>
973 ...
974 DESCRIPTION = "Customized utility"
975 ...
976 EXTRA_OECONF = "--enable-something --enable-somethingelse"
977 ...
978 </literallayout></para></listitem>
979 </itemizedlist></para></listitem>
980 </itemizedlist>
981 </para>
982 </section>
983
984 <section id='creating-a-general-layer-using-the-yocto-layer-script'>
985 <title>Creating a General Layer Using the yocto-layer Script</title>
986
987 <para>
988 The <filename>yocto-layer</filename> script simplifies
989 creating a new general layer.
990 <note>
991 For information on BSP layers, see the
992 "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>"
993 section in the Yocto Project Board Specific (BSP)
994 Developer's Guide.
995 </note>
996 The default mode of the script's operation is to prompt you for
997 information needed to generate the layer:
998 <itemizedlist>
999 <listitem><para>The layer priority.
1000 </para></listitem>
1001 <listitem><para>Whether or not to create a sample recipe.
1002 </para></listitem>
1003 <listitem><para>Whether or not to create a sample
1004 append file.
1005 </para></listitem>
1006 </itemizedlist>
1007 </para>
1008
1009 <para>
1010 Use the <filename>yocto-layer create</filename> sub-command
1011 to create a new general layer.
1012 In its simplest form, you can create a layer as follows:
1013 <literallayout class='monospaced'>
1014 $ yocto-layer create mylayer
1015 </literallayout>
1016 The previous example creates a layer named
1017 <filename>meta-mylayer</filename> in the current directory.
1018 </para>
1019
1020 <para>
1021 As the <filename>yocto-layer create</filename> command runs,
1022 default values for the prompts appear in brackets.
1023 Pressing enter without supplying anything for the prompts
1024 or pressing enter and providing an invalid response causes the
1025 script to accept the default value.
1026 Once the script completes, the new layer
1027 is created in the current working directory.
1028 The script names the layer by prepending
1029 <filename>meta-</filename> to the name you provide.
1030 </para>
1031
1032 <para>
1033 Minimally, the script creates the following within the layer:
1034 <itemizedlist>
1035 <listitem><para><emphasis>The <filename>conf</filename>
1036 directory:</emphasis>
1037 This directory contains the layer's configuration file.
1038 The root name for the file is the same as the root name
1039 your provided for the layer (e.g.
1040 <filename><replaceable>layer</replaceable>.conf</filename>).
1041 </para></listitem>
1042 <listitem><para><emphasis>The
1043 <filename>COPYING.MIT</filename> file:</emphasis>
1044 The copyright and use notice for the software.
1045 </para></listitem>
1046 <listitem><para><emphasis>The <filename>README</filename>
1047 file:</emphasis>
1048 A file describing the contents of your new layer.
1049 </para></listitem>
1050 </itemizedlist>
1051 </para>
1052
1053 <para>
1054 If you choose to generate a sample recipe file, the script
1055 prompts you for the name for the recipe and then creates it
1056 in <filename><replaceable>layer</replaceable>/recipes-example/example/</filename>.
1057 The script creates a <filename>.bb</filename> file and a
1058 directory, which contains a sample
1059 <filename>helloworld.c</filename> source file, along with
1060 a sample patch file.
1061 If you do not provide a recipe name, the script uses
1062 "example".
1063 </para>
1064
1065 <para>
1066 If you choose to generate a sample append file, the script
1067 prompts you for the name for the file and then creates it
1068 in <filename><replaceable>layer</replaceable>/recipes-example-bbappend/example-bbappend/</filename>.
1069 The script creates a <filename>.bbappend</filename> file and a
1070 directory, which contains a sample patch file.
1071 If you do not provide a recipe name, the script uses
1072 "example".
1073 The script also prompts you for the version of the append file.
1074 The version should match the recipe to which the append file
1075 is associated.
1076 </para>
1077
1078 <para>
1079 The easiest way to see how the <filename>yocto-layer</filename>
1080 script works is to experiment with the script.
1081 You can also read the usage information by entering the
1082 following:
1083 <literallayout class='monospaced'>
1084 $ yocto-layer help
1085 </literallayout>
1086 </para>
1087
1088 <para>
1089 Once you create your general layer, you must add it to your
1090 <filename>bblayers.conf</filename> file.
1091 Here is an example where a layer named
1092 <filename>meta-mylayer</filename> is added:
1093 <literallayout class='monospaced'>
1094 BBLAYERS = ?" \
1095 /usr/local/src/yocto/meta \
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05001096 /usr/local/src/yocto/meta-poky \
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001097 /usr/local/src/yocto/meta-yocto-bsp \
1098 /usr/local/src/yocto/meta-mylayer \
1099 "
1100 </literallayout>
1101 Adding the layer to this file enables the build system to
1102 locate the layer during the build.
1103 </para>
1104 </section>
1105 </section>
1106
1107 <section id='usingpoky-extend-customimage'>
1108 <title>Customizing Images</title>
1109
1110 <para>
1111 You can customize images to satisfy particular requirements.
1112 This section describes several methods and provides guidelines for each.
1113 </para>
1114
1115 <section id='usingpoky-extend-customimage-localconf'>
1116 <title>Customizing Images Using <filename>local.conf</filename></title>
1117
1118 <para>
1119 Probably the easiest way to customize an image is to add a
1120 package by way of the <filename>local.conf</filename>
1121 configuration file.
1122 Because it is limited to local use, this method generally only
1123 allows you to add packages and is not as flexible as creating
1124 your own customized image.
1125 When you add packages using local variables this way, you need
1126 to realize that these variable changes are in effect for every
1127 build and consequently affect all images, which might not
1128 be what you require.
1129 </para>
1130
1131 <para>
1132 To add a package to your image using the local configuration
1133 file, use the
1134 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename>
1135 variable with the <filename>_append</filename> operator:
1136 <literallayout class='monospaced'>
1137 IMAGE_INSTALL_append = " strace"
1138 </literallayout>
1139 Use of the syntax is important - specifically, the space between
1140 the quote and the package name, which is
1141 <filename>strace</filename> in this example.
1142 This space is required since the <filename>_append</filename>
1143 operator does not add the space.
1144 </para>
1145
1146 <para>
1147 Furthermore, you must use <filename>_append</filename> instead
1148 of the <filename>+=</filename> operator if you want to avoid
1149 ordering issues.
1150 The reason for this is because doing so unconditionally appends
1151 to the variable and avoids ordering problems due to the
1152 variable being set in image recipes and
1153 <filename>.bbclass</filename> files with operators like
1154 <filename>?=</filename>.
1155 Using <filename>_append</filename> ensures the operation takes
1156 affect.
1157 </para>
1158
1159 <para>
1160 As shown in its simplest use,
1161 <filename>IMAGE_INSTALL_append</filename> affects all images.
1162 It is possible to extend the syntax so that the variable
1163 applies to a specific image only.
1164 Here is an example:
1165 <literallayout class='monospaced'>
1166 IMAGE_INSTALL_append_pn-core-image-minimal = " strace"
1167 </literallayout>
1168 This example adds <filename>strace</filename> to the
1169 <filename>core-image-minimal</filename> image only.
1170 </para>
1171
1172 <para>
1173 You can add packages using a similar approach through the
1174 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-CORE_IMAGE_EXTRA_INSTALL'>CORE_IMAGE_EXTRA_INSTALL</ulink></filename>
1175 variable.
1176 If you use this variable, only
1177 <filename>core-image-*</filename> images are affected.
1178 </para>
1179 </section>
1180
1181 <section id='usingpoky-extend-customimage-imagefeatures'>
1182 <title>Customizing Images Using Custom <filename>IMAGE_FEATURES</filename> and
1183 <filename>EXTRA_IMAGE_FEATURES</filename></title>
1184
1185 <para>
1186 Another method for customizing your image is to enable or
1187 disable high-level image features by using the
1188 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
1189 and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink>
1190 variables.
1191 Although the functions for both variables are nearly equivalent,
1192 best practices dictate using <filename>IMAGE_FEATURES</filename>
1193 from within a recipe and using
1194 <filename>EXTRA_IMAGE_FEATURES</filename> from within
1195 your <filename>local.conf</filename> file, which is found in the
1196 <link linkend='build-directory'>Build Directory</link>.
1197 </para>
1198
1199 <para>
1200 To understand how these features work, the best reference is
1201 <filename>meta/classes/core-image.bbclass</filename>.
1202 This class lists out the available
1203 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
1204 of which most map to package groups while some, such as
1205 <filename>debug-tweaks</filename> and
1206 <filename>read-only-rootfs</filename>, resolve as general
1207 configuration settings.
1208 </para>
1209
1210 <para>
1211 In summary, the file looks at the contents of the
1212 <filename>IMAGE_FEATURES</filename> variable and then maps
1213 or configures the feature accordingly.
1214 Based on this information, the build system automatically
1215 adds the appropriate packages or configurations to the
1216 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink>
1217 variable.
1218 Effectively, you are enabling extra features by extending the
1219 class or creating a custom class for use with specialized image
1220 <filename>.bb</filename> files.
1221 </para>
1222
1223 <para>
1224 Use the <filename>EXTRA_IMAGE_FEATURES</filename> variable
1225 from within your local configuration file.
1226 Using a separate area from which to enable features with
1227 this variable helps you avoid overwriting the features in the
1228 image recipe that are enabled with
1229 <filename>IMAGE_FEATURES</filename>.
1230 The value of <filename>EXTRA_IMAGE_FEATURES</filename> is added
1231 to <filename>IMAGE_FEATURES</filename> within
1232 <filename>meta/conf/bitbake.conf</filename>.
1233 </para>
1234
1235 <para>
1236 To illustrate how you can use these variables to modify your
1237 image, consider an example that selects the SSH server.
1238 The Yocto Project ships with two SSH servers you can use
1239 with your images: Dropbear and OpenSSH.
1240 Dropbear is a minimal SSH server appropriate for
1241 resource-constrained environments, while OpenSSH is a
1242 well-known standard SSH server implementation.
1243 By default, the <filename>core-image-sato</filename> image
1244 is configured to use Dropbear.
1245 The <filename>core-image-full-cmdline</filename> and
1246 <filename>core-image-lsb</filename> images both
1247 include OpenSSH.
1248 The <filename>core-image-minimal</filename> image does not
1249 contain an SSH server.
1250 </para>
1251
1252 <para>
1253 You can customize your image and change these defaults.
1254 Edit the <filename>IMAGE_FEATURES</filename> variable
1255 in your recipe or use the
1256 <filename>EXTRA_IMAGE_FEATURES</filename> in your
1257 <filename>local.conf</filename> file so that it configures the
1258 image you are working with to include
1259 <filename>ssh-server-dropbear</filename> or
1260 <filename>ssh-server-openssh</filename>.
1261 </para>
1262
1263 <note>
1264 See the
1265 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>"
1266 section in the Yocto Project Reference Manual for a complete
1267 list of image features that ship with the Yocto Project.
1268 </note>
1269 </section>
1270
1271 <section id='usingpoky-extend-customimage-custombb'>
1272 <title>Customizing Images Using Custom .bb Files</title>
1273
1274 <para>
1275 You can also customize an image by creating a custom recipe
1276 that defines additional software as part of the image.
1277 The following example shows the form for the two lines you need:
1278 <literallayout class='monospaced'>
1279 IMAGE_INSTALL = "packagegroup-core-x11-base package1 package2"
1280
1281 inherit core-image
1282 </literallayout>
1283 </para>
1284
1285 <para>
1286 Defining the software using a custom recipe gives you total
1287 control over the contents of the image.
1288 It is important to use the correct names of packages in the
1289 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename>
1290 variable.
1291 You must use the OpenEmbedded notation and not the Debian notation for the names
1292 (e.g. <filename>glibc-dev</filename> instead of <filename>libc6-dev</filename>).
1293 </para>
1294
1295 <para>
1296 The other method for creating a custom image is to base it on an existing image.
1297 For example, if you want to create an image based on <filename>core-image-sato</filename>
1298 but add the additional package <filename>strace</filename> to the image,
1299 copy the <filename>meta/recipes-sato/images/core-image-sato.bb</filename> to a
1300 new <filename>.bb</filename> and add the following line to the end of the copy:
1301 <literallayout class='monospaced'>
1302 IMAGE_INSTALL += "strace"
1303 </literallayout>
1304 </para>
1305 </section>
1306
1307 <section id='usingpoky-extend-customimage-customtasks'>
1308 <title>Customizing Images Using Custom Package Groups</title>
1309
1310 <para>
1311 For complex custom images, the best approach for customizing
1312 an image is to create a custom package group recipe that is
1313 used to build the image or images.
1314 A good example of a package group recipe is
1315 <filename>meta/recipes-core/packagegroups/packagegroup-base.bb</filename>.
1316 </para>
1317
1318 <para>
1319 If you examine that recipe, you see that the
1320 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'>PACKAGES</ulink></filename>
1321 variable lists the package group packages to produce.
1322 The <filename>inherit packagegroup</filename> statement
1323 sets appropriate default values and automatically adds
1324 <filename>-dev</filename>, <filename>-dbg</filename>, and
1325 <filename>-ptest</filename> complementary packages for each
1326 package specified in the <filename>PACKAGES</filename>
1327 statement.
1328 <note>
1329 The <filename>inherit packages</filename> should be
1330 located near the top of the recipe, certainly before
1331 the <filename>PACKAGES</filename> statement.
1332 </note>
1333 </para>
1334
1335 <para>
1336 For each package you specify in <filename>PACKAGES</filename>,
1337 you can use
1338 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'>RDEPENDS</ulink></filename>
1339 and
1340 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'>RRECOMMENDS</ulink></filename>
1341 entries to provide a list of packages the parent task package
1342 should contain.
1343 You can see examples of these further down in the
1344 <filename>packagegroup-base.bb</filename> recipe.
1345 </para>
1346
1347 <para>
1348 Here is a short, fabricated example showing the same basic
1349 pieces:
1350 <literallayout class='monospaced'>
1351 DESCRIPTION = "My Custom Package Groups"
1352
1353 inherit packagegroup
1354
1355 PACKAGES = "\
1356 packagegroup-custom-apps \
1357 packagegroup-custom-tools \
1358 "
1359
1360 RDEPENDS_packagegroup-custom-apps = "\
1361 dropbear \
1362 portmap \
1363 psplash"
1364
1365 RDEPENDS_packagegroup-custom-tools = "\
1366 oprofile \
1367 oprofileui-server \
1368 lttng-tools"
1369
1370 RRECOMMENDS_packagegroup-custom-tools = "\
1371 kernel-module-oprofile"
1372 </literallayout>
1373 </para>
1374
1375 <para>
1376 In the previous example, two package group packages are created with their dependencies and their
1377 recommended package dependencies listed: <filename>packagegroup-custom-apps</filename>, and
1378 <filename>packagegroup-custom-tools</filename>.
1379 To build an image using these package group packages, you need to add
1380 <filename>packagegroup-custom-apps</filename> and/or
1381 <filename>packagegroup-custom-tools</filename> to
1382 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename>.
1383 For other forms of image dependencies see the other areas of this section.
1384 </para>
1385 </section>
1386
1387 <section id='usingpoky-extend-customimage-image-name'>
1388 <title>Customizing an Image Hostname</title>
1389
1390 <para>
1391 By default, the configured hostname (i.e.
1392 <filename>/etc/hostname</filename>) in an image is the
1393 same as the machine name.
1394 For example, if
1395 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
1396 equals "qemux86", the configured hostname written to
1397 <filename>/etc/hostname</filename> is "qemux86".
1398 </para>
1399
1400 <para>
1401 You can customize this name by altering the value of the
1402 "hostname" variable in the
1403 <filename>base-files</filename> recipe using either
1404 an append file or a configuration file.
1405 Use the following in an append file:
1406 <literallayout class='monospaced'>
1407 hostname="myhostname"
1408 </literallayout>
1409 Use the following in a configuration file:
1410 <literallayout class='monospaced'>
1411 hostname_pn-base-files = "myhostname"
1412 </literallayout>
1413 </para>
1414
1415 <para>
1416 Changing the default value of the variable "hostname" can be
1417 useful in certain situations.
1418 For example, suppose you need to do extensive testing on an
1419 image and you would like to easily identify the image
1420 under test from existing images with typical default
1421 hostnames.
1422 In this situation, you could change the default hostname to
1423 "testme", which results in all the images using the name
1424 "testme".
1425 Once testing is complete and you do not need to rebuild the
1426 image for test any longer, you can easily reset the default
1427 hostname.
1428 </para>
1429
1430 <para>
1431 Another point of interest is that if you unset the variable,
1432 the image will have no default hostname in the filesystem.
1433 Here is an example that unsets the variable in a
1434 configuration file:
1435 <literallayout class='monospaced'>
1436 hostname_pn-base-files = ""
1437 </literallayout>
1438 Having no default hostname in the filesystem is suitable for
1439 environments that use dynamic hostnames such as virtual
1440 machines.
1441 </para>
1442 </section>
1443 </section>
1444
1445 <section id='new-recipe-writing-a-new-recipe'>
1446 <title>Writing a New Recipe</title>
1447
1448 <para>
1449 Recipes (<filename>.bb</filename> files) are fundamental components
1450 in the Yocto Project environment.
1451 Each software component built by the OpenEmbedded build system
1452 requires a recipe to define the component.
1453 This section describes how to create, write, and test a new
1454 recipe.
1455 <note>
1456 For information on variables that are useful for recipes and
1457 for information about recipe naming issues, see the
1458 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-varlocality-recipe-required'>Required</ulink>"
1459 section of the Yocto Project Reference Manual.
1460 </note>
1461 </para>
1462
1463 <section id='new-recipe-overview'>
1464 <title>Overview</title>
1465
1466 <para>
1467 The following figure shows the basic process for creating a
1468 new recipe.
1469 The remainder of the section provides details for the steps.
1470 <imagedata fileref="figures/recipe-workflow.png" width="6in" depth="7in" align="center" scalefit="1" />
1471 </para>
1472 </section>
1473
1474 <section id='new-recipe-locate-or-automatically-create-a-base-recipe'>
1475 <title>Locate or Automatically Create a Base Recipe</title>
1476
1477 <para>
1478 You can always write a recipe from scratch.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001479 However, three choices exist that can help you quickly get a
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001480 start on a new recipe:
1481 <itemizedlist>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001482 <listitem><para>
1483 <emphasis><filename>devtool add</filename>:</emphasis>
1484 A command that assists in creating a recipe and
1485 an environment conducive to development.
1486 </para></listitem>
1487 <listitem><para>
1488 <emphasis><filename>recipetool create</filename>:</emphasis>
1489 A command provided by the Yocto Project that automates
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001490 creation of a base recipe based on the source
1491 files.
1492 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001493 <listitem><para>
1494 <emphasis>Existing Recipes:</emphasis>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001495 Location and modification of an existing recipe that is
1496 similar in function to the recipe you need.
1497 </para></listitem>
1498 </itemizedlist>
1499 </para>
1500
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001501 <section id='new-recipe-creating-the-base-recipe-using-devtool'>
1502 <title>Creating the Base Recipe Using <filename>devtool add</filename></title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001503
1504 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001505 The <filename>devtool add</filename> command uses the same
1506 logic for auto-creating the recipe as
1507 <filename>recipetool create</filename>, which is listed
1508 below.
1509 Additionally, however, <filename>devtool add</filename>
1510 sets up an environment that makes it easy for you to
1511 patch the source and to make changes to the recipe as
1512 is often necessary when adding a recipe to build a new
1513 piece of software to be included in a build.
1514 </para>
1515
1516 <para>
1517 You can find a complete description of the
1518 <filename>devtool add</filename> command in the
1519 "<link linkend='use-devtool-to-integrate-new-code'>Use <filename>devtool add</filename> to Add an Application</link>"
1520 section.
1521 </para>
1522 </section>
1523
1524 <section id='new-recipe-creating-the-base-recipe-using-recipetool'>
1525 <title>Creating the Base Recipe Using <filename>recipetool create</filename></title>
1526
1527 <para>
1528 <filename>recipetool create</filename> automates creation
1529 of a base recipe given a set of source code files.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001530 As long as you can extract or point to the source files,
1531 the tool will construct a recipe and automatically
1532 configure all pre-build information into the recipe.
1533 For example, suppose you have an application that builds
1534 using Autotools.
1535 Creating the base recipe using
1536 <filename>recipetool</filename> results in a recipe
1537 that has the pre-build dependencies, license requirements,
1538 and checksums configured.
1539 </para>
1540
1541 <para>
1542 To run the tool, you just need to be in your
1543 <link linkend='build-directory'>Build Directory</link>
1544 and have sourced the build environment setup script
1545 (i.e.
1546 <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>oe-init-build-env</filename></ulink>
1547 or
1548 <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>).
1549 Here is the basic <filename>recipetool</filename> syntax:
1550 <note>
1551 Running <filename>recipetool -h</filename> or
1552 <filename>recipetool create -h</filename> produces the
1553 Python-generated help, which presented differently
1554 than what follows here.
1555 </note>
1556 <literallayout class='monospaced'>
1557 recipetool -h
1558 recipetool create [-h]
1559 recipetool [-d] [-q] [--color auto | always | never ] create -o <replaceable>OUTFILE</replaceable> [-m] [-x <replaceable>EXTERNALSRC</replaceable>] <replaceable>source</replaceable>
1560
1561 -d Enables debug output.
1562 -q Outputs only errors (quiet mode).
1563 --color Colorizes the output automatically, always, or never.
1564 -h Displays Python generated syntax for recipetool.
1565 create Causes recipetool to create a base recipe. The create
1566 command is further defined with these options:
1567
1568 -o <replaceable>OUTFILE</replaceable> Specifies the full path and filename for the generated
1569 recipe.
1570 -m Causes the recipe to be machine-specific rather than
1571 architecture-specific (default).
1572 -x <replaceable>EXTERNALSRC</replaceable> Fetches and extracts source files from <replaceable>source</replaceable>
1573 and places them in <replaceable>EXTERNALSRC</replaceable>.
1574 <replaceable>source</replaceable> must be a URL.
1575 -h Displays Python-generated syntax for create.
1576 <replaceable>source</replaceable> Specifies the source code on which to base the
1577 recipe.
1578 </literallayout>
1579 </para>
1580
1581 <para>
1582 Running <filename>recipetool create -o</filename>&nbsp;<replaceable>OUTFILE</replaceable>
1583 creates the base recipe and locates it properly in the
1584 layer that contains your source files.
1585 Following are some syntax examples:
1586 </para>
1587
1588 <para>
1589 Use this syntax to generate a recipe based on <replaceable>source</replaceable>.
1590 Once generated, the recipe resides in the existing source
1591 code layer:
1592 <literallayout class='monospaced'>
1593 recipetool create -o <replaceable>OUTFILE</replaceable>&nbsp;<replaceable>source</replaceable>
1594 </literallayout>
1595 Use this syntax to generate a recipe using code that you
1596 extract from <replaceable>source</replaceable>.
1597 The extracted code is placed in its own layer defined
1598 by <replaceable>EXTERNALSRC</replaceable>.
1599 <literallayout class='monospaced'>
1600 recipetool create -o <replaceable>OUTFILE</replaceable> -x <replaceable>EXTERNALSRC</replaceable> <replaceable>source</replaceable>
1601 </literallayout>
1602 Use this syntax to generate a recipe based on <replaceable>source</replaceable>.
1603 The options direct <filename>recipetool</filename> to
Brad Bishop37a0e4d2017-12-04 01:01:44 -05001604 generate debugging information.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001605 Once generated, the recipe resides in the existing source
1606 code layer:
1607 <literallayout class='monospaced'>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05001608 recipetool create -d -o <replaceable>OUTFILE</replaceable> <replaceable>source</replaceable>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001609 </literallayout>
1610 </para>
1611 </section>
1612
1613 <section id='new-recipe-locating-and-using-a-similar-recipe'>
1614 <title>Locating and Using a Similar Recipe</title>
1615
1616 <para>
1617 Before writing a recipe from scratch, it is often useful to
1618 discover whether someone else has already written one that
1619 meets (or comes close to meeting) your needs.
1620 The Yocto Project and OpenEmbedded communities maintain many
1621 recipes that might be candidates for what you are doing.
1622 You can find a good central index of these recipes in the
1623 <ulink url='http://layers.openembedded.org'>OpenEmbedded metadata index</ulink>.
1624 </para>
1625
1626 <para>
1627 Working from an existing recipe or a skeleton recipe is the
1628 best way to get started.
1629 Here are some points on both methods:
1630 <itemizedlist>
1631 <listitem><para><emphasis>Locate and modify a recipe that
1632 is close to what you want to do:</emphasis>
1633 This method works when you are familiar with the
1634 current recipe space.
1635 The method does not work so well for those new to
1636 the Yocto Project or writing recipes.</para>
1637 <para>Some risks associated with this method are
1638 using a recipe that has areas totally unrelated to
1639 what you are trying to accomplish with your recipe,
1640 not recognizing areas of the recipe that you might
1641 have to add from scratch, and so forth.
1642 All these risks stem from unfamiliarity with the
1643 existing recipe space.</para></listitem>
1644 <listitem><para><emphasis>Use and modify the following
1645 skeleton recipe:</emphasis>
1646 If for some reason you do not want to use
1647 <filename>recipetool</filename> and you cannot
1648 find an existing recipe that is close to meeting
1649 your needs, you can use the following structure to
1650 provide the fundamental areas of a new recipe.
1651 <literallayout class='monospaced'>
1652 DESCRIPTION = ""
1653 HOMEPAGE = ""
1654 LICENSE = ""
1655 SECTION = ""
1656 DEPENDS = ""
1657 LIC_FILES_CHKSUM = ""
1658
1659 SRC_URI = ""
1660 </literallayout>
1661 </para></listitem>
1662 </itemizedlist>
1663 </para>
1664 </section>
1665 </section>
1666
1667 <section id='new-recipe-storing-and-naming-the-recipe'>
1668 <title>Storing and Naming the Recipe</title>
1669
1670 <para>
1671 Once you have your base recipe, you should put it in your
1672 own layer and name it appropriately.
1673 Locating it correctly ensures that the OpenEmbedded build
1674 system can find it when you use BitBake to process the
1675 recipe.
1676 </para>
1677
1678 <itemizedlist>
1679 <listitem><para><emphasis>Storing Your Recipe:</emphasis>
1680 The OpenEmbedded build system locates your recipe
1681 through the layer's <filename>conf/layer.conf</filename>
1682 file and the
1683 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILES'><filename>BBFILES</filename></ulink>
1684 variable.
1685 This variable sets up a path from which the build system can
1686 locate recipes.
1687 Here is the typical use:
1688 <literallayout class='monospaced'>
1689 BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
1690 ${LAYERDIR}/recipes-*/*/*.bbappend"
1691 </literallayout>
1692 Consequently, you need to be sure you locate your new recipe
1693 inside your layer such that it can be found.</para>
1694 <para>You can find more information on how layers are
1695 structured in the
1696 "<link linkend='understanding-and-creating-layers'>Understanding and Creating Layers</link>"
1697 section.</para></listitem>
1698 <listitem><para><emphasis>Naming Your Recipe:</emphasis>
1699 When you name your recipe, you need to follow this naming
1700 convention:
1701 <literallayout class='monospaced'>
1702 <replaceable>basename</replaceable>_<replaceable>version</replaceable>.bb
1703 </literallayout>
1704 Use lower-cased characters and do not include the reserved
1705 suffixes <filename>-native</filename>,
1706 <filename>-cross</filename>, <filename>-initial</filename>,
1707 or <filename>-dev</filename> casually (i.e. do not use them
1708 as part of your recipe name unless the string applies).
1709 Here are some examples:
1710 <literallayout class='monospaced'>
1711 cups_1.7.0.bb
1712 gawk_4.0.2.bb
1713 irssi_0.8.16-rc1.bb
1714 </literallayout></para></listitem>
1715 </itemizedlist>
1716 </section>
1717
1718 <section id='understanding-recipe-syntax'>
1719 <title>Understanding Recipe Syntax</title>
1720
1721 <para>
1722 Understanding recipe file syntax is important for
1723 writing recipes.
1724 The following list overviews the basic items that make up a
1725 BitBake recipe file.
1726 For more complete BitBake syntax descriptions, see the
1727 "<ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata'>Syntax and Operators</ulink>"
1728 chapter of the BitBake User Manual.
1729 <itemizedlist>
1730 <listitem><para><emphasis>Variable Assignments and Manipulations:</emphasis>
1731 Variable assignments allow a value to be assigned to a
1732 variable.
1733 The assignment can be static text or might include
1734 the contents of other variables.
1735 In addition to the assignment, appending and prepending
1736 operations are also supported.</para>
1737 <para>The following example shows some of the ways
1738 you can use variables in recipes:
1739 <literallayout class='monospaced'>
1740 S = "${WORKDIR}/postfix-${PV}"
1741 CFLAGS += "-DNO_ASM"
1742 SRC_URI_append = " file://fixup.patch"
1743 </literallayout>
1744 </para></listitem>
1745 <listitem><para><emphasis>Functions:</emphasis>
1746 Functions provide a series of actions to be performed.
1747 You usually use functions to override the default
1748 implementation of a task function or to complement
1749 a default function (i.e. append or prepend to an
1750 existing function).
1751 Standard functions use <filename>sh</filename> shell
1752 syntax, although access to OpenEmbedded variables and
1753 internal methods are also available.</para>
1754 <para>The following is an example function from the
1755 <filename>sed</filename> recipe:
1756 <literallayout class='monospaced'>
1757 do_install () {
1758 autotools_do_install
1759 install -d ${D}${base_bindir}
1760 mv ${D}${bindir}/sed ${D}${base_bindir}/sed
1761 rmdir ${D}${bindir}/
1762 }
1763 </literallayout>
1764 It is also possible to implement new functions that
1765 are called between existing tasks as long as the
1766 new functions are not replacing or complementing the
1767 default functions.
1768 You can implement functions in Python
1769 instead of shell.
1770 Both of these options are not seen in the majority of
1771 recipes.</para></listitem>
1772 <listitem><para><emphasis>Keywords:</emphasis>
1773 BitBake recipes use only a few keywords.
1774 You use keywords to include common
1775 functions (<filename>inherit</filename>), load parts
1776 of a recipe from other files
1777 (<filename>include</filename> and
1778 <filename>require</filename>) and export variables
1779 to the environment (<filename>export</filename>).</para>
1780 <para>The following example shows the use of some of
1781 these keywords:
1782 <literallayout class='monospaced'>
1783 export POSTCONF = "${STAGING_BINDIR}/postconf"
1784 inherit autoconf
1785 require otherfile.inc
1786 </literallayout>
1787 </para></listitem>
1788 <listitem><para><emphasis>Comments:</emphasis>
1789 Any lines that begin with the hash character
1790 (<filename>#</filename>) are treated as comment lines
1791 and are ignored:
1792 <literallayout class='monospaced'>
1793 # This is a comment
1794 </literallayout>
1795 </para></listitem>
1796 </itemizedlist>
1797 </para>
1798
1799 <para>
1800 This next list summarizes the most important and most commonly
1801 used parts of the recipe syntax.
1802 For more information on these parts of the syntax, you can
1803 reference the
1804 <ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata'>Syntax and Operators</ulink>
1805 chapter in the BitBake User Manual.
1806 <itemizedlist>
1807 <listitem><para><emphasis>Line Continuation: <filename>\</filename></emphasis> -
1808 Use the backward slash (<filename>\</filename>)
1809 character to split a statement over multiple lines.
1810 Place the slash character at the end of the line that
1811 is to be continued on the next line:
1812 <literallayout class='monospaced'>
1813 VAR = "A really long \
1814 line"
1815 </literallayout>
1816 <note>
1817 You cannot have any characters including spaces
1818 or tabs after the slash character.
1819 </note>
1820 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001821 <listitem><para>
1822 <emphasis>Using Variables: <filename>${...}</filename></emphasis> -
1823 Use the <filename>${<replaceable>VARNAME</replaceable>}</filename> syntax to
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001824 access the contents of a variable:
1825 <literallayout class='monospaced'>
1826 SRC_URI = "${SOURCEFORGE_MIRROR}/libpng/zlib-${PV}.tar.gz"
1827 </literallayout>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06001828 <note>
1829 It is important to understand that the value of a
1830 variable expressed in this form does not get
1831 substituted automatically.
1832 The expansion of these expressions happens
1833 on-demand later (e.g. usually when a function that
1834 makes reference to the variable executes).
1835 This behavior ensures that the values are most
1836 appropriate for the context in which they are
1837 finally used.
1838 On the rare occasion that you do need the variable
1839 expression to be expanded immediately, you can use
1840 the <filename>:=</filename> operator instead of
1841 <filename>=</filename> when you make the
1842 assignment, but this is not generally needed.
1843 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001844 </para></listitem>
1845 <listitem><para><emphasis>Quote All Assignments: <filename>"<replaceable>value</replaceable>"</filename></emphasis> -
1846 Use double quotes around the value in all variable
1847 assignments.
1848 <literallayout class='monospaced'>
1849 VAR1 = "${OTHERVAR}"
1850 VAR2 = "The version is ${PV}"
1851 </literallayout>
1852 </para></listitem>
1853 <listitem><para><emphasis>Conditional Assignment: <filename>?=</filename></emphasis> -
1854 Conditional assignment is used to assign a value to
1855 a variable, but only when the variable is currently
1856 unset.
1857 Use the question mark followed by the equal sign
1858 (<filename>?=</filename>) to make a "soft" assignment
1859 used for conditional assignment.
1860 Typically, "soft" assignments are used in the
1861 <filename>local.conf</filename> file for variables
1862 that are allowed to come through from the external
1863 environment.
1864 </para>
1865 <para>Here is an example where
1866 <filename>VAR1</filename> is set to "New value" if
1867 it is currently empty.
1868 However, if <filename>VAR1</filename> has already been
1869 set, it remains unchanged:
1870 <literallayout class='monospaced'>
1871 VAR1 ?= "New value"
1872 </literallayout>
1873 In this next example, <filename>VAR1</filename>
1874 is left with the value "Original value":
1875 <literallayout class='monospaced'>
1876 VAR1 = "Original value"
1877 VAR1 ?= "New value"
1878 </literallayout>
1879 </para></listitem>
1880 <listitem><para><emphasis>Appending: <filename>+=</filename></emphasis> -
1881 Use the plus character followed by the equals sign
1882 (<filename>+=</filename>) to append values to existing
1883 variables.
1884 <note>
1885 This operator adds a space between the existing
1886 content of the variable and the new content.
1887 </note></para>
1888 <para>Here is an example:
1889 <literallayout class='monospaced'>
1890 SRC_URI += "file://fix-makefile.patch"
1891 </literallayout>
1892 </para></listitem>
1893 <listitem><para><emphasis>Prepending: <filename>=+</filename></emphasis> -
1894 Use the equals sign followed by the plus character
1895 (<filename>=+</filename>) to prepend values to existing
1896 variables.
1897 <note>
1898 This operator adds a space between the new content
1899 and the existing content of the variable.
1900 </note></para>
1901 <para>Here is an example:
1902 <literallayout class='monospaced'>
1903 VAR =+ "Starts"
1904 </literallayout>
1905 </para></listitem>
1906 <listitem><para><emphasis>Appending: <filename>_append</filename></emphasis> -
1907 Use the <filename>_append</filename> operator to
1908 append values to existing variables.
1909 This operator does not add any additional space.
1910 Also, the operator is applied after all the
1911 <filename>+=</filename>, and
1912 <filename>=+</filename> operators have been applied and
1913 after all <filename>=</filename> assignments have
1914 occurred.
1915 </para>
1916 <para>The following example shows the space being
1917 explicitly added to the start to ensure the appended
1918 value is not merged with the existing value:
1919 <literallayout class='monospaced'>
1920 SRC_URI_append = " file://fix-makefile.patch"
1921 </literallayout>
1922 You can also use the <filename>_append</filename>
1923 operator with overrides, which results in the actions
1924 only being performed for the specified target or
1925 machine:
1926 <literallayout class='monospaced'>
1927 SRC_URI_append_sh4 = " file://fix-makefile.patch"
1928 </literallayout>
1929 </para></listitem>
1930 <listitem><para><emphasis>Prepending: <filename>_prepend</filename></emphasis> -
1931 Use the <filename>_prepend</filename> operator to
1932 prepend values to existing variables.
1933 This operator does not add any additional space.
1934 Also, the operator is applied after all the
1935 <filename>+=</filename>, and
1936 <filename>=+</filename> operators have been applied and
1937 after all <filename>=</filename> assignments have
1938 occurred.
1939 </para>
1940 <para>The following example shows the space being
1941 explicitly added to the end to ensure the prepended
1942 value is not merged with the existing value:
1943 <literallayout class='monospaced'>
1944 CFLAGS_prepend = "-I${S}/myincludes "
1945 </literallayout>
1946 You can also use the <filename>_prepend</filename>
1947 operator with overrides, which results in the actions
1948 only being performed for the specified target or
1949 machine:
1950 <literallayout class='monospaced'>
1951 CFLAGS_prepend_sh4 = "-I${S}/myincludes "
1952 </literallayout>
1953 </para></listitem>
1954 <listitem><para><emphasis>Overrides:</emphasis> -
1955 You can use overrides to set a value conditionally,
1956 typically based on how the recipe is being built.
1957 For example, to set the
1958 <ulink url='&YOCTO_DOCS_REF_URL;#var-KBRANCH'><filename>KBRANCH</filename></ulink>
1959 variable's value to "standard/base" for any target
1960 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>,
1961 except for qemuarm where it should be set to
1962 "standard/arm-versatile-926ejs", you would do the
1963 following:
1964 <literallayout class='monospaced'>
1965 KBRANCH = "standard/base"
1966 KBRANCH_qemuarm = "standard/arm-versatile-926ejs"
1967 </literallayout>
1968 Overrides are also used to separate alternate values
1969 of a variable in other situations.
1970 For example, when setting variables such as
1971 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink>
1972 and
1973 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>
1974 that are specific to individual packages produced by
1975 a recipe, you should always use an override that
1976 specifies the name of the package.
1977 </para></listitem>
1978 <listitem><para><emphasis>Indentation:</emphasis>
1979 Use spaces for indentation rather than than tabs.
1980 For shell functions, both currently work.
1981 However, it is a policy decision of the Yocto Project
1982 to use tabs in shell functions.
1983 Realize that some layers have a policy to use spaces
1984 for all indentation.
1985 </para></listitem>
1986 <listitem><para><emphasis>Using Python for Complex Operations: <filename>${@<replaceable>python_code</replaceable>}</filename></emphasis> -
1987 For more advanced processing, it is possible to use
1988 Python code during variable assignments (e.g.
1989 search and replacement on a variable).</para>
1990 <para>You indicate Python code using the
1991 <filename>${@<replaceable>python_code</replaceable>}</filename>
1992 syntax for the variable assignment:
1993 <literallayout class='monospaced'>
1994 SRC_URI = "ftp://ftp.info-zip.org/pub/infozip/src/zip${@d.getVar('PV',1).replace('.', '')}.tgz
1995 </literallayout>
1996 </para></listitem>
1997 <listitem><para><emphasis>Shell Function Syntax:</emphasis>
1998 Write shell functions as if you were writing a shell
1999 script when you describe a list of actions to take.
2000 You should ensure that your script works with a generic
2001 <filename>sh</filename> and that it does not require
2002 any <filename>bash</filename> or other shell-specific
2003 functionality.
2004 The same considerations apply to various system
2005 utilities (e.g. <filename>sed</filename>,
2006 <filename>grep</filename>, <filename>awk</filename>,
2007 and so forth) that you might wish to use.
2008 If in doubt, you should check with multiple
2009 implementations - including those from BusyBox.
2010 </para></listitem>
2011 </itemizedlist>
2012 </para>
2013 </section>
2014
2015 <section id='new-recipe-running-a-build-on-the-recipe'>
2016 <title>Running a Build on the Recipe</title>
2017
2018 <para>
2019 Creating a new recipe is usually an iterative process that
2020 requires using BitBake to process the recipe multiple times in
2021 order to progressively discover and add information to the
2022 recipe file.
2023 </para>
2024
2025 <para>
2026 Assuming you have sourced a build environment setup script (i.e.
2027 <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
2028 or
2029 <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>)
2030 and you are in the
2031 <link linkend='build-directory'>Build Directory</link>,
2032 use BitBake to process your recipe.
2033 All you need to provide is the
2034 <filename><replaceable>basename</replaceable></filename> of the recipe as described
2035 in the previous section:
2036 <literallayout class='monospaced'>
2037 $ bitbake <replaceable>basename</replaceable>
2038 </literallayout>
2039
2040 </para>
2041
2042 <para>
2043 During the build, the OpenEmbedded build system creates a
2044 temporary work directory for each recipe
2045 (<filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}</filename>)
2046 where it keeps extracted source files, log files, intermediate
2047 compilation and packaging files, and so forth.
2048 </para>
2049
2050 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002051 The path to the per-recipe temporary work directory depends
2052 on the context in which it is being built.
2053 The quickest way to find this path is to have BitBake return it
2054 by running the following:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002055 <literallayout class='monospaced'>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002056 $ bitbake -e <replaceable>basename</replaceable> | grep ^WORKDIR=
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002057 </literallayout>
2058 As an example, assume a Source Directory top-level folder named
2059 <filename>poky</filename>, a default Build Directory at
2060 <filename>poky/build</filename>, and a
2061 <filename>qemux86-poky-linux</filename> machine target system.
2062 Furthermore, suppose your recipe is named
2063 <filename>foo_1.3.0.bb</filename>.
2064 In this case, the work directory the build system uses to
2065 build the package would be as follows:
2066 <literallayout class='monospaced'>
2067 poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0
2068 </literallayout>
2069 Inside this directory you can find sub-directories such as
2070 <filename>image</filename>, <filename>packages-split</filename>,
2071 and <filename>temp</filename>.
2072 After the build, you can examine these to determine how well
2073 the build went.
2074 <note>
2075 You can find log files for each task in the recipe's
2076 <filename>temp</filename> directory (e.g.
2077 <filename>poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0/temp</filename>).
2078 Log files are named <filename>log.<replaceable>taskname</replaceable></filename>
2079 (e.g. <filename>log.do_configure</filename>,
2080 <filename>log.do_fetch</filename>, and
2081 <filename>log.do_compile</filename>).
2082 </note>
2083 </para>
2084
2085 <para>
2086 You can find more information about the build process in the
2087 "<ulink url='&YOCTO_DOCS_REF_URL;#closer-look'>A Closer Look at the Yocto Project Development Environment</ulink>"
2088 chapter of the Yocto Project Reference Manual.
2089 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002090 </section>
2091
2092 <section id='new-recipe-fetching-code'>
2093 <title>Fetching Code</title>
2094
2095 <para>
2096 The first thing your recipe must do is specify how to fetch
2097 the source files.
2098 Fetching is controlled mainly through the
2099 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
2100 variable.
2101 Your recipe must have a <filename>SRC_URI</filename> variable
2102 that points to where the source is located.
2103 For a graphical representation of source locations, see the
2104 "<ulink url='&YOCTO_DOCS_REF_URL;#sources-dev-environment'>Sources</ulink>"
2105 section in the Yocto Project Reference Manual.
2106 </para>
2107
2108 <para>
2109 The
2110 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-fetch'><filename>do_fetch</filename></ulink>
2111 task uses the prefix of each entry in the
2112 <filename>SRC_URI</filename> variable value to determine which
2113 fetcher to use to get your source files.
2114 It is the <filename>SRC_URI</filename> variable that triggers
2115 the fetcher.
2116 The
2117 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink>
2118 task uses the variable after source is fetched to apply
2119 patches.
2120 The OpenEmbedded build system uses
2121 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESOVERRIDES'><filename>FILESOVERRIDES</filename></ulink>
2122 for scanning directory locations for local files in
2123 <filename>SRC_URI</filename>.
2124 </para>
2125
2126 <para>
2127 The <filename>SRC_URI</filename> variable in your recipe must
2128 define each unique location for your source files.
2129 It is good practice to not hard-code pathnames in an URL used
2130 in <filename>SRC_URI</filename>.
2131 Rather than hard-code these paths, use
2132 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink><filename>}</filename>,
2133 which causes the fetch process to use the version specified in
2134 the recipe filename.
2135 Specifying the version in this manner means that upgrading the
2136 recipe to a future version is as simple as renaming the recipe
2137 to match the new version.
2138 </para>
2139
2140 <para>
2141 Here is a simple example from the
2142 <filename>meta/recipes-devtools/cdrtools/cdrtools-native_3.01a20.bb</filename>
2143 recipe where the source comes from a single tarball.
2144 Notice the use of the
2145 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>
2146 variable:
2147 <literallayout class='monospaced'>
2148 SRC_URI = "ftp://ftp.berlios.de/pub/cdrecord/alpha/cdrtools-${PV}.tar.bz2"
2149 </literallayout>
2150 </para>
2151
2152 <para>
2153 Files mentioned in <filename>SRC_URI</filename> whose names end
2154 in a typical archive extension (e.g. <filename>.tar</filename>,
2155 <filename>.tar.gz</filename>, <filename>.tar.bz2</filename>,
2156 <filename>.zip</filename>, and so forth), are automatically
2157 extracted during the
2158 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-unpack'><filename>do_unpack</filename></ulink>
2159 task.
2160 For another example that specifies these types of files, see
2161 the
2162 "<link linkend='new-recipe-autotooled-package'>Autotooled Package</link>"
2163 section.
2164 </para>
2165
2166 <para>
2167 Another way of specifying source is from an SCM.
2168 For Git repositories, you must specify
2169 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink>
2170 and you should specify
2171 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>
2172 to include the revision with
2173 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCPV'><filename>SRCPV</filename></ulink>.
2174 Here is an example from the recipe
2175 <filename>meta/recipes-kernel/blktrace/blktrace_git.bb</filename>:
2176 <literallayout class='monospaced'>
2177 SRCREV = "d6918c8832793b4205ed3bfede78c2f915c23385"
2178
2179 PR = "r6"
2180 PV = "1.0.5+git${SRCPV}"
2181
2182 SRC_URI = "git://git.kernel.dk/blktrace.git \
2183 file://ldflags.patch"
2184 </literallayout>
2185 </para>
2186
2187 <para>
2188 If your <filename>SRC_URI</filename> statement includes
2189 URLs pointing to individual files fetched from a remote server
2190 other than a version control system, BitBake attempts to
2191 verify the files against checksums defined in your recipe to
2192 ensure they have not been tampered with or otherwise modified
2193 since the recipe was written.
2194 Two checksums are used:
2195 <filename>SRC_URI[md5sum]</filename> and
2196 <filename>SRC_URI[sha256sum]</filename>.
2197 </para>
2198
2199 <para>
2200 If your <filename>SRC_URI</filename> variable points to
2201 more than a single URL (excluding SCM URLs), you need to
2202 provide the <filename>md5</filename> and
2203 <filename>sha256</filename> checksums for each URL.
2204 For these cases, you provide a name for each URL as part of
2205 the <filename>SRC_URI</filename> and then reference that name
2206 in the subsequent checksum statements.
2207 Here is an example:
2208 <literallayout class='monospaced'>
2209 SRC_URI = "${DEBIAN_MIRROR}/main/a/apmd/apmd_3.2.2.orig.tar.gz;name=tarball \
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002210 ${DEBIAN_MIRROR}/main/a/apmd/apmd_${PV}.diff.gz;name=patch"
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002211
2212 SRC_URI[tarball.md5sum] = "b1e6309e8331e0f4e6efd311c2d97fa8"
2213 SRC_URI[tarball.sha256sum] = "7f7d9f60b7766b852881d40b8ff91d8e39fccb0d1d913102a5c75a2dbb52332d"
2214
2215 SRC_URI[patch.md5sum] = "57e1b689264ea80f78353519eece0c92"
2216 SRC_URI[patch.sha256sum] = "7905ff96be93d725544d0040e425c42f9c05580db3c272f11cff75b9aa89d430"
2217 </literallayout>
2218 </para>
2219
2220 <para>
2221 Proper values for <filename>md5</filename> and
2222 <filename>sha256</filename> checksums might be available
2223 with other signatures on the download page for the upstream
2224 source (e.g. <filename>md5</filename>,
2225 <filename>sha1</filename>, <filename>sha256</filename>,
2226 <filename>GPG</filename>, and so forth).
2227 Because the OpenEmbedded build system only deals with
2228 <filename>sha256sum</filename> and <filename>md5sum</filename>,
2229 you should verify all the signatures you find by hand.
2230 </para>
2231
2232 <para>
2233 If no <filename>SRC_URI</filename> checksums are specified
2234 when you attempt to build the recipe, or you provide an
2235 incorrect checksum, the build will produce an error for each
2236 missing or incorrect checksum.
2237 As part of the error message, the build system provides
2238 the checksum string corresponding to the fetched file.
2239 Once you have the correct checksums, you can copy and paste
2240 them into your recipe and then run the build again to continue.
2241 <note>
2242 As mentioned, if the upstream source provides signatures
2243 for verifying the downloaded source code, you should
2244 verify those manually before setting the checksum values
2245 in the recipe and continuing with the build.
2246 </note>
2247 </para>
2248
2249 <para>
2250 This final example is a bit more complicated and is from the
2251 <filename>meta/recipes-sato/rxvt-unicode/rxvt-unicode_9.20.bb</filename>
2252 recipe.
2253 The example's <filename>SRC_URI</filename> statement identifies
2254 multiple files as the source files for the recipe: a tarball, a
2255 patch file, a desktop file, and an icon.
2256 <literallayout class='monospaced'>
2257 SRC_URI = "http://dist.schmorp.de/rxvt-unicode/Attic/rxvt-unicode-${PV}.tar.bz2 \
2258 file://xwc.patch \
2259 file://rxvt.desktop \
2260 file://rxvt.png"
2261 </literallayout>
2262 </para>
2263
2264 <para>
2265 When you specify local files using the
2266 <filename>file://</filename> URI protocol, the build system
2267 fetches files from the local machine.
2268 The path is relative to the
2269 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink>
2270 variable and searches specific directories in a certain order:
2271 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BP'><filename>BP</filename></ulink><filename>}</filename>,
2272 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BPN'><filename>BPN</filename></ulink><filename>}</filename>,
2273 and <filename>files</filename>.
2274 The directories are assumed to be subdirectories of the
2275 directory in which the recipe or append file resides.
2276 For another example that specifies these types of files, see the
2277 "<link linkend='new-recipe-single-c-file-package-hello-world'>Single .c File Package (Hello World!)</link>"
2278 section.
2279 </para>
2280
2281 <para>
2282 The previous example also specifies a patch file.
2283 Patch files are files whose names usually end in
2284 <filename>.patch</filename> or <filename>.diff</filename> but
2285 can end with compressed suffixes such as
2286 <filename>diff.gz</filename> and
2287 <filename>patch.bz2</filename>, for example.
2288 The build system automatically applies patches as described
2289 in the
2290 "<link linkend='new-recipe-patching-code'>Patching Code</link>" section.
2291 </para>
2292 </section>
2293
2294 <section id='new-recipe-unpacking-code'>
2295 <title>Unpacking Code</title>
2296
2297 <para>
2298 During the build, the
2299 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-unpack'><filename>do_unpack</filename></ulink>
2300 task unpacks the source with
2301 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink><filename>}</filename>
2302 pointing to where it is unpacked.
2303 </para>
2304
2305 <para>
2306 If you are fetching your source files from an upstream source
2307 archived tarball and the tarball's internal structure matches
2308 the common convention of a top-level subdirectory named
2309 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BPN'><filename>BPN</filename></ulink><filename>}-${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink><filename>}</filename>,
2310 then you do not need to set <filename>S</filename>.
2311 However, if <filename>SRC_URI</filename> specifies to fetch
2312 source from an archive that does not use this convention,
2313 or from an SCM like Git or Subversion, your recipe needs to
2314 define <filename>S</filename>.
2315 </para>
2316
2317 <para>
2318 If processing your recipe using BitBake successfully unpacks
2319 the source files, you need to be sure that the directory
2320 pointed to by <filename>${S}</filename> matches the structure
2321 of the source.
2322 </para>
2323 </section>
2324
2325 <section id='new-recipe-patching-code'>
2326 <title>Patching Code</title>
2327
2328 <para>
2329 Sometimes it is necessary to patch code after it has been
2330 fetched.
2331 Any files mentioned in <filename>SRC_URI</filename> whose
2332 names end in <filename>.patch</filename> or
2333 <filename>.diff</filename> or compressed versions of these
2334 suffixes (e.g. <filename>diff.gz</filename> are treated as
2335 patches.
2336 The
2337 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink>
2338 task automatically applies these patches.
2339 </para>
2340
2341 <para>
2342 The build system should be able to apply patches with the "-p1"
2343 option (i.e. one directory level in the path will be stripped
2344 off).
2345 If your patch needs to have more directory levels stripped off,
2346 specify the number of levels using the "striplevel" option in
2347 the <filename>SRC_URI</filename> entry for the patch.
2348 Alternatively, if your patch needs to be applied in a specific
2349 subdirectory that is not specified in the patch file, use the
2350 "patchdir" option in the entry.
2351 </para>
2352
2353 <para>
2354 As with all local files referenced in
2355 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
2356 using <filename>file://</filename>, you should place
2357 patch files in a directory next to the recipe either
2358 named the same as the base name of the recipe
2359 (<ulink url='&YOCTO_DOCS_REF_URL;#var-BP'><filename>BP</filename></ulink>
2360 and
2361 <ulink url='&YOCTO_DOCS_REF_URL;#var-BPN'><filename>BPN</filename></ulink>)
2362 or "files".
2363 </para>
2364 </section>
2365
2366 <section id='new-recipe-licensing'>
2367 <title>Licensing</title>
2368
2369 <para>
2370 Your recipe needs to have both the
2371 <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE'><filename>LICENSE</filename></ulink>
2372 and
2373 <ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'><filename>LIC_FILES_CHKSUM</filename></ulink>
2374 variables:
2375 <itemizedlist>
2376 <listitem><para><emphasis><filename>LICENSE</filename>:</emphasis>
2377 This variable specifies the license for the software.
2378 If you do not know the license under which the software
2379 you are building is distributed, you should go to the
2380 source code and look for that information.
2381 Typical files containing this information include
2382 <filename>COPYING</filename>,
2383 <filename>LICENSE</filename>, and
2384 <filename>README</filename> files.
2385 You could also find the information near the top of
2386 a source file.
2387 For example, given a piece of software licensed under
2388 the GNU General Public License version 2, you would
2389 set <filename>LICENSE</filename> as follows:
2390 <literallayout class='monospaced'>
2391 LICENSE = "GPLv2"
2392 </literallayout></para>
2393 <para>The licenses you specify within
2394 <filename>LICENSE</filename> can have any name as long
2395 as you do not use spaces, since spaces are used as
2396 separators between license names.
2397 For standard licenses, use the names of the files in
2398 <filename>meta/files/common-licenses/</filename>
2399 or the <filename>SPDXLICENSEMAP</filename> flag names
2400 defined in <filename>meta/conf/licenses.conf</filename>.
2401 </para></listitem>
2402 <listitem><para><emphasis><filename>LIC_FILES_CHKSUM</filename>:</emphasis>
2403 The OpenEmbedded build system uses this variable to
2404 make sure the license text has not changed.
2405 If it has, the build produces an error and it affords
2406 you the chance to figure it out and correct the problem.
2407 </para>
2408 <para>You need to specify all applicable licensing
2409 files for the software.
2410 At the end of the configuration step, the build process
2411 will compare the checksums of the files to be sure
2412 the text has not changed.
2413 Any differences result in an error with the message
2414 containing the current checksum.
2415 For more explanation and examples of how to set the
2416 <filename>LIC_FILES_CHKSUM</filename> variable, see the
2417 "<ulink url='&YOCTO_DOCS_REF_URL;#usingpoky-configuring-LIC_FILES_CHKSUM'>Tracking License Changes</ulink>"
2418 section in the Yocto Project Reference Manual.</para>
2419 <para>To determine the correct checksum string, you
2420 can list the appropriate files in the
2421 <filename>LIC_FILES_CHKSUM</filename> variable with
2422 incorrect md5 strings, attempt to build the software,
2423 and then note the resulting error messages that will
2424 report the correct md5 strings.
2425 See the
2426 "<link linkend='new-recipe-fetching-code'>Fetching Code</link>"
2427 section for additional information.
2428 </para>
2429
2430 <para>
2431 Here is an example that assumes the software has a
2432 <filename>COPYING</filename> file:
2433 <literallayout class='monospaced'>
2434 LIC_FILES_CHKSUM = "file://COPYING;md5=xxx"
2435 </literallayout>
2436 When you try to build the software, the build system
2437 will produce an error and give you the correct string
2438 that you can substitute into the recipe file for a
2439 subsequent build.
2440 </para></listitem>
2441 </itemizedlist>
2442 </para>
2443
2444<!--
2445
2446 <para>
2447 For trying this out I created a new recipe named
2448 <filename>htop_1.0.2.bb</filename> and put it in
2449 <filename>poky/meta/recipes-extended/htop</filename>.
2450 There are two license type statements in my very simple
2451 recipe:
2452 <literallayout class='monospaced'>
2453 LICENSE = ""
2454
2455 LIC_FILES_CHKSUM = ""
2456
2457 SRC_URI[md5sum] = ""
2458 SRC_URI[sha256sum] = ""
2459 </literallayout>
2460 Evidently, you need to run a <filename>bitbake -c cleanall htop</filename>.
2461 Next, you delete or comment out the two <filename>SRC_URI</filename>
2462 lines at the end and then attempt to build the software with
2463 <filename>bitbake htop</filename>.
2464 Doing so causes BitBake to report some errors and and give
2465 you the actual strings you need for the last two
2466 <filename>SRC_URI</filename> lines.
2467 Prior to this, you have to dig around in the home page of the
2468 source for <filename>htop</filename> and determine that the
2469 software is released under GPLv2.
2470 You can provide that in the <filename>LICENSE</filename>
2471 statement.
2472 Now you edit your recipe to have those two strings for
2473 the <filename>SRC_URI</filename> statements:
2474 <literallayout class='monospaced'>
2475 LICENSE = "GPLv2"
2476
2477 LIC_FILES_CHKSUM = ""
2478
2479 SRC_URI = "${SOURCEFORGE_MIRROR}/htop/htop-${PV}.tar.gz"
2480 SRC_URI[md5sum] = "0d01cca8df3349c74569cefebbd9919e"
2481 SRC_URI[sha256sum] = "ee60657b044ece0df096c053060df7abf3cce3a568ab34d260049e6a37ccd8a1"
2482 </literallayout>
2483 At this point, you can build the software again using the
2484 <filename>bitbake htop</filename> command.
2485 There is just a set of errors now associated with the
2486 empty <filename>LIC_FILES_CHKSUM</filename> variable now.
2487 </para>
2488-->
2489
2490 </section>
2491
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002492 <section id='new-dependencies'>
2493 <title>Dependencies</title>
2494
2495 <para>
2496 Most software packages have a short list of other packages
2497 that they require, which are called dependencies.
2498 These dependencies fall into two main categories: build-time
2499 dependencies, which are required when the software is built;
2500 and runtime dependencies, which are required to be installed
2501 on the target in order for the software to run.
2502 </para>
2503
2504 <para>
2505 Within a recipe, you specify build-time dependencies using the
2506 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
2507 variable.
2508 Although nuances exist, items specified in
2509 <filename>DEPENDS</filename> should be names of other recipes.
2510 It is important that you specify all build-time dependencies
2511 explicitly.
2512 If you do not, due to the parallel nature of BitBake's
2513 execution, you can end up with a race condition where the
2514 dependency is present for one task of a recipe (e.g.
2515 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink>)
2516 and then gone when the next task runs (e.g.
2517 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink>).
2518 </para>
2519
2520 <para>
2521 Another consideration is that configure scripts might
2522 automatically check for optional dependencies and enable
2523 corresponding functionality if those dependencies are found.
2524 This behavior means that to ensure deterministic results and
2525 thus avoid more race conditions, you need to either explicitly
2526 specify these dependencies as well, or tell the configure
2527 script explicitly to disable the functionality.
2528 If you wish to make a recipe that is more generally useful
2529 (e.g. publish the recipe in a layer for others to use),
2530 instead of hard-disabling the functionality, you can use the
2531 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG'><filename>PACKAGECONFIG</filename></ulink>
2532 variable to allow functionality and the corresponding
2533 dependencies to be enabled and disabled easily by other
2534 users of the recipe.
2535 </para>
2536
2537 <para>
2538 Similar to build-time dependencies, you specify runtime
2539 dependencies through a variable -
2540 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>,
2541 which is package-specific.
2542 All variables that are package-specific need to have the name
2543 of the package added to the end as an override.
2544 Since the main package for a recipe has the same name as the
2545 recipe, and the recipe's name can be found through the
2546 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename>
2547 variable, then you specify the dependencies for the main
2548 package by setting <filename>RDEPENDS_${PN}</filename>.
2549 If the package were named <filename>${PN}-tools</filename>,
2550 then you would set <filename>RDEPENDS_${PN}-tools</filename>,
2551 and so forth.
2552 </para>
2553
2554 <para>
2555 Some runtime dependencies will be set automatically at
2556 packaging time.
2557 These dependencies include any shared library dependencies
2558 (i.e. if a package "example" contains "libexample" and
2559 another package "mypackage" contains a binary that links to
2560 "libexample" then the OpenEmbedded build system will
2561 automatically add a runtime dependency to "mypackage" on
2562 "example").
2563 See the
2564 "<ulink url='&YOCTO_DOCS_REF_URL;#automatically-added-runtime-dependencies'>Automatically Added Runtime Dependencies</ulink>"
2565 in the Yocto Project Reference Manual for further details.
2566 </para>
2567 </section>
2568
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002569 <section id='new-recipe-configuring-the-recipe'>
2570 <title>Configuring the Recipe</title>
2571
2572 <para>
2573 Most software provides some means of setting build-time
2574 configuration options before compilation.
2575 Typically, setting these options is accomplished by running a
2576 configure script with some options, or by modifying a build
2577 configuration file.
Patrick Williamsf1e5d692016-03-30 15:21:19 -05002578 <note>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002579 As of Yocto Project Release 1.7, some of the core recipes
Patrick Williamsf1e5d692016-03-30 15:21:19 -05002580 that package binary configuration scripts now disable the
2581 scripts due to the scripts previously requiring error-prone
2582 path substitution.
2583 The OpenEmbedded build system uses
2584 <filename>pkg-config</filename> now, which is much more
2585 robust.
2586 You can find a list of the <filename>*-config</filename>
2587 scripts that are disabled list in the
2588 "<ulink url='&YOCTO_DOCS_REF_URL;#migration-1.7-binary-configuration-scripts-disabled'>Binary Configuration Scripts Disabled</ulink>"
2589 section in the Yocto Project Reference Manual.
2590 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002591 </para>
2592
2593 <para>
2594 A major part of build-time configuration is about checking for
2595 build-time dependencies and possibly enabling optional
2596 functionality as a result.
2597 You need to specify any build-time dependencies for the
2598 software you are building in your recipe's
2599 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
2600 value, in terms of other recipes that satisfy those
2601 dependencies.
2602 You can often find build-time or runtime
2603 dependencies described in the software's documentation.
2604 </para>
2605
2606 <para>
2607 The following list provides configuration items of note based
2608 on how your software is built:
2609 <itemizedlist>
2610 <listitem><para><emphasis>Autotools:</emphasis>
2611 If your source files have a
2612 <filename>configure.ac</filename> file, then your
2613 software is built using Autotools.
2614 If this is the case, you just need to worry about
2615 modifying the configuration.</para>
2616 <para>When using Autotools, your recipe needs to inherit
2617 the
2618 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-autotools'><filename>autotools</filename></ulink>
2619 class and your recipe does not have to contain a
2620 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink>
2621 task.
2622 However, you might still want to make some adjustments.
2623 For example, you can set
2624 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OECONF'><filename>EXTRA_OECONF</filename></ulink>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06002625 or
2626 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG_CONFARGS'><filename>PACKAGECONFIG_CONFARGS</filename></ulink>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002627 to pass any needed configure options that are specific
2628 to the recipe.</para></listitem>
2629 <listitem><para><emphasis>CMake:</emphasis>
2630 If your source files have a
2631 <filename>CMakeLists.txt</filename> file, then your
2632 software is built using CMake.
2633 If this is the case, you just need to worry about
2634 modifying the configuration.</para>
2635 <para>When you use CMake, your recipe needs to inherit
2636 the
2637 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-cmake'><filename>cmake</filename></ulink>
2638 class and your recipe does not have to contain a
2639 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink>
2640 task.
2641 You can make some adjustments by setting
2642 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OECMAKE'><filename>EXTRA_OECMAKE</filename></ulink>
2643 to pass any needed configure options that are specific
2644 to the recipe.</para></listitem>
2645 <listitem><para><emphasis>Other:</emphasis>
2646 If your source files do not have a
2647 <filename>configure.ac</filename> or
2648 <filename>CMakeLists.txt</filename> file, then your
2649 software is built using some method other than Autotools
2650 or CMake.
2651 If this is the case, you normally need to provide a
2652 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink>
2653 task in your recipe
2654 unless, of course, there is nothing to configure.
2655 </para>
2656 <para>Even if your software is not being built by
2657 Autotools or CMake, you still might not need to deal
2658 with any configuration issues.
2659 You need to determine if configuration is even a required step.
2660 You might need to modify a Makefile or some configuration file
2661 used for the build to specify necessary build options.
2662 Or, perhaps you might need to run a provided, custom
2663 configure script with the appropriate options.</para>
2664 <para>For the case involving a custom configure
2665 script, you would run
2666 <filename>./configure --help</filename> and look for
2667 the options you need to set.</para></listitem>
2668 </itemizedlist>
2669 </para>
2670
2671 <para>
2672 Once configuration succeeds, it is always good practice to
2673 look at the <filename>log.do_configure</filename> file to
2674 ensure that the appropriate options have been enabled and no
2675 additional build-time dependencies need to be added to
2676 <filename>DEPENDS</filename>.
2677 For example, if the configure script reports that it found
2678 something not mentioned in <filename>DEPENDS</filename>, or
2679 that it did not find something that it needed for some
2680 desired optional functionality, then you would need to add
2681 those to <filename>DEPENDS</filename>.
2682 Looking at the log might also reveal items being checked for,
2683 enabled, or both that you do not want, or items not being found
2684 that are in <filename>DEPENDS</filename>, in which case
2685 you would need to look at passing extra options to the
2686 configure script as needed.
2687 For reference information on configure options specific to the
2688 software you are building, you can consult the output of the
2689 <filename>./configure --help</filename> command within
2690 <filename>${S}</filename> or consult the software's upstream
2691 documentation.
2692 </para>
2693 </section>
2694
Brad Bishop6e60e8b2018-02-01 10:27:11 -05002695 <section id='new-recipe-using-headers-to-interface-with-devices'>
2696 <title>Using Headers to Interface with Devices</title>
2697
2698 <para>
2699 If your recipe builds an application that needs to
2700 communicate with some device or needs an API into a custom
2701 kernel, you will need to provide appropriate header files.
2702 Under no circumstances should you ever modify the existing
2703 <filename>meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc</filename>
2704 file.
2705 These headers are used to build <filename>libc</filename> and
2706 must not be compromised with custom or machine-specific
2707 header information.
2708 If you customize <filename>libc</filename> through modified
2709 headers all other applications that use
2710 <filename>libc</filename> thus become affected.
2711 <note><title>Warning</title>
2712 Never copy and customize the <filename>libc</filename>
2713 header file (i.e.
2714 <filename>meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc</filename>).
2715 </note>
2716 The correct way to interface to a device or custom kernel is
2717 to use a separate package that provides the additional headers
2718 for the driver or other unique interfaces.
2719 When doing so, your application also becomes responsible for
2720 creating a dependency on that specific provider.
2721 </para>
2722
2723 <para>
2724 Consider the following:
2725 <itemizedlist>
2726 <listitem><para>
2727 Never modify
2728 <filename>linux-libc-headers.inc</filename>.
2729 Consider that file to be part of the
2730 <filename>libc</filename> system, and not something
2731 you use to access the kernel directly.
2732 You should access <filename>libc</filename> through
2733 specific <filename>libc</filename> calls.
2734 </para></listitem>
2735 <listitem><para>
2736 Applications that must talk directly to devices
2737 should either provide necessary headers themselves,
2738 or establish a dependency on a special headers package
2739 that is specific to that driver.
2740 </para></listitem>
2741 </itemizedlist>
2742 </para>
2743
2744 <para>
2745 For example, suppose you want to modify an existing header
2746 that adds I/O control or network support.
2747 If the modifications are used by a small number programs,
2748 providing a unique version of a header is easy and has little
2749 impact.
2750 When doing so, bear in mind the guidelines in the previous
2751 list.
2752 <note>
2753 If for some reason your changes need to modify the behavior
2754 of the <filename>libc</filename>, and subsequently all
2755 other applications on the system, use a
2756 <filename>.bbappend</filename> to modify the
2757 <filename>linux-kernel-headers.inc</filename> file.
2758 However, take care to not make the changes
2759 machine specific.
2760 </note>
2761 </para>
2762
2763 <para>
2764 Consider a case where your kernel is older and you need
2765 an older <filename>libc</filename> ABI.
2766 The headers installed by your recipe should still be a
2767 standard mainline kernel, not your own custom one.
2768 </para>
2769
2770 <para>
2771 When you use custom kernel headers you need to get them from
2772 <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_KERNEL_DIR'><filename>STAGING_KERNEL_DIR</filename></ulink>,
2773 which is the directory with kernel headers that are
2774 required to build out-of-tree modules.
2775 Your recipe will also need the following:
2776 <literallayout class='monospaced'>
2777 do_configure[depends] += "virtual/kernel:do_shared_workdir"
2778 </literallayout>
2779 </para>
2780 </section>
2781
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002782 <section id='new-recipe-compilation'>
2783 <title>Compilation</title>
2784
2785 <para>
2786 During a build, the <filename>do_compile</filename> task
2787 happens after source is fetched, unpacked, and configured.
2788 If the recipe passes through <filename>do_compile</filename>
2789 successfully, nothing needs to be done.
2790 </para>
2791
2792 <para>
2793 However, if the compile step fails, you need to diagnose the
2794 failure.
Patrick Williamsf1e5d692016-03-30 15:21:19 -05002795 Here are some common issues that cause failures.
2796 <note>
2797 For cases where improper paths are detected for
2798 configuration files or for when libraries/headers cannot
2799 be found, be sure you are using the more robust
2800 <filename>pkg-config</filename>.
2801 See the note in section
2802 "<link linkend='new-recipe-configuring-the-recipe'>Configuring the Recipe</link>"
2803 for additional information.
2804 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05002805 <itemizedlist>
2806 <listitem><para><emphasis>Parallel build failures:</emphasis>
2807 These failures manifest themselves as intermittent
2808 errors, or errors reporting that a file or directory
2809 that should be created by some other part of the build
2810 process could not be found.
2811 This type of failure can occur even if, upon inspection,
2812 the file or directory does exist after the build has
2813 failed, because that part of the build process happened
2814 in the wrong order.</para>
2815 <para>To fix the problem, you need to either satisfy
2816 the missing dependency in the Makefile or whatever
2817 script produced the Makefile, or (as a workaround)
2818 set
2819 <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></ulink>
2820 to an empty string:
2821 <literallayout class='monospaced'>
2822 PARALLEL_MAKE = ""
2823 </literallayout></para>
2824 <para>
2825 For information on parallel Makefile issues, see the
2826 "<link linkend='debugging-parallel-make-races'>Debugging Parallel Make Races</link>"
2827 section.
2828 </para></listitem>
2829 <listitem><para><emphasis>Improper host path usage:</emphasis>
2830 This failure applies to recipes building for the target
2831 or <filename>nativesdk</filename> only.
2832 The failure occurs when the compilation process uses
2833 improper headers, libraries, or other files from the
2834 host system when cross-compiling for the target.
2835 </para>
2836 <para>To fix the problem, examine the
2837 <filename>log.do_compile</filename> file to identify
2838 the host paths being used (e.g.
2839 <filename>/usr/include</filename>,
2840 <filename>/usr/lib</filename>, and so forth) and then
2841 either add configure options, apply a patch, or do both.
2842 </para></listitem>
2843 <listitem><para><emphasis>Failure to find required
2844 libraries/headers:</emphasis>
2845 If a build-time dependency is missing because it has
2846 not been declared in
2847 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>,
2848 or because the dependency exists but the path used by
2849 the build process to find the file is incorrect and the
2850 configure step did not detect it, the compilation
2851 process could fail.
2852 For either of these failures, the compilation process
2853 notes that files could not be found.
2854 In these cases, you need to go back and add additional
2855 options to the configure script as well as possibly
2856 add additional build-time dependencies to
2857 <filename>DEPENDS</filename>.</para>
2858 <para>Occasionally, it is necessary to apply a patch
2859 to the source to ensure the correct paths are used.
2860 If you need to specify paths to find files staged
2861 into the sysroot from other recipes, use the variables
2862 that the OpenEmbedded build system provides
2863 (e.g.
2864 <filename>STAGING_BINDIR</filename>,
2865 <filename>STAGING_INCDIR</filename>,
2866 <filename>STAGING_DATADIR</filename>, and so forth).
2867<!--
2868 (e.g.
2869 <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_BINDIR'><filename>STAGING_BINDIR</filename></ulink>,
2870 <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_INCDIR'><filename>STAGING_INCDIR</filename></ulink>,
2871 <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_DATADIR'><filename>STAGING_DATADIR</filename></ulink>,
2872 and so forth).
2873-->
2874 </para></listitem>
2875 </itemizedlist>
2876 </para>
2877 </section>
2878
2879 <section id='new-recipe-installing'>
2880 <title>Installing</title>
2881
2882 <para>
2883 During <filename>do_install</filename>, the task copies the
2884 built files along with their hierarchy to locations that
2885 would mirror their locations on the target device.
2886 The installation process copies files from the
2887 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink><filename>}</filename>,
2888 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-B'><filename>B</filename></ulink><filename>}</filename>,
2889 and
2890 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}</filename>
2891 directories to the
2892 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink><filename>}</filename>
2893 directory to create the structure as it should appear on the
2894 target system.
2895 </para>
2896
2897 <para>
2898 How your software is built affects what you must do to be
2899 sure your software is installed correctly.
2900 The following list describes what you must do for installation
2901 depending on the type of build system used by the software
2902 being built:
2903 <itemizedlist>
2904 <listitem><para><emphasis>Autotools and CMake:</emphasis>
2905 If the software your recipe is building uses Autotools
2906 or CMake, the OpenEmbedded build
2907 system understands how to install the software.
2908 Consequently, you do not have to have a
2909 <filename>do_install</filename> task as part of your
2910 recipe.
2911 You just need to make sure the install portion of the
2912 build completes with no issues.
2913 However, if you wish to install additional files not
2914 already being installed by
2915 <filename>make install</filename>, you should do this
2916 using a <filename>do_install_append</filename> function
2917 using the install command as described in
2918 the "Manual" bulleted item later in this list.
2919 </para></listitem>
2920 <listitem><para><emphasis>Other (using
2921 <filename>make install</filename>):</emphasis>
2922 You need to define a
2923 <filename>do_install</filename> function in your
2924 recipe.
2925 The function should call
2926 <filename>oe_runmake install</filename> and will likely
2927 need to pass in the destination directory as well.
2928 How you pass that path is dependent on how the
2929 <filename>Makefile</filename> being run is written
2930 (e.g. <filename>DESTDIR=${D}</filename>,
2931 <filename>PREFIX=${D}</filename>,
2932 <filename>INSTALLROOT=${D}</filename>, and so forth).
2933 </para>
2934 <para>For an example recipe using
2935 <filename>make install</filename>, see the
2936 "<link linkend='new-recipe-makefile-based-package'>Makefile-Based Package</link>"
2937 section.</para></listitem>
2938 <listitem><para><emphasis>Manual:</emphasis>
2939 You need to define a
2940 <filename>do_install</filename> function in your
2941 recipe.
2942 The function must first use
2943 <filename>install -d</filename> to create the
2944 directories under
2945 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink><filename>}</filename>.
2946 Once the directories exist, your function can use
2947 <filename>install</filename> to manually install the
2948 built software into the directories.</para>
2949 <para>You can find more information on
2950 <filename>install</filename> at
2951 <ulink url='http://www.gnu.org/software/coreutils/manual/html_node/install-invocation.html'></ulink>.
2952 </para></listitem>
2953 </itemizedlist>
2954 </para>
2955
2956 <para>
2957 For the scenarios that do not use Autotools or
2958 CMake, you need to track the installation
2959 and diagnose and fix any issues until everything installs
2960 correctly.
2961 You need to look in the default location of
2962 <filename>${D}</filename>, which is
2963 <filename>${WORKDIR}/image</filename>, to be sure your
2964 files have been installed correctly.
2965 </para>
2966
2967 <note><title>Notes</title>
2968 <itemizedlist>
2969 <listitem><para>
2970 During the installation process, you might need to
2971 modify some of the installed files to suit the target
2972 layout.
2973 For example, you might need to replace hard-coded paths
2974 in an initscript with values of variables provided by
2975 the build system, such as replacing
2976 <filename>/usr/bin/</filename> with
2977 <filename>${bindir}</filename>.
2978 If you do perform such modifications during
2979 <filename>do_install</filename>, be sure to modify the
2980 destination file after copying rather than before
2981 copying.
2982 Modifying after copying ensures that the build system
2983 can re-execute <filename>do_install</filename> if
2984 needed.
2985 </para></listitem>
2986 <listitem><para>
2987 <filename>oe_runmake install</filename>, which can be
2988 run directly or can be run indirectly by the
2989 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-autotools'><filename>autotools</filename></ulink>
2990 and
2991 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-cmake'><filename>cmake</filename></ulink>
2992 classes, runs <filename>make install</filename> in
2993 parallel.
2994 Sometimes, a Makefile can have missing dependencies
2995 between targets that can result in race conditions.
2996 If you experience intermittent failures during
2997 <filename>do_install</filename>, you might be able to
2998 work around them by disabling parallel Makefile
2999 installs by adding the following to the recipe:
3000 <literallayout class='monospaced'>
3001 PARALLEL_MAKEINST = ""
3002 </literallayout>
3003 See
3004 <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKEINST'><filename>PARALLEL_MAKEINST</filename></ulink>
3005 for additional information.
3006 </para></listitem>
3007 </itemizedlist>
3008 </note>
3009 </section>
3010
3011 <section id='new-recipe-enabling-system-services'>
3012 <title>Enabling System Services</title>
3013
3014 <para>
3015 If you want to install a service, which is a process that
3016 usually starts on boot and runs in the background, then
3017 you must include some additional definitions in your recipe.
3018 </para>
3019
3020 <para>
3021 If you are adding services and the service initialization
3022 script or the service file itself is not installed, you must
3023 provide for that installation in your recipe using a
3024 <filename>do_install_append</filename> function.
3025 If your recipe already has a <filename>do_install</filename>
3026 function, update the function near its end rather than
3027 adding an additional <filename>do_install_append</filename>
3028 function.
3029 </para>
3030
3031 <para>
3032 When you create the installation for your services, you need
3033 to accomplish what is normally done by
3034 <filename>make install</filename>.
3035 In other words, make sure your installation arranges the output
3036 similar to how it is arranged on the target system.
3037 </para>
3038
3039 <para>
3040 The OpenEmbedded build system provides support for starting
3041 services two different ways:
3042 <itemizedlist>
3043 <listitem><para><emphasis>SysVinit:</emphasis>
3044 SysVinit is a system and service manager that
3045 manages the init system used to control the very basic
3046 functions of your system.
3047 The init program is the first program
3048 started by the Linux kernel when the system boots.
3049 Init then controls the startup, running and shutdown
3050 of all other programs.</para>
3051 <para>To enable a service using SysVinit, your recipe
3052 needs to inherit the
3053 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-update-rc.d'><filename>update-rc.d</filename></ulink>
3054 class.
3055 The class helps facilitate safely installing the
3056 package on the target.</para>
3057 <para>You will need to set the
3058 <ulink url='&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_PACKAGES'><filename>INITSCRIPT_PACKAGES</filename></ulink>,
3059 <ulink url='&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_NAME'><filename>INITSCRIPT_NAME</filename></ulink>,
3060 and
3061 <ulink url='&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_PARAMS'><filename>INITSCRIPT_PARAMS</filename></ulink>
3062 variables within your recipe.</para></listitem>
3063 <listitem><para><emphasis>systemd:</emphasis>
3064 System Management Daemon (systemd) was designed to
3065 replace SysVinit and to provide
3066 enhanced management of services.
3067 For more information on systemd, see the systemd
3068 homepage at
3069 <ulink url='http://freedesktop.org/wiki/Software/systemd/'></ulink>.
3070 </para>
3071 <para>To enable a service using systemd, your recipe
3072 needs to inherit the
3073 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-systemd'><filename>systemd</filename></ulink>
3074 class.
3075 See the <filename>systemd.bbclass</filename> file
3076 located in your
3077 <link linkend='source-directory'>Source Directory</link>.
3078 section for more information.
3079 </para></listitem>
3080 </itemizedlist>
3081 </para>
3082 </section>
3083
3084 <section id='new-recipe-packaging'>
3085 <title>Packaging</title>
3086
3087 <para>
3088 Successful packaging is a combination of automated processes
3089 performed by the OpenEmbedded build system and some
3090 specific steps you need to take.
3091 The following list describes the process:
3092 <itemizedlist>
3093 <listitem><para><emphasis>Splitting Files</emphasis>:
3094 The <filename>do_package</filename> task splits the
3095 files produced by the recipe into logical components.
3096 Even software that produces a single binary might
3097 still have debug symbols, documentation, and other
3098 logical components that should be split out.
3099 The <filename>do_package</filename> task ensures
3100 that files are split up and packaged correctly.
3101 </para></listitem>
3102 <listitem><para><emphasis>Running QA Checks</emphasis>:
3103 The
3104 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-insane'><filename>insane</filename></ulink>
3105 class adds a step to
3106 the package generation process so that output quality
3107 assurance checks are generated by the OpenEmbedded
3108 build system.
3109 This step performs a range of checks to be sure the
3110 build's output is free of common problems that show
3111 up during runtime.
3112 For information on these checks, see the
3113 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-insane'><filename>insane</filename></ulink>
3114 class and the
3115 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-qa-checks'>QA Error and Warning Messages</ulink>"
3116 chapter in the Yocto Project Reference Manual.
3117 </para></listitem>
3118 <listitem><para><emphasis>Hand-Checking Your Packages</emphasis>:
3119 After you build your software, you need to be sure
3120 your packages are correct.
3121 Examine the
3122 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}/packages-split</filename>
3123 directory and make sure files are where you expect
3124 them to be.
3125 If you discover problems, you can set
3126 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>,
3127 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink>,
3128 <filename>do_install(_append)</filename>, and so forth as
3129 needed.
3130 </para></listitem>
3131 <listitem><para><emphasis>Splitting an Application into Multiple Packages</emphasis>:
3132 If you need to split an application into several
3133 packages, see the
3134 "<link linkend='splitting-an-application-into-multiple-packages'>Splitting an Application into Multiple Packages</link>"
3135 section for an example.
3136 </para></listitem>
3137 <listitem><para><emphasis>Installing a Post-Installation Script</emphasis>:
3138 For an example showing how to install a
3139 post-installation script, see the
3140 "<link linkend='new-recipe-post-installation-scripts'>Post-Installation Scripts</link>"
3141 section.
3142 </para></listitem>
3143 <listitem><para><emphasis>Marking Package Architecture</emphasis>:
3144 Depending on what your recipe is building and how it
3145 is configured, it might be important to mark the
3146 packages produced as being specific to a particular
3147 machine, or to mark them as not being specific to
Patrick Williamsf1e5d692016-03-30 15:21:19 -05003148 a particular machine or architecture at all.</para>
3149 <para>By default, packages apply to any machine with the
3150 same architecture as the target machine.
3151 When a recipe produces packages that are
3152 machine-specific (e.g. the
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003153 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
3154 value is passed into the configure script or a patch
Patrick Williamsf1e5d692016-03-30 15:21:19 -05003155 is applied only for a particular machine), you should
3156 mark them as such by adding the following to the
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003157 recipe:
3158 <literallayout class='monospaced'>
3159 PACKAGE_ARCH = "${MACHINE_ARCH}"
Patrick Williamsf1e5d692016-03-30 15:21:19 -05003160 </literallayout></para>
3161 <para>On the other hand, if the recipe produces packages
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003162 that do not contain anything specific to the target
3163 machine or architecture at all (e.g. recipes
3164 that simply package script files or configuration
3165 files), you should use the
3166 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-allarch'><filename>allarch</filename></ulink>
3167 class to do this for you by adding this to your
3168 recipe:
3169 <literallayout class='monospaced'>
3170 inherit allarch
3171 </literallayout>
3172 Ensuring that the package architecture is correct is
3173 not critical while you are doing the first few builds
3174 of your recipe.
3175 However, it is important in order
3176 to ensure that your recipe rebuilds (or does not
3177 rebuild) appropriately in response to changes in
3178 configuration, and to ensure that you get the
3179 appropriate packages installed on the target machine,
3180 particularly if you run separate builds for more
3181 than one target machine.
3182 </para></listitem>
3183 </itemizedlist>
3184 </para>
3185 </section>
3186
Patrick Williamsc0f7c042017-02-23 20:41:17 -06003187 <section id='new-sharing-files-between-recipes'>
3188 <title>Sharing Files Between Recipes</title>
3189
3190 <para>
3191 Recipes often need to use files provided by other recipes on
3192 the build host.
3193 For example, an application linking to a common library needs
3194 access to the library itself and its associated headers.
Brad Bishop6e60e8b2018-02-01 10:27:11 -05003195 The way this access is accomplished is by populating a sysroot
Patrick Williamsc0f7c042017-02-23 20:41:17 -06003196 with files.
Brad Bishop6e60e8b2018-02-01 10:27:11 -05003197 Each recipe has two sysroots in its work directory, one for
3198 target files
3199 (<filename>recipe-sysroot</filename>) and one for files that
3200 are native to the build host
3201 (<filename>recipe-sysroot-native</filename>).
Patrick Williamsc0f7c042017-02-23 20:41:17 -06003202 <note>
3203 You could find the term "staging" used within the Yocto
Brad Bishop6e60e8b2018-02-01 10:27:11 -05003204 project regarding files populating sysroots (e.g. the
Brad Bishop37a0e4d2017-12-04 01:01:44 -05003205 <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_DIR'><filename>STAGING_DIR</filename></ulink>
3206 variable).
Patrick Williamsc0f7c042017-02-23 20:41:17 -06003207 </note>
3208 </para>
3209
3210 <para>
3211 Recipes should never populate the sysroot directly (i.e. write
3212 files into sysroot).
3213 Instead, files should be installed into standard locations
3214 during the
3215 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>
3216 task within the
3217 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink><filename>}</filename>
3218 directory.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06003219 The reason for this limitation is that almost all files that
3220 populate the sysroot are cataloged in manifests in order to
3221 ensure the files can be removed later when a recipe is either
3222 modified or removed.
3223 Thus, the sysroot is able to remain free from stale files.
3224 </para>
3225
3226 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05003227 A subset of the files installed by the
3228 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>
3229 task are used by the
3230 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></ulink>
3231 task as defined by the the
3232 <ulink url='&YOCTO_DOCS_REF_URL;#var-SYSROOT_DIRS'><filename>SYSROOT_DIRS</filename></ulink>
3233 variable to automatically populate the sysroot.
Brad Bishop37a0e4d2017-12-04 01:01:44 -05003234 It is possible to modify the list of directories that populate
3235 the sysroot.
3236 The following example shows how you could add the
3237 <filename>/opt</filename> directory to the list of
Brad Bishop6e60e8b2018-02-01 10:27:11 -05003238 directories within a recipe:
Brad Bishop37a0e4d2017-12-04 01:01:44 -05003239 <literallayout class='monospaced'>
3240 SYSROOT_DIRS += "/opt"
3241 </literallayout>
3242 </para>
3243
3244 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05003245 For a more complete description of the
3246 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></ulink>
3247 task and its associated functions, see the
3248 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-staging'><filename>staging</filename></ulink>
3249 class.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06003250 </para>
3251 </section>
3252
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003253 <section id='properly-versioning-pre-release-recipes'>
3254 <title>Properly Versioning Pre-Release Recipes</title>
3255
3256 <para>
3257 Sometimes the name of a recipe can lead to versioning
3258 problems when the recipe is upgraded to a final release.
3259 For example, consider the
3260 <filename>irssi_0.8.16-rc1.bb</filename> recipe file in
3261 the list of example recipes in the
3262 "<link linkend='new-recipe-storing-and-naming-the-recipe'>Storing and Naming the Recipe</link>"
3263 section.
3264 This recipe is at a release candidate stage (i.e.
3265 "rc1").
3266 When the recipe is released, the recipe filename becomes
3267 <filename>irssi_0.8.16.bb</filename>.
3268 The version change from <filename>0.8.16-rc1</filename>
3269 to <filename>0.8.16</filename> is seen as a decrease by the
3270 build system and package managers, so the resulting packages
3271 will not correctly trigger an upgrade.
3272 </para>
3273
3274 <para>
3275 In order to ensure the versions compare properly, the
3276 recommended convention is to set
3277 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>
3278 within the recipe to
3279 "<replaceable>previous_version</replaceable>+<replaceable>current_version</replaceable>".
3280 You can use an additional variable so that you can use the
3281 current version elsewhere.
3282 Here is an example:
3283 <literallayout class='monospaced'>
3284 REALPV = "0.8.16-rc1"
3285 PV = "0.8.15+${REALPV}"
3286 </literallayout>
3287 </para>
3288 </section>
3289
3290 <section id='new-recipe-post-installation-scripts'>
3291 <title>Post-Installation Scripts</title>
3292
3293 <para>
3294 Post-installation scripts run immediately after installing
3295 a package on the target or during image creation when a
3296 package is included in an image.
3297 To add a post-installation script to a package, add a
Brad Bishop6e60e8b2018-02-01 10:27:11 -05003298 <filename>pkg_postinst_</filename><replaceable>PACKAGENAME</replaceable><filename>()</filename> function to
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003299 the recipe file (<filename>.bb</filename>) and replace
Brad Bishop6e60e8b2018-02-01 10:27:11 -05003300 <replaceable>PACKAGENAME</replaceable> with the name of the package
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003301 you want to attach to the <filename>postinst</filename>
3302 script.
3303 To apply the post-installation script to the main package
3304 for the recipe, which is usually what is required, specify
3305 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05003306 in place of <replaceable>PACKAGENAME</replaceable>.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003307 </para>
3308
3309 <para>
3310 A post-installation function has the following structure:
3311 <literallayout class='monospaced'>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05003312 pkg_postinst_<replaceable>PACKAGENAME</replaceable>() {
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003313 # Commands to carry out
3314 }
3315 </literallayout>
3316 </para>
3317
3318 <para>
3319 The script defined in the post-installation function is
3320 called when the root filesystem is created.
3321 If the script succeeds, the package is marked as installed.
3322 If the script fails, the package is marked as unpacked and
3323 the script is executed when the image boots again.
Brad Bishop37a0e4d2017-12-04 01:01:44 -05003324 <note>
3325 Any RPM post-installation script that runs on the target
3326 should return a 0 exit code.
3327 RPM does not allow non-zero exit codes for these scripts,
3328 and the RPM package manager will cause the package to fail
3329 installation on the target.
3330 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003331 </para>
3332
3333 <para>
3334 Sometimes it is necessary for the execution of a
3335 post-installation script to be delayed until the first boot.
3336 For example, the script might need to be executed on the
3337 device itself.
3338 To delay script execution until boot time, use the following
3339 structure in the post-installation script:
3340 <literallayout class='monospaced'>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05003341 pkg_postinst_<replaceable>PACKAGENAME</replaceable>() {
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003342 if [ x"$D" = "x" ]; then
3343 # Actions to carry out on the device go here
3344 else
3345 exit 1
3346 fi
3347 }
3348 </literallayout>
3349 </para>
3350
3351 <para>
3352 The previous example delays execution until the image boots
3353 again because the environment variable <filename>D</filename>
3354 points to the directory containing the image when
3355 the root filesystem is created at build time but is unset
3356 when executed on the first boot.
3357 </para>
3358
Brad Bishop6e60e8b2018-02-01 10:27:11 -05003359 <para>
3360 If you have recipes that use <filename>pkg_postinst</filename>
3361 scripts and they require the use of non-standard native
3362 tools that have dependencies during rootfs construction, you
3363 need to use the
3364 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_WRITE_DEPS'><filename>PACKAGE_WRITE_DEPS</filename></ulink>
3365 variable in your recipe to list these tools.
3366 If you do not use this variable, the tools might be missing and
3367 execution of the post-installation script is deferred until
3368 first boot.
3369 Deferring the script to first boot is undesirable and for
3370 read-only rootfs impossible.
3371 </para>
3372
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003373 <note>
3374 Equivalent support for pre-install, pre-uninstall, and
3375 post-uninstall scripts exist by way of
3376 <filename>pkg_preinst</filename>,
3377 <filename>pkg_prerm</filename>, and
3378 <filename>pkg_postrm</filename>, respectively.
3379 These scrips work in exactly the same way as does
3380 <filename>pkg_postinst</filename> with the exception that they
3381 run at different times.
3382 Also, because of when they run, they are not applicable to
3383 being run at image creation time like
3384 <filename>pkg_postinst</filename>.
3385 </note>
3386 </section>
3387
3388 <section id='new-recipe-testing'>
3389 <title>Testing</title>
3390
3391 <para>
3392 The final step for completing your recipe is to be sure that
3393 the software you built runs correctly.
3394 To accomplish runtime testing, add the build's output
3395 packages to your image and test them on the target.
3396 </para>
3397
3398 <para>
3399 For information on how to customize your image by adding
3400 specific packages, see the
3401 "<link linkend='usingpoky-extend-customimage'>Customizing Images</link>"
3402 section.
3403 </para>
3404 </section>
3405
3406 <section id='new-recipe-testing-examples'>
3407 <title>Examples</title>
3408
3409 <para>
3410 To help summarize how to write a recipe, this section provides
3411 some examples given various scenarios:
3412 <itemizedlist>
3413 <listitem><para>Recipes that use local files</para></listitem>
3414 <listitem><para>Using an Autotooled package</para></listitem>
3415 <listitem><para>Using a Makefile-based package</para></listitem>
3416 <listitem><para>Splitting an application into multiple packages</para></listitem>
3417 <listitem><para>Adding binaries to an image</para></listitem>
3418 </itemizedlist>
3419 </para>
3420
3421 <section id='new-recipe-single-c-file-package-hello-world'>
3422 <title>Single .c File Package (Hello World!)</title>
3423
3424 <para>
3425 Building an application from a single file that is stored
3426 locally (e.g. under <filename>files</filename>) requires
3427 a recipe that has the file listed in the
3428 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename>
3429 variable.
3430 Additionally, you need to manually write the
3431 <filename>do_compile</filename> and
3432 <filename>do_install</filename> tasks.
3433 The <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'>S</ulink></filename>
3434 variable defines the directory containing the source code,
3435 which is set to
3436 <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>
3437 in this case - the directory BitBake uses for the build.
3438 <literallayout class='monospaced'>
3439 SUMMARY = "Simple helloworld application"
3440 SECTION = "examples"
3441 LICENSE = "MIT"
3442 LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
3443
3444 SRC_URI = "file://helloworld.c"
3445
3446 S = "${WORKDIR}"
3447
3448 do_compile() {
3449 ${CC} helloworld.c -o helloworld
3450 }
3451
3452 do_install() {
3453 install -d ${D}${bindir}
3454 install -m 0755 helloworld ${D}${bindir}
3455 }
3456 </literallayout>
3457 </para>
3458
3459 <para>
3460 By default, the <filename>helloworld</filename>,
3461 <filename>helloworld-dbg</filename>, and
3462 <filename>helloworld-dev</filename> packages are built.
3463 For information on how to customize the packaging process,
3464 see the
3465 "<link linkend='splitting-an-application-into-multiple-packages'>Splitting an Application into Multiple Packages</link>"
3466 section.
3467 </para>
3468 </section>
3469
3470 <section id='new-recipe-autotooled-package'>
3471 <title>Autotooled Package</title>
3472 <para>
3473 Applications that use Autotools such as <filename>autoconf</filename> and
3474 <filename>automake</filename> require a recipe that has a source archive listed in
3475 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename> and
3476 also inherit the
3477 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-autotools'><filename>autotools</filename></ulink>
3478 class, which contains the definitions of all the steps
3479 needed to build an Autotool-based application.
3480 The result of the build is automatically packaged.
3481 And, if the application uses NLS for localization, packages with local information are
3482 generated (one package per language).
3483 Following is one example: (<filename>hello_2.3.bb</filename>)
3484 <literallayout class='monospaced'>
3485 SUMMARY = "GNU Helloworld application"
3486 SECTION = "examples"
3487 LICENSE = "GPLv2+"
3488 LIC_FILES_CHKSUM = "file://COPYING;md5=751419260aa954499f7abaabaa882bbe"
3489
3490 SRC_URI = "${GNU_MIRROR}/hello/hello-${PV}.tar.gz"
3491
3492 inherit autotools gettext
3493 </literallayout>
3494 </para>
3495
3496 <para>
3497 The variable
3498 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</ulink></filename>
3499 is used to track source license changes as described in the
3500 "<ulink url='&YOCTO_DOCS_REF_URL;#usingpoky-configuring-LIC_FILES_CHKSUM'>Tracking License Changes</ulink>" section.
3501 You can quickly create Autotool-based recipes in a manner similar to the previous example.
3502 </para>
3503 </section>
3504
3505 <section id='new-recipe-makefile-based-package'>
3506 <title>Makefile-Based Package</title>
3507
3508 <para>
3509 Applications that use GNU <filename>make</filename> also require a recipe that has
3510 the source archive listed in
3511 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename>.
3512 You do not need to add a <filename>do_compile</filename> step since by default BitBake
3513 starts the <filename>make</filename> command to compile the application.
3514 If you need additional <filename>make</filename> options, you should store them in the
Patrick Williamsc0f7c042017-02-23 20:41:17 -06003515 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OEMAKE'><filename>EXTRA_OEMAKE</filename></ulink>
3516 or
3517 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG_CONFARGS'><filename>PACKAGECONFIG_CONFARGS</filename></ulink>
3518 variables.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003519 BitBake passes these options into the GNU <filename>make</filename> invocation.
3520 Note that a <filename>do_install</filename> task is still required.
3521 Otherwise, BitBake runs an empty <filename>do_install</filename> task by default.
3522 </para>
3523
3524 <para>
3525 Some applications might require extra parameters to be passed to the compiler.
3526 For example, the application might need an additional header path.
3527 You can accomplish this by adding to the
3528 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-CFLAGS'>CFLAGS</ulink></filename> variable.
3529 The following example shows this:
3530 <literallayout class='monospaced'>
3531 CFLAGS_prepend = "-I ${S}/include "
3532 </literallayout>
3533 </para>
3534
3535 <para>
3536 In the following example, <filename>mtd-utils</filename> is a makefile-based package:
3537 <literallayout class='monospaced'>
3538 SUMMARY = "Tools for managing memory technology devices"
3539 SECTION = "base"
3540 DEPENDS = "zlib lzo e2fsprogs util-linux"
3541 HOMEPAGE = "http://www.linux-mtd.infradead.org/"
3542 LICENSE = "GPLv2+"
3543 LIC_FILES_CHKSUM = "file://COPYING;md5=0636e73ff0215e8d672dc4c32c317bb3 \
3544 file://include/common.h;beginline=1;endline=17;md5=ba05b07912a44ea2bf81ce409380049c"
3545
3546 # Use the latest version at 26 Oct, 2013
3547 SRCREV = "9f107132a6a073cce37434ca9cda6917dd8d866b"
3548 SRC_URI = "git://git.infradead.org/mtd-utils.git \
3549 file://add-exclusion-to-mkfs-jffs2-git-2.patch \
3550 "
3551
3552 PV = "1.5.1+git${SRCPV}"
3553
Patrick Williamsc0f7c042017-02-23 20:41:17 -06003554 S = "${WORKDIR}/git"
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003555
3556 EXTRA_OEMAKE = "'CC=${CC}' 'RANLIB=${RANLIB}' 'AR=${AR}' 'CFLAGS=${CFLAGS} -I${S}/include -DWITHOUT_XATTR' 'BUILDDIR=${S}'"
3557
3558 do_install () {
3559 oe_runmake install DESTDIR=${D} SBINDIR=${sbindir} MANDIR=${mandir} INCLUDEDIR=${includedir}
3560 }
3561
3562 PACKAGES =+ "mtd-utils-jffs2 mtd-utils-ubifs mtd-utils-misc"
3563
3564 FILES_mtd-utils-jffs2 = "${sbindir}/mkfs.jffs2 ${sbindir}/jffs2dump ${sbindir}/jffs2reader ${sbindir}/sumtool"
3565 FILES_mtd-utils-ubifs = "${sbindir}/mkfs.ubifs ${sbindir}/ubi*"
3566 FILES_mtd-utils-misc = "${sbindir}/nftl* ${sbindir}/ftl* ${sbindir}/rfd* ${sbindir}/doc* ${sbindir}/serve_image ${sbindir}/recv_image"
3567
3568 PARALLEL_MAKE = ""
3569
3570 BBCLASSEXTEND = "native"
3571 </literallayout>
3572 </para>
3573 </section>
3574
3575 <section id='splitting-an-application-into-multiple-packages'>
3576 <title>Splitting an Application into Multiple Packages</title>
3577
3578 <para>
3579 You can use the variables
3580 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'>PACKAGES</ulink></filename> and
3581 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'>FILES</ulink></filename>
3582 to split an application into multiple packages.
3583 </para>
3584
3585 <para>
3586 Following is an example that uses the <filename>libxpm</filename> recipe.
3587 By default, this recipe generates a single package that contains the library along
3588 with a few binaries.
3589 You can modify the recipe to split the binaries into separate packages:
3590 <literallayout class='monospaced'>
3591 require xorg-lib-common.inc
3592
3593 SUMMARY = "Xpm: X Pixmap extension library"
3594 LICENSE = "BSD"
3595 LIC_FILES_CHKSUM = "file://COPYING;md5=51f4270b012ecd4ab1a164f5f4ed6cf7"
3596 DEPENDS += "libxext libsm libxt"
3597 PE = "1"
3598
3599 XORG_PN = "libXpm"
3600
3601 PACKAGES =+ "sxpm cxpm"
3602 FILES_cxpm = "${bindir}/cxpm"
3603 FILES_sxpm = "${bindir}/sxpm"
3604 </literallayout>
3605 </para>
3606
3607 <para>
3608 In the previous example, we want to ship the <filename>sxpm</filename>
3609 and <filename>cxpm</filename> binaries in separate packages.
3610 Since <filename>bindir</filename> would be packaged into the main
3611 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'>PN</ulink></filename>
3612 package by default, we prepend the <filename>PACKAGES</filename>
3613 variable so additional package names are added to the start of list.
3614 This results in the extra <filename>FILES_*</filename>
3615 variables then containing information that define which files and
3616 directories go into which packages.
3617 Files included by earlier packages are skipped by latter packages.
3618 Thus, the main <filename>PN</filename> package
3619 does not include the above listed files.
3620 </para>
3621 </section>
3622
3623 <section id='packaging-externally-produced-binaries'>
3624 <title>Packaging Externally Produced Binaries</title>
3625
3626 <para>
3627 Sometimes, you need to add pre-compiled binaries to an
3628 image.
3629 For example, suppose that binaries for proprietary code
3630 exist, which are created by a particular division of a
3631 company.
3632 Your part of the company needs to use those binaries as
3633 part of an image that you are building using the
3634 OpenEmbedded build system.
3635 Since you only have the binaries and not the source code,
3636 you cannot use a typical recipe that expects to fetch the
3637 source specified in
3638 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
3639 and then compile it.
3640 </para>
3641
3642 <para>
3643 One method is to package the binaries and then install them
3644 as part of the image.
3645 Generally, it is not a good idea to package binaries
3646 since, among other things, it can hinder the ability to
3647 reproduce builds and could lead to compatibility problems
3648 with ABI in the future.
3649 However, sometimes you have no choice.
3650 </para>
3651
3652 <para>
3653 The easiest solution is to create a recipe that uses
3654 the
3655 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-bin-package'><filename>bin_package</filename></ulink>
3656 class and to be sure that you are using default locations
3657 for build artifacts.
3658 In most cases, the <filename>bin_package</filename> class
3659 handles "skipping" the configure and compile steps as well
3660 as sets things up to grab packages from the appropriate
3661 area.
3662 In particular, this class sets <filename>noexec</filename>
3663 on both the
3664 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink>
3665 and
3666 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink>
3667 tasks, sets
3668 <filename>FILES_${PN}</filename> to "/" so that it picks
3669 up all files, and sets up a
3670 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>
3671 task, which effectively copies all files from
3672 <filename>${S}</filename> to <filename>${D}</filename>.
3673 The <filename>bin_package</filename> class works well when
3674 the files extracted into <filename>${S}</filename> are
3675 already laid out in the way they should be laid out
3676 on the target.
3677 For more information on these variables, see the
3678 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink>,
3679 <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink>,
3680 <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink>,
3681 and
3682 <ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink>
3683 variables in the Yocto Project Reference Manual's variable
3684 glossary.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06003685 <note><title>Notes</title>
3686 <itemizedlist>
3687 <listitem><para>
3688 Using
3689 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
3690 is a good idea even for components distributed
3691 in binary form, and is often necessary for
3692 shared libraries.
3693 For a shared library, listing the library
3694 dependencies in
3695 <filename>DEPENDS</filename> makes sure that
3696 the libraries are available in the staging
3697 sysroot when other recipes link against the
3698 library, which might be necessary for
3699 successful linking.
3700 </para></listitem>
3701 <listitem><para>
3702 Using <filename>DEPENDS</filename> also
3703 allows runtime dependencies between packages
3704 to be added automatically.
3705 See the
3706 "<ulink url='&YOCTO_DOCS_REF_URL;#automatically-added-runtime-dependencies'>Automatically Added Runtime Dependencies</ulink>"
3707 section in the Yocto Project Reference Manual
3708 for more information.
3709 </para></listitem>
3710 </itemizedlist>
3711 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003712 </para>
3713
3714 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06003715 If you cannot use the <filename>bin_package</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003716 class, you need to be sure you are doing the following:
3717 <itemizedlist>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06003718 <listitem><para>
3719 Create a recipe where the
3720 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink>
3721 and
3722 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink>
3723 tasks do nothing:
3724 It is usually sufficient to just not define these
3725 tasks in the recipe, because the default
3726 implementations do nothing unless a Makefile is
3727 found in
3728 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink><filename>}</filename>.
3729 </para>
3730
3731 <para>If
3732 <filename>${S}</filename> might contain a Makefile,
3733 or if you inherit some class that replaces
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003734 <filename>do_configure</filename> and
Patrick Williamsc0f7c042017-02-23 20:41:17 -06003735 <filename>do_compile</filename> with custom
3736 versions, then you can use the
3737 <filename>[</filename><ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'><filename>noexec</filename></ulink><filename>]</filename>
3738 flag to turn the tasks into no-ops, as follows:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003739 <literallayout class='monospaced'>
3740 do_configure[noexec] = "1"
3741 do_compile[noexec] = "1"
3742 </literallayout>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06003743 Unlike
3744 <ulink url='&YOCTO_DOCS_BB_URL;#deleting-a-task'><filename>deleting the tasks</filename></ulink>,
3745 using the flag preserves the dependency chain from
3746 the
3747 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-fetch'><filename>do_fetch</filename></ulink>, <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-unpack'><filename>do_unpack</filename></ulink>,
3748 and
3749 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink>
3750 tasks to the
3751 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>
3752 task.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003753 </para></listitem>
3754 <listitem><para>Make sure your
3755 <filename>do_install</filename> task installs the
3756 binaries appropriately.
3757 </para></listitem>
3758 <listitem><para>Ensure that you set up
Patrick Williamsc0f7c042017-02-23 20:41:17 -06003759 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink>
3760 (usually
3761 <filename>FILES_${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename>)
3762 to point to the files you have installed, which of
3763 course depends on where you have installed them
3764 and whether those files are in different locations
3765 than the defaults.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003766 </para></listitem>
3767 </itemizedlist>
3768 </para>
3769 </section>
3770 </section>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06003771
3772 <section id="following-recipe-style-guidelines">
3773 <title>Following Recipe Style Guidelines</title>
3774
3775 <para>
3776 When writing recipes, it is good to conform to existing
3777 style guidelines.
3778 The
3779 <ulink url='http://www.openembedded.org/wiki/Styleguide'>OpenEmbedded Styleguide</ulink>
3780 wiki page provides rough guidelines for preferred recipe style.
3781 </para>
3782
3783 <para>
3784 It is common for existing recipes to deviate a bit from this
3785 style.
3786 However, aiming for at least a consistent style is a good idea.
3787 Some practices, such as omitting spaces around
3788 <filename>=</filename> operators in assignments or ordering
3789 recipe components in an erratic way, are widely seen as poor
3790 style.
3791 </para>
3792 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05003793 </section>
3794
3795 <section id="platdev-newmachine">
3796 <title>Adding a New Machine</title>
3797
3798 <para>
3799 Adding a new machine to the Yocto Project is a straightforward
3800 process.
3801 This section describes how to add machines that are similar
3802 to those that the Yocto Project already supports.
3803 <note>
3804 Although well within the capabilities of the Yocto Project,
3805 adding a totally new architecture might require
3806 changes to <filename>gcc/glibc</filename> and to the site
3807 information, which is beyond the scope of this manual.
3808 </note>
3809 </para>
3810
3811 <para>
3812 For a complete example that shows how to add a new machine,
3813 see the
3814 "<ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-yocto-bsp-script'>Creating a New BSP Layer Using the yocto-bsp Script</ulink>"
3815 section in the Yocto Project Board Support Package (BSP) Developer's Guide.
3816 </para>
3817
3818 <section id="platdev-newmachine-conffile">
3819 <title>Adding the Machine Configuration File</title>
3820
3821 <para>
3822 To add a new machine, you need to add a new machine
3823 configuration file to the layer's
3824 <filename>conf/machine</filename> directory.
3825 This configuration file provides details about the device
3826 you are adding.
3827 </para>
3828
3829 <para>
3830 The OpenEmbedded build system uses the root name of the
3831 machine configuration file to reference the new machine.
3832 For example, given a machine configuration file named
3833 <filename>crownbay.conf</filename>, the build system
3834 recognizes the machine as "crownbay".
3835 </para>
3836
3837 <para>
3838 The most important variables you must set in your machine
3839 configuration file or include from a lower-level configuration
3840 file are as follows:
3841 <itemizedlist>
3842 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-TARGET_ARCH'>TARGET_ARCH</ulink></filename>
3843 (e.g. "arm")</para></listitem>
3844 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PREFERRED_PROVIDER'>PREFERRED_PROVIDER</ulink>_virtual/kernel</filename>
3845 </para></listitem>
3846 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES'>MACHINE_FEATURES</ulink></filename>
3847 (e.g. "apm screen wifi")</para></listitem>
3848 </itemizedlist>
3849 </para>
3850
3851 <para>
3852 You might also need these variables:
3853 <itemizedlist>
3854 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SERIAL_CONSOLES'>SERIAL_CONSOLES</ulink></filename>
3855 (e.g. "115200;ttyS0 115200;ttyS1")</para></listitem>
3856 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-KERNEL_IMAGETYPE'>KERNEL_IMAGETYPE</ulink></filename>
3857 (e.g. "zImage")</para></listitem>
3858 <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES'>IMAGE_FSTYPES</ulink></filename>
3859 (e.g. "tar.gz jffs2")</para></listitem>
3860 </itemizedlist>
3861 </para>
3862
3863 <para>
3864 You can find full details on these variables in the reference
3865 section.
3866 You can leverage existing machine <filename>.conf</filename>
3867 files from <filename>meta-yocto-bsp/conf/machine/</filename>.
3868 </para>
3869 </section>
3870
3871 <section id="platdev-newmachine-kernel">
3872 <title>Adding a Kernel for the Machine</title>
3873
3874 <para>
3875 The OpenEmbedded build system needs to be able to build a kernel
3876 for the machine.
3877 You need to either create a new kernel recipe for this machine,
3878 or extend an existing kernel recipe.
3879 You can find several kernel recipe examples in the
3880 Source Directory at
3881 <filename>meta/recipes-kernel/linux</filename>
3882 that you can use as references.
3883 </para>
3884
3885 <para>
3886 If you are creating a new kernel recipe, normal recipe-writing
3887 rules apply for setting up a
3888 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename>.
3889 Thus, you need to specify any necessary patches and set
3890 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'>S</ulink></filename>
3891 to point at the source code.
3892 You need to create a <filename>do_configure</filename> task that
3893 configures the unpacked kernel with a
3894 <filename>defconfig</filename> file.
3895 You can do this by using a <filename>make defconfig</filename>
3896 command or, more commonly, by copying in a suitable
3897 <filename>defconfig</filename> file and then running
3898 <filename>make oldconfig</filename>.
3899 By making use of <filename>inherit kernel</filename> and
3900 potentially some of the <filename>linux-*.inc</filename> files,
3901 most other functionality is centralized and the defaults of the
3902 class normally work well.
3903 </para>
3904
3905 <para>
3906 If you are extending an existing kernel recipe, it is usually
3907 a matter of adding a suitable <filename>defconfig</filename>
3908 file.
3909 The file needs to be added into a location similar to
3910 <filename>defconfig</filename> files used for other machines
3911 in a given kernel recipe.
3912 A possible way to do this is by listing the file in the
3913 <filename>SRC_URI</filename> and adding the machine to the
3914 expression in
3915 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-COMPATIBLE_MACHINE'>COMPATIBLE_MACHINE</ulink></filename>:
3916 <literallayout class='monospaced'>
3917 COMPATIBLE_MACHINE = '(qemux86|qemumips)'
3918 </literallayout>
3919 For more information on <filename>defconfig</filename> files,
3920 see the
3921 "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#changing-the-configuration'>Changing the Configuration</ulink>"
3922 section in the Yocto Project Linux Kernel Development Manual.
3923 </para>
3924 </section>
3925
3926 <section id="platdev-newmachine-formfactor">
3927 <title>Adding a Formfactor Configuration File</title>
3928
3929 <para>
3930 A formfactor configuration file provides information about the
3931 target hardware for which the image is being built and information that
3932 the build system cannot obtain from other sources such as the kernel.
3933 Some examples of information contained in a formfactor configuration file include
3934 framebuffer orientation, whether or not the system has a keyboard,
3935 the positioning of the keyboard in relation to the screen, and
3936 the screen resolution.
3937 </para>
3938
3939 <para>
3940 The build system uses reasonable defaults in most cases.
3941 However, if customization is
3942 necessary, you need to create a <filename>machconfig</filename> file
3943 in the <filename>meta/recipes-bsp/formfactor/files</filename>
3944 directory.
3945 This directory contains directories for specific machines such as
3946 <filename>qemuarm</filename> and <filename>qemux86</filename>.
3947 For information about the settings available and the defaults, see the
3948 <filename>meta/recipes-bsp/formfactor/files/config</filename> file found in the
3949 same area.
3950 </para>
3951
3952 <para>
3953 Following is an example for "qemuarm" machine:
3954 <literallayout class='monospaced'>
3955 HAVE_TOUCHSCREEN=1
3956 HAVE_KEYBOARD=1
3957
3958 DISPLAY_CAN_ROTATE=0
3959 DISPLAY_ORIENTATION=0
3960 #DISPLAY_WIDTH_PIXELS=640
3961 #DISPLAY_HEIGHT_PIXELS=480
3962 #DISPLAY_BPP=16
3963 DISPLAY_DPI=150
3964 DISPLAY_SUBPIXEL_ORDER=vrgb
3965 </literallayout>
3966 </para>
3967 </section>
3968 </section>
3969
Patrick Williamsc0f7c042017-02-23 20:41:17 -06003970 <section id='platdev-building-targets-with-multiple-configurations'>
3971 <title>Building Targets with Multiple Configurations</title>
3972
3973 <para>
3974 Bitbake also has functionality that allows you to build
3975 multiple targets at the same time, where each target uses
3976 a different configuration.
3977 </para>
3978
3979 <para>
3980 In order to accomplish this, you setup each of the configurations
3981 you need to use in parallel by placing the configuration files in
3982 your current build directory alongside the usual
3983 <filename>local.conf</filename> file.
3984 </para>
3985
3986 <para>
3987 Follow these guidelines to create an environment that supports
3988 multiple configurations:
3989 <itemizedlist>
3990 <listitem><para>
3991 <emphasis>Create Configuration Files</emphasis>:
3992 You need to create a single configuration file for each
3993 configuration for which you want to add support.
3994 These files would contain lines such as the following:
3995 <literallayout class='monospaced'>
3996 MACHINE = "A"
3997 </literallayout>
3998 The files would contain any other variables that can
3999 be set and built in the same directory.
4000 <note>
4001 You can change the
4002 <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>
4003 to not conflict.
4004 </note></para>
4005
4006 <para>
4007 Furthermore, the configuration file must be located in the
4008 current build directory in a directory named
4009 <filename>multiconfig</filename> under the build's
4010 <filename>conf</filename> directory where
4011 <filename>local.conf</filename> resides.
4012 The reason for this restriction is because the
4013 <filename>BBPATH</filename> variable is not constructed
4014 until the layers are parsed.
4015 Consequently, using the configuration file as a
4016 pre-configuration file is not possible unless it is
4017 located in the current working directory.
4018 </para></listitem>
4019 <listitem><para>
4020 <emphasis>Add the BitBake Multi-Config Variable to you Local Configuration File</emphasis>:
4021 Use the
4022 <filename>BBMULTICONFIG</filename>
4023 variable in your <filename>conf/local.conf</filename>
4024 configuration file to specify each separate configuration.
4025 For example, the following line tells BitBake it should load
4026 <filename>conf/multiconfig/configA.conf</filename>,
4027 <filename>conf/multiconfig/configB.conf</filename>, and
4028 <filename>conf/multiconfig/configC.conf</filename>.
4029 <literallayout class='monospaced'>
4030 BBMULTICONFIG = "configA configB configC"
4031 </literallayout>
4032 </para></listitem>
4033 <listitem><para>
4034 <emphasis>Launch BitBake</emphasis>:
4035 Use the following BitBake command form to launch the
4036 build:
4037 <literallayout class='monospaced'>
4038 $ bitbake [multiconfig:<replaceable>multiconfigname</replaceable>:]<replaceable>target</replaceable> [[[multiconfig:<replaceable>multiconfigname</replaceable>:]<replaceable>target</replaceable>] ... ]
4039 </literallayout>
4040 Following is an example that supports building a minimal
4041 image for configuration A alongside a standard
4042 <filename>core-image-sato</filename>, which takes its
4043 configuration from <filename>local.conf</filename>:
4044 <literallayout class='monospaced'>
4045 $ bitbake multiconfig:configA:core-image-minimal core-image-sato
4046 </literallayout>
4047 </para></listitem>
4048 </itemizedlist>
4049 </para>
4050
4051 <para>
4052 Support for multiple configurations in this current release of
4053 the Yocto Project (&DISTRO_NAME; &DISTRO;) has some known issues:
4054 <itemizedlist>
4055 <listitem><para>
4056 No inter-multi-configuration dependencies exist.
4057 </para></listitem>
4058 <listitem><para>
4059 Shared State (sstate) optimizations do not exist.
4060 Consequently, if the build uses the same object twice
4061 in, for example, two different
4062 <filename>TMPDIR</filename> directories, the build
4063 will either load from an existing sstate cache at the
4064 start or build the object twice.
4065 </para></listitem>
4066 </itemizedlist>
4067 </para>
4068 </section>
4069
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004070 <section id="platdev-working-with-libraries">
4071 <title>Working With Libraries</title>
4072
4073 <para>
4074 Libraries are an integral part of your system.
4075 This section describes some common practices you might find
4076 helpful when working with libraries to build your system:
4077 <itemizedlist>
4078 <listitem><para><link linkend='including-static-library-files'>How to include static library files</link>
4079 </para></listitem>
4080 <listitem><para><link linkend='combining-multiple-versions-library-files-into-one-image'>How to use the Multilib feature to combine multiple versions of library files into a single image</link>
4081 </para></listitem>
4082 <listitem><para><link linkend='installing-multiple-versions-of-the-same-library'>How to install multiple versions of the same library in parallel on the same system</link>
4083 </para></listitem>
4084 </itemizedlist>
4085 </para>
4086
4087 <section id='including-static-library-files'>
4088 <title>Including Static Library Files</title>
4089
4090 <para>
4091 If you are building a library and the library offers static linking, you can control
4092 which static library files (<filename>*.a</filename> files) get included in the
4093 built library.
4094 </para>
4095
4096 <para>
4097 The <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>
4098 and <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES_*</filename></ulink>
4099 variables in the
4100 <filename>meta/conf/bitbake.conf</filename> configuration file define how files installed
4101 by the <filename>do_install</filename> task are packaged.
4102 By default, the <filename>PACKAGES</filename> variable includes
4103 <filename>${PN}-staticdev</filename>, which represents all static library files.
4104 <note>
4105 Some previously released versions of the Yocto Project
4106 defined the static library files through
4107 <filename>${PN}-dev</filename>.
4108 </note>
4109 Following is part of the BitBake configuration file, where
4110 you can see how the static library files are defined:
4111 <literallayout class='monospaced'>
4112 PACKAGE_BEFORE_PN ?= ""
4113 PACKAGES = "${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale ${PACKAGE_BEFORE_PN} ${PN}"
4114 PACKAGES_DYNAMIC = "^${PN}-locale-.*"
4115 FILES = ""
4116
4117 FILES_${PN} = "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} \
4118 ${sysconfdir} ${sharedstatedir} ${localstatedir} \
4119 ${base_bindir}/* ${base_sbindir}/* \
4120 ${base_libdir}/*${SOLIBS} \
4121 ${base_prefix}/lib/udev/rules.d ${prefix}/lib/udev/rules.d \
4122 ${datadir}/${BPN} ${libdir}/${BPN}/* \
4123 ${datadir}/pixmaps ${datadir}/applications \
4124 ${datadir}/idl ${datadir}/omf ${datadir}/sounds \
4125 ${libdir}/bonobo/servers"
4126
4127 FILES_${PN}-bin = "${bindir}/* ${sbindir}/*"
4128
4129 FILES_${PN}-doc = "${docdir} ${mandir} ${infodir} ${datadir}/gtk-doc \
4130 ${datadir}/gnome/help"
4131 SECTION_${PN}-doc = "doc"
4132
4133 FILES_SOLIBSDEV ?= "${base_libdir}/lib*${SOLIBSDEV} ${libdir}/lib*${SOLIBSDEV}"
4134 FILES_${PN}-dev = "${includedir} ${FILES_SOLIBSDEV} ${libdir}/*.la \
4135 ${libdir}/*.o ${libdir}/pkgconfig ${datadir}/pkgconfig \
4136 ${datadir}/aclocal ${base_libdir}/*.o \
4137 ${libdir}/${BPN}/*.la ${base_libdir}/*.la"
4138 SECTION_${PN}-dev = "devel"
4139 ALLOW_EMPTY_${PN}-dev = "1"
4140 RDEPENDS_${PN}-dev = "${PN} (= ${EXTENDPKGV})"
4141
4142 FILES_${PN}-staticdev = "${libdir}/*.a ${base_libdir}/*.a ${libdir}/${BPN}/*.a"
4143 SECTION_${PN}-staticdev = "devel"
4144 RDEPENDS_${PN}-staticdev = "${PN}-dev (= ${EXTENDPKGV})"
4145 </literallayout>
4146 </para>
4147 </section>
4148
4149 <section id="combining-multiple-versions-library-files-into-one-image">
4150 <title>Combining Multiple Versions of Library Files into One Image</title>
4151
4152 <para>
4153 The build system offers the ability to build libraries with different
4154 target optimizations or architecture formats and combine these together
4155 into one system image.
4156 You can link different binaries in the image
4157 against the different libraries as needed for specific use cases.
4158 This feature is called "Multilib."
4159 </para>
4160
4161 <para>
4162 An example would be where you have most of a system compiled in 32-bit
4163 mode using 32-bit libraries, but you have something large, like a database
4164 engine, that needs to be a 64-bit application and uses 64-bit libraries.
4165 Multilib allows you to get the best of both 32-bit and 64-bit libraries.
4166 </para>
4167
4168 <para>
4169 While the Multilib feature is most commonly used for 32 and 64-bit differences,
4170 the approach the build system uses facilitates different target optimizations.
4171 You could compile some binaries to use one set of libraries and other binaries
4172 to use a different set of libraries.
4173 The libraries could differ in architecture, compiler options, or other
4174 optimizations.
4175 </para>
4176
4177 <para>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05004178 Several examples exist in the
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004179 <filename>meta-skeleton</filename> layer found in the
4180 <link linkend='source-directory'>Source Directory</link>:
4181 <itemizedlist>
4182 <listitem><para><filename>conf/multilib-example.conf</filename>
4183 configuration file</para></listitem>
4184 <listitem><para><filename>conf/multilib-example2.conf</filename>
4185 configuration file</para></listitem>
4186 <listitem><para><filename>recipes-multilib/images/core-image-multilib-example.bb</filename>
4187 recipe</para></listitem>
4188 </itemizedlist>
4189 </para>
4190
4191 <section id='preparing-to-use-multilib'>
4192 <title>Preparing to Use Multilib</title>
4193
4194 <para>
4195 User-specific requirements drive the Multilib feature.
4196 Consequently, there is no one "out-of-the-box" configuration that likely
4197 exists to meet your needs.
4198 </para>
4199
4200 <para>
4201 In order to enable Multilib, you first need to ensure your recipe is
4202 extended to support multiple libraries.
4203 Many standard recipes are already extended and support multiple libraries.
4204 You can check in the <filename>meta/conf/multilib.conf</filename>
4205 configuration file in the
4206 <link linkend='source-directory'>Source Directory</link> to see how this is
4207 done using the
4208 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBCLASSEXTEND'><filename>BBCLASSEXTEND</filename></ulink>
4209 variable.
4210 Eventually, all recipes will be covered and this list will
4211 not be needed.
4212 </para>
4213
4214 <para>
4215 For the most part, the Multilib class extension works automatically to
4216 extend the package name from <filename>${PN}</filename> to
4217 <filename>${MLPREFIX}${PN}</filename>, where <filename>MLPREFIX</filename>
4218 is the particular multilib (e.g. "lib32-" or "lib64-").
4219 Standard variables such as
4220 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>,
4221 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>,
4222 <ulink url='&YOCTO_DOCS_REF_URL;#var-RPROVIDES'><filename>RPROVIDES</filename></ulink>,
4223 <ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'><filename>RRECOMMENDS</filename></ulink>,
4224 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>, and
4225 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES_DYNAMIC'><filename>PACKAGES_DYNAMIC</filename></ulink>
4226 are automatically extended by the system.
4227 If you are extending any manual code in the recipe, you can use the
4228 <filename>${MLPREFIX}</filename> variable to ensure those names are extended
4229 correctly.
4230 This automatic extension code resides in <filename>multilib.bbclass</filename>.
4231 </para>
4232 </section>
4233
4234 <section id='using-multilib'>
4235 <title>Using Multilib</title>
4236
4237 <para>
4238 After you have set up the recipes, you need to define the actual
4239 combination of multiple libraries you want to build.
4240 You accomplish this through your <filename>local.conf</filename>
4241 configuration file in the
4242 <link linkend='build-directory'>Build Directory</link>.
4243 An example configuration would be as follows:
4244 <literallayout class='monospaced'>
4245 MACHINE = "qemux86-64"
4246 require conf/multilib.conf
4247 MULTILIBS = "multilib:lib32"
4248 DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
Patrick Williamsf1e5d692016-03-30 15:21:19 -05004249 IMAGE_INSTALL_append = " lib32-glib-2.0"
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004250 </literallayout>
4251 This example enables an
4252 additional library named <filename>lib32</filename> alongside the
4253 normal target packages.
4254 When combining these "lib32" alternatives, the example uses "x86" for tuning.
4255 For information on this particular tuning, see
4256 <filename>meta/conf/machine/include/ia32/arch-ia32.inc</filename>.
4257 </para>
4258
4259 <para>
Patrick Williamsf1e5d692016-03-30 15:21:19 -05004260 The example then includes <filename>lib32-glib-2.0</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004261 in all the images, which illustrates one method of including a
4262 multiple library dependency.
4263 You can use a normal image build to include this dependency,
4264 for example:
4265 <literallayout class='monospaced'>
4266 $ bitbake core-image-sato
4267 </literallayout>
4268 You can also build Multilib packages specifically with a command like this:
4269 <literallayout class='monospaced'>
Patrick Williamsf1e5d692016-03-30 15:21:19 -05004270 $ bitbake lib32-glib-2.0
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004271 </literallayout>
4272 </para>
4273 </section>
4274
4275 <section id='additional-implementation-details'>
4276 <title>Additional Implementation Details</title>
4277
4278 <para>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05004279 Generic implementation details as well as details that are
4280 specific to package management systems exist.
4281 Following are implementation details that exist regardless
4282 of the package management system:
4283 <itemizedlist>
4284 <listitem><para>The typical convention used for the
4285 class extension code as used by
4286 Multilib assumes that all package names specified
4287 in
4288 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>
4289 that contain <filename>${PN}</filename> have
4290 <filename>${PN}</filename> at the start of the name.
4291 When that convention is not followed and
4292 <filename>${PN}</filename> appears at
4293 the middle or the end of a name, problems occur.
4294 </para></listitem>
4295 <listitem><para>The
4296 <ulink url='&YOCTO_DOCS_REF_URL;#var-TARGET_VENDOR'><filename>TARGET_VENDOR</filename></ulink>
4297 value under Multilib will be extended to
4298 "-<replaceable>vendor</replaceable>ml<replaceable>multilib</replaceable>"
4299 (e.g. "-pokymllib32" for a "lib32" Multilib with
4300 Poky).
4301 The reason for this slightly unwieldy contraction
4302 is that any "-" characters in the vendor
4303 string presently break Autoconf's
4304 <filename>config.sub</filename>, and
4305 other separators are problematic for different
4306 reasons.
4307 </para></listitem>
4308 </itemizedlist>
4309 </para>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004310
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05004311 <para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004312 For the RPM Package Management System, the following implementation details
4313 exist:
4314 <itemizedlist>
4315 <listitem><para>A unique architecture is defined for the Multilib packages,
4316 along with creating a unique deploy folder under
4317 <filename>tmp/deploy/rpm</filename> in the
4318 <link linkend='build-directory'>Build Directory</link>.
4319 For example, consider <filename>lib32</filename> in a
4320 <filename>qemux86-64</filename> image.
4321 The possible architectures in the system are "all", "qemux86_64",
4322 "lib32_qemux86_64", and "lib32_x86".</para></listitem>
4323 <listitem><para>The <filename>${MLPREFIX}</filename> variable is stripped from
4324 <filename>${PN}</filename> during RPM packaging.
4325 The naming for a normal RPM package and a Multilib RPM package in a
4326 <filename>qemux86-64</filename> system resolves to something similar to
4327 <filename>bash-4.1-r2.x86_64.rpm</filename> and
4328 <filename>bash-4.1.r2.lib32_x86.rpm</filename>, respectively.
4329 </para></listitem>
4330 <listitem><para>When installing a Multilib image, the RPM backend first
4331 installs the base image and then installs the Multilib libraries.
4332 </para></listitem>
4333 <listitem><para>The build system relies on RPM to resolve the identical files in the
4334 two (or more) Multilib packages.</para></listitem>
4335 </itemizedlist>
4336 </para>
4337
4338 <para>
4339 For the IPK Package Management System, the following implementation details exist:
4340 <itemizedlist>
4341 <listitem><para>The <filename>${MLPREFIX}</filename> is not stripped from
4342 <filename>${PN}</filename> during IPK packaging.
4343 The naming for a normal RPM package and a Multilib IPK package in a
4344 <filename>qemux86-64</filename> system resolves to something like
4345 <filename>bash_4.1-r2.x86_64.ipk</filename> and
4346 <filename>lib32-bash_4.1-rw_x86.ipk</filename>, respectively.
4347 </para></listitem>
4348 <listitem><para>The IPK deploy folder is not modified with
4349 <filename>${MLPREFIX}</filename> because packages with and without
4350 the Multilib feature can exist in the same folder due to the
4351 <filename>${PN}</filename> differences.</para></listitem>
4352 <listitem><para>IPK defines a sanity check for Multilib installation
4353 using certain rules for file comparison, overridden, etc.
4354 </para></listitem>
4355 </itemizedlist>
4356 </para>
4357 </section>
4358 </section>
4359
4360 <section id='installing-multiple-versions-of-the-same-library'>
4361 <title>Installing Multiple Versions of the Same Library</title>
4362
4363 <para>
4364 Situations can exist where you need to install and use
4365 multiple versions of the same library on the same system
4366 at the same time.
4367 These situations almost always exist when a library API
4368 changes and you have multiple pieces of software that
4369 depend on the separate versions of the library.
4370 To accommodate these situations, you can install multiple
4371 versions of the same library in parallel on the same system.
4372 </para>
4373
4374 <para>
4375 The process is straightforward as long as the libraries use
4376 proper versioning.
4377 With properly versioned libraries, all you need to do to
4378 individually specify the libraries is create separate,
4379 appropriately named recipes where the
4380 <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink> part of the
4381 name includes a portion that differentiates each library version
4382 (e.g.the major part of the version number).
4383 Thus, instead of having a single recipe that loads one version
4384 of a library (e.g. <filename>clutter</filename>), you provide
4385 multiple recipes that result in different versions
4386 of the libraries you want.
4387 As an example, the following two recipes would allow the
4388 two separate versions of the <filename>clutter</filename>
4389 library to co-exist on the same system:
4390 <literallayout class='monospaced'>
4391 clutter-1.6_1.6.20.bb
4392 clutter-1.8_1.8.4.bb
4393 </literallayout>
4394 Additionally, if you have other recipes that depend on a given
4395 library, you need to use the
4396 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
4397 variable to create the dependency.
4398 Continuing with the same example, if you want to have a recipe
4399 depend on the 1.8 version of the <filename>clutter</filename>
4400 library, use the following in your recipe:
4401 <literallayout class='monospaced'>
4402 DEPENDS = "clutter-1.8"
4403 </literallayout>
4404 </para>
4405 </section>
4406 </section>
4407
Patrick Williamsc0f7c042017-02-23 20:41:17 -06004408 <section id='enabling-gobject-introspection-support'>
4409 <title>Enabling GObject Introspection Support</title>
4410
4411 <para>
4412 <ulink url='https://wiki.gnome.org/Projects/GObjectIntrospection'>GObject introspection</ulink>
4413 is the standard mechanism for accessing GObject-based software
4414 from runtime environments.
4415 GObject is a feature of the GLib library that provides an object
4416 framework for the GNOME desktop and related software.
4417 GObject Introspection adds information to GObject that allows
4418 objects created within it to be represented across different
4419 programming languages.
4420 If you want to construct GStreamer pipelines using Python, or
4421 control UPnP infrastructure using Javascript and GUPnP,
4422 GObject introspection is the only way to do it.
4423 </para>
4424
4425 <para>
4426 This section describes the Yocto Project support for generating
4427 and packaging GObject introspection data.
4428 GObject introspection data is a description of the
4429 API provided by libraries built on top of GLib framework,
4430 and, in particular, that framework's GObject mechanism.
4431 GObject Introspection Repository (GIR) files go to
4432 <filename>-dev</filename> packages,
4433 <filename>typelib</filename> files go to main packages as they
4434 are packaged together with libraries that are introspected.
4435 </para>
4436
4437 <para>
4438 The data is generated when building such a library, by linking
4439 the library with a small executable binary that asks the library
4440 to describe itself, and then executing the binary and
4441 processing its output.
4442 </para>
4443
4444 <para>
4445 Generating this data in a cross-compilation environment
4446 is difficult because the library is produced for the target
4447 architecture, but its code needs to be executed on the build host.
4448 This problem is solved with the OpenEmbedded build system by
4449 running the code through QEMU, which allows precisely that.
4450 Unfortunately, QEMU does not always work perfectly as mentioned
4451 in the xxx section.
4452 </para>
4453
4454 <section id='enabling-the-generation-of-introspection-data'>
4455 <title>Enabling the Generation of Introspection Data</title>
4456
4457 <para>
4458 Enabling the generation of introspection data (GIR files)
4459 in your library package involves the following:
4460 <orderedlist>
4461 <listitem><para>
4462 Inherit the
4463 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-gobject-introspection'><filename>gobject-introspection</filename></ulink>
4464 class.
4465 </para></listitem>
4466 <listitem><para>
4467 Make sure introspection is not disabled anywhere in
4468 the recipe or from anything the recipe includes.
4469 Also, make sure that "gobject-introspection-data" is
4470 not in
4471 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED'><filename>DISTRO_FEATURES_BACKFILL_CONSIDERED</filename></ulink>
4472 and that "qemu-usermode" is not in
4473 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES_BACKFILL_CONSIDERED'><filename>MACHINE_FEATURES_BACKFILL_CONSIDERED</filename></ulink>.
4474 If either of these conditions exist, nothing will
4475 happen.
4476 </para></listitem>
4477 <listitem><para>
4478 Try to build the recipe.
4479 If you encounter build errors that look like
4480 something is unable to find
4481 <filename>.so</filename> libraries, check where these
4482 libraries are located in the source tree and add
4483 the following to the recipe:
4484 <literallayout class='monospaced'>
4485 GIR_EXTRA_LIBS_PATH = "${B}/<replaceable>something</replaceable>/.libs"
4486 </literallayout>
4487 <note>
4488 See recipes in the <filename>oe-core</filename>
4489 repository that use that
4490 <filename>GIR_EXTRA_LIBS_PATH</filename> variable
4491 as an example.
4492 </note>
4493 </para></listitem>
4494 <listitem><para>
4495 Look for any other errors, which probably mean that
4496 introspection support in a package is not entirely
4497 standard, and thus breaks down in a cross-compilation
4498 environment.
4499 For such cases, custom-made fixes are needed.
4500 A good place to ask and receive help in these cases
4501 is the
4502 <ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Yocto Project mailing lists</ulink>.
4503 </para></listitem>
4504 </orderedlist>
4505 <note>
4506 Using a library that no longer builds against the latest
4507 Yocto Project release and prints introspection related
4508 errors is a good candidate for the previous procedure.
4509 </note>
4510 </para>
4511 </section>
4512
4513 <section id='disabling-the-generation-of-introspection-data'>
4514 <title>Disabling the Generation of Introspection Data</title>
4515
4516 <para>
4517 You might find that you do not want to generate
4518 introspection data.
4519 Or, perhaps QEMU does not work on your build host and
4520 target architecture combination.
4521 If so, you can use either of the following methods to
4522 disable GIR file generations:
4523 <itemizedlist>
4524 <listitem><para>
4525 Add the following to your distro configuration:
4526 <literallayout class='monospaced'>
4527 DISTRO_FEATURES_BACKFILL_CONSIDERED = "gobject-introspection-data"
4528 </literallayout>
4529 Adding this statement disables generating
4530 introspection data using QEMU but will still enable
4531 building introspection tools and libraries
4532 (i.e. building them does not require the use of QEMU).
4533 </para></listitem>
4534 <listitem><para>
4535 Add the following to your machine configuration:
4536 <literallayout class='monospaced'>
4537 MACHINE_FEATURES_BACKFILL_CONSIDERED = "qemu-usermode"
4538 </literallayout>
4539 Adding this statement disables the use of QEMU
4540 when building packages for your machine.
4541 Currently, this feature is used only by introspection
4542 recipes and has the same effect as the previously
4543 described option.
4544 <note>
4545 Future releases of the Yocto Project might have
4546 other features affected by this option.
4547 </note>
4548 </para></listitem>
4549 </itemizedlist>
4550 If you disable introspection data, you can still
4551 obtain it through other means such as copying the data
4552 from a suitable sysroot, or by generating it on the
4553 target hardware.
4554 The OpenEmbedded build system does not currently
4555 provide specific support for these techniques.
4556 </para>
4557 </section>
4558
4559 <section id='testing-that-introspection-works-in-an-image'>
4560 <title>Testing that Introspection Works in an Image</title>
4561
4562 <para>
4563 Use the following procedure to test if generating
4564 introspection data is working in an image:
4565 <orderedlist>
4566 <listitem><para>
4567 Make sure that "gobject-introspection-data" is not in
4568 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED'><filename>DISTRO_FEATURES_BACKFILL_CONSIDERED</filename></ulink>
4569 and that "qemu-usermode" is not in
4570 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES_BACKFILL_CONSIDERED'><filename>MACHINE_FEATURES_BACKFILL_CONSIDERED</filename></ulink>.
4571 </para></listitem>
4572 <listitem><para>
4573 Build <filename>core-image-sato</filename>.
4574 </para></listitem>
4575 <listitem><para>
4576 Launch a Terminal and then start Python in the
4577 terminal.
4578 </para></listitem>
4579 <listitem><para>
4580 Enter the following in the terminal:
4581 <literallayout class='monospaced'>
4582 >>> from gi.repository import GLib
4583 >>> GLib.get_host_name()
4584 </literallayout>
4585 </para></listitem>
4586 <listitem><para>
4587 For something a little more advanced, enter the
4588 following:
4589 <literallayout class='monospaced'>
4590 http://python-gtk-3-tutorial.readthedocs.org/en/latest/introduction.html
4591 </literallayout>
4592 </para></listitem>
4593 </orderedlist>
4594 </para>
4595 </section>
4596
4597 <section id='known-issues'>
4598 <title>Known Issues</title>
4599
4600 <para>
4601 The following know issues exist for
4602 GObject Introspection Support:
4603 <itemizedlist>
4604 <listitem><para>
4605 <filename>qemu-ppc64</filename> immediately crashes.
4606 Consequently, you cannot build introspection data on
4607 that architecture.
4608 </para></listitem>
4609 <listitem><para>
4610 x32 is not supported by QEMU.
4611 Consequently, introspection data is disabled.
4612 </para></listitem>
4613 <listitem><para>
4614 musl causes transient GLib binaries to crash on
4615 assertion failures.
4616 Consequently, generating introspection data is
4617 disabled.
4618 </para></listitem>
4619 <listitem><para>
4620 Because QEMU is not able to run the binaries correctly,
4621 introspection is disabled for some specific packages
4622 under specific architectures (e.g.
4623 <filename>gcr</filename>,
4624 <filename>libsecret</filename>, and
4625 <filename>webkit</filename>).
4626 </para></listitem>
4627 <listitem><para>
4628 QEMU usermode might not work properly when running
4629 64-bit binaries under 32-bit host machines.
4630 In particular, "qemumips64" is known to not work under
4631 i686.
4632 </para></listitem>
4633 </itemizedlist>
4634 </para>
4635 </section>
4636 </section>
4637
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05004638 <section id='dev-optionally-using-an-external-toolchain'>
4639 <title>Optionally Using an External Toolchain</title>
4640
4641 <para>
4642 You might want to use an external toolchain as part of your
4643 development.
4644 If this is the case, the fundamental steps you need to accomplish
4645 are as follows:
4646 <itemizedlist>
4647 <listitem><para>
4648 Understand where the installed toolchain resides.
4649 For cases where you need to build the external toolchain,
4650 you would need to take separate steps to build and install
4651 the toolchain.
4652 </para></listitem>
4653 <listitem><para>
4654 Make sure you add the layer that contains the toolchain to
4655 your <filename>bblayers.conf</filename> file through the
4656 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBLAYERS'><filename>BBLAYERS</filename></ulink>
4657 variable.
4658 </para></listitem>
4659 <listitem><para>
4660 Set the
4661 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNAL_TOOLCHAIN'><filename>EXTERNAL_TOOLCHAIN</filename></ulink>
4662 variable in your <filename>local.conf</filename> file
4663 to the location in which you installed the toolchain.
4664 </para></listitem>
4665 </itemizedlist>
4666 A good example of an external toolchain used with the Yocto Project
4667 is <trademark class='registered'>Mentor Graphics</trademark>
4668 Sourcery G++ Toolchain.
4669 You can see information on how to use that particular layer in the
4670 <filename>README</filename> file at
4671 <ulink url='http://github.com/MentorEmbedded/meta-sourcery/'></ulink>.
4672 You can find further information by reading about the
4673 <ulink url='&YOCTO_DOCS_REF_URL;#var-TCMODE'><filename>TCMODE</filename></ulink>
4674 variable in the Yocto Project Reference Manual's variable glossary.
4675 </para>
4676 </section>
4677
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004678 <section id='creating-partitioned-images'>
4679 <title>Creating Partitioned Images</title>
4680
4681 <para>
4682 Creating an image for a particular hardware target using the
4683 OpenEmbedded build system does not necessarily mean you can boot
4684 that image as is on your device.
4685 Physical devices accept and boot images in various ways depending
4686 on the specifics of the device.
4687 Usually, information about the hardware can tell you what image
4688 format the device requires.
4689 Should your device require multiple partitions on an SD card, flash,
4690 or an HDD, you can use the OpenEmbedded Image Creator,
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004691 Wic, to create the properly partitioned image.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004692 </para>
4693
4694 <para>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004695 You can generate partitioned images
4696 (<replaceable>image</replaceable><filename>.wic</filename>)
4697 two ways: using the OpenEmbedded build system and by running
4698 the OpenEmbedded Image Creator Wic directly.
4699 The former way is preferable as it is easier to use and understand.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004700 </para>
4701
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004702 <section id='creating-wic-images-oe'>
4703 <title>Creating Partitioned Images</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004704
4705 <para>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004706 The OpenEmbedded build system can generate
4707 partitioned images the same way as it generates
4708 any other image type.
4709 To generate a partitioned image, you need to modify
4710 two variables.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004711 <itemizedlist>
4712 <listitem><para>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004713 Include "wic" as part of the
4714 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></ulink>
4715 variable.
4716 </para></listitem>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004717 <listitem><para>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004718 Include the name of the
4719 <link linkend='openembedded-kickstart-wks-reference'>wic kickstart file</link>
4720 as part of the
4721 <ulink url='&YOCTO_DOCS_REF_URL;#var-WKS_FILE'><filename>WKS_FILE</filename></ulink>
4722 variable
4723 </para></listitem>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004724 </itemizedlist>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004725 Further steps to generate a partitioned image
4726 are the same as for any other image type.
4727 For information on image types, see the
4728 "<link linkend='building-images'>Building Images</link>"
4729 section.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004730 </para>
4731 </section>
4732
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004733 <section id='create-wic-images-wic'>
4734 <title>Using OpenEmbedded Image Creator Wic to Generate Partitioned Images</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004735
4736 <para>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004737 The <filename>wic</filename> command generates partitioned
4738 images from existing OpenEmbedded build artifacts.
4739 Image generation is driven by partitioning commands
4740 contained in an Openembedded kickstart file
4741 (<filename>.wks</filename>) specified either directly on
4742 the command line or as one of a selection of canned
4743 <filename>.wks</filename> files as shown with the
4744 <filename>wic list images</filename> command in the
4745 "<link linkend='using-a-provided-kickstart-file'>Using an Existing Kickstart File</link>"
4746 section.
4747 When you apply the command to a given set of build
4748 artifacts, the result is an image or set of images that
4749 can be directly written onto media and used on a particular
4750 system.
4751 </para>
4752
4753 <para>
4754 The <filename>wic</filename> command and the infrastructure
4755 it is based on is by definition incomplete.
4756 The purpose of the command is to allow the generation of
4757 customized images, and as such, was designed to be
4758 completely extensible through a plug-in interface.
4759 See the
4760 "<link linkend='openembedded-kickstart-plugins'>Plug-ins</link>"
4761 section for information on these plug-ins.
4762 </para>
4763
4764 <para>
4765 This section provides some background information on Wic,
4766 describes what you need to have in
4767 place to run the tool, provides instruction on how to use
4768 the <filename>wic</filename> utility,
4769 and provides several examples.
4770 </para>
4771
4772 <section id='wic-background'>
4773 <title>Background</title>
4774
4775 <para>
4776 This section provides some background on the
4777 <filename>wic</filename> utility.
4778 While none of this information is required to use
4779 Wic, you might find it interesting.
4780 <itemizedlist>
4781 <listitem><para>
4782 The name "Wic" is derived from OpenEmbedded
4783 Image Creator (oeic).
4784 The "oe" diphthong in "oeic" was promoted to the
4785 letter "w", because "oeic" is both difficult to
4786 remember and to pronounce.
4787 </para></listitem>
4788 <listitem><para>
4789 Wic is loosely based on the
4790 Meego Image Creator (<filename>mic</filename>)
4791 framework.
4792 The Wic implementation has been
4793 heavily modified to make direct use of OpenEmbedded
4794 build artifacts instead of package installation and
4795 configuration, which are already incorporated within
4796 the OpenEmbedded artifacts.
4797 </para></listitem>
4798 <listitem><para>
4799 Wic is a completely independent
4800 standalone utility that initially provides
Brad Bishop6e60e8b2018-02-01 10:27:11 -05004801 easier-to-use and more flexible replacements for an
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004802 existing functionality in OE Core's
4803 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-image-live'><filename>image-live</filename></ulink>
4804 class and <filename>mkefidisk.sh</filename> script.
4805 The difference between
4806 Wic and those examples is
4807 that with Wic the
4808 functionality of those scripts is implemented
4809 by a general-purpose partitioning language, which is
4810 based on Redhat kickstart syntax.</para></listitem>
4811 </itemizedlist>
4812 </para>
4813 </section>
4814
4815 <section id='wic-requirements'>
4816 <title>Requirements</title>
4817
4818 <para>
4819 In order to use the <filename>wic</filename> utility
4820 with the OpenEmbedded Build system, your system needs
4821 to meet the following requirements:
4822 <itemizedlist>
4823 <listitem><para>The Linux distribution on your
4824 development host must support the Yocto Project.
4825 See the
4826 "<ulink url='&YOCTO_DOCS_REF_URL;#detailed-supported-distros'>Supported Linux Distributions</ulink>"
4827 section in the Yocto Project Reference Manual for
4828 the list of distributions that support the
4829 Yocto Project.
4830 </para></listitem>
4831 <listitem><para>
4832 The standard system utilities, such as
4833 <filename>cp</filename>, must be installed on your
4834 development host system.
4835 </para></listitem>
4836 <listitem><para>
4837 You need to have the build artifacts already
4838 available, which typically means that you must
4839 have already created an image using the
4840 Openembedded build system (e.g.
4841 <filename>core-image-minimal</filename>).
4842 While it might seem redundant to generate an image
4843 in order to create an image using
4844 Wic, the current version of
4845 Wic requires the artifacts
4846 in the form generated by the build system.
4847 </para></listitem>
4848 <listitem><para>
4849 You must build several native tools, which are tools
4850 built to run on the build system:
4851 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004852 $ bitbake parted-native dosfstools-native mtools-native
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004853 </literallayout>
4854 </para></listitem>
4855 <listitem><para>
4856 You must have sourced one of the build environment
4857 setup scripts (i.e.
4858 <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
4859 or
4860 <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>)
4861 found in the
4862 <link linkend='build-directory'>Build Directory</link>.
4863 </para></listitem>
4864 </itemizedlist>
4865 </para>
4866 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004867
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004868 <section id='wic-getting-help'>
4869 <title>Getting Help</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004870
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004871 <para>
4872 You can get general help for the <filename>wic</filename>
4873 command by entering the <filename>wic</filename> command
4874 by itself or by entering the command with a help argument
4875 as follows:
4876 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004877 $ wic -h
4878 $ wic --help
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004879 </literallayout>
4880 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004881
4882 <para>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004883 Currently, Wic supports two commands:
4884 <filename>create</filename> and <filename>list</filename>.
4885 You can get help for these commands as follows:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004886 <literallayout class='monospaced'>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004887 $ wic help <replaceable>command</replaceable>
4888 with <replaceable>command</replaceable> being either
4889 <filename>create</filename> or <filename>list</filename>.
4890 </literallayout>
4891 </para>
4892
4893 <para>
4894 You can also get detailed help on a number of topics
4895 from the help system.
4896 The output of <filename>wic --help</filename>
4897 displays a list of available help
4898 topics under a "Help topics" heading.
4899 You can have the help system display the help text for
4900 a given topic by prefacing the topic with
4901 <filename>wic help</filename>:
4902 <literallayout class='monospaced'>
4903 $ wic help <replaceable>help_topic</replaceable>
4904 </literallayout>
4905 </para>
4906
4907 <para>
4908 You can find out more about the images
4909 Wic creates using the existing
4910 kickstart files with the following form of the command:
4911 <literallayout class='monospaced'>
4912 $ wic list <replaceable>image</replaceable> help
4913 </literallayout>
4914 with <filename><replaceable>image</replaceable></filename>
4915 being either <filename>directdisk</filename> or
4916 <filename>mkefidisk</filename>.
4917 </para>
4918 </section>
4919
4920 <section id='operational-modes'>
4921 <title>Operational Modes</title>
4922
4923 <para>
4924 You can use Wic in two different
4925 modes, depending on how much control you need for
4926 specifying the Openembedded build artifacts that are
4927 used for creating the image: Raw and Cooked:
4928 <itemizedlist>
4929 <listitem><para>
4930 <emphasis>Raw Mode:</emphasis>
4931 You explicitly specify build artifacts through
4932 command-line arguments.
4933 </para></listitem>
4934 <listitem><para>
4935 <emphasis>Cooked Mode:</emphasis>
4936 The current
4937 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
4938 setting and image name are used to automatically
4939 locate and provide the build artifacts.
4940 </para></listitem>
4941 </itemizedlist>
4942 </para>
4943
4944 <para>
4945 Regardless of the mode you use, you need to have the build
4946 artifacts ready and available.
4947 Additionally, the environment must be set up using the
4948 <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
4949 or
4950 <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>
4951 script found in the
4952 <link linkend='build-directory'>Build Directory</link>.
4953 </para>
4954
4955 <section id='raw-mode'>
4956 <title>Raw Mode</title>
4957
4958 <para>
4959 The general form of the
4960 <filename>wic</filename> command in raw mode is:
4961 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004962 $ wic create <replaceable>image_name</replaceable>.wks [<replaceable>options</replaceable>] [...]
4963
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004964 Where:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004965
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004966 <replaceable>image_name</replaceable>.wks
4967 An OpenEmbedded kickstart file. You can provide
4968 your own custom file or use a file from a set of
4969 existing files as described by further options.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004970
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004971 -o <replaceable>OUTDIR</replaceable>, --outdir=<replaceable>OUTDIR</replaceable>
4972 The name of a directory in which to create image.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004973
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004974 -i <replaceable>PROPERTIES_FILE</replaceable>, --infile=<replaceable>PROPERTIES_FILE</replaceable>
4975 The name of a file containing the values for image
4976 properties as a JSON file.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004977
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004978 -e <replaceable>IMAGE_NAME</replaceable>, --image-name=<replaceable>IMAGE_NAME</replaceable>
4979 The name of the image from which to use the artifacts
4980 (e.g. <filename>core-image-sato</filename>).
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004981
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004982 -r <replaceable>ROOTFS_DIR</replaceable>, --rootfs-dir=<replaceable>ROOTFS_DIR</replaceable>
4983 The path to the <filename>/rootfs</filename> directory to use as the
4984 <filename>.wks</filename> rootfs source.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004985
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004986 -b <replaceable>BOOTIMG_DIR</replaceable>, --bootimg-dir=<replaceable>BOOTIMG_DIR</replaceable>
4987 The path to the directory containing the boot artifacts
4988 (e.g. <filename>/EFI</filename> or <filename>/syslinux</filename>) to use as the <filename>.wks</filename> bootimg
4989 source.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004990
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004991 -k <replaceable>KERNEL_DIR</replaceable>, --kernel-dir=<replaceable>KERNEL_DIR</replaceable>
4992 The path to the directory containing the kernel to use
4993 in the <filename>.wks</filename> boot image.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004994
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004995 -n <replaceable>NATIVE_SYSROOT</replaceable>, --native-sysroot=<replaceable>NATIVE_SYSROOT</replaceable>
4996 The path to the native sysroot containing the tools to use
4997 to build the image.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05004998
Brad Bishop37a0e4d2017-12-04 01:01:44 -05004999 -s, --skip-build-check
5000 Skips the build check.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005001
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005002 -D, --debug
5003 Output debug information.
5004 </literallayout>
5005 <note>
5006 You do not need root privileges to run
5007 Wic.
5008 In fact, you should not run as root when using the
5009 utility.
5010 </note>
5011 </para>
5012 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005013
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005014 <section id='cooked-mode'>
5015 <title>Cooked Mode</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005016
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005017 <para>
5018 The general form of the <filename>wic</filename> command
5019 using Cooked Mode is:
5020 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005021 $ wic create <replaceable>kickstart_file</replaceable> -e <replaceable>image_name</replaceable>
5022
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005023 Where:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005024
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005025 <replaceable>kickstart_file</replaceable>
5026 An OpenEmbedded kickstart file. You can provide your own
5027 custom file or a supplied file.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005028
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005029 <replaceable>image_name</replaceable>
5030 Specifies the image built using the OpenEmbedded build
5031 system.
5032 </literallayout>
5033 This form is the simplest and most user-friendly, as it
5034 does not require specifying all individual parameters.
5035 All you need to provide is your own
5036 <filename>.wks</filename> file or one provided with the
5037 release.
5038 </para>
5039 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005040 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005041
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005042 <section id='using-a-provided-kickstart-file'>
5043 <title>Using an Existing Kickstart File</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005044
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005045 <para>
5046 If you do not want to create your own
5047 <filename>.wks</filename> file, you can use an existing
5048 file provided by the Wic installation.
5049 Use the following command to list the available files:
5050 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005051 $ wic list images
5052 directdisk Create a 'pcbios' direct disk image
5053 mkefidisk Create an EFI disk image
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005054 </literallayout>
5055 When you use an existing file, you do not have to use the
5056 <filename>.wks</filename> extension.
5057 Here is an example in Raw Mode that uses the
5058 <filename>directdisk</filename> file:
5059 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005060 $ wic create directdisk -r <replaceable>rootfs_dir</replaceable> -b <replaceable>bootimg_dir</replaceable> \
5061 -k <replaceable>kernel_dir</replaceable> -n <replaceable>native_sysroot</replaceable>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005062 </literallayout>
5063 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005064
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005065 <para>
5066 Here are the actual partition language commands
5067 used in the <filename>mkefidisk.wks</filename> file to
5068 generate an image:
5069 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005070 # short-description: Create an EFI disk image
5071 # long-description: Creates a partitioned EFI disk image that the user
5072 # can directly dd to boot media.
5073
5074 part /boot --source bootimg-efi --ondisk sda --label msdos --active --align 1024
5075
5076 part / --source rootfs --ondisk sda --fstype=ext3 --label platform --align 1024
5077
5078 part swap --ondisk sda --size 44 --label swap1 --fstype=swap
5079
5080 bootloader --timeout=10 --append="rootwait rootfstype=ext3 console=ttyPCH0,115200 console=tty0 vmalloc=256MB snd-hda-intel.enable_msi=0"
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005081 </literallayout>
5082 </para>
5083 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005084
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005085 <section id='wic-usage-examples'>
5086 <title>Examples</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005087
5088 <para>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005089 This section provides several examples that show how to use
5090 the <filename>wic</filename> utility.
5091 All the examples assume the list of requirements in the
5092 "<link linkend='wic-requirements'>Requirements</link>"
5093 section have been met.
5094 The examples assume the previously generated image is
5095 <filename>core-image-minimal</filename>.
5096 </para>
5097
5098 <section id='generate-an-image-using-a-provided-kickstart-file'>
5099 <title>Generate an Image using an Existing Kickstart File</title>
5100
5101 <para>
5102 This example runs in Cooked Mode and uses the
5103 <filename>mkefidisk</filename> kickstart file:
5104 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005105 $ wic create mkefidisk -e core-image-minimal
5106 Checking basic build environment...
5107 Done.
5108
5109 Creating image(s)...
5110
5111 Info: The new image(s) can be found here:
Brad Bishop6e60e8b2018-02-01 10:27:11 -05005112 <replaceable>current_directory</replaceable>/build/mkefidisk-201310230946-sda.direct
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005113
5114 The following build artifacts were used to create the image(s):
5115 ROOTFS_DIR: /home/trz/yocto/yocto-image/build/tmp/work/minnow-poky-linux/core-image-minimal/1.0-r0/rootfs
5116 BOOTIMG_DIR: /home/trz/yocto/yocto-image/build/tmp/work/minnow-poky-linux/core-image-minimal/1.0-r0/core-image-minimal-1.0/hddimg
5117 KERNEL_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/minnow/usr/src/kernel
5118 NATIVE_SYSROOT: /home/trz/yocto/yocto-image/build/tmp/sysroots/x86_64-linux
5119
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005120 The image(s) were created using OE kickstart file:
5121 /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/mkefidisk.wks
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005122 </literallayout>
5123 The previous example shows the easiest way to create
5124 an image by running in Cooked Mode and using the
5125 <filename>-e</filename> option with an existing
5126 kickstart file.
5127 All that is necessary is to specify the image used to
5128 generate the artifacts.
5129 Your <filename>local.conf</filename> needs to have the
5130 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
5131 variable set to the machine you are using, which is
5132 "minnow" in this example.
5133 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005134
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005135 <para>
5136 The output specifies the exact image created as well as
Brad Bishop6e60e8b2018-02-01 10:27:11 -05005137 where it was created, which is in the current
5138 directory by default.
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005139 The output also names the artifacts used and the exact
5140 <filename>.wks</filename> script that was used to
5141 generate the image.
5142 <note>
5143 You should always verify the details provided in the
5144 output to make sure that the image was indeed
5145 created exactly as expected.
5146 </note>
5147 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005148
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005149 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05005150 Continuing with the example, you can now write the
5151 image to a USB stick, or whatever media for which you
5152 built your image, and boot the resulting media.
5153 You can write the image by using
5154 <filename>bmaptool</filename> or
5155 <filename>dd</filename>:
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005156 <literallayout class='monospaced'>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05005157 $ oe-run-native bmaptool copy build/mkefidisk-201310230946-sda.direct /dev/sd<replaceable>X</replaceable>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005158 </literallayout>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05005159 or
5160 <literallayout class='monospaced'>
5161 $ sudo dd if=build/mkefidisk-201310230946-sda.direct of=/dev/sd<replaceable>X</replaceable>
5162 </literallayout>
5163 <note>
5164 For more information on how to use the
5165 <filename>bmaptool</filename> to flash a device
5166 with an image, see the
5167 "<link linkend='flashing-images-using-bmaptool'>Flashing Images Using <filename>bmaptool</filename></link>"
5168 section.
5169 </note>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005170 </para>
5171 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005172
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005173 <section id='using-a-modified-kickstart-file'>
5174 <title>Using a Modified Kickstart File</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005175
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005176 <para>
5177 Because partitioned image creation is
5178 driven by the kickstart file, it is easy to affect
5179 image creation by changing the parameters in the file.
5180 This next example demonstrates that through modification
5181 of the <filename>directdisk</filename> kickstart file.
5182 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005183
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005184 <para>
5185 As mentioned earlier, you can use the command
5186 <filename>wic list images</filename> to show the list
5187 of existing kickstart files.
5188 The directory in which these files reside is
5189 <filename>scripts/lib/image/canned-wks/</filename>
5190 located in the
5191 <link linkend='source-directory'>Source Directory</link>.
5192 Because the available files reside in this directory,
5193 you can create and add your own custom files to the
5194 directory.
5195 Subsequent use of the
5196 <filename>wic list images</filename> command would then
5197 include your kickstart files.
5198 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005199
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005200 <para>
5201 In this example, the existing
5202 <filename>directdisk</filename> file already does most
5203 of what is needed.
5204 However, for the hardware in this example, the image
5205 will need to boot from <filename>sdb</filename> instead
5206 of <filename>sda</filename>, which is what the
5207 <filename>directdisk</filename> kickstart file uses.
5208 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005209
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005210 <para>
5211 The example begins by making a copy of the
5212 <filename>directdisk.wks</filename> file in the
5213 <filename>scripts/lib/image/canned-wks</filename>
5214 directory and then by changing the lines that specify
5215 the target disk from which to boot.
5216 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005217 $ cp /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/directdisk.wks \
5218 /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/directdisksdb.wks
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005219 </literallayout>
5220 Next, the example modifies the
5221 <filename>directdisksdb.wks</filename> file and changes
5222 all instances of "<filename>--ondisk sda</filename>"
5223 to "<filename>--ondisk sdb</filename>".
5224 The example changes the following two lines and leaves
5225 the remaining lines untouched:
5226 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005227 part /boot --source bootimg-pcbios --ondisk sdb --label boot --active --align 1024
5228 part / --source rootfs --ondisk sdb --fstype=ext3 --label platform --align 1024
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005229 </literallayout>
5230 Once the lines are changed, the example generates the
5231 <filename>directdisksdb</filename> image.
5232 The command points the process at the
5233 <filename>core-image-minimal</filename> artifacts for
5234 the Next Unit of Computing (nuc)
5235 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
5236 the <filename>local.conf</filename>.
5237 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005238 $ wic create directdisksdb -e core-image-minimal
5239 Checking basic build environment...
5240 Done.
5241
5242 Creating image(s)...
5243
5244 Info: The new image(s) can be found here:
Brad Bishop6e60e8b2018-02-01 10:27:11 -05005245 <replaceable>current_directory</replaceable>/build/directdisksdb-201310231131-sdb.direct
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005246
5247 The following build artifacts were used to create the image(s):
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005248
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005249 ROOTFS_DIR: /home/trz/yocto/yocto-image/build/tmp/work/nuc-poky-linux/core-image-minimal/1.0-r0/rootfs
5250 BOOTIMG_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/nuc/usr/share
5251 KERNEL_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/nuc/usr/src/kernel
5252 NATIVE_SYSROOT: /home/trz/yocto/yocto-image/build/tmp/sysroots/x86_64-linux
5253
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005254 The image(s) were created using OE kickstart file:
5255 /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/directdisksdb.wks
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005256 </literallayout>
5257 Continuing with the example, you can now directly
5258 <filename>dd</filename> the image to a USB stick, or
5259 whatever media for which you built your image,
5260 and boot the resulting media:
5261 <literallayout class='monospaced'>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05005262 $ sudo dd if=build/directdisksdb-201310231131-sdb.direct of=/dev/sdb
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005263 86018+0 records in
5264 86018+0 records out
5265 44041216 bytes (44 MB) copied, 13.0734 s, 3.4 MB/s
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005266 [trz at empanada tmp]$ sudo eject /dev/sdb
5267 </literallayout>
5268 </para>
5269 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005270
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005271 <section id='creating-an-image-based-on-core-image-minimal-and-crownbay-noemgd'>
5272 <title>Creating an Image Based on <filename>core-image-minimal</filename> and <filename>crownbay-noemgd</filename></title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005273
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005274 <para>
5275 This example creates an image based on
5276 <filename>core-image-minimal</filename> and a
5277 <filename>crownbay-noemgd</filename>
5278 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
5279 that works right out of the box.
5280 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005281 $ wic create directdisk -e core-image-minimal
5282
5283 Checking basic build environment...
5284 Done.
5285
5286 Creating image(s)...
5287
5288 Info: The new image(s) can be found here:
Brad Bishop6e60e8b2018-02-01 10:27:11 -05005289 <replaceable>current_directory</replaceable>/build/directdisk-201309252350-sda.direct
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005290
5291 The following build artifacts were used to create the image(s):
5292
5293 ROOTFS_DIR: /home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs
5294 BOOTIMG_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share
5295 KERNEL_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
5296 NATIVE_SYSROOT: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
5297
5298 The image(s) were created using OE kickstart file:
5299 /home/trz/yocto/yocto-image/scripts/lib/image/canned-wks/directdisk.wks
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005300 </literallayout>
5301 </para>
5302 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005303
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005304 <section id='using-a-modified-kickstart-file-and-running-in-raw-mode'>
5305 <title>Using a Modified Kickstart File and Running in Raw Mode</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005306
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005307 <para>
5308 This next example manually specifies each build artifact
5309 (runs in Raw Mode) and uses a modified kickstart file.
5310 The example also uses the <filename>-o</filename> option
5311 to cause Wic to create the output
Brad Bishop6e60e8b2018-02-01 10:27:11 -05005312 somewhere other than the default output directory,
5313 which is the current directory:
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005314 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005315 $ wic create ~/test.wks -o /home/trz/testwic --rootfs-dir \
5316 /home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs \
5317 --bootimg-dir /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share \
5318 --kernel-dir /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel \
5319 --native-sysroot /home/trz/yocto/yocto-image/build/tmp/sysroots/x86_64-linux
5320
5321 Creating image(s)...
5322
5323 Info: The new image(s) can be found here:
5324 /home/trz/testwic/build/test-201309260032-sda.direct
5325
5326 The following build artifacts were used to create the image(s):
5327
5328 ROOTFS_DIR: /home/trz/yocto/yocto-image/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs
5329 BOOTIMG_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/share
5330 KERNEL_DIR: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
5331 NATIVE_SYSROOT: /home/trz/yocto/yocto-image/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel
5332
5333 The image(s) were created using OE kickstart file:
5334 /home/trz/test.wks
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005335 </literallayout>
5336 For this example,
5337 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
5338 did not have to be specified in the
5339 <filename>local.conf</filename> file since the
5340 artifact is manually specified.
5341 </para>
5342 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005343 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005344
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005345 <section id='openembedded-kickstart-plugins'>
5346 <title>Plug-ins</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005347
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005348 <para>
5349 Plug-ins allow Wic functionality to
5350 be extended and specialized by users.
5351 This section documents the plug-in interface, which is
5352 currently restricted to source plug-ins.
5353 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005354
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005355 <para>
5356 Source plug-ins provide a mechanism to customize
5357 various aspects of the image generation process in
5358 Wic, mainly the contents of
5359 partitions.
5360 The plug-ins provide a mechanism for mapping values
5361 specified in <filename>.wks</filename> files using the
5362 <filename>--source</filename> keyword to a
5363 particular plug-in implementation that populates a
5364 corresponding partition.
Brad Bishop6e60e8b2018-02-01 10:27:11 -05005365 <note>
5366 If you use plug-ins that have build-time dependencies
5367 (e.g. native tools, bootloaders, and so forth)
5368 when building a Wic image, you need to specify those
5369 dependencies using the
5370 <ulink url='&YOCTO_DOCS_REF_URL;#var-WKS_FILE_DEPENDS'><filename>WKS_FILE_DEPENDS</filename></ulink>
5371 variable.
5372 </note>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005373 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005374
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005375 <para>
5376 A source plug-in is created as a subclass of
5377 <filename>SourcePlugin</filename>.
5378 The plug-in file containing it is added to
5379 <filename>scripts/lib/wic/plugins/source/</filename> to
5380 make the plug-in implementation available to the
5381 Wic implementation.
5382 For more information, see
5383 <filename>scripts/lib/wic/pluginbase.py</filename>.
5384 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005385
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005386 <para>
5387 Source plug-ins can also be implemented and added by
5388 external layers.
5389 As such, any plug-ins found in a
5390 <filename>scripts/lib/wic/plugins/source/</filename>
5391 directory in an external layer are also made
5392 available.
5393 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005394
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005395 <para>
5396 When the Wic implementation needs
5397 to invoke a partition-specific implementation, it looks
5398 for the plug-in that has the same name as the
5399 <filename>--source</filename> parameter given to
5400 that partition.
5401 For example, if the partition is set up as follows:
5402 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005403 part /boot --source bootimg-pcbios ...
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005404 </literallayout>
5405 The methods defined as class members of the plug-in
5406 having the matching <filename>bootimg-pcbios.name</filename>
5407 class member are used.
5408 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005409
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005410 <para>
5411 To be more concrete, here is the plug-in definition that
5412 matches a
5413 <filename>--source bootimg-pcbios</filename> usage,
5414 along with an example
5415 method called by the Wic implementation
5416 when it needs to invoke an implementation-specific
5417 partition-preparation function:
5418 <literallayout class='monospaced'>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005419 class BootimgPcbiosPlugin(SourcePlugin):
5420 name = 'bootimg-pcbios'
5421
5422 @classmethod
5423 def do_prepare_partition(self, part, ...)
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005424 </literallayout>
5425 If the subclass itself does not implement a function, a
5426 default version in a superclass is located and
5427 used, which is why all plug-ins must be derived from
5428 <filename>SourcePlugin</filename>.
5429 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005430
5431 <para>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005432 The <filename>SourcePlugin</filename> class defines the
5433 following methods, which is the current set of methods
5434 that can be implemented or overridden by
5435 <filename>--source</filename> plug-ins.
5436 Any methods not implemented by a
5437 <filename>SourcePlugin</filename> subclass inherit the
5438 implementations present in the
5439 <filename>SourcePlugin</filename> class.
5440 For more information, see the
5441 <filename>SourcePlugin</filename> source for details:
5442 </para>
5443
5444 <para>
5445 <itemizedlist>
5446 <listitem><para>
5447 <emphasis><filename>do_prepare_partition()</filename>:</emphasis>
5448 Called to do the actual content population for a
5449 partition.
5450 In other words, the method prepares the final
5451 partition image that is incorporated into the
5452 disk image.
5453 </para></listitem>
5454 <listitem><para>
5455 <emphasis><filename>do_configure_partition()</filename>:</emphasis>
5456 Called before
5457 <filename>do_prepare_partition()</filename>.
5458 This method is typically used to create custom
5459 configuration files for a partition (e.g. syslinux
5460 or grub configuration files).
5461 </para></listitem>
5462 <listitem><para>
5463 <emphasis><filename>do_install_disk()</filename>:</emphasis>
5464 Called after all partitions have been prepared and
5465 assembled into a disk image.
5466 This method provides a hook to allow finalization
5467 of a disk image, (e.g. writing an MBR).
5468 </para></listitem>
5469 <listitem><para>
5470 <emphasis><filename>do_stage_partition()</filename>:</emphasis>
5471 Special content-staging hook called before
5472 <filename>do_prepare_partition()</filename>.
5473 This method is normally empty.</para>
5474 <para>Typically, a partition just uses the passed-in
5475 parameters (e.g. the unmodified value of
5476 <filename>bootimg_dir</filename>).
5477 However, in some cases things might need to be
5478 more tailored.
5479 As an example, certain files might additionally
5480 need to be taken from
5481 <filename>bootimg_dir + /boot</filename>.
5482 This hook allows those files to be staged in a
5483 customized fashion.
5484 <note>
5485 <filename>get_bitbake_var()</filename>
5486 allows you to access non-standard variables
5487 that you might want to use for this.
5488 </note>
5489 </para></listitem>
5490 </itemizedlist>
5491 </para>
5492
5493 <para>
5494 This scheme is extensible.
5495 Adding more hooks is a simple matter of adding more
5496 plug-in methods to <filename>SourcePlugin</filename> and
5497 derived classes.
5498 The code that then needs to call the plug-in methods uses
5499 <filename>plugin.get_source_plugin_methods()</filename>
5500 to find the method or methods needed by the call.
5501 Retrieval of those methods is accomplished
5502 by filling up a dict with keys
5503 containing the method names of interest.
5504 On success, these will be filled in with the actual
5505 methods.
5506 Please see the Wic
5507 implementation for examples and details.
5508 </para>
5509 </section>
5510
5511 <section id='openembedded-kickstart-wks-reference'>
5512 <title>OpenEmbedded Kickstart (<filename>.wks</filename>) Reference</title>
5513
5514 <para>
5515 The current Wic implementation supports
5516 only the basic kickstart partitioning commands:
5517 <filename>partition</filename> (or <filename>part</filename>
5518 for short) and <filename>bootloader</filename>.
5519 <note>
5520 Future updates will implement more commands and options.
5521 If you use anything that is not specifically
5522 supported, results can be unpredictable.
5523 </note>
5524 </para>
5525
5526 <para>
5527 The following is a list of the commands, their syntax,
5528 and meanings.
5529 The commands are based on the Fedora
5530 kickstart versions but with modifications to
5531 reflect Wic capabilities.
5532 You can see the original documentation for those commands
5533 at the following links:
5534 <itemizedlist>
5535 <listitem><para>
5536 <ulink url='http://fedoraproject.org/wiki/Anaconda/Kickstart#part_or_partition'>http://fedoraproject.org/wiki/Anaconda/Kickstart#part_or_partition</ulink>
5537 </para></listitem>
5538 <listitem><para>
5539 <ulink url='http://fedoraproject.org/wiki/Anaconda/Kickstart#bootloader'>http://fedoraproject.org/wiki/Anaconda/Kickstart#bootloader</ulink>
5540 </para></listitem>
5541 </itemizedlist>
5542 </para>
5543
5544 <section id='command-part-or-partition'>
5545 <title>Command: part or partition</title>
5546
5547 <para>
5548 Either of these commands create a partition on the system
5549 and use the following syntax:
5550 <literallayout class='monospaced'>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05005551 part [<replaceable>mntpoint</replaceable>]
5552 partition [<replaceable>mntpoint</replaceable>]
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005553 </literallayout>
5554 If you do not provide
5555 <replaceable>mntpoint</replaceable>, Wic creates a
5556 partition but does not mount it.
5557 </para>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05005558
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005559 <para>
5560 The
5561 <filename><replaceable>mntpoint</replaceable></filename>
5562 is where the partition will be mounted and must be of
5563 one of the following forms:
5564 <itemizedlist>
5565 <listitem><para>
5566 <filename>/<replaceable>path</replaceable></filename>:
Brad Bishop6e60e8b2018-02-01 10:27:11 -05005567 For example, "/", "/usr", or "/home"
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005568 </para></listitem>
5569 <listitem><para>
5570 <filename>swap</filename>:
5571 The created partition is used as swap space.
5572 </para></listitem>
5573 </itemizedlist>
5574 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005575
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005576 <para>
5577 Specifying a <replaceable>mntpoint</replaceable> causes
5578 the partition to automatically be mounted.
5579 Wic achieves this by adding entries to the filesystem
5580 table (fstab) during image generation.
5581 In order for wic to generate a valid fstab, you must
5582 also provide one of the <filename>--ondrive</filename>,
5583 <filename>--ondisk</filename>, or
5584 <filename>--use-uuid</filename> partition options as
5585 part of the command.
5586 Here is an example using "/" as the mountpoint.
5587 The command uses "--ondisk" to force the partition onto
5588 the <filename>sdb</filename> disk:
5589 <literallayout class='monospaced'>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05005590 part / --source rootfs --ondisk sdb --fstype=ext3 --label platform --align 1024
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005591 </literallayout>
5592 </para>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05005593
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005594 <para>
5595 Here is a list that describes other supported options
5596 you can use with the <filename>part</filename> and
5597 <filename>partition</filename> commands:
5598 <itemizedlist>
5599 <listitem><para>
5600 <emphasis><filename>--size</filename>:</emphasis>
5601 The minimum partition size in MBytes.
5602 Specify an integer value such as 500.
5603 Do not append the number with "MB".
5604 You do not need this option if you use
5605 <filename>--source</filename>.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005606 </para></listitem>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005607 <listitem><para>
5608 <emphasis><filename>--source</filename>:</emphasis>
5609 This option is a
5610 Wic-specific option that
5611 names the source of the data that populates
5612 the partition.
5613 The most common value for this option is
5614 "rootfs", but you can use any value that maps to
5615 a valid source plug-in.
5616 For information on the source plug-ins, see the
5617 "<link linkend='openembedded-kickstart-plugins'>Plug-ins</link>"
5618 section.</para>
5619 <para>If you use
5620 <filename>--source rootfs</filename>,
5621 Wic creates a partition as
5622 large as needed and to fill it with the contents
5623 of the root filesystem pointed to by the
5624 <filename>-r</filename> command-line option
5625 or the equivalent rootfs derived from the
5626 <filename>-e</filename> command-line
5627 option.
5628 The filesystem type used to create the
5629 partition is driven by the value of the
5630 <filename>--fstype</filename> option
5631 specified for the partition.
5632 See the entry on
5633 <filename>--fstype</filename> that
5634 follows for more information.
5635 </para>
5636 <para>If you use
5637 <filename>--source <replaceable>plugin-name</replaceable></filename>,
5638 Wic creates a partition as
5639 large as needed and fills it with the contents
5640 of the partition that is generated by the
5641 specified plug-in name using the data pointed
5642 to by the <filename>-r</filename> command-line
5643 option or the equivalent rootfs derived from the
5644 <filename>-e</filename> command-line
5645 option.
5646 Exactly what those contents and filesystem type
5647 end up being are dependent on the given plug-in
5648 implementation.
5649 </para>
5650 <para>If you do not use the
5651 <filename>--source</filename> option, the
5652 <filename>wic</filename> command creates an
5653 empty partition.
5654 Consequently, you must use the
5655 <filename>--size</filename> option to specify
5656 the size of the empty partition.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005657 </para></listitem>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005658 <listitem><para>
5659 <emphasis><filename>--ondisk</filename> or <filename>--ondrive</filename>:</emphasis>
5660 Forces the partition to be created on a
5661 particular disk.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005662 </para></listitem>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005663 <listitem><para>
5664 <emphasis><filename>--fstype</filename>:</emphasis>
5665 Sets the file system type for the partition.
5666 Valid values are:
5667 <itemizedlist>
5668 <listitem><para><filename>ext4</filename>
5669 </para></listitem>
5670 <listitem><para><filename>ext3</filename>
5671 </para></listitem>
5672 <listitem><para><filename>ext2</filename>
5673 </para></listitem>
5674 <listitem><para><filename>btrfs</filename>
5675 </para></listitem>
5676 <listitem><para><filename>squashfs</filename>
5677 </para></listitem>
5678 <listitem><para><filename>swap</filename>
5679 </para></listitem>
5680 </itemizedlist>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005681 </para></listitem>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005682 <listitem><para>
5683 <emphasis><filename>--fsoptions</filename>:</emphasis>
5684 Specifies a free-form string of options to be
5685 used when mounting the filesystem.
5686 This string will be copied into the
5687 <filename>/etc/fstab</filename> file of the
5688 installed system and should be enclosed in
5689 quotes.
5690 If not specified, the default string
5691 is "defaults".
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005692 </para></listitem>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005693 <listitem><para>
5694 <emphasis><filename>--label label</filename>:</emphasis>
5695 Specifies the label to give to the filesystem to
5696 be made on the partition.
5697 If the given label is already in use by another
5698 filesystem, a new label is created for the
5699 partition.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005700 </para></listitem>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005701 <listitem><para>
5702 <emphasis><filename>--active</filename>:</emphasis>
5703 Marks the partition as active.
5704 </para></listitem>
5705 <listitem><para>
5706 <emphasis><filename>--align (in KBytes)</filename>:</emphasis>
5707 This option is a
5708 Wic-specific option that
5709 says to start a partition on an
5710 <replaceable>x</replaceable> KBytes
5711 boundary.</para></listitem>
5712 <listitem><para>
5713 <emphasis><filename>--no-table</filename>:</emphasis>
5714 This option is a
5715 Wic-specific option.
5716 Using the option reserves space for the
5717 partition and causes it to become populated.
5718 However, the partition is not added to the
5719 partition table.
5720 </para></listitem>
5721 <listitem><para>
5722 <emphasis><filename>--extra-space</filename>:</emphasis>
5723 This option is a
5724 Wic-specific option that
5725 adds extra space after the space filled by the
5726 content of the partition.
5727 The final size can go beyond the size specified
5728 by the <filename>--size</filename> option.
5729 The default value is 10 Mbytes.
5730 </para></listitem>
5731 <listitem><para>
5732 <emphasis><filename>--overhead-factor</filename>:</emphasis>
5733 This option is a
5734 Wic-specific option that
5735 multiplies the size of the partition by the
5736 option's value.
5737 You must supply a value greater than or equal to
5738 "1".
5739 The default value is "1.3".
5740 </para></listitem>
5741 <listitem><para>
5742 <emphasis><filename>--part-type</filename>:</emphasis>
5743 This option is a
5744 Wic-specific option that
5745 specifies the partition type globally
5746 unique identifier (GUID) for GPT partitions.
5747 You can find the list of partition type GUIDs
5748 at
5749 <ulink url='http://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_type_GUIDs'></ulink>.
5750 </para></listitem>
5751 <listitem><para>
5752 <emphasis><filename>--use-uuid</filename>:</emphasis>
5753 This option is a
5754 Wic-specific option that
5755 causes Wic to generate a
5756 random GUID for the partition.
5757 The generated identifier is used in the
5758 bootloader configuration to specify the root
5759 partition.
5760 </para></listitem>
5761 <listitem><para>
5762 <emphasis><filename>--uuid</filename>:</emphasis>
5763 This option is a
5764 Wic-specific
5765 option that specifies the partition UUID.
5766 </para></listitem>
5767 </itemizedlist>
5768 </para>
5769 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005770
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005771 <section id='command-bootloader'>
5772 <title>Command: bootloader</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005773
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005774 <para>
5775 This command specifies how the bootloader should be
5776 configured and supports the following options:
5777 <note>
5778 Bootloader functionality and boot partitions are
5779 implemented by the various
5780 <filename>--source</filename>
5781 plug-ins that implement bootloader functionality.
5782 The bootloader command essentially provides a
5783 means of modifying bootloader configuration.
5784 </note>
5785 <itemizedlist>
5786 <listitem><para>
5787 <emphasis><filename>--timeout</filename>:</emphasis>
5788 Specifies the number of seconds before the
5789 bootloader times out and boots the default
5790 option.
5791 </para></listitem>
5792 <listitem><para>
5793 <emphasis><filename>--append</filename>:</emphasis>
5794 Specifies kernel parameters.
5795 These parameters will be added to the syslinux
5796 <filename>APPEND</filename> or
5797 <filename>grub</filename> kernel command line.
5798 </para></listitem>
5799 <listitem><para>
5800 <emphasis><filename>--configfile</filename>:</emphasis>
5801 Specifies a user-defined configuration file for
5802 the bootloader.
5803 You can provide a full pathname for the file or
5804 a file that exists in the
5805 <filename>canned-wks</filename> folder.
5806 This option overrides all other bootloader
5807 options.
5808 </para></listitem>
5809 </itemizedlist>
5810 </para>
5811 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005812 </section>
5813 </section>
5814 </section>
5815
Brad Bishop37a0e4d2017-12-04 01:01:44 -05005816 <section id='building-an-initramfs-image'>
5817 <title>Building an Initial RAM Filesystem (initramfs) Image</title>
5818
5819 <para>
5820 initramfs is the successor of Initial RAM Disk (initrd).
5821 It is a "copy in and out" (cpio) archive of the initial file system
5822 that gets loaded into memory during the Linux startup process.
5823 Because Linux uses the contents of the archive during
5824 initialization, the initramfs needs to contain all of the device
5825 drivers and tools needed to mount the final root filesystem.
5826 </para>
5827
5828 <para>
5829 To build an initramfs image and bundle it into the kernel, set the
5830 <ulink url='&YOCTO_DOCS_REF_URL;#var-INITRAMFS_IMAGE_BUNDLE'><filename>INITRAMFS_IMAGE_BUNDLE</filename></ulink>
5831 variable in your <filename>local.conf</filename> file, and set the
5832 <ulink url='&YOCTO_DOCS_REF_URL;#var-INITRAMFS_IMAGE'><filename>INITRAMFS_IMAGE</filename></ulink>
5833 variable in your <filename>machine.conf</filename> file:
5834 <literallayout class='monospaced'>
5835 INITRAMFS_IMAGE_BUNDLE = "1"
5836 INITRAMFS_IMAGE = "<replaceable>image_recipe_name</replaceable>"
5837 </literallayout>
5838 Setting the <filename>INITRAMFS_IMAGE_BUNDLE</filename>
5839 flag causes the initramfs created by the recipe and defined by
5840 <filename>INITRAMFS_IMAGE</filename> to be unpacked into the
5841 <filename>${B}/usr/</filename> directory.
5842 The unpacked initramfs is then passed to the kernel's
5843 <filename>Makefile</filename> using the
5844 <ulink url='&YOCTO_DOCS_REF_URL;#var-CONFIG_INITRAMFS_SOURCE'><filename>CONFIG_INITRAMFS_SOURCE</filename></ulink>
5845 variable, allowing initramfs to be built in to the kernel
5846 normally.
5847 <note>
5848 The preferred method is to use the
5849 <filename>INITRAMFS_IMAGE</filename> variable rather than the
5850 <filename>INITRAMFS_TASK</filename> variable.
5851 Setting <filename>INITRAMFS_TASK</filename> is supported for
5852 backward compatibility.
5853 However, use of this variable has circular dependency
5854 problems.
5855 See the
5856 <ulink url='&YOCTO_DOCS_REF_URL;#var-INITRAMFS_IMAGE_BUNDLE'><filename>INITRAMFS_IMAGE_BUNDLE</filename></ulink>
5857 variable for additional information on these dependency
5858 problems.
5859 </note>
5860 </para>
5861
5862 <para>
5863 The recipe that <filename>INITRAMFS_IMAGE</filename>
5864 points to must produce a <filename>.cpio.gz</filename>,
5865 <filename>.cpio.tar</filename>, <filename>.cpio.lz4</filename>,
5866 <filename>.cpio.lzma</filename>, or
5867 <filename>.cpio.xz</filename> file.
5868 You can ensure you produce one of these <filename>.cpio.*</filename>
5869 files by setting the
5870 <ulink url='&YOCTO_DOCS_REF_URL;#var-INITRAMFS_FSTYPES'><filename>INITRAMFS_FSTYPES</filename></ulink>
5871 variable in your configuration file to one or more of the above
5872 file types.
5873 <note>
5874 If you add items to the initramfs image by way of its recipe,
5875 you should use
5876 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_INSTALL'><filename>PACKAGE_INSTALL</filename></ulink>
5877 rather than
5878 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink>.
5879 <filename>PACKAGE_INSTALL</filename> gives more direct control
5880 of what is added to the image as compared to the defaults you
5881 might not necessarily want that are set by the
5882 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-image'><filename>image</filename></ulink>
5883 or
5884 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-core-image'><filename>core-image</filename></ulink>
5885 classes.
5886 </note>
5887 </para>
5888 </section>
5889
Patrick Williamsc124f4f2015-09-15 14:41:29 -05005890 <section id='configuring-the-kernel'>
5891 <title>Configuring the Kernel</title>
5892
5893 <para>
5894 Configuring the Yocto Project kernel consists of making sure the
5895 <filename>.config</filename> file has all the right information
5896 in it for the image you are building.
5897 You can use the <filename>menuconfig</filename> tool and
5898 configuration fragments to make sure your
5899 <filename>.config</filename> file is just how you need it.
5900 You can also save known configurations in a
5901 <filename>defconfig</filename> file that the build system can use
5902 for kernel configuration.
5903 </para>
5904
5905 <para>
5906 This section describes how to use <filename>menuconfig</filename>,
5907 create and use configuration fragments, and how to interactively
5908 modify your <filename>.config</filename> file to create the
5909 leanest kernel configuration file possible.
5910 </para>
5911
5912 <para>
5913 For more information on kernel configuration, see the
5914 "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#changing-the-configuration'>Changing the Configuration</ulink>"
5915 section in the Yocto Project Linux Kernel Development Manual.
5916 </para>
5917
5918 <section id='using-menuconfig'>
5919 <title>Using&nbsp;&nbsp;<filename>menuconfig</filename></title>
5920
5921 <para>
5922 The easiest way to define kernel configurations is to set them through the
5923 <filename>menuconfig</filename> tool.
5924 This tool provides an interactive method with which
5925 to set kernel configurations.
5926 For general information on <filename>menuconfig</filename>, see
5927 <ulink url='http://en.wikipedia.org/wiki/Menuconfig'></ulink>.
5928 </para>
5929
5930 <para>
5931 To use the <filename>menuconfig</filename> tool in the Yocto Project development
5932 environment, you must launch it using BitBake.
5933 Thus, the environment must be set up using the
5934 <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
5935 or
5936 <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>
5937 script found in the
5938 <link linkend='build-directory'>Build Directory</link>.
5939 You must also be sure of the state of your build in the
5940 <link linkend='source-directory'>Source Directory</link>.
5941 The following commands run <filename>menuconfig</filename>
5942 assuming the Source Directory's top-level folder is
5943 <filename>~/poky</filename>:
5944 <literallayout class='monospaced'>
5945 $ cd poky
5946 $ source oe-init-build-env
5947 $ bitbake linux-yocto -c kernel_configme -f
5948 $ bitbake linux-yocto -c menuconfig
5949 </literallayout>
5950 Once <filename>menuconfig</filename> comes up, its standard
5951 interface allows you to interactively examine and configure
5952 all the kernel configuration parameters.
5953 After making your changes, simply exit the tool and save your
5954 changes to create an updated version of the
5955 <filename>.config</filename> configuration file.
5956 </para>
5957
5958 <para>
5959 Consider an example that configures the <filename>linux-yocto-3.14</filename>
5960 kernel.
5961 The OpenEmbedded build system recognizes this kernel as
5962 <filename>linux-yocto</filename>.
5963 Thus, the following commands from the shell in which you previously sourced the
5964 environment initialization script cleans the shared state cache and the
5965 <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>
5966 directory and then runs <filename>menuconfig</filename>:
5967 <literallayout class='monospaced'>
5968 $ bitbake linux-yocto -c menuconfig
5969 </literallayout>
5970 </para>
5971
5972 <para>
5973 Once <filename>menuconfig</filename> launches, use the interface
5974 to navigate through the selections to find the configuration settings in
5975 which you are interested.
5976 For example, consider the <filename>CONFIG_SMP</filename> configuration setting.
5977 You can find it at <filename>Processor Type and Features</filename> under
5978 the configuration selection <filename>Symmetric Multi-processing Support</filename>.
5979 After highlighting the selection, use the arrow keys to select or deselect
5980 the setting.
5981 When you are finished with all your selections, exit out and save them.
5982 </para>
5983
5984 <para>
5985 Saving the selections updates the <filename>.config</filename> configuration file.
5986 This is the file that the OpenEmbedded build system uses to configure the
5987 kernel during the build.
5988 You can find and examine this file in the Build Directory in
5989 <filename>tmp/work/</filename>.
5990 The actual <filename>.config</filename> is located in the area where the
5991 specific kernel is built.
5992 For example, if you were building a Linux Yocto kernel based on the
5993 Linux 3.14 kernel and you were building a QEMU image targeted for
5994 <filename>x86</filename> architecture, the
5995 <filename>.config</filename> file would be located here:
5996 <literallayout class='monospaced'>
5997 poky/build/tmp/work/qemux86-poky-linux/linux-yocto-3.14.11+git1+84f...
5998 ...656ed30-r1/linux-qemux86-standard-build
5999 </literallayout>
6000 <note>
6001 The previous example directory is artificially split and many of the characters
6002 in the actual filename are omitted in order to make it more readable.
6003 Also, depending on the kernel you are using, the exact pathname
6004 for <filename>linux-yocto-3.14...</filename> might differ.
6005 </note>
6006 </para>
6007
6008 <para>
6009 Within the <filename>.config</filename> file, you can see the kernel settings.
6010 For example, the following entry shows that symmetric multi-processor support
6011 is not set:
6012 <literallayout class='monospaced'>
6013 # CONFIG_SMP is not set
6014 </literallayout>
6015 </para>
6016
6017 <para>
6018 A good method to isolate changed configurations is to use a combination of the
6019 <filename>menuconfig</filename> tool and simple shell commands.
6020 Before changing configurations with <filename>menuconfig</filename>, copy the
6021 existing <filename>.config</filename> and rename it to something else,
6022 use <filename>menuconfig</filename> to make
6023 as many changes as you want and save them, then compare the renamed configuration
6024 file against the newly created file.
6025 You can use the resulting differences as your base to create configuration fragments
6026 to permanently save in your kernel layer.
6027 <note>
6028 Be sure to make a copy of the <filename>.config</filename> and don't just
6029 rename it.
6030 The build system needs an existing <filename>.config</filename>
6031 from which to work.
6032 </note>
6033 </para>
6034 </section>
6035
6036 <section id='creating-a-defconfig-file'>
6037 <title>Creating a&nbsp;&nbsp;<filename>defconfig</filename> File</title>
6038
6039 <para>
6040 A <filename>defconfig</filename> file is simply a
6041 <filename>.config</filename> renamed to "defconfig".
6042 You can use a <filename>defconfig</filename> file
6043 to retain a known set of kernel configurations from which the
6044 OpenEmbedded build system can draw to create the final
6045 <filename>.config</filename> file.
6046 <note>
6047 Out-of-the-box, the Yocto Project never ships a
6048 <filename>defconfig</filename> or
6049 <filename>.config</filename> file.
6050 The OpenEmbedded build system creates the final
6051 <filename>.config</filename> file used to configure the
6052 kernel.
6053 </note>
6054 </para>
6055
6056 <para>
6057 To create a <filename>defconfig</filename>, start with a
6058 complete, working Linux kernel <filename>.config</filename>
6059 file.
6060 Copy that file to the appropriate
6061 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename>
6062 directory in your layer's
6063 <filename>recipes-kernel/linux</filename> directory, and rename
6064 the copied file to "defconfig".
6065 Then, add the following lines to the linux-yocto
6066 <filename>.bbappend</filename> file in your layer:
6067 <literallayout class='monospaced'>
6068 FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
6069 SRC_URI += "file://defconfig"
6070 </literallayout>
6071 The
6072 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
6073 tells the build system how to search for the file, while the
6074 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink>
6075 extends the
6076 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink>
6077 variable (search directories) to include the
6078 <filename>${PN}</filename> directory you created to hold the
6079 configuration changes.
6080 <note>
6081 The build system applies the configurations from the
6082 <filename>defconfig</filename> file before applying any
6083 subsequent configuration fragments.
6084 The final kernel configuration is a combination of the
6085 configurations in the <filename>defconfig</filename>
6086 file and any configuration fragments you provide.
6087 You need to realize that if you have any configuration
6088 fragments, the build system applies these on top of and
6089 after applying the existing defconfig file configurations.
6090 </note>
6091 For more information on configuring the kernel, see the
6092 "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#changing-the-configuration'>Changing the Configuration</ulink>"
6093 and
6094 "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#generating-configuration-files'>Generating Configuration Files</ulink>"
6095 sections, both in the Yocto Project Linux Kernel Development
6096 Manual.
6097 </para>
6098 </section>
6099
6100 <section id='creating-config-fragments'>
6101 <title>Creating Configuration Fragments</title>
6102
6103 <para>
6104 Configuration fragments are simply kernel options that appear in a file
6105 placed where the OpenEmbedded build system can find and apply them.
6106 Syntactically, the configuration statement is identical to what would appear
6107 in the <filename>.config</filename> file, which is in the
6108 <link linkend='build-directory'>Build Directory</link>:
6109 <literallayout class='monospaced'>
6110 tmp/work/<replaceable>arch</replaceable>-poky-linux/linux-yocto-<replaceable>release_specific_string</replaceable>/linux-<replaceable>arch</replaceable>-<replaceable>build_type</replaceable>
6111 </literallayout>
6112 </para>
6113
6114 <para>
6115 It is simple to create a configuration fragment.
6116 For example, issuing the following from the shell creates a configuration fragment
6117 file named <filename>my_smp.cfg</filename> that enables multi-processor support
6118 within the kernel:
6119 <literallayout class='monospaced'>
6120 $ echo "CONFIG_SMP=y" >> my_smp.cfg
6121 </literallayout>
6122 <note>
6123 All configuration fragment files must use the
6124 <filename>.cfg</filename> extension in order for the
6125 OpenEmbedded build system to recognize them as a
6126 configuration fragment.
6127 </note>
6128 </para>
6129
6130 <para>
6131 Where do you put your configuration fragment files?
6132 You can place these files in the same area pointed to by
6133 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>.
6134 The OpenEmbedded build system picks up the configuration and
6135 adds it to the kernel's configuration.
6136 For example, suppose you had a set of configuration options
6137 in a file called <filename>myconfig.cfg</filename>.
6138 If you put that file inside a directory named
6139 <filename>linux-yocto</filename> that resides in the same
6140 directory as the kernel's append file and then add a
6141 <filename>SRC_URI</filename> statement such as the following
6142 to the kernel's append file, those configuration options
6143 will be picked up and applied when the kernel is built.
6144 <literallayout class='monospaced'>
6145 SRC_URI += "file://myconfig.cfg"
6146 </literallayout>
6147 </para>
6148
6149 <para>
6150 As mentioned earlier, you can group related configurations into multiple files and
6151 name them all in the <filename>SRC_URI</filename> statement as well.
6152 For example, you could group separate configurations specifically for Ethernet and graphics
6153 into their own files and add those by using a <filename>SRC_URI</filename> statement like the
6154 following in your append file:
6155 <literallayout class='monospaced'>
6156 SRC_URI += "file://myconfig.cfg \
6157 file://eth.cfg \
6158 file://gfx.cfg"
6159 </literallayout>
6160 </para>
6161 </section>
6162
6163 <section id='fine-tuning-the-kernel-configuration-file'>
6164 <title>Fine-Tuning the Kernel Configuration File</title>
6165
6166 <para>
6167 You can make sure the <filename>.config</filename> file is as lean or efficient as
6168 possible by reading the output of the kernel configuration fragment audit,
6169 noting any issues, making changes to correct the issues, and then repeating.
6170 </para>
6171
6172 <para>
6173 As part of the kernel build process, the
6174 <filename>do_kernel_configcheck</filename> task runs.
6175 This task validates the kernel configuration by checking the final
6176 <filename>.config</filename> file against the input files.
6177 During the check, the task produces warning messages for the following
6178 issues:
6179 <itemizedlist>
6180 <listitem><para>Requested options that did not make the final
6181 <filename>.config</filename> file.</para></listitem>
6182 <listitem><para>Configuration items that appear twice in the same
6183 configuration fragment.</para></listitem>
6184 <listitem><para>Configuration items tagged as "required" that were overridden.
6185 </para></listitem>
6186 <listitem><para>A board overrides a non-board specific option.</para></listitem>
6187 <listitem><para>Listed options not valid for the kernel being processed.
6188 In other words, the option does not appear anywhere.</para></listitem>
6189 </itemizedlist>
6190 <note>
6191 The <filename>do_kernel_configcheck</filename> task can
6192 also optionally report if an option is overridden during
6193 processing.
6194 </note>
6195 </para>
6196
6197 <para>
6198 For each output warning, a message points to the file
6199 that contains a list of the options and a pointer to the
6200 configuration fragment that defines them.
6201 Collectively, the files are the key to streamlining the
6202 configuration.
6203 </para>
6204
6205 <para>
6206 To streamline the configuration, do the following:
6207 <orderedlist>
6208 <listitem><para>Start with a full configuration that you
6209 know works - it builds and boots successfully.
6210 This configuration file will be your baseline.
6211 </para></listitem>
6212 <listitem><para>Separately run the
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05006213 <filename>do_kernel_configme</filename> and
Patrick Williamsc124f4f2015-09-15 14:41:29 -05006214 <filename>do_kernel_configcheck</filename> tasks.
6215 </para></listitem>
6216 <listitem><para>Take the resulting list of files from the
6217 <filename>do_kernel_configcheck</filename> task
6218 warnings and do the following:
6219 <itemizedlist>
6220 <listitem><para>
6221 Drop values that are redefined in the fragment
6222 but do not change the final
6223 <filename>.config</filename> file.
6224 </para></listitem>
6225 <listitem><para>
6226 Analyze and potentially drop values from the
6227 <filename>.config</filename> file that override
6228 required configurations.
6229 </para></listitem>
6230 <listitem><para>
6231 Analyze and potentially remove non-board
6232 specific options.
6233 </para></listitem>
6234 <listitem><para>
6235 Remove repeated and invalid options.
6236 </para></listitem>
6237 </itemizedlist></para></listitem>
6238 <listitem><para>
6239 After you have worked through the output of the kernel
6240 configuration audit, you can re-run the
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05006241 <filename>do_kernel_configme</filename> and
Patrick Williamsc124f4f2015-09-15 14:41:29 -05006242 <filename>do_kernel_configcheck</filename> tasks to
6243 see the results of your changes.
6244 If you have more issues, you can deal with them as
6245 described in the previous step.
6246 </para></listitem>
6247 </orderedlist>
6248 </para>
6249
6250 <para>
6251 Iteratively working through steps two through four eventually yields
6252 a minimal, streamlined configuration file.
6253 Once you have the best <filename>.config</filename>, you can build the Linux
6254 Yocto kernel.
6255 </para>
6256 </section>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05006257
6258 <section id='determining-hardware-and-non-hardware-features-for-the-kernel-configuration-audit-phase'>
6259 <title>Determining Hardware and Non-Hardware Features for the Kernel Configuration Audit Phase</title>
6260
6261 <para>
6262 This section describes part of the kernel configuration audit
6263 phase that most developers can ignore.
6264 During this part of the audit phase, the contents of the final
6265 <filename>.config</filename> file are compared against the
6266 fragments specified by the system.
6267 These fragments can be system fragments, distro fragments,
6268 or user specified configuration elements.
6269 Regardless of their origin, the OpenEmbedded build system
6270 warns the user if a specific option is not included in the
6271 final kernel configuration.
6272 </para>
6273
6274 <para>
6275 In order to not overwhelm the user with configuration warnings,
6276 by default the system only reports on missing "hardware"
6277 options because a missing hardware option could mean a boot
6278 failure or that important hardware is not available.
6279 </para>
6280
6281 <para>
6282 To determine whether or not a given option is "hardware" or
6283 "non-hardware", the kernel Metadata contains files that
6284 classify individual or groups of options as either hardware
6285 or non-hardware.
6286 To better show this, consider a situation where the
6287 Yocto Project kernel cache contains the following files:
6288 <literallayout class='monospaced'>
6289 kernel-cache/features/drm-psb/hardware.cfg
6290 kernel-cache/features/kgdb/hardware.cfg
6291 kernel-cache/ktypes/base/hardware.cfg
6292 kernel-cache/bsp/mti-malta32/hardware.cfg
6293 kernel-cache/bsp/fsl-mpc8315e-rdb/hardware.cfg
6294 kernel-cache/bsp/qemu-ppc32/hardware.cfg
6295 kernel-cache/bsp/qemuarma9/hardware.cfg
6296 kernel-cache/bsp/mti-malta64/hardware.cfg
6297 kernel-cache/bsp/arm-versatile-926ejs/hardware.cfg
6298 kernel-cache/bsp/common-pc/hardware.cfg
6299 kernel-cache/bsp/common-pc-64/hardware.cfg
6300 kernel-cache/features/rfkill/non-hardware.cfg
6301 kernel-cache/ktypes/base/non-hardware.cfg
6302 kernel-cache/features/aufs/non-hardware.kcf
6303 kernel-cache/features/ocf/non-hardware.kcf
6304 kernel-cache/ktypes/base/non-hardware.kcf
6305 kernel-cache/ktypes/base/hardware.kcf
6306 kernel-cache/bsp/qemu-ppc32/hardware.kcf
6307 </literallayout>
6308 The following list provides explanations for the various
6309 files:
6310 <itemizedlist>
6311 <listitem><para><filename>hardware.kcf</filename>:
6312 Specifies a list of kernel Kconfig files that contain
6313 hardware options only.
6314 </para></listitem>
6315 <listitem><para><filename>non-hardware.kcf</filename>:
6316 Specifies a list of kernel Kconfig files that contain
6317 non-hardware options only.
6318 </para></listitem>
6319 <listitem><para><filename>hardware.cfg</filename>:
6320 Specifies a list of kernel
6321 <filename>CONFIG_</filename> options that are hardware,
6322 regardless of whether or not they are within a Kconfig
6323 file specified by a hardware or non-hardware
6324 Kconfig file (i.e. <filename>hardware.kcf</filename> or
6325 <filename>non-hardware.kcf</filename>).
6326 </para></listitem>
6327 <listitem><para><filename>non-hardware.cfg</filename>:
6328 Specifies a list of kernel
6329 <filename>CONFIG_</filename> options that are
6330 not hardware, regardless of whether or not they are
6331 within a Kconfig file specified by a hardware or
6332 non-hardware Kconfig file (i.e.
6333 <filename>hardware.kcf</filename> or
6334 <filename>non-hardware.kcf</filename>).
6335 </para></listitem>
6336 </itemizedlist>
6337 Here is a specific example using the
6338 <filename>kernel-cache/bsp/mti-malta32/hardware.cfg</filename>:
6339 <literallayout class='monospaced'>
6340 CONFIG_SERIAL_8250
6341 CONFIG_SERIAL_8250_CONSOLE
6342 CONFIG_SERIAL_8250_NR_UARTS
6343 CONFIG_SERIAL_8250_PCI
6344 CONFIG_SERIAL_CORE
6345 CONFIG_SERIAL_CORE_CONSOLE
6346 CONFIG_VGA_ARB
6347 </literallayout>
6348 The kernel configuration audit automatically detects these
6349 files (hence the names must be exactly the ones discussed here),
6350 and uses them as inputs when generating warnings about the
6351 final <filename>.config</filename> file.
6352 </para>
6353
6354 <para>
6355 A user-specified kernel Metadata repository, or recipe space
6356 feature, can use these same files to classify options that are
6357 found within its <filename>.cfg</filename> files as hardware
6358 or non-hardware, to prevent the OpenEmbedded build system from
6359 producing an error or warning when an option is not in the
6360 final <filename>.config</filename> file.
6361 </para>
6362 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05006363 </section>
6364
6365 <section id="patching-the-kernel">
6366 <title>Patching the Kernel</title>
6367
6368 <para>
6369 Patching the kernel involves changing or adding configurations to an existing kernel,
6370 changing or adding recipes to the kernel that are needed to support specific hardware features,
6371 or even altering the source code itself.
6372 <note>
6373 You can use the <filename>yocto-kernel</filename> script
6374 found in the <link linkend='source-directory'>Source Directory</link>
6375 under <filename>scripts</filename> to manage kernel patches and configuration.
6376 See the "<ulink url='&YOCTO_DOCS_BSP_URL;#managing-kernel-patches-and-config-items-with-yocto-kernel'>Managing kernel Patches and Config Items with yocto-kernel</ulink>"
6377 section in the Yocto Project Board Support Packages (BSP) Developer's Guide for
6378 more information.</note>
6379 </para>
6380
6381 <para>
6382 This example creates a simple patch by adding some QEMU emulator console
6383 output at boot time through <filename>printk</filename> statements in the kernel's
6384 <filename>calibrate.c</filename> source code file.
6385 Applying the patch and booting the modified image causes the added
6386 messages to appear on the emulator's console.
6387 </para>
6388
6389 <para>
6390 The example assumes a clean build exists for the <filename>qemux86</filename>
6391 machine in a
6392 <link linkend='source-directory'>Source Directory</link>
6393 named <filename>poky</filename>.
6394 Furthermore, the <link linkend='build-directory'>Build Directory</link> is
6395 <filename>build</filename> and is located in <filename>poky</filename> and
6396 the kernel is based on the Linux 3.4 kernel.
6397 </para>
6398
6399 <para>
6400 Also, for more information on patching the kernel, see the
6401 "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#applying-patches'>Applying Patches</ulink>"
6402 section in the Yocto Project Linux Kernel Development Manual.
6403 </para>
6404
6405 <section id='create-a-layer-for-your-changes'>
6406 <title>Create a Layer for your Changes</title>
6407
6408 <para>
6409 The first step is to create a layer so you can isolate your
6410 changes.
6411 Rather than use the <filename>yocto-layer</filename> script
6412 to create the layer, this example steps through the process
6413 by hand.
6414 If you want information on the script that creates a general
6415 layer, see the
6416 "<link linkend='creating-a-general-layer-using-the-yocto-layer-script'>Creating a General Layer Using the yocto-layer Script</link>"
6417 section.
6418 </para>
6419
6420 <para>
6421 These two commands create a directory you can use for your
6422 layer:
6423 <literallayout class='monospaced'>
6424 $ cd ~/poky
6425 $ mkdir meta-mylayer
6426 </literallayout>
6427 Creating a directory that follows the Yocto Project layer naming
6428 conventions sets up the layer for your changes.
6429 The layer is where you place your configuration files, append
6430 files, and patch files.
6431 To learn more about creating a layer and filling it with the
6432 files you need, see the "<link linkend='understanding-and-creating-layers'>Understanding
6433 and Creating Layers</link>" section.
6434 </para>
6435 </section>
6436
6437 <section id='finding-the-kernel-source-code'>
6438 <title>Finding the Kernel Source Code</title>
6439
6440 <para>
6441 Each time you build a kernel image, the kernel source code is fetched
6442 and unpacked into the following directory:
6443 <literallayout class='monospaced'>
6444 ${S}/linux
6445 </literallayout>
6446 See the "<link linkend='finding-the-temporary-source-code'>Finding Temporary Source Code</link>"
6447 section and the
6448 <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink> variable
6449 for more information about where source is kept during a build.
6450 </para>
6451
6452 <para>
6453 For this example, we are going to patch the
6454 <filename>init/calibrate.c</filename> file
6455 by adding some simple console <filename>printk</filename> statements that we can
6456 see when we boot the image using QEMU.
6457 </para>
6458 </section>
6459
6460 <section id='creating-the-patch'>
6461 <title>Creating the Patch</title>
6462
6463 <para>
6464 Two methods exist by which you can create the patch:
6465 <link linkend='using-devtool-in-your-workflow'><filename>devtool</filename></link> and
6466 <link linkend='using-a-quilt-workflow'>Quilt</link>.
6467 For kernel patches, the Git workflow is more appropriate.
6468 This section assumes the Git workflow and shows the steps specific to
6469 this example.
6470 <orderedlist>
6471 <listitem><para><emphasis>Change the working directory</emphasis>:
6472 Change to where the kernel source code is before making
6473 your edits to the <filename>calibrate.c</filename> file:
6474 <literallayout class='monospaced'>
6475 $ cd ~/poky/build/tmp/work/qemux86-poky-linux/linux-yocto-${PV}-${PR}/linux
6476 </literallayout>
6477 Because you are working in an established Git repository,
6478 you must be in this directory in order to commit your changes
6479 and create the patch file.
6480 <note>The <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink> and
6481 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink> variables
6482 represent the version and revision for the
6483 <filename>linux-yocto</filename> recipe.
6484 The <filename>PV</filename> variable includes the Git meta and machine
6485 hashes, which make the directory name longer than you might
6486 expect.
6487 </note></para></listitem>
6488 <listitem><para><emphasis>Edit the source file</emphasis>:
6489 Edit the <filename>init/calibrate.c</filename> file to have the
6490 following changes:
6491 <literallayout class='monospaced'>
6492 void calibrate_delay(void)
6493 {
6494 unsigned long lpj;
6495 static bool printed;
6496 int this_cpu = smp_processor_id();
6497
6498 printk("*************************************\n");
6499 printk("* *\n");
6500 printk("* HELLO YOCTO KERNEL *\n");
6501 printk("* *\n");
6502 printk("*************************************\n");
6503
6504 if (per_cpu(cpu_loops_per_jiffy, this_cpu)) {
6505 .
6506 .
6507 .
6508 </literallayout></para></listitem>
6509 <listitem><para><emphasis>Stage and commit your changes</emphasis>:
6510 These Git commands display the modified file, stage it, and then
6511 commit the file:
6512 <literallayout class='monospaced'>
6513 $ git status
6514 $ git add init/calibrate.c
6515 $ git commit -m "calibrate: Add printk example"
6516 </literallayout></para></listitem>
6517 <listitem><para><emphasis>Generate the patch file</emphasis>:
6518 This Git command creates the a patch file named
6519 <filename>0001-calibrate-Add-printk-example.patch</filename>
6520 in the current directory.
6521 <literallayout class='monospaced'>
6522 $ git format-patch -1
6523 </literallayout>
6524 </para></listitem>
6525 </orderedlist>
6526 </para>
6527 </section>
6528
6529 <section id='set-up-your-layer-for-the-build'>
6530 <title>Set Up Your Layer for the Build</title>
6531
6532 <para>These steps get your layer set up for the build:
6533 <orderedlist>
6534 <listitem><para><emphasis>Create additional structure</emphasis>:
6535 Create the additional layer structure:
6536 <literallayout class='monospaced'>
6537 $ cd ~/poky/meta-mylayer
6538 $ mkdir conf
6539 $ mkdir recipes-kernel
6540 $ mkdir recipes-kernel/linux
6541 $ mkdir recipes-kernel/linux/linux-yocto
6542 </literallayout>
6543 The <filename>conf</filename> directory holds your configuration files, while the
6544 <filename>recipes-kernel</filename> directory holds your append file and
6545 your patch file.</para></listitem>
6546 <listitem><para><emphasis>Create the layer configuration file</emphasis>:
6547 Move to the <filename>meta-mylayer/conf</filename> directory and create
6548 the <filename>layer.conf</filename> file as follows:
6549 <literallayout class='monospaced'>
6550 # We have a conf and classes directory, add to BBPATH
6551 BBPATH .= ":${LAYERDIR}"
6552
6553 # We have recipes-* directories, add to BBFILES
6554 BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
6555 ${LAYERDIR}/recipes-*/*/*.bbappend"
6556
6557 BBFILE_COLLECTIONS += "mylayer"
6558 BBFILE_PATTERN_mylayer = "^${LAYERDIR}/"
6559 BBFILE_PRIORITY_mylayer = "5"
6560 </literallayout>
6561 Notice <filename>mylayer</filename> as part of the last three
6562 statements.</para></listitem>
6563 <listitem><para><emphasis>Create the kernel recipe append file</emphasis>:
6564 Move to the <filename>meta-mylayer/recipes-kernel/linux</filename> directory and create
6565 the <filename>linux-yocto_3.4.bbappend</filename> file as follows:
6566 <literallayout class='monospaced'>
6567 FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
6568
6569 SRC_URI += "file://0001-calibrate-Add-printk-example.patch"
6570 </literallayout>
6571 The <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink>
6572 and <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
6573 statements enable the OpenEmbedded build system to find the patch file.
6574 For more information on using append files, see the
Brad Bishop6e60e8b2018-02-01 10:27:11 -05006575 "<link linkend='using-bbappend-files'>Using .bbappend Files in Your Layer</link>"
Patrick Williamsc124f4f2015-09-15 14:41:29 -05006576 section.
6577 </para></listitem>
6578 <listitem><para><emphasis>Put the patch file in your layer</emphasis>:
6579 Move the <filename>0001-calibrate-Add-printk-example.patch</filename> file to
6580 the <filename>meta-mylayer/recipes-kernel/linux/linux-yocto</filename>
6581 directory.</para></listitem>
6582 </orderedlist>
6583 </para>
6584 </section>
6585
6586 <section id='set-up-for-the-build'>
6587 <title>Set Up for the Build</title>
6588
6589 <para>
6590 Do the following to make sure the build parameters are set up for the example.
6591 Once you set up these build parameters, they do not have to change unless you
6592 change the target architecture of the machine you are building:
6593 <itemizedlist>
6594 <listitem><para><emphasis>Build for the correct target architecture:</emphasis> Your
6595 selected <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
6596 definition within the <filename>local.conf</filename> file in the
6597 <link linkend='build-directory'>Build Directory</link>
6598 specifies the target architecture used when building the Linux kernel.
6599 By default, <filename>MACHINE</filename> is set to
6600 <filename>qemux86</filename>, which specifies a 32-bit
6601 <trademark class='registered'>Intel</trademark> Architecture
6602 target machine suitable for the QEMU emulator.</para></listitem>
6603 <listitem><para><emphasis>Identify your <filename>meta-mylayer</filename>
6604 layer:</emphasis> The
6605 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBLAYERS'><filename>BBLAYERS</filename></ulink>
6606 variable in the
6607 <filename>bblayers.conf</filename> file found in the
6608 <filename>poky/build/conf</filename> directory needs to have the path to your local
6609 <filename>meta-mylayer</filename> layer.
6610 By default, the <filename>BBLAYERS</filename> variable contains paths to
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05006611 <filename>meta</filename>, <filename>meta-poky</filename>, and
Patrick Williamsc124f4f2015-09-15 14:41:29 -05006612 <filename>meta-yocto-bsp</filename> in the
6613 <filename>poky</filename> Git repository.
6614 Add the path to your <filename>meta-mylayer</filename> location:
6615 <literallayout class='monospaced'>
6616 BBLAYERS ?= " \
6617 $HOME/poky/meta \
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05006618 $HOME/poky/meta-poky \
Patrick Williamsc124f4f2015-09-15 14:41:29 -05006619 $HOME/poky/meta-yocto-bsp \
6620 $HOME/poky/meta-mylayer \
6621 "
6622 </literallayout></para></listitem>
6623 </itemizedlist>
6624 </para>
6625 </section>
6626
6627 <section id='build-the-modified-qemu-kernel-image'>
6628 <title>Build the Modified QEMU Kernel Image</title>
6629
6630 <para>
6631 The following steps build your modified kernel image:
6632 <orderedlist>
6633 <listitem><para><emphasis>Be sure your build environment is initialized</emphasis>:
6634 Your environment should be set up since you previously sourced
6635 the
6636 <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
6637 script.
6638 If it is not, source the script again from <filename>poky</filename>.
6639 <literallayout class='monospaced'>
6640 $ cd ~/poky
6641 $ source &OE_INIT_FILE;
6642 </literallayout>
6643 </para></listitem>
6644 <listitem><para><emphasis>Clean up</emphasis>:
6645 Be sure to clean the shared state out by using BitBake
6646 to run from within the Build Directory the
6647 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-cleansstate'><filename>do_cleansstate</filename></ulink>
6648 task as follows:
6649 <literallayout class='monospaced'>
6650 $ bitbake -c cleansstate linux-yocto
6651 </literallayout></para>
6652 <para>
6653 <note>
6654 Never remove any files by hand from the
6655 <filename>tmp/deploy</filename>
6656 directory inside the
6657 <link linkend='build-directory'>Build Directory</link>.
6658 Always use the various BitBake clean tasks to
6659 clear out previous build artifacts.
6660 For information on the clean tasks, see the
6661 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-clean'><filename>do_clean</filename></ulink>",
6662 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-cleanall'><filename>do_cleanall</filename></ulink>",
6663 and
6664 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-cleansstate'><filename>do_cleansstate</filename></ulink>"
6665 sections all in the Yocto Project Reference
6666 Manual.
6667 </note>
6668 </para></listitem>
6669 <listitem><para><emphasis>Build the image</emphasis>:
6670 Next, build the kernel image using this command:
6671 <literallayout class='monospaced'>
6672 $ bitbake -k linux-yocto
6673 </literallayout></para></listitem>
6674 </orderedlist>
6675 </para>
6676 </section>
6677
6678 <section id='boot-the-image-and-verify-your-changes'>
6679 <title>Boot the Image and Verify Your Changes</title>
6680
6681 <para>
6682 These steps boot the image and allow you to see the changes
6683 <orderedlist>
6684 <listitem><para><emphasis>Boot the image</emphasis>:
6685 Boot the modified image in the QEMU emulator
6686 using this command:
6687 <literallayout class='monospaced'>
6688 $ runqemu qemux86
6689 </literallayout></para></listitem>
6690 <listitem><para><emphasis>Verify the changes</emphasis>:
6691 Log into the machine using <filename>root</filename> with no password and then
6692 use the following shell command to scroll through the console's boot output.
6693 <literallayout class='monospaced'>
6694 # dmesg | less
6695 </literallayout>
6696 You should see the results of your <filename>printk</filename> statements
6697 as part of the output.</para></listitem>
6698 </orderedlist>
6699 </para>
6700 </section>
6701 </section>
6702
6703 <section id='making-images-more-secure'>
6704 <title>Making Images More Secure</title>
6705
6706 <para>
6707 Security is of increasing concern for embedded devices.
6708 Consider the issues and problems discussed in just this
6709 sampling of work found across the Internet:
6710 <itemizedlist>
6711 <listitem><para><emphasis>
6712 "<ulink url='https://www.schneier.com/blog/archives/2014/01/security_risks_9.html'>Security Risks of Embedded Systems</ulink>"</emphasis>
6713 by Bruce Schneier
6714 </para></listitem>
6715 <listitem><para><emphasis>
6716 "<ulink url='http://internetcensus2012.bitbucket.org/paper.html'>Internet Census 2012</ulink>"</emphasis>
6717 by Carna Botnet</para></listitem>
6718 <listitem><para><emphasis>
6719 "<ulink url='http://elinux.org/images/6/6f/Security-issues.pdf'>Security Issues for Embedded Devices</ulink>"</emphasis>
6720 by Jake Edge
6721 </para></listitem>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05006722 </itemizedlist>
6723 </para>
6724
6725 <para>
6726 When securing your image is of concern, there are steps, tools,
6727 and variables that you can consider to help you reach the
6728 security goals you need for your particular device.
6729 Not all situations are identical when it comes to making an
6730 image secure.
6731 Consequently, this section provides some guidance and suggestions
6732 for consideration when you want to make your image more secure.
6733 <note>
6734 Because the security requirements and risks are
6735 different for every type of device, this section cannot
6736 provide a complete reference on securing your custom OS.
6737 It is strongly recommended that you also consult other sources
6738 of information on embedded Linux system hardening and on
6739 security.
6740 </note>
6741 </para>
6742
6743 <section id='general-considerations'>
6744 <title>General Considerations</title>
6745
6746 <para>
6747 General considerations exist that help you create more
6748 secure images.
6749 You should consider the following suggestions to help
6750 make your device more secure:
6751 <itemizedlist>
6752 <listitem><para>
6753 Scan additional code you are adding to the system
6754 (e.g. application code) by using static analysis
6755 tools.
6756 Look for buffer overflows and other potential
6757 security problems.
6758 </para></listitem>
6759 <listitem><para>
6760 Pay particular attention to the security for
6761 any web-based administration interface.
6762 </para>
6763 <para>Web interfaces typically need to perform
6764 administrative functions and tend to need to run with
6765 elevated privileges.
6766 Thus, the consequences resulting from the interface's
6767 security becoming compromised can be serious.
6768 Look for common web vulnerabilities such as
6769 cross-site-scripting (XSS), unvalidated inputs,
6770 and so forth.</para>
6771 <para>As with system passwords, the default credentials
6772 for accessing a web-based interface should not be the
6773 same across all devices.
6774 This is particularly true if the interface is enabled
6775 by default as it can be assumed that many end-users
6776 will not change the credentials.
6777 </para></listitem>
6778 <listitem><para>
6779 Ensure you can update the software on the device to
6780 mitigate vulnerabilities discovered in the future.
6781 This consideration especially applies when your
6782 device is network-enabled.
6783 </para></listitem>
6784 <listitem><para>
6785 Ensure you remove or disable debugging functionality
6786 before producing the final image.
6787 For information on how to do this, see the
6788 "<link linkend='considerations-specific-to-the-openembedded-build-system'>Considerations Specific to the OpenEmbedded Build System</link>"
6789 section.
6790 </para></listitem>
6791 <listitem><para>
6792 Ensure you have no network services listening that
6793 are not needed.
6794 </para></listitem>
6795 <listitem><para>
6796 Remove any software from the image that is not needed.
6797 </para></listitem>
6798 <listitem><para>
6799 Enable hardware support for secure boot functionality
6800 when your device supports this functionality.
6801 </para></listitem>
6802 </itemizedlist>
6803 </para>
6804 </section>
6805
6806 <section id='security-flags'>
6807 <title>Security Flags</title>
6808
6809 <para>
6810 The Yocto Project has security flags that you can enable that
6811 help make your build output more secure.
6812 The security flags are in the
6813 <filename>meta/conf/distro/include/security_flags.inc</filename>
6814 file in your
6815 <link linkend='source-directory'>Source Directory</link>
6816 (e.g. <filename>poky</filename>).
6817 <note>
6818 Depending on the recipe, certain security flags are enabled
6819 and disabled by default.
6820 </note>
6821 </para>
6822
6823 <para>
6824<!--
6825 The GCC/LD flags in <filename>security_flags.inc</filename>
6826 enable more secure code generation.
6827 By including the <filename>security_flags.inc</filename>
6828 file, you enable flags to the compiler and linker that cause
6829 them to generate more secure code.
6830 <note>
6831 The GCC/LD flags are enabled by default in the
6832 <filename>poky-lsb</filename> distribution.
6833 </note>
6834-->
6835 Use the following line in your
6836 <filename>local.conf</filename> file or in your custom
6837 distribution configuration file to enable the security
6838 compiler and linker flags for your build:
6839 <literallayout class='monospaced'>
6840 require conf/distro/include/security_flags.inc
6841 </literallayout>
6842 </para>
6843 </section>
6844
6845 <section id='considerations-specific-to-the-openembedded-build-system'>
6846 <title>Considerations Specific to the OpenEmbedded Build System</title>
6847
6848 <para>
6849 You can take some steps that are specific to the
6850 OpenEmbedded build system to make your images more secure:
6851 <itemizedlist>
6852 <listitem><para>
6853 Ensure "debug-tweaks" is not one of your selected
6854 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>.
6855 When creating a new project, the default is to provide you
6856 with an initial <filename>local.conf</filename> file that
6857 enables this feature using the
6858 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink> variable with the line:
6859 <literallayout class='monospaced'>
6860 EXTRA_IMAGE_FEATURES = "debug-tweaks"
6861 </literallayout>
6862 To disable that feature, simply comment out that line in your
6863 <filename>local.conf</filename> file, or
6864 make sure <filename>IMAGE_FEATURES</filename> does not contain
6865 "debug-tweaks" before producing your final image.
6866 Among other things, leaving this in place sets the
6867 root password as blank, which makes logging in for
6868 debugging or inspection easy during
6869 development but also means anyone can easily log in
6870 during production.
6871 </para></listitem>
6872 <listitem><para>
6873 It is possible to set a root password for the image
6874 and also to set passwords for any extra users you might
6875 add (e.g. administrative or service type users).
6876 When you set up passwords for multiple images or
6877 users, you should not duplicate passwords.
6878 </para>
6879 <para>
6880 To set up passwords, use the
6881 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-extrausers'><filename>extrausers</filename></ulink>
6882 class, which is the preferred method.
6883 For an example on how to set up both root and user
6884 passwords, see the
6885 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-extrausers'><filename>extrausers.bbclass</filename></ulink>"
6886 section.
6887 <note>
6888 When adding extra user accounts or setting a
6889 root password, be cautious about setting the
6890 same password on every device.
6891 If you do this, and the password you have set
6892 is exposed, then every device is now potentially
6893 compromised.
6894 If you need this access but want to ensure
6895 security, consider setting a different,
6896 random password for each device.
6897 Typically, you do this as a separate step after
6898 you deploy the image onto the device.
6899 </note>
6900 </para></listitem>
6901 <listitem><para>
6902 Consider enabling a Mandatory Access Control (MAC)
6903 framework such as SMACK or SELinux and tuning it
6904 appropriately for your device's usage.
6905 You can find more information in the
6906 <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/meta-selinux/'><filename>meta-selinux</filename></ulink>
6907 layer.
6908 </para></listitem>
6909 </itemizedlist>
6910 </para>
6911
6912 <para>
6913 </para>
6914 </section>
6915
6916 <section id='tools-for-hardening-your-image'>
6917 <title>Tools for Hardening Your Image</title>
6918
6919 <para>
6920 The Yocto Project provides tools for making your image
6921 more secure.
6922 You can find these tools in the
6923 <filename>meta-security</filename> layer of the
6924 <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'>Yocto Project Source Repositories</ulink>.
6925 </para>
6926 </section>
6927 </section>
6928
6929 <section id='creating-your-own-distribution'>
6930 <title>Creating Your Own Distribution</title>
6931
6932 <para>
6933 When you build an image using the Yocto Project and
6934 do not alter any distribution
6935 <link linkend='metadata'>Metadata</link>, you are creating a
6936 Poky distribution.
6937 If you wish to gain more control over package alternative
6938 selections, compile-time options, and other low-level
6939 configurations, you can create your own distribution.
6940 </para>
6941
6942 <para>
6943 To create your own distribution, the basic steps consist of
6944 creating your own distribution layer, creating your own
6945 distribution configuration file, and then adding any needed
6946 code and Metadata to the layer.
6947 The following steps provide some more detail:
6948 <itemizedlist>
6949 <listitem><para><emphasis>Create a layer for your new distro:</emphasis>
6950 Create your distribution layer so that you can keep your
6951 Metadata and code for the distribution separate.
6952 It is strongly recommended that you create and use your own
6953 layer for configuration and code.
6954 Using your own layer as compared to just placing
6955 configurations in a <filename>local.conf</filename>
6956 configuration file makes it easier to reproduce the same
6957 build configuration when using multiple build machines.
6958 See the
6959 "<link linkend='creating-a-general-layer-using-the-yocto-layer-script'>Creating a General Layer Using the yocto-layer Script</link>"
6960 section for information on how to quickly set up a layer.
6961 </para></listitem>
6962 <listitem><para><emphasis>Create the distribution configuration file:</emphasis>
6963 The distribution configuration file needs to be created in
6964 the <filename>conf/distro</filename> directory of your
6965 layer.
6966 You need to name it using your distribution name
6967 (e.g. <filename>mydistro.conf</filename>).
6968 <note>
6969 The
6970 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink>
6971 variable in your
6972 <filename>local.conf</filename> file determines the
6973 name of your distribution.
6974 </note></para>
6975 <para>You can split out parts of your configuration file
6976 into include files and then "require" them from within
6977 your distribution configuration file.
6978 Be sure to place the include files in the
6979 <filename>conf/distro/include</filename> directory of
6980 your layer.
6981 A common example usage of include files would be to
6982 separate out the selection of desired version and revisions
6983 for individual recipes.
6984</para>
6985 <para>Your configuration file needs to set the following
6986 required variables:
6987 <literallayout class='monospaced'>
6988 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_NAME'><filename>DISTRO_NAME</filename></ulink>
6989 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_VERSION'><filename>DISTRO_VERSION</filename></ulink>
6990 </literallayout>
6991 These following variables are optional and you typically
6992 set them from the distribution configuration file:
6993 <literallayout class='monospaced'>
6994 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>
6995 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_EXTRA_RDEPENDS'><filename>DISTRO_EXTRA_RDEPENDS</filename></ulink>
6996 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_EXTRA_RRECOMMENDS'><filename>DISTRO_EXTRA_RRECOMMENDS</filename></ulink>
6997 <ulink url='&YOCTO_DOCS_REF_URL;#var-TCLIBC'><filename>TCLIBC</filename></ulink>
6998 </literallayout>
6999 <tip>
7000 If you want to base your distribution configuration file
7001 on the very basic configuration from OE-Core, you
7002 can use
7003 <filename>conf/distro/defaultsetup.conf</filename> as
7004 a reference and just include variables that differ
7005 as compared to <filename>defaultsetup.conf</filename>.
7006 Alternatively, you can create a distribution
7007 configuration file from scratch using the
7008 <filename>defaultsetup.conf</filename> file
7009 or configuration files from other distributions
7010 such as Poky or Angstrom as references.
7011 </tip></para></listitem>
7012 <listitem><para><emphasis>Provide miscellaneous variables:</emphasis>
7013 Be sure to define any other variables for which you want to
7014 create a default or enforce as part of the distribution
7015 configuration.
7016 You can include nearly any variable from the
7017 <filename>local.conf</filename> file.
7018 The variables you use are not limited to the list in the
7019 previous bulleted item.</para></listitem>
7020 <listitem><para><emphasis>Point to Your distribution configuration file:</emphasis>
7021 In your <filename>local.conf</filename> file in the
7022 <link linkend='build-directory'>Build Directory</link>,
7023 set your
7024 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink>
7025 variable to point to your distribution's configuration file.
7026 For example, if your distribution's configuration file is
7027 named <filename>mydistro.conf</filename>, then you point
7028 to it as follows:
7029 <literallayout class='monospaced'>
7030 DISTRO = "mydistro"
7031 </literallayout></para></listitem>
7032 <listitem><para><emphasis>Add more to the layer if necessary:</emphasis>
7033 Use your layer to hold other information needed for the
7034 distribution:
7035 <itemizedlist>
7036 <listitem><para>Add recipes for installing
7037 distro-specific configuration files that are not
7038 already installed by another recipe.
7039 If you have distro-specific configuration files
7040 that are included by an existing recipe, you should
7041 add an append file (<filename>.bbappend</filename>)
7042 for those.
7043 For general information and recommendations
7044 on how to add recipes to your layer, see the
7045 "<link linkend='creating-your-own-layer'>Creating Your Own Layer</link>"
7046 and
7047 "<link linkend='best-practices-to-follow-when-creating-layers'>Best Practices to Follow When Creating Layers</link>"
7048 sections.</para></listitem>
7049 <listitem><para>Add any image recipes that are specific
7050 to your distribution.</para></listitem>
7051 <listitem><para>Add a <filename>psplash</filename>
7052 append file for a branded splash screen.
7053 For information on append files, see the
Brad Bishop6e60e8b2018-02-01 10:27:11 -05007054 "<link linkend='using-bbappend-files'>Using .bbappend Files in Your Layer</link>"
Patrick Williamsc124f4f2015-09-15 14:41:29 -05007055 section.</para></listitem>
7056 <listitem><para>Add any other append files to make
7057 custom changes that are specific to individual
7058 recipes.</para></listitem>
7059 </itemizedlist></para></listitem>
7060 </itemizedlist>
7061 </para>
7062 </section>
7063
7064 <section id='creating-a-custom-template-configuration-directory'>
7065 <title>Creating a Custom Template Configuration Directory</title>
7066
7067 <para>
7068 If you are producing your own customized version
7069 of the build system for use by other users, you might
7070 want to customize the message shown by the setup script or
7071 you might want to change the template configuration files (i.e.
7072 <filename>local.conf</filename> and
7073 <filename>bblayers.conf</filename>) that are created in
7074 a new build directory.
7075 </para>
7076
7077 <para>
7078 The OpenEmbedded build system uses the environment variable
7079 <filename>TEMPLATECONF</filename> to locate the directory
7080 from which it gathers configuration information that ultimately
7081 ends up in the
7082 <link linkend='build-directory'>Build Directory's</link>
7083 <filename>conf</filename> directory.
7084 By default, <filename>TEMPLATECONF</filename> is set as
7085 follows in the <filename>poky</filename> repository:
7086 <literallayout class='monospaced'>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05007087 TEMPLATECONF=${TEMPLATECONF:-meta-poky/conf}
Patrick Williamsc124f4f2015-09-15 14:41:29 -05007088 </literallayout>
7089 This is the directory used by the build system to find templates
7090 from which to build some key configuration files.
7091 If you look at this directory, you will see the
7092 <filename>bblayers.conf.sample</filename>,
7093 <filename>local.conf.sample</filename>, and
7094 <filename>conf-notes.txt</filename> files.
7095 The build system uses these files to form the respective
7096 <filename>bblayers.conf</filename> file,
7097 <filename>local.conf</filename> file, and display the list of
7098 BitBake targets when running the setup script.
7099 </para>
7100
7101 <para>
7102 To override these default configuration files with
7103 configurations you want used within every new
7104 Build Directory, simply set the
7105 <filename>TEMPLATECONF</filename> variable to your directory.
7106 The <filename>TEMPLATECONF</filename> variable is set in the
7107 <filename>.templateconf</filename> file, which is in the
7108 top-level
7109 <link linkend='source-directory'>Source Directory</link>
7110 folder (e.g. <filename>poky</filename>).
7111 Edit the <filename>.templateconf</filename> so that it can locate
7112 your directory.
7113 </para>
7114
7115 <para>
7116 Best practices dictate that you should keep your
7117 template configuration directory in your custom distribution layer.
7118 For example, suppose you have a layer named
7119 <filename>meta-mylayer</filename> located in your home directory
7120 and you want your template configuration directory named
7121 <filename>myconf</filename>.
7122 Changing the <filename>.templateconf</filename> as follows
7123 causes the OpenEmbedded build system to look in your directory
7124 and base its configuration files on the
7125 <filename>*.sample</filename> configuration files it finds.
7126 The final configuration files (i.e.
7127 <filename>local.conf</filename> and
7128 <filename>bblayers.conf</filename> ultimately still end up in
7129 your Build Directory, but they are based on your
7130 <filename>*.sample</filename> files.
7131 <literallayout class='monospaced'>
7132 TEMPLATECONF=${TEMPLATECONF:-meta-mylayer/myconf}
7133 </literallayout>
7134 </para>
7135
7136 <para>
7137 Aside from the <filename>*.sample</filename> configuration files,
7138 the <filename>conf-notes.txt</filename> also resides in the
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05007139 default <filename>meta-poky/conf</filename> directory.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05007140 The scripts that set up the build environment
7141 (i.e.
7142 <ulink url="&YOCTO_DOCS_REF_URL;#structure-core-script"><filename>&OE_INIT_FILE;</filename></ulink>
7143 and
7144 <ulink url="&YOCTO_DOCS_REF_URL;#structure-memres-core-script"><filename>oe-init-build-env-memres</filename></ulink>)
7145 use this file to display BitBake targets as part of the script
7146 output.
7147 Customizing this <filename>conf-notes.txt</filename> file is a
7148 good way to make sure your list of custom targets appears
7149 as part of the script's output.
7150 </para>
7151
7152 <para>
7153 Here is the default list of targets displayed as a result of
7154 running either of the setup scripts:
7155 <literallayout class='monospaced'>
7156 You can now run 'bitbake &lt;target&gt;'
7157
7158 Common targets are:
7159 core-image-minimal
7160 core-image-sato
7161 meta-toolchain
Patrick Williamsc124f4f2015-09-15 14:41:29 -05007162 meta-ide-support
7163 </literallayout>
7164 </para>
7165
7166 <para>
7167 Changing the listed common targets is as easy as editing your
7168 version of <filename>conf-notes.txt</filename> in your
7169 custom template configuration directory and making sure you
7170 have <filename>TEMPLATECONF</filename> set to your directory.
7171 </para>
7172 </section>
7173
7174 <section id='building-a-tiny-system'>
7175 <title>Building a Tiny System</title>
7176
7177 <para>
7178 Very small distributions have some significant advantages such
7179 as requiring less on-die or in-package memory (cheaper), better
7180 performance through efficient cache usage, lower power requirements
7181 due to less memory, faster boot times, and reduced development
7182 overhead.
7183 Some real-world examples where a very small distribution gives
7184 you distinct advantages are digital cameras, medical devices,
7185 and small headless systems.
7186 </para>
7187
7188 <para>
7189 This section presents information that shows you how you can
7190 trim your distribution to even smaller sizes than the
7191 <filename>poky-tiny</filename> distribution, which is around
7192 5 Mbytes, that can be built out-of-the-box using the Yocto Project.
7193 </para>
7194
7195 <section id='tiny-system-overview'>
7196 <title>Overview</title>
7197
7198 <para>
7199 The following list presents the overall steps you need to
7200 consider and perform to create distributions with smaller
7201 root filesystems, achieve faster boot times, maintain your critical
7202 functionality, and avoid initial RAM disks:
7203 <itemizedlist>
7204 <listitem><para>
7205 <link linkend='goals-and-guiding-principles'>Determine your goals and guiding principles.</link>
7206 </para></listitem>
7207 <listitem><para>
7208 <link linkend='understand-what-gives-your-image-size'>Understand what contributes to your image size.</link>
7209 </para></listitem>
7210 <listitem><para>
7211 <link linkend='trim-the-root-filesystem'>Reduce the size of the root filesystem.</link>
7212 </para></listitem>
7213 <listitem><para>
7214 <link linkend='trim-the-kernel'>Reduce the size of the kernel.</link>
7215 </para></listitem>
7216 <listitem><para>
7217 <link linkend='remove-package-management-requirements'>Eliminate packaging requirements.</link>
7218 </para></listitem>
7219 <listitem><para>
7220 <link linkend='look-for-other-ways-to-minimize-size'>Look for other ways to minimize size.</link>
7221 </para></listitem>
7222 <listitem><para>
7223 <link linkend='iterate-on-the-process'>Iterate on the process.</link>
7224 </para></listitem>
7225 </itemizedlist>
7226 </para>
7227 </section>
7228
7229 <section id='goals-and-guiding-principles'>
7230 <title>Goals and Guiding Principles</title>
7231
7232 <para>
7233 Before you can reach your destination, you need to know
7234 where you are going.
7235 Here is an example list that you can use as a guide when
7236 creating very small distributions:
7237 <itemizedlist>
7238 <listitem><para>Determine how much space you need
7239 (e.g. a kernel that is 1 Mbyte or less and
7240 a root filesystem that is 3 Mbytes or less).
7241 </para></listitem>
7242 <listitem><para>Find the areas that are currently
7243 taking 90% of the space and concentrate on reducing
7244 those areas.
7245 </para></listitem>
7246 <listitem><para>Do not create any difficult "hacks"
7247 to achieve your goals.</para></listitem>
7248 <listitem><para>Leverage the device-specific
7249 options.</para></listitem>
7250 <listitem><para>Work in a separate layer so that you
7251 keep changes isolated.
7252 For information on how to create layers, see
7253 the "<link linkend='understanding-and-creating-layers'>Understanding and Creating Layers</link>" section.
7254 </para></listitem>
7255 </itemizedlist>
7256 </para>
7257 </section>
7258
7259 <section id='understand-what-gives-your-image-size'>
7260 <title>Understand What Contributes to Your Image Size</title>
7261
7262 <para>
7263 It is easiest to have something to start with when creating
7264 your own distribution.
7265 You can use the Yocto Project out-of-the-box to create the
7266 <filename>poky-tiny</filename> distribution.
7267 Ultimately, you will want to make changes in your own
7268 distribution that are likely modeled after
7269 <filename>poky-tiny</filename>.
7270 <note>
7271 To use <filename>poky-tiny</filename> in your build,
7272 set the
7273 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink>
7274 variable in your
7275 <filename>local.conf</filename> file to "poky-tiny"
7276 as described in the
7277 "<link linkend='creating-your-own-distribution'>Creating Your Own Distribution</link>"
7278 section.
7279 </note>
7280 </para>
7281
7282 <para>
7283 Understanding some memory concepts will help you reduce the
7284 system size.
7285 Memory consists of static, dynamic, and temporary memory.
7286 Static memory is the TEXT (code), DATA (initialized data
7287 in the code), and BSS (uninitialized data) sections.
7288 Dynamic memory represents memory that is allocated at runtime:
7289 stacks, hash tables, and so forth.
7290 Temporary memory is recovered after the boot process.
7291 This memory consists of memory used for decompressing
7292 the kernel and for the <filename>__init__</filename>
7293 functions.
7294 </para>
7295
7296 <para>
7297 To help you see where you currently are with kernel and root
7298 filesystem sizes, you can use two tools found in the
7299 <link linkend='source-directory'>Source Directory</link> in
7300 the <filename>scripts/tiny/</filename> directory:
7301 <itemizedlist>
7302 <listitem><para><filename>ksize.py</filename>: Reports
7303 component sizes for the kernel build objects.
7304 </para></listitem>
7305 <listitem><para><filename>dirsize.py</filename>: Reports
7306 component sizes for the root filesystem.</para></listitem>
7307 </itemizedlist>
7308 This next tool and command help you organize configuration
7309 fragments and view file dependencies in a human-readable form:
7310 <itemizedlist>
7311 <listitem><para><filename>merge_config.sh</filename>:
7312 Helps you manage configuration files and fragments
7313 within the kernel.
7314 With this tool, you can merge individual configuration
7315 fragments together.
7316 The tool allows you to make overrides and warns you
7317 of any missing configuration options.
7318 The tool is ideal for allowing you to iterate on
7319 configurations, create minimal configurations, and
7320 create configuration files for different machines
7321 without having to duplicate your process.</para>
7322 <para>The <filename>merge_config.sh</filename> script is
7323 part of the Linux Yocto kernel Git repositories
7324 (i.e. <filename>linux-yocto-3.14</filename>,
7325 <filename>linux-yocto-3.10</filename>,
7326 <filename>linux-yocto-3.8</filename>, and so forth)
7327 in the
7328 <filename>scripts/kconfig</filename> directory.</para>
7329 <para>For more information on configuration fragments,
7330 see the
7331 "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#generating-configuration-files'>Generating Configuration Files</ulink>"
7332 section of the Yocto Project Linux Kernel Development
7333 Manual and the "<link linkend='creating-config-fragments'>Creating Configuration Fragments</link>"
7334 section, which is in this manual.</para></listitem>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05007335 <listitem><para><filename>bitbake -u taskexp -g <replaceable>bitbake_target</replaceable></filename>:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05007336 Using the BitBake command with these options brings up
7337 a Dependency Explorer from which you can view file
7338 dependencies.
7339 Understanding these dependencies allows you to make
7340 informed decisions when cutting out various pieces of the
7341 kernel and root filesystem.</para></listitem>
7342 </itemizedlist>
7343 </para>
7344 </section>
7345
7346 <section id='trim-the-root-filesystem'>
7347 <title>Trim the Root Filesystem</title>
7348
7349 <para>
7350 The root filesystem is made up of packages for booting,
7351 libraries, and applications.
7352 To change things, you can configure how the packaging happens,
7353 which changes the way you build them.
7354 You can also modify the filesystem itself or select a different
7355 filesystem.
7356 </para>
7357
7358 <para>
7359 First, find out what is hogging your root filesystem by running the
7360 <filename>dirsize.py</filename> script from your root directory:
7361 <literallayout class='monospaced'>
7362 $ cd <replaceable>root-directory-of-image</replaceable>
7363 $ dirsize.py 100000 > dirsize-100k.log
7364 $ cat dirsize-100k.log
7365 </literallayout>
7366 You can apply a filter to the script to ignore files under
7367 a certain size.
7368 The previous example filters out any files below 100 Kbytes.
7369 The sizes reported by the tool are uncompressed, and thus
7370 will be smaller by a relatively constant factor in a
7371 compressed root filesystem.
7372 When you examine your log file, you can focus on areas of the
7373 root filesystem that take up large amounts of memory.
7374 </para>
7375
7376 <para>
7377 You need to be sure that what you eliminate does not cripple
7378 the functionality you need.
7379 One way to see how packages relate to each other is by using
7380 the Dependency Explorer UI with the BitBake command:
7381 <literallayout class='monospaced'>
7382 $ cd <replaceable>image-directory</replaceable>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05007383 $ bitbake -u taskexp -g <replaceable>image</replaceable>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05007384 </literallayout>
7385 Use the interface to select potential packages you wish to
7386 eliminate and see their dependency relationships.
7387 </para>
7388
7389 <para>
7390 When deciding how to reduce the size, get rid of packages that
7391 result in minimal impact on the feature set.
7392 For example, you might not need a VGA display.
7393 Or, you might be able to get by with <filename>devtmpfs</filename>
7394 and <filename>mdev</filename> instead of
7395 <filename>udev</filename>.
7396 </para>
7397
7398 <para>
7399 Use your <filename>local.conf</filename> file to make changes.
7400 For example, to eliminate <filename>udev</filename> and
7401 <filename>glib</filename>, set the following in the
7402 local configuration file:
7403 <literallayout class='monospaced'>
7404 VIRTUAL-RUNTIME_dev_manager = ""
7405 </literallayout>
7406 </para>
7407
7408 <para>
7409 Finally, you should consider exactly the type of root
7410 filesystem you need to meet your needs while also reducing
7411 its size.
7412 For example, consider <filename>cramfs</filename>,
7413 <filename>squashfs</filename>, <filename>ubifs</filename>,
7414 <filename>ext2</filename>, or an <filename>initramfs</filename>
7415 using <filename>initramfs</filename>.
7416 Be aware that <filename>ext3</filename> requires a 1 Mbyte
7417 journal.
7418 If you are okay with running read-only, you do not need this
7419 journal.
7420 </para>
7421
7422 <note>
7423 After each round of elimination, you need to rebuild your
7424 system and then use the tools to see the effects of your
7425 reductions.
7426 </note>
7427
7428
7429 </section>
7430
7431 <section id='trim-the-kernel'>
7432 <title>Trim the Kernel</title>
7433
7434 <para>
7435 The kernel is built by including policies for hardware-independent
7436 aspects.
7437 What subsystems do you enable?
7438 For what architecture are you building?
7439 Which drivers do you build by default?
7440 <note>You can modify the kernel source if you want to help
7441 with boot time.
7442 </note>
7443 </para>
7444
7445 <para>
7446 Run the <filename>ksize.py</filename> script from the top-level
7447 Linux build directory to get an idea of what is making up
7448 the kernel:
7449 <literallayout class='monospaced'>
7450 $ cd <replaceable>top-level-linux-build-directory</replaceable>
7451 $ ksize.py > ksize.log
7452 $ cat ksize.log
7453 </literallayout>
7454 When you examine the log, you will see how much space is
7455 taken up with the built-in <filename>.o</filename> files for
7456 drivers, networking, core kernel files, filesystem, sound,
7457 and so forth.
7458 The sizes reported by the tool are uncompressed, and thus
7459 will be smaller by a relatively constant factor in a compressed
7460 kernel image.
7461 Look to reduce the areas that are large and taking up around
7462 the "90% rule."
7463 </para>
7464
7465 <para>
7466 To examine, or drill down, into any particular area, use the
7467 <filename>-d</filename> option with the script:
7468 <literallayout class='monospaced'>
7469 $ ksize.py -d > ksize.log
7470 </literallayout>
7471 Using this option breaks out the individual file information
7472 for each area of the kernel (e.g. drivers, networking, and
7473 so forth).
7474 </para>
7475
7476 <para>
7477 Use your log file to see what you can eliminate from the kernel
7478 based on features you can let go.
7479 For example, if you are not going to need sound, you do not
7480 need any drivers that support sound.
7481 </para>
7482
7483 <para>
7484 After figuring out what to eliminate, you need to reconfigure
7485 the kernel to reflect those changes during the next build.
7486 You could run <filename>menuconfig</filename> and make all your
7487 changes at once.
7488 However, that makes it difficult to see the effects of your
7489 individual eliminations and also makes it difficult to replicate
7490 the changes for perhaps another target device.
7491 A better method is to start with no configurations using
7492 <filename>allnoconfig</filename>, create configuration
7493 fragments for individual changes, and then manage the
7494 fragments into a single configuration file using
7495 <filename>merge_config.sh</filename>.
7496 The tool makes it easy for you to iterate using the
7497 configuration change and build cycle.
7498 </para>
7499
7500 <para>
7501 Each time you make configuration changes, you need to rebuild
7502 the kernel and check to see what impact your changes had on
7503 the overall size.
7504 </para>
7505 </section>
7506
7507 <section id='remove-package-management-requirements'>
7508 <title>Remove Package Management Requirements</title>
7509
7510 <para>
7511 Packaging requirements add size to the image.
7512 One way to reduce the size of the image is to remove all the
7513 packaging requirements from the image.
7514 This reduction includes both removing the package manager
7515 and its unique dependencies as well as removing the package
7516 management data itself.
7517 </para>
7518
7519 <para>
7520 To eliminate all the packaging requirements for an image,
7521 be sure that "package-management" is not part of your
7522 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
7523 statement for the image.
7524 When you remove this feature, you are removing the package
7525 manager as well as its dependencies from the root filesystem.
7526 </para>
7527 </section>
7528
7529 <section id='look-for-other-ways-to-minimize-size'>
7530 <title>Look for Other Ways to Minimize Size</title>
7531
7532 <para>
7533 Depending on your particular circumstances, other areas that you
7534 can trim likely exist.
7535 The key to finding these areas is through tools and methods
7536 described here combined with experimentation and iteration.
7537 Here are a couple of areas to experiment with:
7538 <itemizedlist>
7539 <listitem><para><filename>glibc</filename>:
7540 In general, follow this process:
7541 <orderedlist>
7542 <listitem><para>Remove <filename>glibc</filename>
7543 features from
7544 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>
7545 that you think you do not need.</para></listitem>
7546 <listitem><para>Build your distribution.
7547 </para></listitem>
7548 <listitem><para>If the build fails due to missing
7549 symbols in a package, determine if you can
7550 reconfigure the package to not need those
7551 features.
7552 For example, change the configuration to not
7553 support wide character support as is done for
7554 <filename>ncurses</filename>.
7555 Or, if support for those characters is needed,
7556 determine what <filename>glibc</filename>
7557 features provide the support and restore the
7558 configuration.
7559 </para></listitem>
7560 <listitem><para>Rebuild and repeat the process.
7561 </para></listitem>
7562 </orderedlist></para></listitem>
7563 <listitem><para><filename>busybox</filename>:
7564 For BusyBox, use a process similar as described for
7565 <filename>glibc</filename>.
7566 A difference is you will need to boot the resulting
7567 system to see if you are able to do everything you
7568 expect from the running system.
7569 You need to be sure to integrate configuration fragments
7570 into Busybox because BusyBox handles its own core
7571 features and then allows you to add configuration
7572 fragments on top.
7573 </para></listitem>
7574 </itemizedlist>
7575 </para>
7576 </section>
7577
7578 <section id='iterate-on-the-process'>
7579 <title>Iterate on the Process</title>
7580
7581 <para>
7582 If you have not reached your goals on system size, you need
7583 to iterate on the process.
7584 The process is the same.
7585 Use the tools and see just what is taking up 90% of the root
7586 filesystem and the kernel.
7587 Decide what you can eliminate without limiting your device
7588 beyond what you need.
7589 </para>
7590
7591 <para>
7592 Depending on your system, a good place to look might be
7593 Busybox, which provides a stripped down
7594 version of Unix tools in a single, executable file.
7595 You might be able to drop virtual terminal services or perhaps
7596 ipv6.
7597 </para>
7598 </section>
7599 </section>
7600
7601 <section id='building-images-for-more-than-one-machine'>
7602 <title>Building Images for More than One Machine</title>
7603
7604 <para>
7605 A common scenario developers face is creating images for several
7606 different machines that use the same software environment.
7607 In this situation, it is tempting to set the
7608 tunings and optimization flags for each build specifically for
7609 the targeted hardware (i.e. "maxing out" the tunings).
7610 Doing so can considerably add to build times and package feed
7611 maintenance collectively for the machines.
7612 For example, selecting tunes that are extremely specific to a
7613 CPU core used in a system might enable some micro optimizations
7614 in GCC for that particular system but would otherwise not gain
7615 you much of a performance difference across the other systems
7616 as compared to using a more general tuning across all the builds
7617 (e.g. setting
Brad Bishop6e60e8b2018-02-01 10:27:11 -05007618 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEFAULTTUNE'><filename>DEFAULTTUNE</filename></ulink>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05007619 specifically for each machine's build).
7620 Rather than "max out" each build's tunings, you can take steps that
7621 cause the OpenEmbedded build system to reuse software across the
7622 various machines where it makes sense.
7623 </para>
7624 <para>
7625 If build speed and package feed maintenance are considerations,
7626 you should consider the points in this section that can help you
7627 optimize your tunings to best consider build times and package
7628 feed maintenance.
7629 <itemizedlist>
7630 <listitem><para><emphasis>Share the Build Directory:</emphasis>
7631 If at all possible, share the
7632 <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>
7633 across builds.
7634 The Yocto Project supports switching between different
7635 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
7636 values in the same <filename>TMPDIR</filename>.
7637 This practice is well supported and regularly used by
7638 developers when building for multiple machines.
7639 When you use the same <filename>TMPDIR</filename> for
7640 multiple machine builds, the OpenEmbedded build system can
7641 reuse the existing native and often cross-recipes for
7642 multiple machines.
7643 Thus, build time decreases.
7644 <note>
7645 If
7646 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink>
7647 settings change or fundamental configuration settings
7648 such as the filesystem layout, you need to work with
7649 a clean <filename>TMPDIR</filename>.
7650 Sharing <filename>TMPDIR</filename> under these
7651 circumstances might work but since it is not
7652 guaranteed, you should use a clean
7653 <filename>TMPDIR</filename>.
7654 </note>
7655 </para></listitem>
7656 <listitem><para><emphasis>Enable the Appropriate Package Architecture:</emphasis>
7657 By default, the OpenEmbedded build system enables three
7658 levels of package architectures: "all", "tune" or "package",
7659 and "machine".
7660 Any given recipe usually selects one of these package
7661 architectures (types) for its output.
7662 Depending for what a given recipe creates packages, making
7663 sure you enable the appropriate package architecture can
7664 directly impact the build time.</para>
7665 <para>A recipe that just generates scripts can enable
7666 "all" architecture because there are no binaries to build.
7667 To specifically enable "all" architecture, be sure your
7668 recipe inherits the
7669 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-allarch'><filename>allarch</filename></ulink>
7670 class.
7671 This class is useful for "all" architectures because it
7672 configures many variables so packages can be used across
7673 multiple architectures.</para>
7674 <para>If your recipe needs to generate packages that are
7675 machine-specific or when one of the build or runtime
7676 dependencies is already machine-architecture dependent,
7677 which makes your recipe also machine-architecture dependent,
7678 make sure your recipe enables the "machine" package
7679 architecture through the
7680 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_ARCH'><filename>MACHINE_ARCH</filename></ulink>
7681 variable:
7682 <literallayout class='monospaced'>
7683 PACKAGE_ARCH = "${MACHINE_ARCH}"
7684 </literallayout>
7685 When you do not specifically enable a package
7686 architecture through the
7687 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink>,
7688 The OpenEmbedded build system defaults to the
7689 <ulink url='&YOCTO_DOCS_REF_URL;#var-TUNE_PKGARCH'><filename>TUNE_PKGARCH</filename></ulink>
7690 setting:
7691 <literallayout class='monospaced'>
7692 PACKAGE_ARCH = "${TUNE_PKGARCH}"
7693 </literallayout>
7694 </para></listitem>
7695 <listitem><para><emphasis>Choose a Generic Tuning File if Possible:</emphasis>
7696 Some tunes are more generic and can run on multiple targets
7697 (e.g. an <filename>armv5</filename> set of packages could
7698 run on <filename>armv6</filename> and
7699 <filename>armv7</filename> processors in most cases).
7700 Similarly, <filename>i486</filename> binaries could work
7701 on <filename>i586</filename> and higher processors.
7702 You should realize, however, that advances on newer
7703 processor versions would not be used.</para>
7704 <para>If you select the same tune for several different
7705 machines, the OpenEmbedded build system reuses software
7706 previously built, thus speeding up the overall build time.
7707 Realize that even though a new sysroot for each machine is
7708 generated, the software is not recompiled and only one
7709 package feed exists.
7710 </para></listitem>
7711 <listitem><para><emphasis>Manage Granular Level Packaging:</emphasis>
7712 Sometimes cases exist where injecting another level
7713 of package architecture beyond the three higher levels
7714 noted earlier can be useful.
7715 For example, consider the <filename>emgd</filename>
7716 graphics stack in the
7717 <filename>meta-intel</filename> layer.
7718 In this layer, a subset of software exists that is
7719 compiled against something different from the rest of the
7720 generic packages.
7721 You can examine the key code in the
7722 <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi'>Source Repositories</ulink>
7723 "daisy" branch in
7724 <filename>classes/emgd-gl.bbclass</filename>.
7725 For a specific set of packages, the code redefines
7726 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink>.
7727 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_EXTRA_ARCHS'><filename>PACKAGE_EXTRA_ARCHS</filename></ulink>
7728 is then appended with this extra tune name in
7729 <filename>meta-intel-emgd.inc</filename>.
7730 The result is that when searching for packages, the
7731 build system uses a four-level search and the packages
7732 in this new level are preferred as compared to the standard
7733 tune.
7734 The overall result is that the build system reuses most
7735 software from the common tune except for specific cases
7736 as needed.
7737 </para></listitem>
7738 <listitem><para><emphasis>Use Tools to Debug Issues:</emphasis>
7739 Sometimes you can run into situations where software is
7740 being rebuilt when you think it should not be.
7741 For example, the OpenEmbedded build system might not be
7742 using shared state between machines when you think it
7743 should be.
7744 These types of situations are usually due to references
7745 to machine-specific variables such as
7746 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>,
7747 <ulink url='&YOCTO_DOCS_REF_URL;#var-SERIAL_CONSOLE'><filename>SERIAL_CONSOLE</filename></ulink>,
7748 <ulink url='&YOCTO_DOCS_REF_URL;#var-XSERVER'><filename>XSERVER</filename></ulink>,
7749 <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES'><filename>MACHINE_FEATURES</filename></ulink>,
7750 and so forth in code that is supposed to only be
7751 tune-specific or when the recipe depends
7752 (<ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>,
7753 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>,
7754 <ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'><filename>RRECOMMENDS</filename></ulink>,
7755 <ulink url='&YOCTO_DOCS_REF_URL;#var-RSUGGESTS'><filename>RSUGGESTS</filename></ulink>,
7756 and so forth) on some other recipe that already has
7757 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink>
7758 defined as "${MACHINE_ARCH}".
7759 <note>
7760 Patches to fix any issues identified are most welcome
7761 as these issues occasionally do occur.
7762 </note></para>
7763 <para>For such cases, you can use some tools to help you
7764 sort out the situation:
7765 <itemizedlist>
7766 <listitem><para><emphasis><filename>sstate-diff-machines.sh</filename>:</emphasis>
7767 You can find this tool in the
7768 <filename>scripts</filename> directory of the
7769 Source Repositories.
7770 See the comments in the script for information on
7771 how to use the tool.
7772 </para></listitem>
7773 <listitem><para><emphasis>BitBake's "-S printdiff" Option:</emphasis>
7774 Using this option causes BitBake to try to
7775 establish the closest signature match it can
7776 (e.g. in the shared state cache) and then run
7777 <filename>bitbake-diffsigs</filename> over the
7778 matches to determine the stamps and delta where
7779 these two stamp trees diverge.
7780 </para></listitem>
7781 </itemizedlist>
7782 </para></listitem>
7783 </itemizedlist>
7784 </para>
7785 </section>
7786
7787 <section id='working-with-packages'>
7788 <title>Working with Packages</title>
7789
7790 <para>
7791 This section describes a few tasks that involve packages:
7792 <itemizedlist>
7793 <listitem><para>
7794 <link linkend='excluding-packages-from-an-image'>Excluding packages from an image</link>
7795 </para></listitem>
7796 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05007797 <link linkend='incrementing-a-binary-package-version'>Incrementing a binary package version</link>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05007798 </para></listitem>
7799 <listitem><para>
7800 <link linkend='handling-optional-module-packaging'>Handling optional module packaging</link>
7801 </para></listitem>
7802 <listitem><para>
7803 <link linkend='using-runtime-package-management'>Using Runtime Package Management</link>
7804 </para></listitem>
7805 <listitem><para>
7806 <link linkend='testing-packages-with-ptest'>Setting up and running package test (ptest)</link>
7807 </para></listitem>
7808 </itemizedlist>
7809 </para>
7810
7811 <section id='excluding-packages-from-an-image'>
7812 <title>Excluding Packages from an Image</title>
7813
7814 <para>
7815 You might find it necessary to prevent specific packages
7816 from being installed into an image.
7817 If so, you can use several variables to direct the build
7818 system to essentially ignore installing recommended packages
7819 or to not install a package at all.
7820 </para>
7821
7822 <para>
7823 The following list introduces variables you can use to
7824 prevent packages from being installed into your image.
7825 Each of these variables only works with IPK and RPM
7826 package types.
7827 Support for Debian packages does not exist.
7828 Also, you can use these variables from your
7829 <filename>local.conf</filename> file or attach them to a
7830 specific image recipe by using a recipe name override.
7831 For more detail on the variables, see the descriptions in the
7832 Yocto Project Reference Manual's glossary chapter.
7833 <itemizedlist>
7834 <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-BAD_RECOMMENDATIONS'><filename>BAD_RECOMMENDATIONS</filename></ulink>:
7835 Use this variable to specify "recommended-only"
7836 packages that you do not want installed.
7837 </para></listitem>
7838 <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-NO_RECOMMENDATIONS'><filename>NO_RECOMMENDATIONS</filename></ulink>:
7839 Use this variable to prevent all "recommended-only"
7840 packages from being installed.
7841 </para></listitem>
7842 <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_EXCLUDE'><filename>PACKAGE_EXCLUDE</filename></ulink>:
7843 Use this variable to prevent specific packages from
7844 being installed regardless of whether they are
7845 "recommended-only" or not.
7846 You need to realize that the build process could
7847 fail with an error when you
7848 prevent the installation of a package whose presence
7849 is required by an installed package.
7850 </para></listitem>
7851 </itemizedlist>
7852 </para>
7853 </section>
7854
Brad Bishop6e60e8b2018-02-01 10:27:11 -05007855 <section id='incrementing-a-binary-package-version'>
7856 <title>Incrementing a Package Version</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05007857
7858 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05007859 This section provides some background on how binary package
7860 versioning is accomplished and presents some of the services,
7861 variables, and terminology involved.
7862 </para>
7863
7864 <para>
7865 In order to understand binary package versioning, you need
7866 to consider the following:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05007867 <itemizedlist>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05007868 <listitem><para>
7869 Binary Package: The binary package that is eventually
7870 built and installed into an image.
7871 </para></listitem>
7872 <listitem><para>
7873 Binary Package Version: The binary package version
7874 is composed of two components - a version and a
7875 revision.
7876 <note>
7877 Technically, a third component, the "epoch" (i.e.
7878 <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>)
7879 is involved but this discussion for the most part
7880 ignores <filename>PE</filename>.
7881 </note>
7882 The version and revision are taken from the
7883 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>
7884 and
7885 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>
7886 variables, respectively.
7887 </para></listitem>
7888 <listitem><para>
7889 <filename>PV</filename>: The recipe version.
7890 <filename>PV</filename> represents the version of the
7891 software being packaged.
7892 Do not confuse <filename>PV</filename> with the
7893 binary package version.
7894 </para></listitem>
7895 <listitem><para>
7896 <filename>PR</filename>: The recipe revision.
7897 </para></listitem>
7898 <listitem><para>
7899 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCPV'><filename>SRCPV</filename></ulink>:
7900 The OpenEmbedded build system uses this string
7901 to help define the value of <filename>PV</filename>
7902 when the source code revision needs to be included
7903 in it.
7904 </para></listitem>
7905 <listitem><para>
7906 <ulink url='https://wiki.yoctoproject.org/wiki/PR_Service'>PR Service</ulink>:
7907 A network-based service that helps automate keeping
7908 package feeds compatible with existing package
7909 manager applications such as RPM, APT, and OPKG.
7910 </para></listitem>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05007911 </itemizedlist>
7912 </para>
7913
7914 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05007915 Whenever the binary package content changes, the binary package
7916 version must change.
7917 Changing the binary package version is accomplished by changing
7918 or "bumping" the <filename>PR</filename> and/or
7919 <filename>PV</filename> values.
7920 Increasing these values occurs one of two ways:
7921 <itemizedlist>
7922 <listitem><para>Automatically using a Package Revision
7923 Service (PR Service).
7924 </para></listitem>
7925 <listitem><para>Manually incrementing the
7926 <filename>PR</filename> and/or
7927 <filename>PV</filename> variables.
7928 </para></listitem>
7929 </itemizedlist>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05007930 </para>
7931
7932 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05007933 Given a primary challenge of any build system and its users
7934 is how to maintain a package feed that is compatible with
7935 existing package manager applications such as RPM, APT, and
7936 OPKG, using an automated system is much preferred over a
7937 manual system.
7938 In either system, the main requirement is that binary package
7939 version numbering increases in a linear fashion and that a
7940 number of version components exist that support that linear
7941 progression.
7942 For information on how to ensure package revisioning remains
7943 linear, see the
7944 "<link linkend='automatically-incrementing-a-binary-package-revision-number'>Automatically Incrementing a Binary Package Revision Number</link>"
7945 section.
7946 </para>
7947
7948 <para>
7949 The following three sections provide related information on the
7950 PR Service, the manual method for "bumping"
7951 <filename>PR</filename> and/or <filename>PV</filename>, and
7952 on how to ensure binary package revisioning remains linear.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05007953 </para>
7954
7955 <section id='working-with-a-pr-service'>
7956 <title>Working With a PR Service</title>
7957
7958 <para>
7959 As mentioned, attempting to maintain revision numbers in the
7960 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>
7961 is error prone, inaccurate, and causes problems for people
7962 submitting recipes.
7963 Conversely, the PR Service automatically generates
7964 increasing numbers, particularly the revision field,
7965 which removes the human element.
7966 <note>
7967 For additional information on using a PR Service, you
7968 can see the
7969 <ulink url='&YOCTO_WIKI_URL;/wiki/PR_Service'>PR Service</ulink>
7970 wiki page.
7971 </note>
7972 </para>
7973
7974 <para>
7975 The Yocto Project uses variables in order of
7976 decreasing priority to facilitate revision numbering (i.e.
7977 <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>,
7978 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>, and
7979 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>
7980 for epoch, version, and revision, respectively).
7981 The values are highly dependent on the policies and
7982 procedures of a given distribution and package feed.
7983 </para>
7984
7985 <para>
7986 Because the OpenEmbedded build system uses
7987 "<ulink url='&YOCTO_DOCS_REF_URL;#checksums'>signatures</ulink>",
7988 which are unique to a given build, the build system
7989 knows when to rebuild packages.
7990 All the inputs into a given task are represented by a
7991 signature, which can trigger a rebuild when different.
7992 Thus, the build system itself does not rely on the
Brad Bishop6e60e8b2018-02-01 10:27:11 -05007993 <filename>PR</filename>, <filename>PV</filename>, and
7994 <filename>PE</filename> numbers to trigger a rebuild.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05007995 The signatures, however, can be used to generate
Brad Bishop6e60e8b2018-02-01 10:27:11 -05007996 these values.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05007997 </para>
7998
7999 <para>
8000 The PR Service works with both
8001 <filename>OEBasic</filename> and
8002 <filename>OEBasicHash</filename> generators.
8003 The value of <filename>PR</filename> bumps when the
8004 checksum changes and the different generator mechanisms
8005 change signatures under different circumstances.
8006 </para>
8007
8008 <para>
8009 As implemented, the build system includes values from
8010 the PR Service into the <filename>PR</filename> field as
8011 an addition using the form "<filename>.x</filename>" so
8012 <filename>r0</filename> becomes <filename>r0.1</filename>,
8013 <filename>r0.2</filename> and so forth.
8014 This scheme allows existing <filename>PR</filename> values
8015 to be used for whatever reasons, which include manual
8016 <filename>PR</filename> bumps, should it be necessary.
8017 </para>
8018
8019 <para>
8020 By default, the PR Service is not enabled or running.
8021 Thus, the packages generated are just "self consistent".
8022 The build system adds and removes packages and
8023 there are no guarantees about upgrade paths but images
8024 will be consistent and correct with the latest changes.
8025 </para>
8026
8027 <para>
8028 The simplest form for a PR Service is for it to exist
8029 for a single host development system that builds the
8030 package feed (building system).
8031 For this scenario, you can enable a local PR Service by
8032 setting
8033 <ulink url='&YOCTO_DOCS_REF_URL;#var-PRSERV_HOST'><filename>PRSERV_HOST</filename></ulink>
8034 in your <filename>local.conf</filename> file in the
8035 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>:
8036 <literallayout class='monospaced'>
8037 PRSERV_HOST = "localhost:0"
8038 </literallayout>
8039 Once the service is started, packages will automatically
8040 get increasing <filename>PR</filename> values and
Brad Bishop6e60e8b2018-02-01 10:27:11 -05008041 BitBake takes care of starting and stopping the server.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008042 </para>
8043
8044 <para>
8045 If you have a more complex setup where multiple host
8046 development systems work against a common, shared package
8047 feed, you have a single PR Service running and it is
8048 connected to each building system.
8049 For this scenario, you need to start the PR Service using
8050 the <filename>bitbake-prserv</filename> command:
8051 <literallayout class='monospaced'>
8052 bitbake-prserv --host <replaceable>ip</replaceable> --port <replaceable>port</replaceable> --start
8053 </literallayout>
8054 In addition to hand-starting the service, you need to
8055 update the <filename>local.conf</filename> file of each
8056 building system as described earlier so each system
8057 points to the server and port.
8058 </para>
8059
8060 <para>
8061 It is also recommended you use build history, which adds
Brad Bishop6e60e8b2018-02-01 10:27:11 -05008062 some sanity checks to binary package versions, in
8063 conjunction with the server that is running the PR Service.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008064 To enable build history, add the following to each building
8065 system's <filename>local.conf</filename> file:
8066 <literallayout class='monospaced'>
8067 # It is recommended to activate "buildhistory" for testing the PR service
8068 INHERIT += "buildhistory"
8069 BUILDHISTORY_COMMIT = "1"
8070 </literallayout>
8071 For information on build history, see the
8072 "<ulink url='&YOCTO_DOCS_REF_URL;#maintaining-build-output-quality'>Maintaining Build Output Quality</ulink>"
8073 section in the Yocto Project Reference Manual.
8074 </para>
8075
8076 <note>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05008077 <para>
8078 The OpenEmbedded build system does not maintain
8079 <filename>PR</filename> information as part of the
8080 shared state (sstate) packages.
8081 If you maintain an sstate feed, its expected that either
8082 all your building systems that contribute to the sstate
8083 feed use a shared PR Service, or you do not run a PR
8084 Service on any of your building systems.
8085 Having some systems use a PR Service while others do
8086 not leads to obvious problems.
8087 </para>
8088
8089 <para>
8090 For more information on shared state, see the
8091 "<ulink url='&YOCTO_DOCS_REF_URL;#shared-state-cache'>Shared State Cache</ulink>"
8092 section in the Yocto Project Reference Manual.
8093 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008094 </note>
8095 </section>
8096
8097 <section id='manually-bumping-pr'>
8098 <title>Manually Bumping PR</title>
8099
8100 <para>
8101 The alternative to setting up a PR Service is to manually
Brad Bishop6e60e8b2018-02-01 10:27:11 -05008102 "bump" the
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008103 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>
8104 variable.
8105 </para>
8106
8107 <para>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05008108 If a committed change results in changing the package
8109 output, then the value of the PR variable needs to be
8110 increased (or "bumped") as part of that commit.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008111 For new recipes you should add the <filename>PR</filename>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05008112 variable and set its initial value equal to "r0", which is
8113 the default.
8114 Even though the default value is "r0", the practice of
8115 adding it to a new recipe makes it harder to forget to bump
8116 the variable when you make changes to the recipe in future.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008117 </para>
8118
8119 <para>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05008120 If you are sharing a common <filename>.inc</filename> file
8121 with multiple recipes, you can also use the
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008122 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-INC_PR'>INC_PR</ulink></filename>
Brad Bishop37a0e4d2017-12-04 01:01:44 -05008123 variable to ensure that the recipes sharing the
8124 <filename>.inc</filename> file are rebuilt when the
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008125 <filename>.inc</filename> file itself is changed.
Brad Bishop37a0e4d2017-12-04 01:01:44 -05008126 The <filename>.inc</filename> file must set
8127 <filename>INC_PR</filename> (initially to "r0"), and all
8128 recipes referring to it should set <filename>PR</filename>
8129 to "${INC_PR}.0" initially, incrementing the last number
8130 when the recipe is changed.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008131 If the <filename>.inc</filename> file is changed then its
8132 <filename>INC_PR</filename> should be incremented.
8133 </para>
8134
8135 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05008136 When upgrading the version of a binary package, assuming the
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008137 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PV'>PV</ulink></filename>
8138 changes, the <filename>PR</filename> variable should be
Brad Bishop37a0e4d2017-12-04 01:01:44 -05008139 reset to "r0" (or "${INC_PR}.0" if you are using
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008140 <filename>INC_PR</filename>).
8141 </para>
8142
8143 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05008144 Usually, version increases occur only to binary packages.
Brad Bishop37a0e4d2017-12-04 01:01:44 -05008145 However, if for some reason <filename>PV</filename> changes
8146 but does not increase, you can increase the
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008147 <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PE'>PE</ulink></filename>
8148 variable (Package Epoch).
8149 The <filename>PE</filename> variable defaults to "0".
8150 </para>
8151
8152 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05008153 Binary package version numbering strives to follow the
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008154 <ulink url='http://www.debian.org/doc/debian-policy/ch-controlfields.html'>
8155 Debian Version Field Policy Guidelines</ulink>.
Brad Bishop37a0e4d2017-12-04 01:01:44 -05008156 These guidelines define how versions are compared and what
8157 "increasing" a version means.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008158 </para>
8159 </section>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05008160
8161 <section id='automatically-incrementing-a-binary-package-revision-number'>
8162 <title>Automatically Incrementing a Package Version Number</title>
8163
8164 <para>
8165 When fetching a repository, BitBake uses the
8166 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink>
8167 variable to determine the specific source code revision
8168 from which to build.
8169 You set the <filename>SRCREV</filename> variable to
8170 <ulink url='&YOCTO_DOCS_REF_URL;#var-AUTOREV'><filename>AUTOREV</filename></ulink>
8171 to cause the OpenEmbedded build system to automatically use the
8172 latest revision of the software:
8173 <literallayout class='monospaced'>
8174 SRCREV = "${AUTOREV}"
8175 </literallayout>
8176 </para>
8177
8178 <para>
8179 Furthermore, you need to reference <filename>SRCPV</filename>
8180 in <filename>PV</filename> in order to automatically update
8181 the version whenever the revision of the source code
8182 changes.
8183 Here is an example:
8184 <literallayout class='monospaced'>
8185 PV = "1.0+git${SRCPV}"
8186 </literallayout>
8187 The OpenEmbedded build system substitutes
8188 <filename>SRCPV</filename> with the following:
8189 <literallayout class='monospaced'>
8190 AUTOINC+<replaceable>source_code_revision</replaceable>
8191 </literallayout>
8192 The build system replaces the <filename>AUTOINC</filename> with
8193 a number.
8194 The number used depends on the state of the PR Service:
8195 <itemizedlist>
8196 <listitem><para>
8197 If PR Service is enabled, the build system increments
8198 the number, which is similar to the behavior of
8199 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>.
8200 This behavior results in linearly increasing package
8201 versions, which is desirable.
8202 Here is an example:
8203 <literallayout class='monospaced'>
8204 hello-world-git_0.0+git0+b6558dd387-r0.0_armv7a-neon.ipk
8205 hello-world-git_0.0+git1+dd2f5c3565-r0.0_armv7a-neon.ipk
8206 </literallayout>
8207 </para></listitem>
8208 <listitem><para>
8209 If PR Service is not enabled, the build system
8210 replaces the <filename>AUTOINC</filename>
8211 placeholder with zero (i.e. "0").
8212 This results in changing the package version since
8213 the source revision is included.
8214 However, package versions are not increased linearly.
8215 Here is an example:
8216 <literallayout class='monospaced'>
8217 hello-world-git_0.0+git0+b6558dd387-r0.0_armv7a-neon.ipk
8218 hello-world-git_0.0+git0+dd2f5c3565-r0.0_armv7a-neon.ipk
8219 </literallayout>
8220 </para></listitem>
8221 </itemizedlist>
8222 </para>
8223
8224 <para>
8225 In summary, the OpenEmbedded build system does not track the
8226 history of binary package versions for this purpose.
8227 <filename>AUTOINC</filename>, in this case, is comparable to
8228 <filename>PR</filename>.
8229 If PR server is not enabled, <filename>AUTOINC</filename>
8230 in the package version is simply replaced by "0".
8231 If PR server is enabled, the build system keeps track of the
8232 package versions and bumps the number when the package
8233 revision changes.
8234 </para>
8235 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008236 </section>
8237
8238 <section id='handling-optional-module-packaging'>
8239 <title>Handling Optional Module Packaging</title>
8240
8241 <para>
8242 Many pieces of software split functionality into optional
8243 modules (or plug-ins) and the plug-ins that are built
8244 might depend on configuration options.
8245 To avoid having to duplicate the logic that determines what
8246 modules are available in your recipe or to avoid having
8247 to package each module by hand, the OpenEmbedded build system
8248 provides functionality to handle module packaging dynamically.
8249 </para>
8250
8251 <para>
8252 To handle optional module packaging, you need to do two things:
8253 <itemizedlist>
8254 <listitem><para>Ensure the module packaging is actually
8255 done.</para></listitem>
8256 <listitem><para>Ensure that any dependencies on optional
8257 modules from other recipes are satisfied by your recipe.
8258 </para></listitem>
8259 </itemizedlist>
8260 </para>
8261
8262 <section id='making-sure-the-packaging-is-done'>
8263 <title>Making Sure the Packaging is Done</title>
8264
8265 <para>
8266 To ensure the module packaging actually gets done, you use
8267 the <filename>do_split_packages</filename> function within
8268 the <filename>populate_packages</filename> Python function
8269 in your recipe.
8270 The <filename>do_split_packages</filename> function
8271 searches for a pattern of files or directories under a
8272 specified path and creates a package for each one it finds
8273 by appending to the
8274 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>
8275 variable and setting the appropriate values for
8276 <filename>FILES_packagename</filename>,
8277 <filename>RDEPENDS_packagename</filename>,
8278 <filename>DESCRIPTION_packagename</filename>, and so forth.
8279 Here is an example from the <filename>lighttpd</filename>
8280 recipe:
8281 <literallayout class='monospaced'>
8282 python populate_packages_prepend () {
8283 lighttpd_libdir = d.expand('${libdir}')
8284 do_split_packages(d, lighttpd_libdir, '^mod_(.*)\.so$',
8285 'lighttpd-module-%s', 'Lighttpd module for %s',
8286 extra_depends='')
8287 }
8288 </literallayout>
8289 The previous example specifies a number of things in the
8290 call to <filename>do_split_packages</filename>.
8291 <itemizedlist>
8292 <listitem><para>A directory within the files installed
8293 by your recipe through <filename>do_install</filename>
8294 in which to search.</para></listitem>
8295 <listitem><para>A regular expression used to match module
8296 files in that directory.
8297 In the example, note the parentheses () that mark
8298 the part of the expression from which the module
8299 name should be derived.</para></listitem>
8300 <listitem><para>A pattern to use for the package names.
8301 </para></listitem>
8302 <listitem><para>A description for each package.
8303 </para></listitem>
8304 <listitem><para>An empty string for
8305 <filename>extra_depends</filename>, which disables
8306 the default dependency on the main
8307 <filename>lighttpd</filename> package.
8308 Thus, if a file in <filename>${libdir}</filename>
8309 called <filename>mod_alias.so</filename> is found,
8310 a package called <filename>lighttpd-module-alias</filename>
8311 is created for it and the
8312 <ulink url='&YOCTO_DOCS_REF_URL;#var-DESCRIPTION'><filename>DESCRIPTION</filename></ulink>
8313 is set to "Lighttpd module for alias".</para></listitem>
8314 </itemizedlist>
8315 </para>
8316
8317 <para>
8318 Often, packaging modules is as simple as the previous
8319 example.
8320 However, more advanced options exist that you can use
8321 within <filename>do_split_packages</filename> to modify its
8322 behavior.
8323 And, if you need to, you can add more logic by specifying
8324 a hook function that is called for each package.
8325 It is also perfectly acceptable to call
8326 <filename>do_split_packages</filename> multiple times if
8327 you have more than one set of modules to package.
8328 </para>
8329
8330 <para>
8331 For more examples that show how to use
8332 <filename>do_split_packages</filename>, see the
8333 <filename>connman.inc</filename> file in the
8334 <filename>meta/recipes-connectivity/connman/</filename>
8335 directory of the <filename>poky</filename>
8336 <link linkend='yocto-project-repositories'>source repository</link>.
8337 You can also find examples in
8338 <filename>meta/classes/kernel.bbclass</filename>.
8339 </para>
8340
8341 <para>
8342 Following is a reference that shows
8343 <filename>do_split_packages</filename> mandatory and
8344 optional arguments:
8345 <literallayout class='monospaced'>
8346 Mandatory arguments
8347
8348 root
8349 The path in which to search
8350 file_regex
8351 Regular expression to match searched files.
8352 Use parentheses () to mark the part of this
8353 expression that should be used to derive the
8354 module name (to be substituted where %s is
8355 used in other function arguments as noted below)
8356 output_pattern
8357 Pattern to use for the package names. Must
8358 include %s.
8359 description
8360 Description to set for each package. Must
8361 include %s.
8362
8363 Optional arguments
8364
8365 postinst
8366 Postinstall script to use for all packages
8367 (as a string)
8368 recursive
8369 True to perform a recursive search - default
8370 False
8371 hook
8372 A hook function to be called for every match.
8373 The function will be called with the following
8374 arguments (in the order listed):
8375
8376 f
8377 Full path to the file/directory match
8378 pkg
8379 The package name
8380 file_regex
8381 As above
8382 output_pattern
8383 As above
8384 modulename
8385 The module name derived using file_regex
8386
8387 extra_depends
8388 Extra runtime dependencies (RDEPENDS) to be
8389 set for all packages. The default value of None
8390 causes a dependency on the main package
8391 (${PN}) - if you do not want this, pass empty
8392 string '' for this parameter.
8393 aux_files_pattern
8394 Extra item(s) to be added to FILES for each
8395 package. Can be a single string item or a list
8396 of strings for multiple items. Must include %s.
8397 postrm
8398 postrm script to use for all packages (as a
8399 string)
8400 allow_dirs
8401 True to allow directories to be matched -
8402 default False
8403 prepend
8404 If True, prepend created packages to PACKAGES
8405 instead of the default False which appends them
8406 match_path
8407 match file_regex on the whole relative path to
8408 the root rather than just the file name
8409 aux_files_pattern_verbatim
8410 Extra item(s) to be added to FILES for each
8411 package, using the actual derived module name
8412 rather than converting it to something legal
8413 for a package name. Can be a single string item
8414 or a list of strings for multiple items. Must
8415 include %s.
8416 allow_links
8417 True to allow symlinks to be matched - default
8418 False
8419 summary
8420 Summary to set for each package. Must include %s;
8421 defaults to description if not set.
8422 </literallayout>
8423 </para>
8424 </section>
8425
8426 <section id='satisfying-dependencies'>
8427 <title>Satisfying Dependencies</title>
8428
8429 <para>
8430 The second part for handling optional module packaging
8431 is to ensure that any dependencies on optional modules
8432 from other recipes are satisfied by your recipe.
8433 You can be sure these dependencies are satisfied by
8434 using the
8435 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES_DYNAMIC'><filename>PACKAGES_DYNAMIC</filename></ulink> variable.
8436 Here is an example that continues with the
8437 <filename>lighttpd</filename> recipe shown earlier:
8438 <literallayout class='monospaced'>
8439 PACKAGES_DYNAMIC = "lighttpd-module-.*"
8440 </literallayout>
8441 The name specified in the regular expression can of
8442 course be anything.
8443 In this example, it is <filename>lighttpd-module-</filename>
8444 and is specified as the prefix to ensure that any
8445 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>
8446 and <ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'><filename>RRECOMMENDS</filename></ulink>
8447 on a package name starting with the prefix are satisfied
8448 during build time.
8449 If you are using <filename>do_split_packages</filename>
8450 as described in the previous section, the value you put in
8451 <filename>PACKAGES_DYNAMIC</filename> should correspond to
8452 the name pattern specified in the call to
8453 <filename>do_split_packages</filename>.
8454 </para>
8455 </section>
8456 </section>
8457
8458 <section id='using-runtime-package-management'>
8459 <title>Using Runtime Package Management</title>
8460
8461 <para>
8462 During a build, BitBake always transforms a recipe into one or
8463 more packages.
8464 For example, BitBake takes the <filename>bash</filename> recipe
8465 and currently produces the <filename>bash-dbg</filename>,
8466 <filename>bash-staticdev</filename>,
8467 <filename>bash-dev</filename>, <filename>bash-doc</filename>,
8468 <filename>bash-locale</filename>, and
8469 <filename>bash</filename> packages.
8470 Not all generated packages are included in an image.
8471 </para>
8472
8473 <para>
8474 In several situations, you might need to update, add, remove,
8475 or query the packages on a target device at runtime
8476 (i.e. without having to generate a new image).
8477 Examples of such situations include:
8478 <itemizedlist>
8479 <listitem><para>
8480 You want to provide in-the-field updates to deployed
8481 devices (e.g. security updates).
8482 </para></listitem>
8483 <listitem><para>
8484 You want to have a fast turn-around development cycle
8485 for one or more applications that run on your device.
8486 </para></listitem>
8487 <listitem><para>
8488 You want to temporarily install the "debug" packages
8489 of various applications on your device so that
8490 debugging can be greatly improved by allowing
8491 access to symbols and source debugging.
8492 </para></listitem>
8493 <listitem><para>
8494 You want to deploy a more minimal package selection of
8495 your device but allow in-the-field updates to add a
8496 larger selection for customization.
8497 </para></listitem>
8498 </itemizedlist>
8499 </para>
8500
8501 <para>
8502 In all these situations, you have something similar to a more
8503 traditional Linux distribution in that in-field devices
8504 are able to receive pre-compiled packages from a server for
8505 installation or update.
8506 Being able to install these packages on a running,
8507 in-field device is what is termed "runtime package
8508 management".
8509 </para>
8510
8511 <para>
8512 In order to use runtime package management, you
8513 need a host/server machine that serves up the pre-compiled
8514 packages plus the required metadata.
8515 You also need package manipulation tools on the target.
8516 The build machine is a likely candidate to act as the server.
8517 However, that machine does not necessarily have to be the
8518 package server.
8519 The build machine could push its artifacts to another machine
8520 that acts as the server (e.g. Internet-facing).
8521 </para>
8522
8523 <para>
8524 A simple build that targets just one device produces
8525 more than one package database.
8526 In other words, the packages produced by a build are separated
8527 out into a couple of different package groupings based on
8528 criteria such as the target's CPU architecture, the target
8529 board, or the C library used on the target.
8530 For example, a build targeting the <filename>qemuarm</filename>
8531 device produces the following three package databases:
8532 <filename>all</filename>, <filename>armv5te</filename>, and
8533 <filename>qemuarm</filename>.
8534 If you wanted your <filename>qemuarm</filename> device to be
8535 aware of all the packages that were available to it,
8536 you would need to point it to each of these databases
8537 individually.
8538 In a similar way, a traditional Linux distribution usually is
8539 configured to be aware of a number of software repositories
8540 from which it retrieves packages.
8541 </para>
8542
8543 <para>
8544 Using runtime package management is completely optional and
8545 not required for a successful build or deployment in any
8546 way.
8547 But if you want to make use of runtime package management,
8548 you need to do a couple things above and beyond the basics.
8549 The remainder of this section describes what you need to do.
8550 </para>
8551
8552 <section id='runtime-package-management-build'>
8553 <title>Build Considerations</title>
8554
8555 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008556 This section describes build considerations of which you
8557 need to be aware in order to provide support for runtime
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008558 package management.
8559 </para>
8560
8561 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008562 When BitBake generates packages, it needs to know
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008563 what format or formats to use.
8564 In your configuration, you use the
8565 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008566 variable to specify the format:
8567 <orderedlist>
8568 <listitem><para>
8569 Open the <filename>local.conf</filename> file
8570 inside your
8571 <link linkend='build-directory'>Build Directory</link>
8572 (e.g. <filename>~/poky/build/conf/local.conf</filename>).
8573 </para></listitem>
8574 <listitem><para>
8575 Select the desired package format as follows:
8576 <literallayout class='monospaced'>
8577 PACKAGE_CLASSES ?= “package_<replaceable>packageformat</replaceable>
8578 </literallayout>
8579 where <replaceable>packageformat</replaceable>
8580 can be "ipk", "rpm", and "deb", which are the
8581 supported package formats.
8582 <note>
8583 Because the Yocto Project supports three
8584 different package formats, you can set the
8585 variable with more than one argument.
8586 However, the OpenEmbedded build system only
8587 uses the first argument when creating an image
8588 or Software Development Kit (SDK).
8589 </note>
8590 </para></listitem>
8591 </orderedlist>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008592 </para>
8593
8594 <para>
8595 If you would like your image to start off with a basic
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008596 package database containing the packages in your current
8597 build as well as to have the relevant tools available on the
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008598 target for runtime package management, you can include
8599 "package-management" in the
8600 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
8601 variable.
8602 Including "package-management" in this
8603 configuration variable ensures that when the image
8604 is assembled for your target, the image includes
8605 the currently-known package databases as well as
8606 the target-specific tools required for runtime
8607 package management to be performed on the target.
8608 However, this is not strictly necessary.
8609 You could start your image off without any databases
8610 but only include the required on-target package
8611 tool(s).
8612 As an example, you could include "opkg" in your
8613 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink>
8614 variable if you are using the IPK package format.
8615 You can then initialize your target's package database(s)
8616 later once your image is up and running.
8617 </para>
8618
8619 <para>
8620 Whenever you perform any sort of build step that can
8621 potentially generate a package or modify an existing
8622 package, it is always a good idea to re-generate the
8623 package index with:
8624 <literallayout class='monospaced'>
8625 $ bitbake package-index
8626 </literallayout>
8627 Realize that it is not sufficient to simply do the
8628 following:
8629 <literallayout class='monospaced'>
8630 $ bitbake <replaceable>some-package</replaceable> package-index
8631 </literallayout>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008632 The reason for this restriction is because BitBake does not
8633 properly schedule the <filename>package-index</filename>
8634 target fully after any other target has completed.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008635 Thus, be sure to run the package update step separately.
8636 </para>
8637
8638 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008639 You can use the
8640 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS'><filename>PACKAGE_FEED_ARCHS</filename></ulink>,
8641 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS'><filename>PACKAGE_FEED_BASE_PATHS</filename></ulink>,
8642 and
8643 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS'><filename>PACKAGE_FEED_URIS</filename></ulink>
8644 variables to pre-configure target images to use a package
8645 feed.
8646 If you do not define these variables, then manual steps
8647 as described in the subsequent sections are necessary to
8648 configure the target.
8649 You should set these variables before building the image
8650 in order to produce a correctly configured image.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008651 </para>
8652
8653 <para>
8654 When your build is complete, your packages reside in the
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008655 <filename>${TMPDIR}/deploy/<replaceable>packageformat</replaceable></filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008656 directory.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008657 For example, if
8658 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink><filename>}</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008659 is <filename>tmp</filename> and your selected package type
8660 is IPK, then your IPK packages are available in
8661 <filename>tmp/deploy/ipk</filename>.
8662 </para>
8663 </section>
8664
8665 <section id='runtime-package-management-server'>
8666 <title>Host or Server Machine Setup</title>
8667
8668 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008669 Although other protocols are possible, a server using HTTP
8670 typically serves packages.
8671 If you want to use HTTP, then set up and configure a
8672 web server such as Apache 2, lighttpd, or
8673 SimpleHTTPServer on the machine serving the packages.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008674 </para>
8675
8676 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008677 To keep things simple, this section describes how to set
8678 up a SimpleHTTPServer web server to share package feeds
8679 from the developer's machine.
8680 Although this server might not be the best for a production
8681 environment, the setup is simple and straight forward.
8682 Should you want to use a different server more suited for
8683 production (e.g. Apache 2, Lighttpd, or Nginx), take the
8684 appropriate steps to do so.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008685 </para>
8686
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008687 <para>
8688 From within the build directory where you have built an
8689 image based on your packaging choice (i.e. the
8690 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink>
8691 setting), simply start the server.
8692 The following example assumes a build directory of
8693 <filename>~/poky/build/tmp/deploy/rpm</filename> and a
8694 <filename>PACKAGE_CLASSES</filename> setting of
8695 "package_rpm":
8696 <literallayout class='monospaced'>
8697 $ cd ~/poky/build/tmp/deploy/rpm
8698 $ python -m SimpleHTTPServer
8699 </literallayout>
8700 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008701 </section>
8702
8703 <section id='runtime-package-management-target'>
8704 <title>Target Setup</title>
8705
8706 <para>
8707 Setting up the target differs depending on the
8708 package management system.
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008709 This section provides information for RPM, IPK, and DEB.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008710 </para>
8711
8712 <section id='runtime-package-management-target-rpm'>
8713 <title>Using RPM</title>
8714
8715 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05008716 The <filename>dnf</filename> application performs
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008717 runtime package management of RPM packages.
8718 You must perform an initial setup for
Brad Bishop6e60e8b2018-02-01 10:27:11 -05008719 <filename>dnf</filename> on the target machine
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008720 if the
8721 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS'><filename>PACKAGE_FEED_ARCHS</filename></ulink>,
Brad Bishop6e60e8b2018-02-01 10:27:11 -05008722 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS'><filename>PACKAGE_FEED_BASE_PATHS</filename></ulink>,
8723 and
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008724 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS'><filename>PACKAGE_FEED_URIS</filename></ulink>
8725 variables have not been set or the target image was
8726 built before the variables were set.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008727 </para>
8728
8729 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008730 As an example, assume the target is able to use the
8731 following package databases:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008732 <filename>all</filename>, <filename>i586</filename>,
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008733 and <filename>qemux86</filename> from a server named
8734 <filename>my.server</filename>.
Brad Bishop6e60e8b2018-02-01 10:27:11 -05008735 You must inform <filename>dnf</filename> of the
8736 availability of these databases by creating a
8737 <filename>/etc/yum.repos.d/oe-packages.repo</filename>
8738 file with the following content:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008739 <literallayout class='monospaced'>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05008740 [oe-packages]
8741 baseurl="http://my.server/rpm/i586 http://my.server/rpm/qemux86 http://my.server/rpm/all"
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008742 </literallayout>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008743 From the target machine, fetch the repository:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008744 <literallayout class='monospaced'>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05008745 # dnf makecache
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008746 </literallayout>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05008747 After everything is set up, <filename>dnf</filename>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008748 is able to find, install, and upgrade packages from
8749 the specified repository.
Brad Bishop6e60e8b2018-02-01 10:27:11 -05008750 <note>
8751 See the
8752 <ulink url='http://dnf.readthedocs.io/en/latest/'>DNF documentation</ulink>
8753 for additional information.
8754 </note>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008755 </para>
8756 </section>
8757
8758 <section id='runtime-package-management-target-ipk'>
8759 <title>Using IPK</title>
8760
8761 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008762 The <filename>opkg</filename> application performs
8763 runtime package management of IPK packages.
8764 You must perform an initial setup for
8765 <filename>opkg</filename> on the target machine
8766 if the
8767 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS'><filename>PACKAGE_FEED_ARCHS</filename></ulink>,
8768 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS'><filename>PACKAGE_FEED_BASE_PATHS</filename></ulink>, and
8769 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS'><filename>PACKAGE_FEED_URIS</filename></ulink>
8770 variables have not been set or the target image was
8771 built before the variables were set.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008772 </para>
8773
8774 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008775 The <filename>opkg</filename> application uses
8776 configuration files to find available package
8777 databases.
8778 Thus, you need to create a configuration file inside
8779 the <filename>/etc/opkg/</filename> direction, which
8780 informs <filename>opkg</filename> of any repository
8781 you want to use.
8782 </para>
8783
8784 <para>
8785 As an example, suppose you are serving packages from a
8786 <filename>ipk/</filename> directory containing the
8787 <filename>i586</filename>,
8788 <filename>all</filename>, and
8789 <filename>qemux86</filename> databases through an
8790 HTTP server named <filename>my.server</filename>.
8791 On the target, create a configuration file
8792 (e.g. <filename>my_repo.conf</filename>) inside the
8793 <filename>/etc/opkg/</filename> directory containing
8794 the following:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008795 <literallayout class='monospaced'>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008796 src/gz all http://my.server/ipk/all
8797 src/gz i586 http://my.server/ipk/i586
8798 src/gz qemux86 http://my.server/ipk/qemux86
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008799 </literallayout>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008800 Next, instruct <filename>opkg</filename> to fetch
8801 the repository information:
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008802 <literallayout class='monospaced'>
8803 # opkg update
8804 </literallayout>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06008805 The <filename>opkg</filename> application is now able
8806 to find, install, and upgrade packages from the
8807 specified repository.
8808 </para>
8809 </section>
8810
8811 <section id='runtime-package-management-target-deb'>
8812 <title>Using DEB</title>
8813
8814 <para>
8815 The <filename>apt</filename> application performs
8816 runtime package management of DEB packages.
8817 This application uses a source list file to find
8818 available package databases.
8819 You must perform an initial setup for
8820 <filename>apt</filename> on the target machine
8821 if the
8822 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS'><filename>PACKAGE_FEED_ARCHS</filename></ulink>,
8823 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS'><filename>PACKAGE_FEED_BASE_PATHS</filename></ulink>, and
8824 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS'><filename>PACKAGE_FEED_URIS</filename></ulink>
8825 variables have not been set or the target image was
8826 built before the variables were set.
8827 </para>
8828
8829 <para>
8830 To inform <filename>apt</filename> of the repository
8831 you want to use, you might create a list file (e.g.
8832 <filename>my_repo.list</filename>) inside the
8833 <filename>/etc/apt/sources.list.d/</filename>
8834 directory.
8835 As an example, suppose you are serving packages from a
8836 <filename>deb/</filename> directory containing the
8837 <filename>i586</filename>,
8838 <filename>all</filename>, and
8839 <filename>qemux86</filename> databases through an
8840 HTTP server named <filename>my.server</filename>.
8841 The list file should contain:
8842 <literallayout class='monospaced'>
8843 deb http://my.server/deb/all ./
8844 deb http://my.server/deb/i586 ./
8845 deb http://my.server/deb/qemux86 ./
8846 </literallayout>
8847 Next, instruct the <filename>apt</filename>
8848 application to fetch the repository information:
8849 <literallayout class='monospaced'>
8850 # apt-get update
8851 </literallayout>
8852 After this step, <filename>apt</filename> is able
8853 to find, install, and upgrade packages from the
8854 specified repository.
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008855 </para>
8856 </section>
8857 </section>
8858 </section>
8859
Brad Bishop6e60e8b2018-02-01 10:27:11 -05008860 <section id='generating-and-using-signed-packages'>
8861 <title>Generating and Using Signed Packages</title>
8862 <para>
8863 In order to add security to RPM packages used during a build,
8864 you can take steps to securely sign them.
8865 Once a signature is verified, the OpenEmbedded build system
8866 can use the package in the build.
8867 If security fails for a signed package, the build system
8868 aborts the build.
8869 </para>
8870
8871 <para>
8872 This section describes how to sign RPM packages during a build
8873 and how to use signed package feeds (repositories) when
8874 doing a build.
8875 </para>
8876
8877 <section id='signing-rpm-packages'>
8878 <title>Signing RPM Packages</title>
8879
8880 <para>
8881 To enable signing RPM packages, you must set up the
8882 following configurations in either your
8883 <filename>local.config</filename> or
8884 <filename>distro.config</filename> file:
8885 <literallayout class='monospaced'>
8886 # Inherit sign_rpm.bbclass to enable signing functionality
8887 INHERIT += " sign_rpm"
8888 # Define the GPG key that will be used for signing.
8889 RPM_GPG_NAME = "<replaceable>key_name</replaceable>"
8890 # Provide passphrase for the key
8891 RPM_GPG_PASSPHRASE = "<replaceable>passphrase</replaceable>"
8892 </literallayout>
8893 <note>
8894 Be sure to supply appropriate values for both
8895 <replaceable>key_name</replaceable> and
8896 <replaceable>passphrase</replaceable>
8897 </note>
8898 Aside from the
8899 <filename>RPM_GPG_NAME</filename> and
8900 <filename>RPM_GPG_PASSPHRASE</filename> variables in the
8901 previous example, two optional variables related to signing
8902 exist:
8903 <itemizedlist>
8904 <listitem><para>
8905 <emphasis><filename>GPG_BIN</filename>:</emphasis>
8906 Specifies a <filename>gpg</filename> binary/wrapper
8907 that is executed when the package is signed.
8908 </para></listitem>
8909 <listitem><para>
8910 <emphasis><filename>GPG_PATH</filename>:</emphasis>
8911 Specifies the <filename>gpg</filename> home
8912 directory used when the package is signed.
8913 </para></listitem>
8914 </itemizedlist>
8915 </para>
8916 </section>
8917
8918 <section id='processing-package-feeds'>
8919 <title>Processing Package Feeds</title>
8920
8921 <para>
8922 In addition to being able to sign RPM packages, you can
8923 also enable the OpenEmbedded build system to be able to
8924 handle previously signed package feeds for IPK
8925 packages.
8926 <note>
8927 The OpenEmbedded build system does not currently
8928 support signed DPKG or RPM package feeds.
8929 </note>
8930 The steps you need to take to enable signed package feed
8931 use are similar to the steps used to sign RPM packages.
8932 You must define the following in your
8933 <filename>local.config</filename> or
8934 <filename>distro.config</filename> file:
8935 <literallayout class='monospaced'>
8936 INHERIT += "sign_package_feed"
8937 PACKAGE_FEED_GPG_NAME = "<replaceable>key_name</replaceable>"
8938 PACKAGE_FEED_GPG_PASSPHRASE_FILE = "<replaceable>path_to_file_containing_passphrase</replaceable>"
8939 </literallayout>
8940 For signed package feeds, the passphrase must exist in a
8941 separate file, which is pointed to by the
8942 <filename>PACKAGE_FEED_GPG_PASSPHRASE_FILE</filename>
8943 variable.
8944 Regarding security, keeping a plain text passphrase out of
8945 the configuration is more secure.
8946 </para>
8947
8948 <para>
8949 Aside from the
8950 <filename>PACKAGE_FEED_GPG_NAME</filename> and
8951 <filename>PACKAGE_FEED_GPG_PASSPHRASE_FILE</filename>
8952 variables, three optional variables related to signed
8953 package feeds exist:
8954 <itemizedlist>
8955 <listitem><para>
8956 <emphasis><filename>GPG_BIN</filename>:</emphasis>
8957 Specifies a <filename>gpg</filename> binary/wrapper
8958 that is executed when the package is signed.
8959 </para></listitem>
8960 <listitem><para>
8961 <emphasis><filename>GPG_PATH</filename>:</emphasis>
8962 Specifies the <filename>gpg</filename> home
8963 directory used when the package is signed.
8964 </para></listitem>
8965 <listitem><para>
8966 <emphasis><filename>PACKAGE_FEED_GPG_SIGNATURE_TYPE</filename>:</emphasis>
8967 Specifies the type of <filename>gpg</filename>
8968 signature.
8969 This variable applies only to RPM and IPK package
8970 feeds.
8971 Allowable values for the
8972 <filename>PACKAGE_FEED_GPG_SIGNATURE_TYPE</filename>
8973 are "ASC", which is the default and specifies ascii
8974 armored, and "BIN", which specifies binary.
8975 </para></listitem>
8976 </itemizedlist>
8977 </para>
8978 </section>
8979 </section>
8980
Patrick Williamsc124f4f2015-09-15 14:41:29 -05008981 <section id='testing-packages-with-ptest'>
8982 <title>Testing Packages With ptest</title>
8983
8984 <para>
8985 A Package Test (ptest) runs tests against packages built
8986 by the OpenEmbedded build system on the target machine.
8987 A ptest contains at least two items: the actual test, and
8988 a shell script (<filename>run-ptest</filename>) that starts
8989 the test.
8990 The shell script that starts the test must not contain
8991 the actual test - the script only starts the test.
8992 On the other hand, the test can be anything from a simple
8993 shell script that runs a binary and checks the output to
8994 an elaborate system of test binaries and data files.
8995 </para>
8996
8997 <para>
8998 The test generates output in the format used by
8999 Automake:
9000 <literallayout class='monospaced'>
9001 <replaceable>result</replaceable>: <replaceable>testname</replaceable>
9002 </literallayout>
9003 where the result can be <filename>PASS</filename>,
9004 <filename>FAIL</filename>, or <filename>SKIP</filename>,
9005 and the testname can be any identifying string.
9006 </para>
9007
9008 <para>
9009 For a list of Yocto Project recipes that are already
9010 enabled with ptest, see the
9011 <ulink url='https://wiki.yoctoproject.org/wiki/Ptest'>Ptest</ulink>
9012 wiki page.
9013 <note>
9014 A recipe is "ptest-enabled" if it inherits the
9015 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-ptest'><filename>ptest</filename></ulink>
9016 class.
9017 </note>
9018 </para>
9019
9020 <section id='adding-ptest-to-your-build'>
9021 <title>Adding ptest to Your Build</title>
9022
9023 <para>
9024 To add package testing to your build, add the
9025 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>
9026 and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink>
9027 variables to your <filename>local.conf</filename> file,
9028 which is found in the
9029 <link linkend='build-directory'>Build Directory</link>:
9030 <literallayout class='monospaced'>
9031 DISTRO_FEATURES_append = " ptest"
9032 EXTRA_IMAGE_FEATURES += "ptest-pkgs"
9033 </literallayout>
9034 Once your build is complete, the ptest files are installed
9035 into the
9036 <filename>/usr/lib/<replaceable>package</replaceable>/ptest</filename>
9037 directory within the image, where
9038 <filename><replaceable>package</replaceable></filename>
9039 is the name of the package.
9040 </para>
9041 </section>
9042
9043 <section id='running-ptest'>
9044 <title>Running ptest</title>
9045
9046 <para>
9047 The <filename>ptest-runner</filename> package installs a
9048 shell script that loops through all installed ptest test
9049 suites and runs them in sequence.
9050 Consequently, you might want to add this package to
9051 your image.
9052 </para>
9053 </section>
9054
9055 <section id='getting-your-package-ready'>
9056 <title>Getting Your Package Ready</title>
9057
9058 <para>
9059 In order to enable a recipe to run installed ptests
9060 on target hardware,
9061 you need to prepare the recipes that build the packages
9062 you want to test.
9063 Here is what you have to do for each recipe:
9064 <itemizedlist>
9065 <listitem><para><emphasis>Be sure the recipe
9066 inherits the
9067 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-ptest'><filename>ptest</filename></ulink>
9068 class:</emphasis>
9069 Include the following line in each recipe:
9070 <literallayout class='monospaced'>
9071 inherit ptest
9072 </literallayout>
9073 </para></listitem>
9074 <listitem><para><emphasis>Create <filename>run-ptest</filename>:</emphasis>
9075 This script starts your test.
9076 Locate the script where you will refer to it
9077 using
9078 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>.
9079 Here is an example that starts a test for
9080 <filename>dbus</filename>:
9081 <literallayout class='monospaced'>
9082 #!/bin/sh
9083 cd test
9084 make -k runtest-TESTS
9085 </literallayout>
9086 </para></listitem>
9087 <listitem><para><emphasis>Ensure dependencies are
9088 met:</emphasis>
9089 If the test adds build or runtime dependencies
9090 that normally do not exist for the package
9091 (such as requiring "make" to run the test suite),
9092 use the
9093 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
9094 and
9095 <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>
9096 variables in your recipe in order for the package
9097 to meet the dependencies.
9098 Here is an example where the package has a runtime
9099 dependency on "make":
9100 <literallayout class='monospaced'>
9101 RDEPENDS_${PN}-ptest += "make"
9102 </literallayout>
9103 </para></listitem>
9104 <listitem><para><emphasis>Add a function to build the
9105 test suite:</emphasis>
9106 Not many packages support cross-compilation of
9107 their test suites.
9108 Consequently, you usually need to add a
9109 cross-compilation function to the package.
9110 </para>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05009111
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009112 <para>Many packages based on Automake compile and
9113 run the test suite by using a single command
9114 such as <filename>make check</filename>.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05009115 However, the host <filename>make check</filename>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009116 builds and runs on the same computer, while
9117 cross-compiling requires that the package is built
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05009118 on the host but executed for the target
9119 architecture (though often, as in the case for
9120 ptest, the execution occurs on the host).
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009121 The built version of Automake that ships with the
9122 Yocto Project includes a patch that separates
9123 building and execution.
9124 Consequently, packages that use the unaltered,
9125 patched version of <filename>make check</filename>
9126 automatically cross-compiles.</para>
9127 <para>Regardless, you still must add a
9128 <filename>do_compile_ptest</filename> function to
9129 build the test suite.
9130 Add a function similar to the following to your
9131 recipe:
9132 <literallayout class='monospaced'>
9133 do_compile_ptest() {
9134 oe_runmake buildtest-TESTS
9135 }
9136 </literallayout>
9137 </para></listitem>
9138 <listitem><para><emphasis>Ensure special configurations
9139 are set:</emphasis>
9140 If the package requires special configurations
9141 prior to compiling the test code, you must
9142 insert a <filename>do_configure_ptest</filename>
9143 function into the recipe.
9144 </para></listitem>
9145 <listitem><para><emphasis>Install the test
9146 suite:</emphasis>
9147 The <filename>ptest</filename> class
9148 automatically copies the file
9149 <filename>run-ptest</filename> to the target and
9150 then runs make <filename>install-ptest</filename>
9151 to run the tests.
9152 If this is not enough, you need to create a
9153 <filename>do_install_ptest</filename> function and
9154 make sure it gets called after the
9155 "make install-ptest" completes.
9156 </para></listitem>
9157 </itemizedlist>
9158 </para>
9159 </section>
9160 </section>
9161 </section>
9162
9163 <section id='working-with-source-files'>
9164 <title>Working with Source Files</title>
9165
9166 <para>
9167 The OpenEmbedded build system works with source files located
9168 through the
9169 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
9170 variable.
9171 When you build something using BitBake, a big part of the operation
9172 is locating and downloading all the source tarballs.
9173 For images, downloading all the source for various packages can
9174 take a significant amount of time.
9175 </para>
9176
9177 <para>
9178 This section presents information for working with source
9179 files that can lead to more efficient use of resources and
9180 time.
9181 </para>
9182
9183 <section id='setting-up-effective-mirrors'>
9184 <title>Setting up Effective Mirrors</title>
9185
9186 <para>
9187 As mentioned, a good deal that goes into a Yocto Project
9188 build is simply downloading all of the source tarballs.
9189 Maybe you have been working with another build system
9190 (OpenEmbedded or Angstrom) for which you have built up a
9191 sizable directory of source tarballs.
9192 Or, perhaps someone else has such a directory for which you
9193 have read access.
9194 If so, you can save time by adding statements to your
9195 configuration file so that the build process checks local
9196 directories first for existing tarballs before checking the
9197 Internet.
9198 </para>
9199
9200 <para>
9201 Here is an efficient way to set it up in your
9202 <filename>local.conf</filename> file:
9203 <literallayout class='monospaced'>
9204 SOURCE_MIRROR_URL ?= "file:///home/you/your-download-dir/"
9205 INHERIT += "own-mirrors"
9206 BB_GENERATE_MIRROR_TARBALLS = "1"
9207 # BB_NO_NETWORK = "1"
9208 </literallayout>
9209 </para>
9210
9211 <para>
9212 In the previous example, the
9213 <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></ulink>
9214 variable causes the OpenEmbedded build system to generate
9215 tarballs of the Git repositories and store them in the
9216 <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink>
9217 directory.
9218 Due to performance reasons, generating and storing these
9219 tarballs is not the build system's default behavior.
9220 </para>
9221
9222 <para>
9223 You can also use the
9224 <ulink url='&YOCTO_DOCS_REF_URL;#var-PREMIRRORS'><filename>PREMIRRORS</filename></ulink>
9225 variable.
9226 For an example, see the variable's glossary entry in the
9227 Yocto Project Reference Manual.
9228 </para>
9229 </section>
9230
9231 <section id='getting-source-files-and-suppressing-the-build'>
9232 <title>Getting Source Files and Suppressing the Build</title>
9233
9234 <para>
9235 Another technique you can use to ready yourself for a
9236 successive string of build operations, is to pre-fetch
9237 all the source files without actually starting a build.
9238 This technique lets you work through any download issues
9239 and ultimately gathers all the source files into your
9240 download directory
9241 <ulink url='&YOCTO_DOCS_REF_URL;#structure-build-downloads'><filename>build/downloads</filename></ulink>,
9242 which is located with
9243 <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink>.
9244 </para>
9245
9246 <para>
9247 Use the following BitBake command form to fetch all the
9248 necessary sources without starting the build:
9249 <literallayout class='monospaced'>
9250 $ bitbake -c fetchall <replaceable>target</replaceable>
9251 </literallayout>
9252 This variation of the BitBake command guarantees that you
9253 have all the sources for that BitBake target should you
9254 disconnect from the Internet and want to do the build
9255 later offline.
9256 </para>
9257 </section>
9258 </section>
9259
9260 <section id="building-software-from-an-external-source">
9261 <title>Building Software from an External Source</title>
9262
9263 <para>
9264 By default, the OpenEmbedded build system uses the
9265 <link linkend='build-directory'>Build Directory</link> when
9266 building source code.
9267 The build process involves fetching the source files, unpacking
9268 them, and then patching them if necessary before the build takes
9269 place.
9270 </para>
9271
9272 <para>
9273 Situations exist where you might want to build software from source
9274 files that are external to and thus outside of the
9275 OpenEmbedded build system.
9276 For example, suppose you have a project that includes a new BSP with
9277 a heavily customized kernel.
9278 And, you want to minimize exposing the build system to the
9279 development team so that they can focus on their project and
9280 maintain everyone's workflow as much as possible.
9281 In this case, you want a kernel source directory on the development
9282 machine where the development occurs.
9283 You want the recipe's
9284 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
9285 variable to point to the external directory and use it as is, not
9286 copy it.
9287 </para>
9288
9289 <para>
9290 To build from software that comes from an external source, all you
9291 need to do is inherit the
9292 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-externalsrc'><filename>externalsrc</filename></ulink>
9293 class and then set the
9294 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC'><filename>EXTERNALSRC</filename></ulink>
9295 variable to point to your external source code.
9296 Here are the statements to put in your
9297 <filename>local.conf</filename> file:
9298 <literallayout class='monospaced'>
9299 INHERIT += "externalsrc"
9300 EXTERNALSRC_pn-<replaceable>myrecipe</replaceable> = "<replaceable>path-to-your-source-tree</replaceable>"
9301 </literallayout>
9302 </para>
9303
9304 <para>
9305 This next example shows how to accomplish the same thing by setting
9306 <filename>EXTERNALSRC</filename> in the recipe itself or in the
9307 recipe's append file:
9308 <literallayout class='monospaced'>
9309 EXTERNALSRC = "<replaceable>path</replaceable>"
9310 EXTERNALSRC_BUILD = "<replaceable>path</replaceable>"
9311 </literallayout>
9312 <note>
9313 In order for these settings to take effect, you must globally
9314 or locally inherit the
9315 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-externalsrc'><filename>externalsrc</filename></ulink>
9316 class.
9317 </note>
9318 </para>
9319
9320 <para>
9321 By default, <filename>externalsrc.bbclass</filename> builds
9322 the source code in a directory separate from the external source
9323 directory as specified by
9324 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC'><filename>EXTERNALSRC</filename></ulink>.
9325 If you need to have the source built in the same directory in
9326 which it resides, or some other nominated directory, you can set
9327 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC_BUILD'><filename>EXTERNALSRC_BUILD</filename></ulink>
9328 to point to that directory:
9329 <literallayout class='monospaced'>
9330 EXTERNALSRC_BUILD_pn-<replaceable>myrecipe</replaceable> = "<replaceable>path-to-your-source-tree</replaceable>"
9331 </literallayout>
9332 </para>
9333 </section>
9334
9335 <section id="selecting-an-initialization-manager">
9336 <title>Selecting an Initialization Manager</title>
9337
9338 <para>
9339 By default, the Yocto Project uses SysVinit as the initialization
9340 manager.
9341 However, support also exists for systemd,
9342 which is a full replacement for init with
9343 parallel starting of services, reduced shell overhead and other
9344 features that are used by many distributions.
9345 </para>
9346
9347 <para>
9348 If you want to use SysVinit, you do
9349 not have to do anything.
9350 But, if you want to use systemd, you must
9351 take some steps as described in the following sections.
9352 </para>
9353
9354 <section id='using-systemd-exclusively'>
9355 <title>Using systemd Exclusively</title>
9356
9357 <para>
9358 Set the these variables in your distribution configuration
9359 file as follows:
9360 <literallayout class='monospaced'>
9361 DISTRO_FEATURES_append = " systemd"
9362 VIRTUAL-RUNTIME_init_manager = "systemd"
9363 </literallayout>
9364 You can also prevent the SysVinit
9365 distribution feature from
9366 being automatically enabled as follows:
9367 <literallayout class='monospaced'>
9368 DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
9369 </literallayout>
9370 Doing so removes any redundant SysVinit scripts.
9371 </para>
9372
9373 <para>
9374 To remove initscripts from your image altogether,
9375 set this variable also:
9376 <literallayout class='monospaced'>
9377 VIRTUAL-RUNTIME_initscripts = ""
9378 </literallayout>
9379 </para>
9380
9381 <para>
9382 For information on the backfill variable, see
9383 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED'><filename>DISTRO_FEATURES_BACKFILL_CONSIDERED</filename></ulink>.
9384 </para>
9385 </section>
9386
9387 <section id='using-systemd-for-the-main-image-and-using-sysvinit-for-the-rescue-image'>
9388 <title>Using systemd for the Main Image and Using SysVinit for the Rescue Image</title>
9389
9390 <para>
9391 Set these variables in your distribution configuration
9392 file as follows:
9393 <literallayout class='monospaced'>
9394 DISTRO_FEATURES_append = " systemd"
9395 VIRTUAL-RUNTIME_init_manager = "systemd"
9396 </literallayout>
9397 Doing so causes your main image to use the
9398 <filename>packagegroup-core-boot.bb</filename> recipe and
9399 systemd.
9400 The rescue/minimal image cannot use this package group.
9401 However, it can install SysVinit
9402 and the appropriate packages will have support for both
9403 systemd and SysVinit.
9404 </para>
9405 </section>
9406 </section>
9407
9408 <section id="selecting-dev-manager">
9409 <title>Selecting a Device Manager</title>
9410
9411 <para>
9412 The Yocto Project provides multiple ways to manage the device
9413 manager (<filename>/dev</filename>):
9414 <itemizedlist>
9415 <listitem><para><emphasis>Persistent and Pre-Populated<filename>/dev</filename>:</emphasis>
9416 For this case, the <filename>/dev</filename> directory
9417 is persistent and the required device nodes are created
9418 during the build.
9419 </para></listitem>
9420 <listitem><para><emphasis>Use <filename>devtmpfs</filename> with a Device Manager:</emphasis>
9421 For this case, the <filename>/dev</filename> directory
9422 is provided by the kernel as an in-memory file system and
9423 is automatically populated by the kernel at runtime.
9424 Additional configuration of device nodes is done in user
9425 space by a device manager like
9426 <filename>udev</filename> or
9427 <filename>busybox-mdev</filename>.
9428 </para></listitem>
9429 </itemizedlist>
9430 </para>
9431
9432 <section id="static-dev-management">
9433 <title>Using Persistent and Pre-Populated<filename>/dev</filename></title>
9434
9435 <para>
9436 To use the static method for device population, you need to
9437 set the
9438 <ulink url='&YOCTO_DOCS_REF_URL;#var-USE_DEVFS'><filename>USE_DEVFS</filename></ulink>
9439 variable to "0" as follows:
9440 <literallayout class='monospaced'>
9441 USE_DEVFS = "0"
9442 </literallayout>
9443 </para>
9444
9445 <para>
9446 The content of the resulting <filename>/dev</filename>
9447 directory is defined in a Device Table file.
9448 The
9449 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_DEVICE_TABLES'><filename>IMAGE_DEVICE_TABLES</filename></ulink>
9450 variable defines the Device Table to use and should be set
9451 in the machine or distro configuration file.
9452 Alternatively, you can set this variable in your
9453 <filename>local.conf</filename> configuration file.
9454 </para>
9455
9456 <para>
9457 If you do not define the
9458 <filename>IMAGE_DEVICE_TABLES</filename> variable, the default
9459 <filename>device_table-minimal.txt</filename> is used:
9460 <literallayout class='monospaced'>
9461 IMAGE_DEVICE_TABLES = "device_table-mymachine.txt"
9462 </literallayout>
9463 </para>
9464
9465 <para>
9466 The population is handled by the <filename>makedevs</filename>
9467 utility during image creation:
9468 </para>
9469 </section>
9470
9471 <section id="devtmpfs-dev-management">
9472 <title>Using <filename>devtmpfs</filename> and a Device Manager</title>
9473
9474 <para>
9475 To use the dynamic method for device population, you need to
9476 use (or be sure to set) the
9477 <ulink url='&YOCTO_DOCS_REF_URL;#var-USE_DEVFS'><filename>USE_DEVFS</filename></ulink>
9478 variable to "1", which is the default:
9479 <literallayout class='monospaced'>
9480 USE_DEVFS = "1"
9481 </literallayout>
9482 With this setting, the resulting <filename>/dev</filename>
9483 directory is populated by the kernel using
9484 <filename>devtmpfs</filename>.
9485 Make sure the corresponding kernel configuration variable
9486 <filename>CONFIG_DEVTMPFS</filename> is set when building
9487 you build a Linux kernel.
9488 </para>
9489
9490 <para>
9491 All devices created by <filename>devtmpfs</filename> will be
9492 owned by <filename>root</filename> and have permissions
9493 <filename>0600</filename>.
9494 </para>
9495
9496 <para>
9497 To have more control over the device nodes, you can use a
9498 device manager like <filename>udev</filename> or
9499 <filename>busybox-mdev</filename>.
9500 You choose the device manager by defining the
9501 <filename>VIRTUAL-RUNTIME_dev_manager</filename> variable
9502 in your machine or distro configuration file.
9503 Alternatively, you can set this variable in your
9504 <filename>local.conf</filename> configuration file:
9505 <literallayout class='monospaced'>
9506 VIRTUAL-RUNTIME_dev_manager = "udev"
9507
9508 # Some alternative values
9509 # VIRTUAL-RUNTIME_dev_manager = "busybox-mdev"
9510 # VIRTUAL-RUNTIME_dev_manager = "systemd"
9511 </literallayout>
9512 </para>
9513 </section>
9514 </section>
9515
9516 <section id="platdev-appdev-srcrev">
9517 <title>Using an External SCM</title>
9518
9519 <para>
9520 If you're working on a recipe that pulls from an external Source
9521 Code Manager (SCM), it is possible to have the OpenEmbedded build
9522 system notice new recipe changes added to the SCM and then build
9523 the resulting packages that depend on the new recipes by using
9524 the latest versions.
9525 This only works for SCMs from which it is possible to get a
9526 sensible revision number for changes.
9527 Currently, you can do this with Apache Subversion (SVN), Git, and
9528 Bazaar (BZR) repositories.
9529 </para>
9530
9531 <para>
9532 To enable this behavior, the
9533 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>
9534 of the recipe needs to reference
9535 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCPV'><filename>SRCPV</filename></ulink>.
9536 Here is an example:
9537 <literallayout class='monospaced'>
9538 PV = "1.2.3+git${SRCPV}"
9539 </literallayout>
9540 Then, you can add the following to your
9541 <filename>local.conf</filename>:
9542 <literallayout class='monospaced'>
9543 SRCREV_pn-<replaceable>PN</replaceable> = "${AUTOREV}"
9544 </literallayout>
9545 <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink>
9546 is the name of the recipe for which you want to enable automatic source
9547 revision updating.
9548 </para>
9549
9550 <para>
9551 If you do not want to update your local configuration file, you can
9552 add the following directly to the recipe to finish enabling
9553 the feature:
9554 <literallayout class='monospaced'>
9555 SRCREV = "${AUTOREV}"
9556 </literallayout>
9557 </para>
9558
9559 <para>
9560 The Yocto Project provides a distribution named
9561 <filename>poky-bleeding</filename>, whose configuration
9562 file contains the line:
9563 <literallayout class='monospaced'>
9564 require conf/distro/include/poky-floating-revisions.inc
9565 </literallayout>
9566 This line pulls in the listed include file that contains
9567 numerous lines of exactly that form:
9568 <literallayout class='monospaced'>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05009569 #SRCREV_pn-opkg-native ?= "${AUTOREV}"
9570 #SRCREV_pn-opkg-sdk ?= "${AUTOREV}"
9571 #SRCREV_pn-opkg ?= "${AUTOREV}"
9572 #SRCREV_pn-opkg-utils-native ?= "${AUTOREV}"
9573 #SRCREV_pn-opkg-utils ?= "${AUTOREV}"
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009574 SRCREV_pn-gconf-dbus ?= "${AUTOREV}"
9575 SRCREV_pn-matchbox-common ?= "${AUTOREV}"
9576 SRCREV_pn-matchbox-config-gtk ?= "${AUTOREV}"
9577 SRCREV_pn-matchbox-desktop ?= "${AUTOREV}"
9578 SRCREV_pn-matchbox-keyboard ?= "${AUTOREV}"
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009579 SRCREV_pn-matchbox-panel-2 ?= "${AUTOREV}"
9580 SRCREV_pn-matchbox-themes-extra ?= "${AUTOREV}"
9581 SRCREV_pn-matchbox-terminal ?= "${AUTOREV}"
9582 SRCREV_pn-matchbox-wm ?= "${AUTOREV}"
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009583 SRCREV_pn-settings-daemon ?= "${AUTOREV}"
9584 SRCREV_pn-screenshot ?= "${AUTOREV}"
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009585 .
9586 .
9587 .
9588 </literallayout>
9589 These lines allow you to experiment with building a
9590 distribution that tracks the latest development source
9591 for numerous packages.
9592 <note><title>Caution</title>
9593 The <filename>poky-bleeding</filename> distribution
9594 is not tested on a regular basis.
9595 Keep this in mind if you use it.
9596 </note>
9597 </para>
9598 </section>
9599
9600 <section id='creating-a-read-only-root-filesystem'>
9601 <title>Creating a Read-Only Root Filesystem</title>
9602
9603 <para>
9604 Suppose, for security reasons, you need to disable
9605 your target device's root filesystem's write permissions
9606 (i.e. you need a read-only root filesystem).
9607 Or, perhaps you are running the device's operating system
9608 from a read-only storage device.
9609 For either case, you can customize your image for
9610 that behavior.
9611 </para>
9612
9613 <note>
9614 Supporting a read-only root filesystem requires that the system and
9615 applications do not try to write to the root filesystem.
9616 You must configure all parts of the target system to write
9617 elsewhere, or to gracefully fail in the event of attempting to
9618 write to the root filesystem.
9619 </note>
9620
9621 <section id='creating-the-root-filesystem'>
9622 <title>Creating the Root Filesystem</title>
9623
9624 <para>
9625 To create the read-only root filesystem, simply add the
9626 "read-only-rootfs" feature to your image.
9627 Using either of the following statements in your
9628 image recipe or from within the
9629 <filename>local.conf</filename> file found in the
9630 <link linkend='build-directory'>Build Directory</link>
9631 causes the build system to create a read-only root filesystem:
9632 <literallayout class='monospaced'>
9633 IMAGE_FEATURES = "read-only-rootfs"
9634 </literallayout>
9635 or
9636 <literallayout class='monospaced'>
9637 EXTRA_IMAGE_FEATURES += "read-only-rootfs"
9638 </literallayout>
9639 </para>
9640
9641 <para>
9642 For more information on how to use these variables, see the
9643 "<link linkend='usingpoky-extend-customimage-imagefeatures'>Customizing Images Using Custom <filename>IMAGE_FEATURES</filename> and <filename>EXTRA_IMAGE_FEATURES</filename></link>"
9644 section.
9645 For information on the variables, see
9646 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
9647 and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink>.
9648 </para>
9649 </section>
9650
9651 <section id='post-installation-scripts'>
9652 <title>Post-Installation Scripts</title>
9653
9654 <para>
9655 It is very important that you make sure all
9656 post-Installation (<filename>pkg_postinst</filename>) scripts
9657 for packages that are installed into the image can be run
9658 at the time when the root filesystem is created during the
9659 build on the host system.
9660 These scripts cannot attempt to run during first-boot on the
9661 target device.
9662 With the "read-only-rootfs" feature enabled,
9663 the build system checks during root filesystem creation to make
9664 sure all post-installation scripts succeed.
9665 If any of these scripts still need to be run after the root
9666 filesystem is created, the build immediately fails.
9667 These build-time checks ensure that the build fails
9668 rather than the target device fails later during its
9669 initial boot operation.
9670 </para>
9671
9672 <para>
9673 Most of the common post-installation scripts generated by the
9674 build system for the out-of-the-box Yocto Project are engineered
9675 so that they can run during root filesystem creation
9676 (e.g. post-installation scripts for caching fonts).
9677 However, if you create and add custom scripts, you need
9678 to be sure they can be run during this file system creation.
9679 </para>
9680
9681 <para>
9682 Here are some common problems that prevent
9683 post-installation scripts from running during root filesystem
9684 creation:
9685 <itemizedlist>
9686 <listitem><para>
9687 <emphasis>Not using $D in front of absolute
9688 paths:</emphasis>
9689 The build system defines
9690 <filename>$</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink>
9691 when the root filesystem is created.
9692 Furthermore, <filename>$D</filename> is blank when the
9693 script is run on the target device.
9694 This implies two purposes for <filename>$D</filename>:
9695 ensuring paths are valid in both the host and target
9696 environments, and checking to determine which
9697 environment is being used as a method for taking
9698 appropriate actions.
9699 </para></listitem>
9700 <listitem><para>
9701 <emphasis>Attempting to run processes that are
9702 specific to or dependent on the target
9703 architecture:</emphasis>
9704 You can work around these attempts by using native
Patrick Williamsd8c66bc2016-06-20 12:57:21 -05009705 tools, which run on the host system,
9706 to accomplish the same tasks, or
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009707 by alternatively running the processes under QEMU,
9708 which has the <filename>qemu_run_binary</filename>
9709 function.
9710 For more information, see the
9711 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-qemu'><filename>qemu</filename></ulink>
9712 class.</para></listitem>
9713 </itemizedlist>
9714 </para>
9715 </section>
9716
9717 <section id='areas-with-write-access'>
9718 <title>Areas With Write Access</title>
9719
9720 <para>
9721 With the "read-only-rootfs" feature enabled,
9722 any attempt by the target to write to the root filesystem at
9723 runtime fails.
9724 Consequently, you must make sure that you configure processes
9725 and applications that attempt these types of writes do so
9726 to directories with write access (e.g.
9727 <filename>/tmp</filename> or <filename>/var/run</filename>).
9728 </para>
9729 </section>
9730 </section>
9731
9732 <section id="performing-automated-runtime-testing">
9733 <title>Performing Automated Runtime Testing</title>
9734
9735 <para>
9736 The OpenEmbedded build system makes available a series of automated
9737 tests for images to verify runtime functionality.
9738 You can run these tests on either QEMU or actual target hardware.
9739 Tests are written in Python making use of the
9740 <filename>unittest</filename> module, and the majority of them
9741 run commands on the target system over SSH.
9742 This section describes how you set up the environment to use these
9743 tests, run available tests, and write and add your own tests.
9744 </para>
9745
Brad Bishop6e60e8b2018-02-01 10:27:11 -05009746 <para>
9747 For information on the test and QA infrastructure available
9748 within the Yocto Project, see the
9749 "<ulink url='&YOCTO_DOCS_REF_URL;#testing-and-quality-assurance'>Testing and Quality Assurance</ulink>"
9750 section in the Yocto Project Reference Manual.
9751 </para>
9752
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009753 <section id='enabling-tests'>
9754 <title>Enabling Tests</title>
9755
9756 <para>
9757 Depending on whether you are planning to run tests using
9758 QEMU or on the hardware, you have to take
9759 different steps to enable the tests.
9760 See the following subsections for information on how to
9761 enable both types of tests.
9762 </para>
9763
9764 <section id='qemu-image-enabling-tests'>
9765 <title>Enabling Runtime Tests on QEMU</title>
9766
9767 <para>
9768 In order to run tests, you need to do the following:
9769 <itemizedlist>
9770 <listitem><para><emphasis>Set up to avoid interaction
9771 with <filename>sudo</filename> for networking:</emphasis>
9772 To accomplish this, you must do one of the
9773 following:
9774 <itemizedlist>
9775 <listitem><para>Add
9776 <filename>NOPASSWD</filename> for your user
9777 in <filename>/etc/sudoers</filename> either for
9778 all commands or just for
9779 <filename>runqemu-ifup</filename>.
9780 You must provide the full path as that can
9781 change if you are using multiple clones of the
9782 source repository.
9783 <note>
9784 On some distributions, you also need to
9785 comment out "Defaults requiretty" in
9786 <filename>/etc/sudoers</filename>.
9787 </note></para></listitem>
9788 <listitem><para>Manually configure a tap interface
9789 for your system.</para></listitem>
9790 <listitem><para>Run as root the script in
9791 <filename>scripts/runqemu-gen-tapdevs</filename>,
9792 which should generate a list of tap devices.
9793 This is the option typically chosen for
9794 Autobuilder-type environments.
9795 </para></listitem>
9796 </itemizedlist></para></listitem>
9797 <listitem><para><emphasis>Set the
9798 <filename>DISPLAY</filename> variable:</emphasis>
9799 You need to set this variable so that you have an X
9800 server available (e.g. start
9801 <filename>vncserver</filename> for a headless machine).
9802 </para></listitem>
9803 <listitem><para><emphasis>Be sure your host's firewall
9804 accepts incoming connections from
9805 192.168.7.0/24:</emphasis>
Brad Bishop6e60e8b2018-02-01 10:27:11 -05009806 Some of the tests (in particular DNF tests) start
9807 an HTTP server on a random high number port,
9808 which is used to serve files to the target.
9809 The DNF module serves
9810 <filename>${WORKDIR}/oe-rootfs-repo</filename>
9811 so it can run DNF channel commands.
9812 That means your host's firewall
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009813 must accept incoming connections from 192.168.7.0/24,
9814 which is the default IP range used for tap devices
9815 by <filename>runqemu</filename>.</para></listitem>
Patrick Williamsf1e5d692016-03-30 15:21:19 -05009816 <listitem><para><emphasis>Be sure your host has the
9817 correct packages installed:</emphasis>
9818 Depending your host's distribution, you need
9819 to have the following packages installed:
9820 <itemizedlist>
9821 <listitem><para>Ubuntu and Debian:
9822 <filename>sysstat</filename> and
9823 <filename>iproute2</filename>
9824 </para></listitem>
9825 <listitem><para>OpenSUSE:
9826 <filename>sysstat</filename> and
9827 <filename>iproute2</filename>
9828 </para></listitem>
9829 <listitem><para>Fedora:
9830 <filename>sysstat</filename> and
9831 <filename>iproute</filename>
9832 </para></listitem>
9833 <listitem><para>CentOS:
9834 <filename>sysstat</filename> and
9835 <filename>iproute</filename>
9836 </para></listitem>
9837 </itemizedlist>
9838 </para></listitem>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009839 </itemizedlist>
9840 </para>
9841
9842 <para>
9843 Once you start running the tests, the following happens:
9844 <orderedlist>
9845 <listitem><para>A copy of the root filesystem is written
9846 to <filename>${WORKDIR}/testimage</filename>.
9847 </para></listitem>
9848 <listitem><para>The image is booted under QEMU using the
9849 standard <filename>runqemu</filename> script.
9850 </para></listitem>
9851 <listitem><para>A default timeout of 500 seconds occurs
9852 to allow for the boot process to reach the login prompt.
9853 You can change the timeout period by setting
9854 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_QEMUBOOT_TIMEOUT'><filename>TEST_QEMUBOOT_TIMEOUT</filename></ulink>
9855 in the <filename>local.conf</filename> file.
9856 </para></listitem>
9857 <listitem><para>Once the boot process is reached and the
9858 login prompt appears, the tests run.
9859 The full boot log is written to
9860 <filename>${WORKDIR}/testimage/qemu_boot_log</filename>.
9861 </para></listitem>
9862 <listitem><para>Each test module loads in the order found
9863 in <filename>TEST_SUITES</filename>.
9864 You can find the full output of the commands run over
9865 SSH in
9866 <filename>${WORKDIR}/testimgage/ssh_target_log</filename>.
9867 </para></listitem>
9868 <listitem><para>If no failures occur, the task running the
9869 tests ends successfully.
9870 You can find the output from the
9871 <filename>unittest</filename> in the task log at
9872 <filename>${WORKDIR}/temp/log.do_testimage</filename>.
9873 </para></listitem>
9874 </orderedlist>
9875 </para>
9876 </section>
9877
9878 <section id='hardware-image-enabling-tests'>
9879 <title>Enabling Runtime Tests on Hardware</title>
9880
9881 <para>
9882 The OpenEmbedded build system can run tests on real
9883 hardware, and for certain devices it can also deploy
9884 the image to be tested onto the device beforehand.
9885 </para>
9886
9887 <para>
9888 For automated deployment, a "master image" is installed
9889 onto the hardware once as part of setup.
9890 Then, each time tests are to be run, the following
9891 occurs:
9892 <orderedlist>
9893 <listitem><para>The master image is booted into and
9894 used to write the image to be tested to
9895 a second partition.
9896 </para></listitem>
9897 <listitem><para>The device is then rebooted using an
9898 external script that you need to provide.
9899 </para></listitem>
9900 <listitem><para>The device boots into the image to be
9901 tested.
9902 </para></listitem>
9903 </orderedlist>
9904 </para>
9905
9906 <para>
9907 When running tests (independent of whether the image
9908 has been deployed automatically or not), the device is
9909 expected to be connected to a network on a
9910 pre-determined IP address.
9911 You can either use static IP addresses written into
9912 the image, or set the image to use DHCP and have your
9913 DHCP server on the test network assign a known IP address
9914 based on the MAC address of the device.
9915 </para>
9916
9917 <para>
9918 In order to run tests on hardware, you need to set
9919 <filename>TEST_TARGET</filename> to an appropriate value.
9920 For QEMU, you do not have to change anything, the default
9921 value is "QemuTarget".
9922 For running tests on hardware, the following options exist:
9923 <itemizedlist>
9924 <listitem><para><emphasis>"SimpleRemoteTarget":</emphasis>
9925 Choose "SimpleRemoteTarget" if you are going to
9926 run tests on a target system that is already
9927 running the image to be tested and is available
9928 on the network.
9929 You can use "SimpleRemoteTarget" in conjunction
9930 with either real hardware or an image running
9931 within a separately started QEMU or any
9932 other virtual machine manager.
9933 </para></listitem>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06009934 <listitem><para><emphasis>"Systemd-bootTarget":</emphasis>
9935 Choose "Systemd-bootTarget" if your hardware is
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009936 an EFI-based machine with
Patrick Williamsc0f7c042017-02-23 20:41:17 -06009937 <filename>systemd-boot</filename> as bootloader and
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009938 <filename>core-image-testmaster</filename>
9939 (or something similar) is installed.
9940 Also, your hardware under test must be in a
9941 DHCP-enabled network that gives it the same IP
9942 address for each reboot.</para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -06009943 <para>If you choose "Systemd-bootTarget", there are
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009944 additional requirements and considerations.
9945 See the
Patrick Williamsc0f7c042017-02-23 20:41:17 -06009946 "<link linkend='selecting-systemd-boottarget'>Selecting Systemd-bootTarget</link>"
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009947 section, which follows, for more information.
9948 </para></listitem>
9949 <listitem><para><emphasis>"BeagleBoneTarget":</emphasis>
9950 Choose "BeagleBoneTarget" if you are deploying
9951 images and running tests on the BeagleBone
9952 "Black" or original "White" hardware.
9953 For information on how to use these tests, see the
9954 comments at the top of the BeagleBoneTarget
9955 <filename>meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py</filename>
9956 file.
9957 </para></listitem>
9958 <listitem><para><emphasis>"EdgeRouterTarget":</emphasis>
9959 Choose "EdgeRouterTarget" is you are deploying
9960 images and running tests on the Ubiquiti Networks
9961 EdgeRouter Lite.
9962 For information on how to use these tests, see the
9963 comments at the top of the EdgeRouterTarget
9964 <filename>meta-yocto-bsp/lib/oeqa/controllers/edgeroutertarget.py</filename>
9965 file.
9966 </para></listitem>
9967 <listitem><para><emphasis>"GrubTarget":</emphasis>
9968 Choose the "supports deploying images and running
9969 tests on any generic PC that boots using GRUB.
9970 For information on how to use these tests, see the
9971 comments at the top of the GrubTarget
9972 <filename>meta-yocto-bsp/lib/oeqa/controllers/grubtarget.py</filename>
9973 file.
9974 </para></listitem>
9975 <listitem><para><emphasis>"<replaceable>your-target</replaceable>":</emphasis>
9976 Create your own custom target if you want to run
9977 tests when you are deploying images and running
9978 tests on a custom machine within your BSP layer.
9979 To do this, you need to add a Python unit that
9980 defines the target class under
9981 <filename>lib/oeqa/controllers/</filename> within
9982 your layer.
9983 You must also provide an empty
9984 <filename>__init__.py</filename>.
9985 For examples, see files in
9986 <filename>meta-yocto-bsp/lib/oeqa/controllers/</filename>.
9987 </para></listitem>
9988 </itemizedlist>
9989 </para>
9990 </section>
9991
Patrick Williamsc0f7c042017-02-23 20:41:17 -06009992 <section id='selecting-systemd-boottarget'>
9993 <title>Selecting Systemd-bootTarget</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009994
9995 <para>
9996 If you did not set <filename>TEST_TARGET</filename> to
Patrick Williamsc0f7c042017-02-23 20:41:17 -06009997 "Systemd-bootTarget", then you do not need any information
Patrick Williamsc124f4f2015-09-15 14:41:29 -05009998 in this section.
9999 You can skip down to the
10000 "<link linkend='qemu-image-running-tests'>Running Tests</link>"
10001 section.
10002 </para>
10003
10004 <para>
10005 If you did set <filename>TEST_TARGET</filename> to
Patrick Williamsc0f7c042017-02-23 20:41:17 -060010006 "Systemd-bootTarget", you also need to perform a one-time
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010007 setup of your master image by doing the following:
10008 <orderedlist>
10009 <listitem><para><emphasis>Set <filename>EFI_PROVIDER</filename>:</emphasis>
10010 Be sure that <filename>EFI_PROVIDER</filename>
10011 is as follows:
10012 <literallayout class='monospaced'>
Patrick Williamsc0f7c042017-02-23 20:41:17 -060010013 EFI_PROVIDER = "systemd-boot"
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010014 </literallayout>
10015 </para></listitem>
10016 <listitem><para><emphasis>Build the master image:</emphasis>
10017 Build the <filename>core-image-testmaster</filename>
10018 image.
10019 The <filename>core-image-testmaster</filename>
10020 recipe is provided as an example for a
10021 "master" image and you can customize the image
10022 recipe as you would any other recipe.
10023 </para>
10024 <para>Here are the image recipe requirements:
10025 <itemizedlist>
10026 <listitem><para>Inherits
10027 <filename>core-image</filename>
10028 so that kernel modules are installed.
10029 </para></listitem>
10030 <listitem><para>Installs normal linux utilities
10031 not busybox ones (e.g.
10032 <filename>bash</filename>,
10033 <filename>coreutils</filename>,
10034 <filename>tar</filename>,
10035 <filename>gzip</filename>, and
10036 <filename>kmod</filename>).
10037 </para></listitem>
10038 <listitem><para>Uses a custom
10039 Initial RAM Disk (initramfs) image with a
10040 custom installer.
10041 A normal image that you can install usually
10042 creates a single rootfs partition.
10043 This image uses another installer that
10044 creates a specific partition layout.
10045 Not all Board Support Packages (BSPs)
10046 can use an installer.
10047 For such cases, you need to manually create
10048 the following partition layout on the
10049 target:
10050 <itemizedlist>
10051 <listitem><para>First partition mounted
10052 under <filename>/boot</filename>,
10053 labeled "boot".
10054 </para></listitem>
10055 <listitem><para>The main rootfs
10056 partition where this image gets
10057 installed, which is mounted under
10058 <filename>/</filename>.
10059 </para></listitem>
10060 <listitem><para>Another partition
10061 labeled "testrootfs" where test
10062 images get deployed.
10063 </para></listitem>
10064 </itemizedlist>
10065 </para></listitem>
10066 </itemizedlist>
10067 </para></listitem>
10068 <listitem><para><emphasis>Install image:</emphasis>
10069 Install the image that you just built on the target
10070 system.
10071 </para></listitem>
10072 </orderedlist>
10073 </para>
10074
10075 <para>
10076 The final thing you need to do when setting
Patrick Williamsc0f7c042017-02-23 20:41:17 -060010077 <filename>TEST_TARGET</filename> to "Systemd-bootTarget" is
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010078 to set up the test image:
10079 <orderedlist>
10080 <listitem><para><emphasis>Set up your <filename>local.conf</filename> file:</emphasis>
10081 Make sure you have the following statements in
10082 your <filename>local.conf</filename> file:
10083 <literallayout class='monospaced'>
10084 IMAGE_FSTYPES += "tar.gz"
10085 INHERIT += "testimage"
Patrick Williamsc0f7c042017-02-23 20:41:17 -060010086 TEST_TARGET = "Systemd-bootTarget"
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010087 TEST_TARGET_IP = "192.168.2.3"
10088 </literallayout>
10089 </para></listitem>
10090 <listitem><para><emphasis>Build your test image:</emphasis>
10091 Use BitBake to build the image:
10092 <literallayout class='monospaced'>
10093 $ bitbake core-image-sato
10094 </literallayout>
10095 </para></listitem>
10096 </orderedlist>
10097 </para>
10098 </section>
10099
10100 <section id='power-control'>
10101 <title>Power Control</title>
10102
10103 <para>
10104 For most hardware targets other than SimpleRemoteTarget,
10105 you can control power:
10106 <itemizedlist>
10107 <listitem><para>
10108 You can use
10109 <filename>TEST_POWERCONTROL_CMD</filename>
10110 together with
10111 <filename>TEST_POWERCONTROL_EXTRA_ARGS</filename>
10112 as a command that runs on the host and does power
10113 cycling.
10114 The test code passes one argument to that command:
10115 off, on or cycle (off then on).
10116 Here is an example that could appear in your
10117 <filename>local.conf</filename> file:
10118 <literallayout class='monospaced'>
10119 TEST_POWERCONTROL_CMD = "powercontrol.exp test 10.11.12.1 nuc1"
10120 </literallayout>
10121 In this example, the expect script does the
10122 following:
10123 <literallayout class='monospaced'>
10124 ssh test@10.11.12.1 "pyctl nuc1 <replaceable>arg</replaceable>"
10125 </literallayout>
10126 It then runs a Python script that controls power
10127 for a label called <filename>nuc1</filename>.
10128 <note>
10129 You need to customize
10130 <filename>TEST_POWERCONTROL_CMD</filename>
10131 and
10132 <filename>TEST_POWERCONTROL_EXTRA_ARGS</filename>
10133 for your own setup.
10134 The one requirement is that it accepts
10135 "on", "off", and "cycle" as the last argument.
10136 </note>
10137 </para></listitem>
10138 <listitem><para>
10139 When no command is defined, it connects to the
10140 device over SSH and uses the classic reboot command
10141 to reboot the device.
10142 Classic reboot is fine as long as the machine
10143 actually reboots (i.e. the SSH test has not
10144 failed).
10145 It is useful for scenarios where you have a simple
10146 setup, typically with a single board, and where
10147 some manual interaction is okay from time to time.
10148 </para></listitem>
10149 </itemizedlist>
10150 If you have no hardware to automatically perform power
10151 control but still wish to experiment with automated
10152 hardware testing, you can use the dialog-power-control
10153 script that shows a dialog prompting you to perform the
10154 required power action.
10155 This script requires either KDialog or Zenity to be
10156 installed.
10157 To use this script, set the
10158 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_POWERCONTROL_CMD'><filename>TEST_POWERCONTROL_CMD</filename></ulink>
10159 variable as follows:
10160 <literallayout class='monospaced'>
10161 TEST_POWERCONTROL_CMD = "${COREBASE}/scripts/contrib/dialog-power-control"
10162 </literallayout>
10163 </para>
10164 </section>
10165
10166 <section id='serial-console-connection'>
10167 <title>Serial Console Connection</title>
10168
10169 <para>
10170 For test target classes requiring a serial console
10171 to interact with the bootloader (e.g. BeagleBoneTarget,
10172 EdgeRouterTarget, and GrubTarget), you need to
10173 specify a command to use to connect to the serial console
10174 of the target machine by using the
10175 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SERIALCONTROL_CMD'><filename>TEST_SERIALCONTROL_CMD</filename></ulink>
10176 variable and optionally the
10177 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SERIALCONTROL_EXTRA_ARGS'><filename>TEST_SERIALCONTROL_EXTRA_ARGS</filename></ulink>
10178 variable.
10179 </para>
10180
10181 <para>
10182 These cases could be a serial terminal program if the
10183 machine is connected to a local serial port, or a
10184 <filename>telnet</filename> or
10185 <filename>ssh</filename> command connecting to a remote
10186 console server.
10187 Regardless of the case, the command simply needs to
10188 connect to the serial console and forward that connection
10189 to standard input and output as any normal terminal
10190 program does.
10191 For example, to use the picocom terminal program on
10192 serial device <filename>/dev/ttyUSB0</filename>
10193 at 115200bps, you would set the variable as follows:
10194 <literallayout class='monospaced'>
10195 TEST_SERIALCONTROL_CMD = "picocom /dev/ttyUSB0 -b 115200"
10196 </literallayout>
10197 For local devices where the serial port device disappears
10198 when the device reboots, an additional "serdevtry" wrapper
10199 script is provided.
10200 To use this wrapper, simply prefix the terminal command
10201 with
10202 <filename>${COREBASE}/scripts/contrib/serdevtry</filename>:
10203 <literallayout class='monospaced'>
10204 TEST_SERIALCONTROL_CMD = "${COREBASE}/scripts/contrib/serdevtry picocom -b
10205115200 /dev/ttyUSB0"
10206 </literallayout>
10207 </para>
10208 </section>
10209 </section>
10210
10211 <section id="qemu-image-running-tests">
10212 <title>Running Tests</title>
10213
10214 <para>
10215 You can start the tests automatically or manually:
10216 <itemizedlist>
10217 <listitem><para><emphasis>Automatically running tests:</emphasis>
10218 To run the tests automatically after the
10219 OpenEmbedded build system successfully creates an image,
10220 first set the
10221 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_IMAGE'><filename>TEST_IMAGE</filename></ulink>
10222 variable to "1" in your <filename>local.conf</filename>
10223 file in the
10224 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>:
10225 <literallayout class='monospaced'>
10226 TEST_IMAGE = "1"
10227 </literallayout>
10228 Next, build your image.
10229 If the image successfully builds, the tests will be
10230 run:
10231 <literallayout class='monospaced'>
10232 bitbake core-image-sato
10233 </literallayout></para></listitem>
10234 <listitem><para><emphasis>Manually running tests:</emphasis>
10235 To manually run the tests, first globally inherit the
Patrick Williamsf1e5d692016-03-30 15:21:19 -050010236 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage*'><filename>testimage</filename></ulink>
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010237 class by editing your <filename>local.conf</filename>
10238 file:
10239 <literallayout class='monospaced'>
10240 INHERIT += "testimage"
10241 </literallayout>
10242 Next, use BitBake to run the tests:
10243 <literallayout class='monospaced'>
10244 bitbake -c testimage <replaceable>image</replaceable>
10245 </literallayout></para></listitem>
10246 </itemizedlist>
10247 </para>
10248
10249 <para>
10250 All test files reside in
10251 <filename>meta/lib/oeqa/runtime</filename> in the
10252 <link linkend='source-directory'>Source Directory</link>.
10253 A test name maps directly to a Python module.
10254 Each test module may contain a number of individual tests.
10255 Tests are usually grouped together by the area
10256 tested (e.g tests for systemd reside in
10257 <filename>meta/lib/oeqa/runtime/systemd.py</filename>).
10258 </para>
10259
10260 <para>
10261 You can add tests to any layer provided you place them in the
10262 proper area and you extend
10263 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>
10264 in the <filename>local.conf</filename> file as normal.
10265 Be sure that tests reside in
10266 <filename><replaceable>layer</replaceable>/lib/oeqa/runtime</filename>.
10267 <note>
10268 Be sure that module names do not collide with module names
10269 used in the default set of test modules in
10270 <filename>meta/lib/oeqa/runtime</filename>.
10271 </note>
10272 </para>
10273
10274 <para>
10275 You can change the set of tests run by appending or overriding
10276 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SUITES'><filename>TEST_SUITES</filename></ulink>
10277 variable in <filename>local.conf</filename>.
10278 Each name in <filename>TEST_SUITES</filename> represents a
10279 required test for the image.
10280 Test modules named within <filename>TEST_SUITES</filename>
10281 cannot be skipped even if a test is not suitable for an image
10282 (e.g. running the RPM tests on an image without
10283 <filename>rpm</filename>).
10284 Appending "auto" to <filename>TEST_SUITES</filename> causes the
10285 build system to try to run all tests that are suitable for the
10286 image (i.e. each test module may elect to skip itself).
10287 </para>
10288
10289 <para>
10290 The order you list tests in <filename>TEST_SUITES</filename>
10291 is important and influences test dependencies.
10292 Consequently, tests that depend on other tests should be added
10293 after the test on which they depend.
10294 For example, since the <filename>ssh</filename> test
10295 depends on the
10296 <filename>ping</filename> test, "ssh" needs to come after
10297 "ping" in the list.
10298 The test class provides no re-ordering or dependency handling.
10299 <note>
10300 Each module can have multiple classes with multiple test
10301 methods.
10302 And, Python <filename>unittest</filename> rules apply.
10303 </note>
10304 </para>
10305
10306 <para>
10307 Here are some things to keep in mind when running tests:
10308 <itemizedlist>
10309 <listitem><para>The default tests for the image are defined
10310 as:
10311 <literallayout class='monospaced'>
Brad Bishop6e60e8b2018-02-01 10:27:11 -050010312 DEFAULT_TEST_SUITES_pn-<replaceable>image</replaceable> = "ping ssh df connman syslog xorg scp vnc date rpm dnf dmesg"
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010313 </literallayout></para></listitem>
10314 <listitem><para>Add your own test to the list of the
10315 by using the following:
10316 <literallayout class='monospaced'>
10317 TEST_SUITES_append = " mytest"
10318 </literallayout></para></listitem>
10319 <listitem><para>Run a specific list of tests as follows:
10320 <literallayout class='monospaced'>
10321 TEST_SUITES = "test1 test2 test3"
10322 </literallayout>
10323 Remember, order is important.
10324 Be sure to place a test that is dependent on another test
10325 later in the order.</para></listitem>
10326 </itemizedlist>
10327 </para>
10328 </section>
10329
10330 <section id="exporting-tests">
10331 <title>Exporting Tests</title>
10332
10333 <para>
10334 You can export tests so that they can run independently of
10335 the build system.
10336 Exporting tests is required if you want to be able to hand
10337 the test execution off to a scheduler.
10338 You can only export tests that are defined in
10339 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SUITES'><filename>TEST_SUITES</filename></ulink>.
10340 </para>
10341
10342 <para>
10343 If your image is already built, make sure the following are set
Brad Bishop37a0e4d2017-12-04 01:01:44 -050010344 in your <filename>local.conf</filename> file:
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010345 <literallayout class='monospaced'>
Patrick Williamsc0f7c042017-02-23 20:41:17 -060010346 INHERIT +="testexport"
Brad Bishop37a0e4d2017-12-04 01:01:44 -050010347 TEST_TARGET_IP = "<replaceable>IP-address-for-the-test-target</replaceable>"
10348 TEST_SERVER_IP = "<replaceable>IP-address-for-the-test-server</replaceable>"
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010349 </literallayout>
Brad Bishop37a0e4d2017-12-04 01:01:44 -050010350 You can then export the tests with the following BitBake
10351 command form:
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010352 <literallayout class='monospaced'>
Brad Bishop37a0e4d2017-12-04 01:01:44 -050010353 $ bitbake <replaceable>image</replaceable> -c testexport
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010354 </literallayout>
10355 Exporting the tests places them in the
10356 <link linkend='build-directory'>Build Directory</link> in
Brad Bishop37a0e4d2017-12-04 01:01:44 -050010357 <filename>tmp/testexport/</filename><replaceable>image</replaceable>,
10358 which is controlled by the
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010359 <filename>TEST_EXPORT_DIR</filename> variable.
10360 </para>
10361
10362 <para>
10363 You can now run the tests outside of the build environment:
10364 <literallayout class='monospaced'>
Brad Bishop37a0e4d2017-12-04 01:01:44 -050010365 $ cd tmp/testexport/<replaceable>image</replaceable>
10366 $ ./runexported.py testdata.json
10367 </literallayout>
10368 </para>
10369
10370 <para>
10371 Here is a complete example that shows IP addresses and uses
10372 the <filename>core-image-sato</filename> image:
10373 <literallayout class='monospaced'>
10374 INHERIT +="testexport"
10375 TEST_TARGET_IP = "192.168.7.2"
10376 TEST_SERVER_IP = "192.168.7.1"
10377 </literallayout>
10378 Use BitBake to export the tests:
10379 <literallayout class='monospaced'>
10380 $ bitbake core-image-sato -c testexport
10381 </literallayout>
10382 Run the tests outside of the build environment using the
10383 following:
10384 <literallayout class='monospaced'>
Patrick Williamsc0f7c042017-02-23 20:41:17 -060010385 $ cd tmp/testexport/core-image-sato
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010386 $ ./runexported.py testdata.json
10387 </literallayout>
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010388 </para>
10389 </section>
10390
10391 <section id="qemu-image-writing-new-tests">
10392 <title>Writing New Tests</title>
10393
10394 <para>
10395 As mentioned previously, all new test files need to be in the
10396 proper place for the build system to find them.
10397 New tests for additional functionality outside of the core
10398 should be added to the layer that adds the functionality, in
10399 <filename><replaceable>layer</replaceable>/lib/oeqa/runtime</filename>
10400 (as long as
10401 <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>
10402 is extended in the layer's
10403 <filename>layer.conf</filename> file as normal).
10404 Just remember the following:
10405 <itemizedlist>
10406 <listitem><para>Filenames need to map directly to test
10407 (module) names.
10408 </para></listitem>
10409 <listitem><para>Do not use module names that
10410 collide with existing core tests.
10411 </para></listitem>
10412 <listitem><para>Minimally, an empty
10413 <filename>__init__.py</filename> file must exist
10414 in the runtime directory.
10415 </para></listitem>
10416 </itemizedlist>
10417 </para>
10418
10419 <para>
10420 To create a new test, start by copying an existing module
10421 (e.g. <filename>syslog.py</filename> or
10422 <filename>gcc.py</filename> are good ones to use).
10423 Test modules can use code from
10424 <filename>meta/lib/oeqa/utils</filename>, which are helper
10425 classes.
10426 </para>
10427
10428 <note>
10429 Structure shell commands such that you rely on them and they
10430 return a single code for success.
10431 Be aware that sometimes you will need to parse the output.
10432 See the <filename>df.py</filename> and
10433 <filename>date.py</filename> modules for examples.
10434 </note>
10435
10436 <para>
10437 You will notice that all test classes inherit
10438 <filename>oeRuntimeTest</filename>, which is found in
10439 <filename>meta/lib/oetest.py</filename>.
10440 This base class offers some helper attributes, which are
10441 described in the following sections:
10442 </para>
10443
10444 <section id='qemu-image-writing-tests-class-methods'>
10445 <title>Class Methods</title>
10446
10447 <para>
10448 Class methods are as follows:
10449 <itemizedlist>
10450 <listitem><para><emphasis><filename>hasPackage(pkg)</filename>:</emphasis>
10451 Returns "True" if <filename>pkg</filename> is in the
10452 installed package list of the image, which is based
10453 on the manifest file that is generated during the
10454 <filename>do_rootfs</filename> task.
10455 </para></listitem>
10456 <listitem><para><emphasis><filename>hasFeature(feature)</filename>:</emphasis>
10457 Returns "True" if the feature is in
10458 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>
10459 or
10460 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>.
10461 </para></listitem>
10462 </itemizedlist>
10463 </para>
10464 </section>
10465
10466 <section id='qemu-image-writing-tests-class-attributes'>
10467 <title>Class Attributes</title>
10468
10469 <para>
10470 Class attributes are as follows:
10471 <itemizedlist>
10472 <listitem><para><emphasis><filename>pscmd</filename>:</emphasis>
10473 Equals "ps -ef" if <filename>procps</filename> is
10474 installed in the image.
10475 Otherwise, <filename>pscmd</filename> equals
10476 "ps" (busybox).
10477 </para></listitem>
10478 <listitem><para><emphasis><filename>tc</filename>:</emphasis>
10479 The called test context, which gives access to the
10480 following attributes:
10481 <itemizedlist>
10482 <listitem><para><emphasis><filename>d</filename>:</emphasis>
10483 The BitBake datastore, which allows you to
10484 use stuff such as
10485 <filename>oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager")</filename>.
10486 </para></listitem>
10487 <listitem><para><emphasis><filename>testslist</filename> and <filename>testsrequired</filename>:</emphasis>
10488 Used internally.
10489 The tests do not need these.
10490 </para></listitem>
10491 <listitem><para><emphasis><filename>filesdir</filename>:</emphasis>
10492 The absolute path to
10493 <filename>meta/lib/oeqa/runtime/files</filename>,
10494 which contains helper files for tests meant
10495 for copying on the target such as small
10496 files written in C for compilation.
10497 </para></listitem>
10498 <listitem><para><emphasis><filename>target</filename>:</emphasis>
10499 The target controller object used to deploy
10500 and start an image on a particular target
10501 (e.g. QemuTarget, SimpleRemote, and
Patrick Williamsc0f7c042017-02-23 20:41:17 -060010502 Systemd-bootTarget).
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010503 Tests usually use the following:
10504 <itemizedlist>
10505 <listitem><para><emphasis><filename>ip</filename>:</emphasis>
10506 The target's IP address.
10507 </para></listitem>
10508 <listitem><para><emphasis><filename>server_ip</filename>:</emphasis>
10509 The host's IP address, which is
Brad Bishop6e60e8b2018-02-01 10:27:11 -050010510 usually used by the DNF test
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010511 suite.
10512 </para></listitem>
10513 <listitem><para><emphasis><filename>run(cmd, timeout=None)</filename>:</emphasis>
10514 The single, most used method.
10515 This command is a wrapper for:
10516 <filename>ssh root@host "cmd"</filename>.
10517 The command returns a tuple:
10518 (status, output), which are what
10519 their names imply - the return code
10520 of "cmd" and whatever output
10521 it produces.
10522 The optional timeout argument
10523 represents the number of seconds the
10524 test should wait for "cmd" to
10525 return.
10526 If the argument is "None", the
10527 test uses the default instance's
10528 timeout period, which is 300
10529 seconds.
10530 If the argument is "0", the test
10531 runs until the command returns.
10532 </para></listitem>
10533 <listitem><para><emphasis><filename>copy_to(localpath, remotepath)</filename>:</emphasis>
10534 <filename>scp localpath root@ip:remotepath</filename>.
10535 </para></listitem>
10536 <listitem><para><emphasis><filename>copy_from(remotepath, localpath)</filename>:</emphasis>
10537 <filename>scp root@host:remotepath localpath</filename>.
10538 </para></listitem>
10539 </itemizedlist></para></listitem>
10540 </itemizedlist></para></listitem>
10541 </itemizedlist>
10542 </para>
10543 </section>
10544
10545 <section id='qemu-image-writing-tests-instance-attributes'>
10546 <title>Instance Attributes</title>
10547
10548 <para>
10549 A single instance attribute exists, which is
10550 <filename>target</filename>.
10551 The <filename>target</filename> instance attribute is
10552 identical to the class attribute of the same name, which
10553 is described in the previous section.
10554 This attribute exists as both an instance and class
10555 attribute so tests can use
10556 <filename>self.target.run(cmd)</filename> in instance
10557 methods instead of
10558 <filename>oeRuntimeTest.tc.target.run(cmd)</filename>.
10559 </para>
10560 </section>
10561 </section>
Patrick Williamsc0f7c042017-02-23 20:41:17 -060010562
10563 <section id='installing-packages-in-the-dut-without-the-package-manager'>
10564 <title>Installing Packages in the DUT Without the Package Manager</title>
10565
10566 <para>
10567 When a test requires a package built by BitBake, it is possible
10568 to install that package.
10569 Installing the package does not require a package manager be
10570 installed in the device under test (DUT).
10571 It does, however, require an SSH connection and the target must
10572 be using the <filename>sshcontrol</filename> class.
10573 <note>
10574 This method uses <filename>scp</filename> to copy files
10575 from the host to the target, which causes permissions and
10576 special attributes to be lost.
10577 </note>
10578 </para>
10579
10580 <para>
10581 A JSON file is used to define the packages needed by a test.
10582 This file must be in the same path as the file used to define
10583 the tests.
10584 Furthermore, the filename must map directly to the test
10585 module name with a <filename>.json</filename> extension.
10586 </para>
10587
10588 <para>
10589 The JSON file must include an object with the test name as
10590 keys of an object or an array.
10591 This object (or array of objects) uses the following data:
10592 <itemizedlist>
10593 <listitem><para>"pkg" - A mandatory string that is the
10594 name of the package to be installed.
10595 </para></listitem>
10596 <listitem><para>"rm" - An optional boolean, which defaults
10597 to "false", that specifies to remove the package after
10598 the test.
10599 </para></listitem>
10600 <listitem><para>"extract" - An optional boolean, which
10601 defaults to "false", that specifies if the package must
10602 be extracted from the package format.
10603 When set to "true", the package is not automatically
10604 installed into the DUT.
10605 </para></listitem>
10606 </itemizedlist>
10607 </para>
10608
10609 <para>
10610 Following is an example JSON file that handles test "foo"
10611 installing package "bar" and test "foobar" installing
10612 packages "foo" and "bar".
10613 Once the test is complete, the packages are removed from the
10614 DUT.
10615 <literallayout class='monospaced'>
10616 {
10617 "foo": {
10618 "pkg": "bar"
10619 },
10620 "foobar": [
10621 {
10622 "pkg": "foo",
10623 "rm": true
10624 },
10625 {
10626 "pkg": "bar",
10627 "rm": true
10628 }
10629 ]
10630 }
10631 </literallayout>
10632 </para>
10633 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010634 </section>
10635
10636 <section id="platdev-gdb-remotedebug">
10637 <title>Debugging With the GNU Project Debugger (GDB) Remotely</title>
10638
10639 <para>
10640 GDB allows you to examine running programs, which in turn helps you to understand and fix problems.
10641 It also allows you to perform post-mortem style analysis of program crashes.
10642 GDB is available as a package within the Yocto Project and is
10643 installed in SDK images by default.
10644 See the "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>" chapter
10645 in the Yocto Project Reference Manual for a description of these images.
10646 You can find information on GDB at <ulink url="http://sourceware.org/gdb/"/>.
10647 </para>
10648
10649 <tip>
10650 For best results, install debug (<filename>-dbg</filename>) packages
10651 for the applications you are going to debug.
10652 Doing so makes extra debug symbols available that give you more
10653 meaningful output.
10654 </tip>
10655
10656 <para>
10657 Sometimes, due to memory or disk space constraints, it is not possible
10658 to use GDB directly on the remote target to debug applications.
10659 These constraints arise because GDB needs to load the debugging information and the
10660 binaries of the process being debugged.
10661 Additionally, GDB needs to perform many computations to locate information such as function
10662 names, variable names and values, stack traces and so forth - even before starting the
10663 debugging process.
10664 These extra computations place more load on the target system and can alter the
10665 characteristics of the program being debugged.
10666 </para>
10667
10668 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -050010669 To help get past the previously mentioned constraints, you can use
10670 gdbserver, which runs on the remote target and does not load any
10671 debugging information from the debugged process.
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010672 Instead, a GDB instance processes the debugging information that is run on a
10673 remote computer - the host GDB.
Brad Bishop6e60e8b2018-02-01 10:27:11 -050010674 The host GDB then sends control commands to gdbserver to make it stop or start the debugged
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010675 program, as well as read or write memory regions of that debugged program.
10676 All the debugging information loaded and processed as well
10677 as all the heavy debugging is done by the host GDB.
Brad Bishop6e60e8b2018-02-01 10:27:11 -050010678 Offloading these processes gives the gdbserver running on the target a chance to remain
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010679 small and fast.
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010680 </para>
10681
10682 <para>
10683 Because the host GDB is responsible for loading the debugging information and
10684 for doing the necessary processing to make actual debugging happen,
10685 you have to make sure the host can access the unstripped binaries complete
10686 with their debugging information and also be sure the target is compiled with no optimizations.
10687 The host GDB must also have local access to all the libraries used by the
10688 debugged program.
Brad Bishop6e60e8b2018-02-01 10:27:11 -050010689 Because gdbserver does not need any local debugging information, the binaries on
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010690 the remote target can remain stripped.
10691 However, the binaries must also be compiled without optimization
10692 so they match the host's binaries.
10693 </para>
10694
10695 <para>
10696 To remain consistent with GDB documentation and terminology, the binary being debugged
10697 on the remote target machine is referred to as the "inferior" binary.
10698 For documentation on GDB see the
10699 <ulink url="http://sourceware.org/gdb/documentation/">GDB site</ulink>.
10700 </para>
10701
10702 <para>
Patrick Williamsc0f7c042017-02-23 20:41:17 -060010703 The following steps show you how to debug using the GNU project
10704 debugger.
10705 <orderedlist>
10706 <listitem><para>
10707 <emphasis>Configure your build system to construct the
10708 companion debug filesystem:</emphasis></para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010709
Patrick Williamsc0f7c042017-02-23 20:41:17 -060010710 <para>In your <filename>local.conf</filename> file, set
10711 the following:
10712 <literallayout class='monospaced'>
10713 IMAGE_GEN_DEBUGFS = "1"
10714 IMAGE_FSTYPES_DEBUGFS = "tar.bz2"
10715 </literallayout>
10716 These options cause the OpenEmbedded build system
10717 to generate a special companion filesystem fragment,
10718 which contains the matching source and debug symbols to
10719 your deployable filesystem.
10720 The build system does this by looking at what is in the
10721 deployed filesystem, and pulling the corresponding
10722 <filename>-dbg</filename> packages.</para>
10723
10724 <para>The companion debug filesystem is not a complete
10725 filesystem, but only contains the debug fragments.
10726 This filesystem must be combined with the full filesystem
10727 for debugging.
10728 Subsequent steps in this procedure show how to combine
10729 the partial filesystem with the full filesystem.
10730 </para></listitem>
10731 <listitem><para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -050010732 <emphasis>Configure the system to include gdbserver in
Patrick Williamsc0f7c042017-02-23 20:41:17 -060010733 the target filesystem:</emphasis></para>
10734
10735 <para>Make the following addition in either your
10736 <filename>local.conf</filename> file or in an image
10737 recipe:
10738 <literallayout class='monospaced'>
10739 IMAGE_INSTALL_append = “ gdbserver"
10740 </literallayout>
10741 The change makes sure the <filename>gdbserver</filename>
10742 package is included.
10743 </para></listitem>
10744 <listitem><para>
10745 <emphasis>Build the environment:</emphasis></para>
10746
10747 <para>Use the following command to construct the image and
10748 the companion Debug Filesystem:
10749 <literallayout class='monospaced'>
10750 $ bitbake <replaceable>image</replaceable>
10751 </literallayout>
10752 Build the cross GDB component and make it available
10753 for debugging.
10754 Build the SDK that matches the image.
10755 Building the SDK is best for a production build
10756 that can be used later for debugging, especially
10757 during long term maintenance:
10758 <literallayout class='monospaced'>
10759 $ bitbake -c populate_sdk <replaceable>image</replaceable>
10760 </literallayout></para>
10761
10762 <para>Alternatively, you can build the minimal
10763 toolchain components that match the target.
10764 Doing so creates a smaller than typical SDK and only
10765 contains a minimal set of components with which to
10766 build simple test applications, as well as run the
10767 debugger:
10768 <literallayout class='monospaced'>
10769 $ bitbake meta-toolchain
10770 </literallayout></para>
10771
10772 <para>A final method is to build Gdb itself within
10773 the build system:
10774 <literallayout class='monospaced'>
10775 $ bitbake gdb-cross-<replaceable>architecture</replaceable>
10776 </literallayout>
10777 Doing so produces a temporary copy of
10778 <filename>cross-gdb</filename> you can use for
10779 debugging during development.
10780 While this is the quickest approach, the two previous
10781 methods in this step are better when considering
10782 long-term maintenance strategies.
10783 <note>
10784 If you run
10785 <filename>bitbake gdb-cross</filename>, the
10786 OpenEmbedded build system suggests the actual
10787 image (e.g. <filename>gdb-cross-i586</filename>).
10788 The suggestion is usually the actual name you want
10789 to use.
10790 </note>
10791 </para></listitem>
10792 <listitem><para>
10793 <emphasis>Set up the</emphasis>&nbsp;<filename>debugfs</filename></para>
10794
10795 <para>Run the following commands to set up the
10796 <filename>debugfs</filename>:
10797 <literallayout class='monospaced'>
10798 $ mkdir debugfs
10799 $ cd debugfs
10800 $ tar xvfj <replaceable>build-dir</replaceable>/tmp-glibc/deploy/images/<replaceable>machine</replaceable>/<replaceable>image</replaceable>.rootfs.tar.bz2
10801 $ tar xvfj <replaceable>build-dir</replaceable>/tmp-glibc/deploy/images/<replaceable>machine</replaceable>/<replaceable>image</replaceable>-dbg.rootfs.tar.bz2
10802 </literallayout>
10803 </para></listitem>
10804 <listitem><para>
10805 <emphasis>Set up GDB</emphasis></para>
10806
10807 <para>Install the SDK (if you built one) and then
10808 source the correct environment file.
10809 Sourcing the environment file puts the SDK in your
10810 <filename>PATH</filename> environment variable.</para>
10811
10812 <para>If you are using the build system, Gdb is
10813 located in
10814 <replaceable>build-dir</replaceable>/tmp/sysroots/<replaceable>host</replaceable>/usr/bin/<replaceable>architecture</replaceable>/<replaceable>architecture</replaceable>-gdb
10815 </para></listitem>
10816 <listitem><para>
10817 <emphasis>Boot the target:</emphasis></para>
10818
10819 <para>For information on how to run QEMU, see the
10820 <ulink url='http://wiki.qemu.org/Documentation/GettingStartedDevelopers'>QEMU Documentation</ulink>.
10821 <note>
10822 Be sure to verify that your host can access the
10823 target via TCP.
10824 </note>
10825 </para></listitem>
10826 <listitem><para>
10827 <emphasis>Debug a program:</emphasis></para>
10828
Brad Bishop6e60e8b2018-02-01 10:27:11 -050010829 <para>Debugging a program involves running gdbserver
Patrick Williamsc0f7c042017-02-23 20:41:17 -060010830 on the target and then running Gdb on the host.
10831 The example in this step debugs
10832 <filename>gzip</filename>:
10833 <literallayout class='monospaced'>
10834 root@qemux86:~# gdbserver localhost:1234 /bin/gzip —help
10835 </literallayout>
Brad Bishop6e60e8b2018-02-01 10:27:11 -050010836 For additional gdbserver options, see the
10837 <ulink url='https://www.gnu.org/software/gdb/documentation/'>GDB Server Documentation</ulink>.
Patrick Williamsc0f7c042017-02-23 20:41:17 -060010838 </para>
10839
Brad Bishop6e60e8b2018-02-01 10:27:11 -050010840 <para>After running gdbserver on the target, you need
Patrick Williamsc0f7c042017-02-23 20:41:17 -060010841 to run Gdb on the host and configure it and connect to
10842 the target.
10843 Use these commands:
10844 <literallayout class='monospaced'>
10845 $ cd <replaceable>directory-holding-the-debugfs-directory</replaceable>
10846 $ <replaceable>arch</replaceable>-gdb
10847
10848 (gdb) set sysroot debugfs
10849 (gdb) set substitute-path /usr/src/debug debugfs/usr/src/debug
10850 (gdb) target remote <replaceable>IP-of-target</replaceable>:1234
10851 </literallayout>
10852 At this point, everything should automatically load
10853 (i.e. matching binaries, symbols and headers).
10854 <note>
10855 The Gdb <filename>set</filename> commands in the
10856 previous example can be placed into the users
10857 <filename>~/.gdbinit</filename> file.
10858 Upon starting, Gdb automatically runs whatever
10859 commands are in that file.
10860 </note>
10861 </para></listitem>
10862 <listitem><para>
10863 <emphasis>Deploying without a full image
10864 rebuild:</emphasis></para>
10865
10866 <para>In many cases, during development you want a
10867 quick method to deploy a new binary to the target and
10868 debug it, without waiting for a full image build.
10869 </para>
10870
10871 <para>One approach to solving this situation is to
10872 just build the component you want to debug.
10873 Once you have built the component, copy the
10874 executable directly to both the target and the
10875 host <filename>debugfs</filename>.</para>
10876
10877 <para>If the binary is processed through the debug
10878 splitting in OpenEmbedded, you should also
10879 copy the debug items (i.e. <filename>.debug</filename>
10880 contents and corresponding
10881 <filename>/usr/src/debug</filename> files)
10882 from the work directory.
10883 Here is an example:
10884 <literallayout class='monospaced'>
10885 $ bitbake bash
10886 $ bitbake -c devshell bash
10887 $ cd ..
10888 $ scp packages-split/bash/bin/bash <replaceable>target</replaceable>:/bin/bash
10889 $ cp -a packages-split/bash-dbg/* <replaceable>path</replaceable>/debugfs
10890 </literallayout>
10891 </para></listitem>
10892 </orderedlist>
10893 </para>
10894 </section>
10895
10896<!--
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010897 <section id='platdev-gdb-remotedebug-setup'>
10898 <title>Set Up the Cross-Development Debugging Environment</title>
10899
10900 <para>
10901 Before you can initiate a remote debugging session, you need
10902 to be sure you have set up the cross-development environment,
10903 toolchain, and sysroot.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -050010904 The <ulink url='&YOCTO_DOCS_SDK_URL;#sdk-intro'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010905 describes this process.
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010906 </para>
10907 </section>
10908
10909 <section id="platdev-gdb-remotedebug-launch-gdbserver">
Brad Bishop6e60e8b2018-02-01 10:27:11 -050010910 <title>Launch gdbserver on the Target</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010911
10912 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -050010913 Make sure gdbserver is installed on the target.
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010914 If it is not, install the package
10915 <filename>gdbserver</filename>, which needs the
10916 <filename>libthread-db1</filename> package.
10917 </para>
10918
10919 <para>
10920 Here is an example, that when entered from the host,
Brad Bishop6e60e8b2018-02-01 10:27:11 -050010921 connects to the target and launches gdbserver in order to
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010922 "debug" a binary named <filename>helloworld</filename>:
10923 <literallayout class='monospaced'>
10924 $ gdbserver localhost:2345 /usr/bin/helloworld
10925 </literallayout>
Brad Bishop6e60e8b2018-02-01 10:27:11 -050010926 gdbserver should now be listening on port 2345 for debugging
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010927 commands coming from a remote GDB process that is running on
10928 the host computer.
Brad Bishop6e60e8b2018-02-01 10:27:11 -050010929 Communication between gdbserver and the host GDB are done
Patrick Williamsc124f4f2015-09-15 14:41:29 -050010930 using TCP.
10931 To use other communication protocols, please refer to the
10932 <ulink url='http://www.gnu.org/software/gdb/'>Gdbserver documentation</ulink>.
10933 </para>
10934 </section>
10935
10936 <section id="platdev-gdb-remotedebug-launch-gdb">
10937 <title>Launch GDB on the Host Computer</title>
10938
10939 <para>
10940 Running GDB on the host computer takes a number of stages, which
10941 this section describes.
10942 </para>
10943
10944 <section id="platdev-gdb-remotedebug-launch-gdb-buildcross">
10945 <title>Build the Cross-GDB Package</title>
10946 <para>
10947 A suitable GDB cross-binary is required that runs on your
10948 host computer but also knows about the the ABI of the
10949 remote target.
10950 You can get this binary from the
10951 <link linkend='cross-development-toolchain'>Cross-Development Toolchain</link>.
10952 Here is an example where the toolchain has been installed
10953 in the default directory
10954 <filename>/opt/poky/&DISTRO;</filename>:
10955 <literallayout class='monospaced'>
10956 /opt/poky/&DISTRO;/sysroots/i686-pokysdk-linux/usr/bin/armv7a-vfp-neon-poky-linux-gnueabi/arm-poky-linux-gnueabi-gdb
10957 </literallayout>
10958 where <filename>arm</filename> is the target architecture
10959 and <filename>linux-gnueabi</filename> is the target ABI.
10960 </para>
10961
10962 <para>
10963 Alternatively, you can use BitBake to build the
10964 <filename>gdb-cross</filename> binary.
10965 Here is an example:
10966 <literallayout class='monospaced'>
10967 $ bitbake gdb-cross
10968 </literallayout>
10969 Once the binary is built, you can find it here:
10970 <literallayout class='monospaced'>
10971 tmp/sysroots/<replaceable>host-arch</replaceable>/usr/bin/<replaceable>target-platform</replaceable>/<replaceable>target-abi</replaceable>-gdb
10972 </literallayout>
10973 </para>
10974 </section>
10975
10976 <section id='create-the-gdb-initialization-file'>
10977 <title>Create the GDB Initialization File and Point to Your Root Filesystem</title>
10978
10979 <para>
10980 Aside from the GDB cross-binary, you also need a GDB
10981 initialization file in the same top directory in which
10982 your binary resides.
10983 When you start GDB on your host development system, GDB
10984 finds this initialization file and executes all the
10985 commands within.
10986 For information on the <filename>.gdbinit</filename>, see
10987 "<ulink url='http://sourceware.org/gdb/onlinedocs/gdb/'>Debugging with GDB</ulink>",
10988 which is maintained by
10989 <ulink url='http://www.sourceware.org'>sourceware.org</ulink>.
10990 </para>
10991
10992 <para>
10993 You need to add a statement in the
10994 <filename>~/.gdbinit</filename> file that points to your
10995 root filesystem.
10996 Here is an example that points to the root filesystem for
10997 an ARM-based target device:
10998 <literallayout class='monospaced'>
10999 set sysroot ~/sysroot_arm
11000 </literallayout>
11001 </para>
11002 </section>
11003
11004 <section id="platdev-gdb-remotedebug-launch-gdb-launchhost">
11005 <title>Launch the Host GDB</title>
11006
11007 <para>
11008 Before launching the host GDB, you need to be sure
11009 you have sourced the cross-debugging environment script,
11010 which if you installed the root filesystem in the default
11011 location is at <filename>/opt/poky/&DISTRO;</filename>
11012 and begins with the string "environment-setup".
11013 For more information, see the
Patrick Williamsd8c66bc2016-06-20 12:57:21 -050011014 <ulink url='&YOCTO_DOCS_SDK_URL;#sdk-manual'>Yocto Project Software Development Kit (SDK) Developer's
11015 Guide</ulink>.
Patrick Williamsc124f4f2015-09-15 14:41:29 -050011016 </para>
11017
11018 <para>
11019 Finally, switch to the directory where the binary resides
11020 and run the <filename>cross-gdb</filename> binary.
11021 Provide the binary file you are going to debug.
11022 For example, the following command continues with the
11023 example used in the previous section by loading
11024 the <filename>helloworld</filename> binary as well as the
11025 debugging information:
11026 <literallayout class='monospaced'>
11027 $ arm-poky-linux-gnuabi-gdb helloworld
11028 </literallayout>
11029 The commands in your <filename>.gdbinit</filename> execute
11030 and the GDB prompt appears.
11031 </para>
11032 </section>
11033 </section>
11034
11035 <section id='platdev-gdb-connect-to-the-remote-gdb-server'>
11036 <title>Connect to the Remote GDB Server</title>
11037
11038 <para>
11039 From the target, you need to connect to the remote GDB
11040 server that is running on the host.
11041 You need to specify the remote host and port.
11042 Here is the command continuing with the example:
11043 <literallayout class='monospaced'>
11044 target remote 192.168.7.2:2345
11045 </literallayout>
11046 </para>
11047 </section>
11048
11049 <section id="platdev-gdb-remotedebug-launch-gdb-using">
11050 <title>Use the Debugger</title>
11051
11052 <para>
11053 You can now proceed with debugging as normal - as if you were debugging
11054 on the local machine.
11055 For example, to instruct GDB to break in the "main" function and then
11056 continue with execution of the inferior binary use the following commands
11057 from within GDB:
11058 <literallayout class='monospaced'>
11059 (gdb) break main
11060 (gdb) continue
11061 </literallayout>
11062 </para>
11063
11064 <para>
11065 For more information about using GDB, see the project's online documentation at
11066 <ulink url="http://sourceware.org/gdb/download/onlinedocs/"/>.
11067 </para>
11068 </section>
11069 </section>
Patrick Williamsc0f7c042017-02-23 20:41:17 -060011070-->
11071
11072 <section id='debugging-with-the-gnu-project-debugger-gdb-on-the-target'>
11073 <title>Debugging with the GNU Project Debugger (GDB) on the Target</title>
11074
11075 <para>
11076 The previous section addressed using GDB remotely for debugging
11077 purposes, which is the most usual case due to the inherent
11078 hardware limitations on many embedded devices.
11079 However, debugging in the target hardware itself is also possible
11080 with more powerful devices.
11081 This section describes what you need to do in order to support
11082 using GDB to debug on the target hardware.
11083 </para>
11084
11085 <para>
11086 To support this kind of debugging, you need do the following:
11087 <itemizedlist>
11088 <listitem><para>
11089 Ensure that GDB is on the target.
11090 You can do this by adding "gdb" to
11091 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink>:
11092 <literallayout class='monospaced'>
11093 IMAGE_INSTALL_append = " gdb"
11094 </literallayout>
11095 Alternatively, you can add "tools-debug" to
11096 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>:
11097 <literallayout class='monospaced'>
11098 IMAGE_FEATURES_append = " tools-debug"
11099 </literallayout>
11100 </para></listitem>
11101 <listitem><para>
11102 Ensure that debug symbols are present.
11103 You can make sure these symbols are present by installing
11104 <filename>-dbg</filename>:
11105 <literallayout class='monospaced'>
11106 IMAGE_INSTALL_append = " <replaceable>packagename</replaceable>-dbg"
11107 </literallayout>
11108 Alternatively, you can do the following to include all the
11109 debug symbols:
11110 <literallayout class='monospaced'>
11111 IMAGE_FEATURES_append = " dbg-pkgs"
11112 </literallayout>
11113 </para></listitem>
11114 </itemizedlist>
11115 <note>
11116 To improve the debug information accuracy, you can reduce the
11117 level of optimization used by the compiler.
11118 For example, when adding the following line to your
11119 <filename>local.conf</filename> file, you will reduce
11120 optimization from
11121 <ulink url='&YOCTO_DOCS_REF_URL;#var-FULL_OPTIMIZATION'><filename>FULL_OPTIMIZATION</filename></ulink>
11122 of "-O2" to
11123 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEBUG_OPTIMIZATION'><filename>DEBUG_OPTIMIZATION</filename></ulink>
11124 of "-O -fno-omit-frame-pointer":
11125 <literallayout class='monospaced'>
11126 DEBUG_BUILD = "1"
11127 </literallayout>
11128 Consider that this will reduce the application's performance
11129 and is recommended only for debugging purposes.
11130 </note>
11131 </para>
11132 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -050011133
11134 <section id='debugging-parallel-make-races'>
11135 <title>Debugging Parallel Make Races</title>
11136
11137 <para>
11138 A parallel <filename>make</filename> race occurs when the build
11139 consists of several parts that are run simultaneously and
11140 a situation occurs when the output or result of one
11141 part is not ready for use with a different part of the build that
11142 depends on that output.
11143 Parallel make races are annoying and can sometimes be difficult
11144 to reproduce and fix.
11145 However, some simple tips and tricks exist that can help
11146 you debug and fix them.
11147 This section presents a real-world example of an error encountered
11148 on the Yocto Project autobuilder and the process used to fix it.
11149 <note>
11150 If you cannot properly fix a <filename>make</filename> race
11151 condition, you can work around it by clearing either the
11152 <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></ulink>
11153 or
11154 <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKEINST'><filename>PARALLEL_MAKEINST</filename></ulink>
11155 variables.
11156 </note>
11157 </para>
11158
11159 <section id='the-failure'>
11160 <title>The Failure</title>
11161
11162 <para>
11163 For this example, assume that you are building an image that
11164 depends on the "neard" package.
11165 And, during the build, BitBake runs into problems and
11166 creates the following output.
11167 <note>
11168 This example log file has longer lines artificially
11169 broken to make the listing easier to read.
11170 </note>
11171 If you examine the output or the log file, you see the
11172 failure during <filename>make</filename>:
11173 <literallayout class='monospaced'>
11174 | DEBUG: SITE files ['endian-little', 'bit-32', 'ix86-common', 'common-linux', 'common-glibc', 'i586-linux', 'common']
11175 | DEBUG: Executing shell function do_compile
11176 | NOTE: make -j 16
11177 | make --no-print-directory all-am
11178 | /bin/mkdir -p include/near
11179 | /bin/mkdir -p include/near
11180 | /bin/mkdir -p include/near
11181 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
11182 0.14-r0/neard-0.14/include/types.h include/near/types.h
11183 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
11184 0.14-r0/neard-0.14/include/log.h include/near/log.h
11185 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
11186 0.14-r0/neard-0.14/include/plugin.h include/near/plugin.h
11187 | /bin/mkdir -p include/near
11188 | /bin/mkdir -p include/near
11189 | /bin/mkdir -p include/near
11190 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
11191 0.14-r0/neard-0.14/include/tag.h include/near/tag.h
11192 | /bin/mkdir -p include/near
11193 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
11194 0.14-r0/neard-0.14/include/adapter.h include/near/adapter.h
11195 | /bin/mkdir -p include/near
11196 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
11197 0.14-r0/neard-0.14/include/ndef.h include/near/ndef.h
11198 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
11199 0.14-r0/neard-0.14/include/tlv.h include/near/tlv.h
11200 | /bin/mkdir -p include/near
11201 | /bin/mkdir -p include/near
11202 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
11203 0.14-r0/neard-0.14/include/setting.h include/near/setting.h
11204 | /bin/mkdir -p include/near
11205 | /bin/mkdir -p include/near
11206 | /bin/mkdir -p include/near
11207 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
11208 0.14-r0/neard-0.14/include/device.h include/near/device.h
11209 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
11210 0.14-r0/neard-0.14/include/nfc_copy.h include/near/nfc_copy.h
11211 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
11212 0.14-r0/neard-0.14/include/snep.h include/near/snep.h
11213 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
11214 0.14-r0/neard-0.14/include/version.h include/near/version.h
11215 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
11216 0.14-r0/neard-0.14/include/dbus.h include/near/dbus.h
11217 | ./src/genbuiltin nfctype1 nfctype2 nfctype3 nfctype4 p2p > src/builtin.h
11218 | i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/
11219 build/build/tmp/sysroots/qemux86 -DHAVE_CONFIG_H -I. -I./include -I./src -I./gdbus -I/home/pokybuild/
11220 yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/glib-2.0
11221 -I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/
11222 lib/glib-2.0/include -I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/
11223 tmp/sysroots/qemux86/usr/include/dbus-1.0 -I/home/pokybuild/yocto-autobuilder/yocto-slave/
11224 nightly-x86/build/build/tmp/sysroots/qemux86/usr/lib/dbus-1.0/include -I/home/pokybuild/yocto-autobuilder/
11225 yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/libnl3
11226 -DNEAR_PLUGIN_BUILTIN -DPLUGINDIR=\""/usr/lib/near/plugins"\"
11227 -DCONFIGDIR=\""/etc/neard\"" -O2 -pipe -g -feliminate-unused-debug-types -c
11228 -o tools/snep-send.o tools/snep-send.c
11229 | In file included from tools/snep-send.c:16:0:
11230 | tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory
11231 | #include &lt;near/dbus.h&gt;
11232 | ^
11233 | compilation terminated.
11234 | make[1]: *** [tools/snep-send.o] Error 1
11235 | make[1]: *** Waiting for unfinished jobs....
11236 | make: *** [all] Error 2
11237 | ERROR: oe_runmake failed
11238 </literallayout>
11239 </para>
11240 </section>
11241
11242 <section id='reproducing-the-error'>
11243 <title>Reproducing the Error</title>
11244
11245 <para>
11246 Because race conditions are intermittent, they do not
11247 manifest themselves every time you do the build.
11248 In fact, most times the build will complete without problems
11249 even though the potential race condition exists.
11250 Thus, once the error surfaces, you need a way to reproduce it.
11251 </para>
11252
11253 <para>
11254 In this example, compiling the "neard" package is causing the
11255 problem.
11256 So the first thing to do is build "neard" locally.
11257 Before you start the build, set the
11258 <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></ulink>
11259 variable in your <filename>local.conf</filename> file to
11260 a high number (e.g. "-j 20").
11261 Using a high value for <filename>PARALLEL_MAKE</filename>
11262 increases the chances of the race condition showing up:
11263 <literallayout class='monospaced'>
11264 $ bitbake neard
11265 </literallayout>
11266 </para>
11267
11268 <para>
11269 Once the local build for "neard" completes, start a
11270 <filename>devshell</filename> build:
11271 <literallayout class='monospaced'>
11272 $ bitbake neard -c devshell
11273 </literallayout>
11274 For information on how to use a
11275 <filename>devshell</filename>, see the
11276 "<link linkend='platdev-appdev-devshell'>Using a Development Shell</link>"
11277 section.
11278 </para>
11279
11280 <para>
11281 In the <filename>devshell</filename>, do the following:
11282 <literallayout class='monospaced'>
11283 $ make clean
11284 $ make tools/snep-send.o
11285 </literallayout>
11286 The <filename>devshell</filename> commands cause the failure
11287 to clearly be visible.
11288 In this case, a missing dependency exists for the "neard"
11289 Makefile target.
11290 Here is some abbreviated, sample output with the
11291 missing dependency clearly visible at the end:
11292 <literallayout class='monospaced'>
11293 i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/scott-lenovo/......
11294 .
11295 .
11296 .
11297 tools/snep-send.c
11298 In file included from tools/snep-send.c:16:0:
11299 tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory
11300 #include &lt;near/dbus.h&gt;
11301 ^
11302 compilation terminated.
11303 make: *** [tools/snep-send.o] Error 1
11304 $
11305 </literallayout>
11306 </para>
11307 </section>
11308
11309 <section id='creating-a-patch-for-the-fix'>
11310 <title>Creating a Patch for the Fix</title>
11311
11312 <para>
11313 Because there is a missing dependency for the Makefile
11314 target, you need to patch the
11315 <filename>Makefile.am</filename> file, which is generated
11316 from <filename>Makefile.in</filename>.
11317 You can use Quilt to create the patch:
11318 <literallayout class='monospaced'>
11319 $ quilt new parallelmake.patch
11320 Patch patches/parallelmake.patch is now on top
11321 $ quilt add Makefile.am
11322 File Makefile.am added to patch patches/parallelmake.patch
11323 </literallayout>
11324 For more information on using Quilt, see the
11325 "<link linkend='using-a-quilt-workflow'>Using Quilt in Your Workflow</link>"
11326 section.
11327 </para>
11328
11329 <para>
11330 At this point you need to make the edits to
11331 <filename>Makefile.am</filename> to add the missing
11332 dependency.
11333 For our example, you have to add the following line
11334 to the file:
11335 <literallayout class='monospaced'>
11336 tools/snep-send.$(OBJEXT): include/near/dbus.h
11337 </literallayout>
11338 </para>
11339
11340 <para>
11341 Once you have edited the file, use the
11342 <filename>refresh</filename> command to create the patch:
11343 <literallayout class='monospaced'>
11344 $ quilt refresh
11345 Refreshed patch patches/parallelmake.patch
11346 </literallayout>
11347 Once the patch file exists, you need to add it back to the
11348 originating recipe folder.
11349 Here is an example assuming a top-level
11350 <link linkend='source-directory'>Source Directory</link>
11351 named <filename>poky</filename>:
11352 <literallayout class='monospaced'>
11353 $ cp patches/parallelmake.patch poky/meta/recipes-connectivity/neard/neard
11354 </literallayout>
11355 The final thing you need to do to implement the fix in the
11356 build is to update the "neard" recipe (i.e.
11357 <filename>neard-0.14.bb</filename>) so that the
11358 <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>
11359 statement includes the patch file.
11360 The recipe file is in the folder above the patch.
11361 Here is what the edited <filename>SRC_URI</filename>
11362 statement would look like:
11363 <literallayout class='monospaced'>
11364 SRC_URI = "${KERNELORG_MIRROR}/linux/network/nfc/${BPN}-${PV}.tar.xz \
11365 file://neard.in \
11366 file://neard.service.in \
11367 file://parallelmake.patch \
11368 "
11369 </literallayout>
11370 </para>
11371
11372 <para>
11373 With the patch complete and moved to the correct folder and
11374 the <filename>SRC_URI</filename> statement updated, you can
11375 exit the <filename>devshell</filename>:
11376 <literallayout class='monospaced'>
11377 $ exit
11378 </literallayout>
11379 </para>
11380 </section>
11381
11382 <section id='testing-the-build'>
11383 <title>Testing the Build</title>
11384
11385 <para>
11386 With everything in place, you can get back to trying the
11387 build again locally:
11388 <literallayout class='monospaced'>
11389 $ bitbake neard
11390 </literallayout>
11391 This build should succeed.
11392 </para>
11393
11394 <para>
11395 Now you can open up a <filename>devshell</filename> again
11396 and repeat the clean and make operations as follows:
11397 <literallayout class='monospaced'>
11398 $ bitbake neard -c devshell
11399 $ make clean
11400 $ make tools/snep-send.o
11401 </literallayout>
11402 The build should work without issue.
11403 </para>
11404
11405 <para>
11406 As with all solved problems, if they originated upstream, you
11407 need to submit the fix for the recipe in OE-Core and upstream
11408 so that the problem is taken care of at its source.
11409 See the
11410 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
11411 section for more information.
11412 </para>
11413 </section>
11414 </section>
11415
Patrick Williamsc124f4f2015-09-15 14:41:29 -050011416 <section id='maintaining-open-source-license-compliance-during-your-products-lifecycle'>
11417 <title>Maintaining Open Source License Compliance During Your Product's Lifecycle</title>
11418
11419 <para>
11420 One of the concerns for a development organization using open source
11421 software is how to maintain compliance with various open source
11422 licensing during the lifecycle of the product.
11423 While this section does not provide legal advice or
11424 comprehensively cover all scenarios, it does
11425 present methods that you can use to
11426 assist you in meeting the compliance requirements during a software
11427 release.
11428 </para>
11429
11430 <para>
11431 With hundreds of different open source licenses that the Yocto
11432 Project tracks, it is difficult to know the requirements of each
11433 and every license.
11434 However, the requirements of the major FLOSS licenses can begin
11435 to be covered by
11436 assuming that three main areas of concern exist:
11437 <itemizedlist>
11438 <listitem><para>Source code must be provided.</para></listitem>
11439 <listitem><para>License text for the software must be
11440 provided.</para></listitem>
11441 <listitem><para>Compilation scripts and modifications to the
11442 source code must be provided.
11443 </para></listitem>
11444 </itemizedlist>
11445 There are other requirements beyond the scope of these
11446 three and the methods described in this section
11447 (e.g. the mechanism through which source code is distributed).
11448 </para>
11449
11450 <para>
11451 As different organizations have different methods of complying with
11452 open source licensing, this section is not meant to imply that
11453 there is only one single way to meet your compliance obligations,
11454 but rather to describe one method of achieving compliance.
11455 The remainder of this section describes methods supported to meet the
11456 previously mentioned three requirements.
11457 Once you take steps to meet these requirements,
11458 and prior to releasing images, sources, and the build system,
11459 you should audit all artifacts to ensure completeness.
11460 <note>
11461 The Yocto Project generates a license manifest during
11462 image creation that is located
11463 in <filename>${DEPLOY_DIR}/licenses/<replaceable>image_name-datestamp</replaceable></filename>
11464 to assist with any audits.
11465 </note>
11466 </para>
11467
11468 <section id='providing-the-source-code'>
11469 <title>Providing the Source Code</title>
11470
11471 <para>
11472 Compliance activities should begin before you generate the
11473 final image.
11474 The first thing you should look at is the requirement that
11475 tops the list for most compliance groups - providing
11476 the source.
11477 The Yocto Project has a few ways of meeting this
11478 requirement.
11479 </para>
11480
11481 <para>
11482 One of the easiest ways to meet this requirement is
11483 to provide the entire
11484 <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink>
11485 used by the build.
11486 This method, however, has a few issues.
11487 The most obvious is the size of the directory since it includes
11488 all sources used in the build and not just the source used in
11489 the released image.
11490 It will include toolchain source, and other artifacts, which
11491 you would not generally release.
11492 However, the more serious issue for most companies is accidental
11493 release of proprietary software.
11494 The Yocto Project provides an
11495 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-archiver'><filename>archiver</filename></ulink>
11496 class to help avoid some of these concerns.
11497 </para>
11498
11499 <para>
11500 Before you employ <filename>DL_DIR</filename> or the
Brad Bishop6e60e8b2018-02-01 10:27:11 -050011501 <filename>archiver</filename> class, you need to decide how
11502 you choose to provide source.
11503 The source <filename>archiver</filename> class can generate
11504 tarballs and SRPMs and can create them with various levels of
11505 compliance in mind.
Patrick Williamsc124f4f2015-09-15 14:41:29 -050011506 </para>
11507
11508 <para>
11509 One way of doing this (but certainly not the only way) is to
11510 release just the source as a tarball.
11511 You can do this by adding the following to the
11512 <filename>local.conf</filename> file found in the
11513 <link linkend='build-directory'>Build Directory</link>:
11514 <literallayout class='monospaced'>
11515 INHERIT += "archiver"
11516 ARCHIVER_MODE[src] = "original"
11517 </literallayout>
11518 During the creation of your image, the source from all
11519 recipes that deploy packages to the image is placed within
11520 subdirectories of
11521 <filename>DEPLOY_DIR/sources</filename> based on the
11522 <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE'><filename>LICENSE</filename></ulink>
11523 for each recipe.
11524 Releasing the entire directory enables you to comply with
11525 requirements concerning providing the unmodified source.
11526 It is important to note that the size of the directory can
11527 get large.
11528 </para>
11529
11530 <para>
11531 A way to help mitigate the size issue is to only release
11532 tarballs for licenses that require the release of
11533 source.
11534 Let us assume you are only concerned with GPL code as
Patrick Williamsc0f7c042017-02-23 20:41:17 -060011535 identified by running the following script:
Patrick Williamsc124f4f2015-09-15 14:41:29 -050011536 <literallayout class='monospaced'>
Patrick Williamsc0f7c042017-02-23 20:41:17 -060011537 # Script to archive a subset of packages matching specific license(s)
11538 # Source and license files are copied into sub folders of package folder
11539 # Must be run from build folder
11540 #!/bin/bash
11541 src_release_dir="source-release"
11542 mkdir -p $src_release_dir
11543 for a in tmp/deploy/sources/*; do
11544 for d in $a/*; do
11545 # Get package name from path
11546 p=`basename $d`
11547 p=${p%-*}
11548 p=${p%-*}
11549 # Only archive GPL packages (update *GPL* regex for your license check)
11550 numfiles=`ls tmp/deploy/licenses/$p/*GPL* 2> /dev/null | wc -l`
11551 if [ $numfiles -gt 1 ]; then
11552 echo Archiving $p
11553 mkdir -p $src_release_dir/$p/source
11554 cp $d/* $src_release_dir/$p/source 2> /dev/null
11555 mkdir -p $src_release_dir/$p/license
11556 cp tmp/deploy/licenses/$p/* $src_release_dir/$p/license 2> /dev/null
11557 fi
11558 done
11559 done </literallayout>
Patrick Williamsc124f4f2015-09-15 14:41:29 -050011560 At this point, you could create a tarball from the
11561 <filename>gpl_source_release</filename> directory and
11562 provide that to the end user.
11563 This method would be a step toward achieving compliance
11564 with section 3a of GPLv2 and with section 6 of GPLv3.
11565 </para>
11566 </section>
11567
11568 <section id='providing-license-text'>
11569 <title>Providing License Text</title>
11570
11571 <para>
11572 One requirement that is often overlooked is inclusion
11573 of license text.
11574 This requirement also needs to be dealt with prior to
11575 generating the final image.
11576 Some licenses require the license text to accompany
11577 the binary.
11578 You can achieve this by adding the following to your
11579 <filename>local.conf</filename> file:
11580 <literallayout class='monospaced'>
11581 COPY_LIC_MANIFEST = "1"
11582 COPY_LIC_DIRS = "1"
Patrick Williamsd8c66bc2016-06-20 12:57:21 -050011583 LICENSE_CREATE_PACKAGE = "1"
Patrick Williamsc124f4f2015-09-15 14:41:29 -050011584 </literallayout>
11585 Adding these statements to the configuration file ensures
11586 that the licenses collected during package generation
11587 are included on your image.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -050011588 <note>
11589 <para>Setting all three variables to "1" results in the
11590 image having two copies of the same license file.
11591 One copy resides in
11592 <filename>/usr/share/common-licenses</filename> and
11593 the other resides in
11594 <filename>/usr/share/license</filename>.</para>
11595
11596 <para>The reason for this behavior is because
11597 <ulink url='&YOCTO_DOCS_REF_URL;#var-COPY_LIC_DIRS'><filename>COPY_LIC_DIRS</filename></ulink>
11598 and
11599 <ulink url='&YOCTO_DOCS_REF_URL;#var-COPY_LIC_MANIFEST'><filename>COPY_LIC_MANIFEST</filename></ulink>
Brad Bishop6e60e8b2018-02-01 10:27:11 -050011600 add a copy of the license when the image is built but do
11601 not offer a path for adding licenses for newly installed
11602 packages to an image.
Patrick Williamsd8c66bc2016-06-20 12:57:21 -050011603 <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE_CREATE_PACKAGE'><filename>LICENSE_CREATE_PACKAGE</filename></ulink>
11604 adds a separate package and an upgrade path for adding
11605 licenses to an image.</para>
11606 </note>
11607 </para>
11608
11609 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -050011610 As the source <filename>archiver</filename> class has already
11611 archived the original
Patrick Williamsc124f4f2015-09-15 14:41:29 -050011612 unmodified source that contains the license files,
11613 you would have already met the requirements for inclusion
11614 of the license information with source as defined by the GPL
11615 and other open source licenses.
11616 </para>
11617 </section>
11618
11619 <section id='providing-compilation-scripts-and-source-code-modifications'>
11620 <title>Providing Compilation Scripts and Source Code Modifications</title>
11621
11622 <para>
Brad Bishop6e60e8b2018-02-01 10:27:11 -050011623 At this point, we have addressed all we need to
Patrick Williamsc124f4f2015-09-15 14:41:29 -050011624 prior to generating the image.
11625 The next two requirements are addressed during the final
11626 packaging of the release.
11627 </para>
11628
11629 <para>
11630 By releasing the version of the OpenEmbedded build system
11631 and the layers used during the build, you will be providing both
11632 compilation scripts and the source code modifications in one
11633 step.
11634 </para>
11635
11636 <para>
11637 If the deployment team has a
11638 <ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP layer</ulink>
11639 and a distro layer, and those those layers are used to patch,
11640 compile, package, or modify (in any way) any open source
11641 software included in your released images, you
Brad Bishop6e60e8b2018-02-01 10:27:11 -050011642 might be required to release those layers under section 3 of
Patrick Williamsc124f4f2015-09-15 14:41:29 -050011643 GPLv2 or section 1 of GPLv3.
11644 One way of doing that is with a clean
11645 checkout of the version of the Yocto Project and layers used
11646 during your build.
11647 Here is an example:
11648 <literallayout class='monospaced'>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -050011649 # We built using the &DISTRO_NAME_NO_CAP; branch of the poky repo
11650 $ git clone -b &DISTRO_NAME_NO_CAP; git://git.yoctoproject.org/poky
Patrick Williamsc124f4f2015-09-15 14:41:29 -050011651 $ cd poky
11652 # We built using the release_branch for our layers
11653 $ git clone -b release_branch git://git.mycompany.com/meta-my-bsp-layer
11654 $ git clone -b release_branch git://git.mycompany.com/meta-my-software-layer
11655 # clean up the .git repos
11656 $ find . -name ".git" -type d -exec rm -rf {} \;
11657 </literallayout>
11658 One thing a development organization might want to consider
11659 for end-user convenience is to modify
Patrick Williamsd8c66bc2016-06-20 12:57:21 -050011660 <filename>meta-poky/conf/bblayers.conf.sample</filename> to
Patrick Williamsc124f4f2015-09-15 14:41:29 -050011661 ensure that when the end user utilizes the released build
11662 system to build an image, the development organization's
11663 layers are included in the <filename>bblayers.conf</filename>
11664 file automatically:
11665 <literallayout class='monospaced'>
11666 # LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
11667 # changes incompatibly
11668 LCONF_VERSION = "6"
11669
11670 BBPATH = "${TOPDIR}"
11671 BBFILES ?= ""
11672
11673 BBLAYERS ?= " \
11674 ##OEROOT##/meta \
Patrick Williamsd8c66bc2016-06-20 12:57:21 -050011675 ##OEROOT##/meta-poky \
Patrick Williamsc124f4f2015-09-15 14:41:29 -050011676 ##OEROOT##/meta-yocto-bsp \
11677 ##OEROOT##/meta-mylayer \
11678 "
11679 </literallayout>
11680 Creating and providing an archive of the
11681 <link linkend='metadata'>Metadata</link> layers
11682 (recipes, configuration files, and so forth)
11683 enables you to meet your
11684 requirements to include the scripts to control compilation
11685 as well as any modifications to the original source.
11686 </para>
11687 </section>
11688 </section>
11689
11690 <section id='using-the-error-reporting-tool'>
11691 <title>Using the Error Reporting Tool</title>
11692
11693 <para>
11694 The error reporting tool allows you to
11695 submit errors encountered during builds to a central database.
11696 Outside of the build environment, you can use a web interface to
11697 browse errors, view statistics, and query for errors.
11698 The tool works using a client-server system where the client
11699 portion is integrated with the installed Yocto Project
11700 <link linkend='source-directory'>Source Directory</link>
11701 (e.g. <filename>poky</filename>).
11702 The server receives the information collected and saves it in a
11703 database.
11704 </para>
11705
11706 <para>
11707 A live instance of the error reporting server exists at
11708 <ulink url='http://errors.yoctoproject.org'></ulink>.
11709 This server exists so that when you want to get help with
11710 build failures, you can submit all of the information on the
11711 failure easily and then point to the URL in your bug report
11712 or send an email to the mailing list.
11713 <note>
11714 If you send error reports to this server, the reports become
11715 publicly visible.
11716 </note>
11717 </para>
11718
11719 <section id='enabling-and-using-the-tool'>
11720 <title>Enabling and Using the Tool</title>
11721
11722 <para>
11723 By default, the error reporting tool is disabled.
11724 You can enable it by inheriting the
11725 <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-report-error'><filename>report-error</filename></ulink>
11726 class by adding the following statement to the end of
11727 your <filename>local.conf</filename> file in your
11728 <link linkend='build-directory'>Build Directory</link>.
11729 <literallayout class='monospaced'>
11730 INHERIT += "report-error"
11731 </literallayout>
11732 </para>
11733
11734 <para>
11735 By default, the error reporting feature stores information in
11736 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LOG_DIR'><filename>LOG_DIR</filename></ulink><filename>}/error-report</filename>.
11737 However, you can specify a directory to use by adding the following
11738 to your <filename>local.conf</filename> file:
11739 <literallayout class='monospaced'>
11740 ERR_REPORT_DIR = "path"
11741 </literallayout>
11742 Enabling error reporting causes the build process to collect
11743 the errors and store them in a file as previously described.
11744 When the build system encounters an error, it includes a
11745 command as part of the console output.
11746 You can run the command to send the error file to the server.
11747 For example, the following command sends the errors to an
11748 upstream server:
11749 <literallayout class='monospaced'>
11750 $ send-error-report /home/brandusa/project/poky/build/tmp/log/error-report/error_report_201403141617.txt
11751 </literallayout>
11752 In the previous example, the errors are sent to a public
11753 database available at
11754 <ulink url='http://errors.yoctoproject.org'></ulink>, which is
11755 used by the entire community.
11756 If you specify a particular server, you can send the errors
11757 to a different database.
11758 Use the following command for more information on available
11759 options:
11760 <literallayout class='monospaced'>
11761 $ send-error-report --help
11762 </literallayout>
11763 </para>
11764
11765 <para>
11766 When sending the error file, you are prompted to review the
11767 data being sent as well as to provide a name and optional
11768 email address.
11769 Once you satisfy these prompts, the command returns a link
11770 from the server that corresponds to your entry in the database.
11771 For example, here is a typical link:
11772 <literallayout class='monospaced'>
11773 http://errors.yoctoproject.org/Errors/Details/9522/
11774 </literallayout>
11775 Following the link takes you to a web interface where you can
11776 browse, query the errors, and view statistics.
11777 </para>
11778 </section>
11779
11780 <section id='disabling-the-tool'>
11781 <title>Disabling the Tool</title>
11782
11783 <para>
11784 To disable the error reporting feature, simply remove or comment
11785 out the following statement from the end of your
11786 <filename>local.conf</filename> file in your
11787 <link linkend='build-directory'>Build Directory</link>.
11788 <literallayout class='monospaced'>
11789 INHERIT += "report-error"
11790 </literallayout>
11791 </para>
11792 </section>
11793
11794 <section id='setting-up-your-own-error-reporting-server'>
11795 <title>Setting Up Your Own Error Reporting Server</title>
11796
11797 <para>
11798 If you want to set up your own error reporting server, you
11799 can obtain the code from the Git repository at
11800 <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/error-report-web/'></ulink>.
11801 Instructions on how to set it up are in the README document.
11802 </para>
11803 </section>
11804 </section>
11805</chapter>
11806
11807<!--
11808vim: expandtab tw=80 ts=4
11809-->