Blame - import-layers/yocto-poky/documentation/ref-manual/usingpoky.xml - openbmc/openbmc

Patrick Williams

c124f4f

2015-09-15 14:41:29 -0500

[diff] [blame]

1

<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"

2

"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"

3

[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >

4

5

6

<title>Using the Yocto Project</title>

7

8

<para>

9

This chapter describes common usage for the Yocto Project.

10

The information is introductory in nature as other manuals in the Yocto Project

11

documentation set provide more details on how to use the Yocto Project.

</para>

<title>Running a Build</title>

16

17

<para>

18

This section provides a summary of the build process and provides information

19

for less obvious aspects of the build process.

20

For general information on how to build an image using the OpenEmbedded build

21

system, see the

22

"<ulink url='&YOCTO_DOCS_QS_URL;#qs-building-images'>Building Images</ulink>"

23

section of the Yocto Project Quick Start.

</para>

<title>Build Overview</title>

28

29

<para>

30

In the development environment you will need to build an image whenever you change hardware

31

support, add or change system libraries, or add or change services that have dependencies.

</para>

</imageobject>

<para>Building an Image</para>

</caption>

</mediaobject>

<para>

The first thing you need to do is set up the OpenEmbedded build

45

environment by sourcing an environment setup script

(i.e.

or

Here is an example:

$ source &OE_INIT_FILE; [<replaceable>build_dir</replaceable>]

</literallayout>

</para>

<para>

The <replaceable>build_dir</replaceable> argument is optional and specifies the directory the

58

OpenEmbedded build system uses for the build -

59

the <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.

60

If you do not specify a Build Directory, it defaults to a directory

61

named <filename>build</filename> in your current working directory.

62

A common practice is to use a different Build Directory for different targets.

63

For example, <filename>~/build/x86</filename> for a <filename>qemux86</filename>

64

target, and <filename>~/build/arm</filename> for a <filename>qemuarm</filename> target.

</para>

<para>

Once the build environment is set up, you can build a target using:

69

70

$ bitbake <replaceable>target</replaceable>

</literallayout>

</para>

<para>

The <replaceable>target</replaceable> is the name of the recipe you want to build.

76

Common targets are the images in <filename>meta/recipes-core/images</filename>,

77

<filename>meta/recipes-sato/images</filename>, etc. all found in the

78

<ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>.

79

Or, the target can be the name of a recipe for a specific piece of software such as

80

BusyBox.

81

For more details about the images the OpenEmbedded build system supports, see the

82

"<link linkend="ref-images">Images</link>" chapter.

</para>

<note>

Building an image without GNU General Public License Version

87

3 (GPLv3), or similarly licensed, components is supported for

88

only minimal and base images.

89

See the "<link linkend='ref-images'>Images</link>" chapter for more information.

</note>

</section>

<title>Building an Image Using GPL Components</title>

95

96

<para>

97

When building an image using GPL components, you need to maintain your original

98

settings and not switch back and forth applying different versions of the GNU

99

General Public License.

100

If you rebuild using different versions of GPL, dependency errors might occur

101

due to some components not being rebuilt.

</para>

</section>

</section>

<title>Installing and Using the Result</title>

108

109

<para>

110

Once an image has been built, it often needs to be installed.

111

The images and kernels built by the OpenEmbedded build system are placed in the

112

<ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink> in

113

<filename class="directory">tmp/deploy/images</filename>.

114

For information on how to run pre-built images such as <filename>qemux86</filename>

115

and <filename>qemuarm</filename>, see the

Patrick Williams

d8c66bc

2016-06-20 12:57:21 -0500

[diff] [blame]

116

<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-manual'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>.

Patrick Williams

c124f4f

2015-09-15 14:41:29 -0500

[diff] [blame]

117

For information about how to install these images, see the documentation for your

118

particular board or machine.

</para>

</section>

<title>Debugging Build Failures</title>

124

125

<para>

126

The exact method for debugging build failures depends on the nature of

127

the problem and on the system's area from which the bug originates.

128

Standard debugging practices such as comparison against the last

129

known working version with examination of the changes and the

130

re-application of steps to identify the one causing the problem are

131

valid for the Yocto Project just as they are for any other system.

132

Even though it is impossible to detail every possible potential failure,

133

this section provides some general tips to aid in debugging.

</para>

<para>

A useful feature for debugging is the error reporting tool.

138

Configuring the Yocto Project to use this tool causes the

139

OpenEmbedded build system to produce error reporting commands as

140

part of the console output.

141

You can enter the commands after the build completes

142

to log error information

143

into a common database, that can help you figure out what might be

144

going wrong.

145

For information on how to enable and use this feature, see the

146

"<ulink url='&YOCTO_DOCS_DEV_URL;#using-the-error-reporting-tool'>Using the Error Reporting Tool</ulink>"

147

section in the Yocto Project Development Manual.

</para>

<para>

For discussions on debugging, see the

Patrick Williams

d8c66bc

2016-06-20 12:57:21 -0500

[diff] [blame]

152

"<ulink url='&YOCTO_DOCS_DEV_URL;#platdev-gdb-remotedebug'>Debugging With the GNU Project Debugger (GDB) Remotely</ulink>" section

153

in the Yocto Project Developer's Manual

154

and the

155

"<ulink url='&YOCTO_DOCS_SDK_URL;#adt-eclipse'>Working within Eclipse</ulink>"

156

section in the Yocto Project Software Development Kit (SDK) Developer's Guide.

Patrick Williams

c124f4f

2015-09-15 14:41:29 -0500

[diff] [blame]

</para>

<note>

The remainder of this section presents many examples of the

161

<filename>bitbake</filename> command.

162

You can learn about BitBake by reading the

163

<ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual'>BitBake User Manual</ulink>.

</note>

<title>Task Failures</title>

169

170

<para>The log file for shell tasks is available in

171

<filename>${WORKDIR}/temp/log.do_<replaceable>taskname</replaceable>.pid</filename>.

172

For example, the <filename>do_compile</filename> task for the QEMU minimal image for the x86

173

machine (<filename>qemux86</filename>) might be

174

<filename>tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_compile.20830</filename>.

175

To see what

176

<ulink url='&YOCTO_DOCS_DEV_URL;#bitbake-term'>BitBake</ulink>

177

runs to generate that log, look at the corresponding

178

<filename>run.do_<replaceable>taskname</replaceable>.pid</filename> file located in the same directory.

</para>

<para>

Presently, the output from Python tasks is sent directly to the console.

</para>

</section>

<title>Running Specific Tasks</title>

188

189

<para>

190

Any given package consists of a set of tasks.

191

The standard BitBake behavior in most cases is:

192

<filename>do_fetch</filename>,

193

<filename>do_unpack</filename>,

194

<filename>do_patch</filename>, <filename>do_configure</filename>,

195

<filename>do_compile</filename>, <filename>do_install</filename>,

196

<filename>do_package</filename>,

197

<filename>do_package_write_*</filename>, and

198

<filename>do_build</filename>.

199

The default task is <filename>do_build</filename> and any tasks

200

on which it depends build first.

201

Some tasks, such as <filename>do_devshell</filename>, are not part

202

of the default build chain.

203

If you wish to run a task that is not part of the default build

204

chain, you can use the <filename>-c</filename> option in BitBake.

205

Here is an example:

206

207

$ bitbake matchbox-desktop -c devshell

</literallayout>

</para>

<para>

If you wish to rerun a task, use the <filename>-f</filename> force

213

option.

214

For example, the following sequence forces recompilation after

215

changing files in the work directory.

216

217

$ bitbake matchbox-desktop

218

.

219

.

220

<replaceable>make some changes to the source code in the work directory</replaceable>

221

.

222

.

223

$ bitbake matchbox-desktop -c compile -f

224

$ bitbake matchbox-desktop

</literallayout>

</para>

<para>

This sequence first builds and then recompiles

230

<filename>matchbox-desktop</filename>.

231

The last command reruns all tasks (basically the packaging tasks)

232

after the compile.

233

BitBake recognizes that the <filename>do_compile</filename>

234

task was rerun and therefore understands that the other tasks

235

also need to be run again.

</para>

<para>

You can view a list of tasks in a given package by running the

240

<filename>do_listtasks</filename> task as follows:

241

242

$ bitbake matchbox-desktop -c listtasks

243

</literallayout>

244

The results appear as output to the console and are also in the

245

file <filename>${WORKDIR}/temp/log.do_listtasks</filename>.

</para>

</section>

<title>Dependency Graphs</title>

251

252

<para>

253

Sometimes it can be hard to see why BitBake wants to build

254

other packages before building a given package you have specified.

255

The <filename>bitbake -g <replaceable>targetname</replaceable></filename> command

256

creates the <filename>pn-buildlist</filename>,

257

<filename>pn-depends.dot</filename>,

258

<filename>package-depends.dot</filename>, and

259

<filename>task-depends.dot</filename> files in the current

260

directory.

261

These files show what will be built and the package and task

262

dependencies, which are useful for debugging problems.

263

You can use the

264

<filename>bitbake -g -u depexp <replaceable>targetname</replaceable></filename>

265

command to display the results in a more human-readable form.

</para>

</section>

<title>General BitBake Problems</title>

271

272

<para>

273

You can see debug output from BitBake by using the <filename>-D</filename> option.

274

The debug output gives more information about what BitBake

275

is doing and the reason behind it.

276

Each <filename>-D</filename> option you use increases the logging level.

277

The most common usage is <filename>-DDD</filename>.

</para>

<para>

The output from <filename>bitbake -DDD -v</filename> <replaceable>targetname</replaceable> can reveal why

282

BitBake chose a certain version of a package or why BitBake

283

picked a certain provider.

284

This command could also help you in a situation where you think BitBake did something

unexpected.

</para>

</section>

<title>Development Host System Issues</title>

291

292

<para>

293

Sometimes issues on the host development system can cause your

294

build to fail.

295

Following are known, host-specific problems.

296

Be sure to always consult the

297

<ulink url='&YOCTO_RELEASE_NOTES;'>Release Notes</ulink>

298

for a look at all release-related issues.

299

300

<listitem><para><emphasis><filename>glibc-initial</filename> fails to build</emphasis>:

301

If your development host system has the unpatched

302

<filename>GNU Make 3.82</filename>,

303

the

304

305

task fails for <filename>glibc-initial</filename> during

306

the build.</para>

307

<para>Typically, every distribution that ships

308

<filename>GNU Make 3.82</filename> as

309

the default already has the patched version.

310

However, some distributions, such as Debian, have

311

<filename>GNU Make 3.82</filename> as an option, which

312

is unpatched.

313

You will see this error on these types of distributions.

314

Switch to <filename>GNU Make 3.81</filename> or patch

315

your <filename>make</filename> to solve the problem.

</para></listitem>

</itemizedlist>

</para>

</section>

<title>Building with No Dependencies</title>

323

<para>

324

To build a specific recipe (<filename>.bb</filename> file),

325

you can use the following command form:

326

327

$ bitbake -b <replaceable>somepath</replaceable>/<replaceable>somerecipe</replaceable>.bb

328

</literallayout>

329

This command form does not check for dependencies.

330

Consequently, you should use it

331

only when you know existing dependencies have been met.

332

<note>

333

You can also specify fragments of the filename.

334

In this case, BitBake checks for a unique match.

</note>

</para>

</section>

<title>Variables</title>

341

<para>

342

You can use the <filename>-e</filename> BitBake option to

343

display the parsing environment for a configuration.

344

The following displays the general parsing environment:

$ bitbake -e

</literallayout>

This next example shows the parsing environment for a specific

349

recipe:

350

351

$ bitbake -e <replaceable>recipename</replaceable>

</literallayout>

</para>

</section>

<title>Recipe Logging Mechanisms</title>

358

<para>

359

Best practices exist while writing recipes that both log build progress and

360

act on build conditions such as warnings and errors.

361

Both Python and Bash language bindings exist for the logging mechanism:

362

363

<listitem><para><emphasis>Python:</emphasis> For Python functions, BitBake

364

supports several loglevels: <filename>bb.fatal</filename>,

365

<filename>bb.error</filename>, <filename>bb.warn</filename>,

366

<filename>bb.note</filename>, <filename>bb.plain</filename>,

367

and <filename>bb.debug</filename>.</para></listitem>

368

<listitem><para><emphasis>Bash:</emphasis> For Bash functions, the same set

369

of loglevels exist and are accessed with a similar syntax:

370

<filename>bbfatal</filename>, <filename>bberror</filename>,

371

<filename>bbwarn</filename>, <filename>bbnote</filename>,

372

<filename>bbplain</filename>, and <filename>bbdebug</filename>.</para></listitem>

</itemizedlist>

</para>

<para>

For guidance on how logging is handled in both Python and Bash recipes, see the

378

<filename>logging.bbclass</filename> file in the

379

<filename>meta/classes</filename> folder of the

380

<ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>.

</para>

<title>Logging With Python</title>

385

<para>

386

When creating recipes using Python and inserting code that handles build logs,

387

keep in mind the goal is to have informative logs while keeping the console as

388

"silent" as possible.

389

Also, if you want status messages in the log, use the "debug" loglevel.

</para>

<para>

Following is an example written in Python.

394

The code handles logging for a function that determines the

395

number of tasks needed to be run.

396

See the

397

"<link linkend='ref-tasks-listtasks'><filename>do_listtasks</filename></link>"

398

section for additional information:

399

400

python do_listtasks() {

401

bb.debug(2, "Starting to figure out the task list")

402

if noteworthy_condition:

403

bb.note("There are 47 tasks to run")

404

bb.debug(2, "Got to point xyz")

405

if warning_trigger:

406

bb.warn("Detected warning_trigger, this might be a problem later.")

407

if recoverable_error:

408

bb.error("Hit recoverable_error, you really need to fix this!")

409

if fatal_error:

410

bb.fatal("fatal_error detected, unable to print the task list")

411

bb.plain("The tasks present are abc")

412

bb.debug(2, "Finished figuring out the tasklist")

}

</literallayout>

</para>

</section>

<title>Logging With Bash</title>

420

<para>

421

When creating recipes using Bash and inserting code that handles build

422

logs, you have the same goals - informative with minimal console output.

423

The syntax you use for recipes written in Bash is similar to that of

424

recipes written in Python described in the previous section.

</para>

<para>

Following is an example written in Bash.

429

The code logs the progress of the <filename>do_my_function</filename> function.

430

431

do_my_function() {

432

bbdebug 2 "Running do_my_function"

433

if [ exceptional_condition ]; then

434

bbnote "Hit exceptional_condition"

435

fi

436

bbdebug 2 "Got to point xyz"

437

if [ warning_trigger ]; then

438

bbwarn "Detected warning_trigger, this might cause a problem later."

439

fi

440

if [ recoverable_error ]; then

441

bberror "Hit recoverable_error, correcting"

442

fi

443

if [ fatal_error ]; then

444

bbfatal "fatal_error detected"

445

fi

446

bbdebug 2 "Completed do_my_function"

}

</literallayout>

</para>

</section>

</section>

<title>Other Tips</title>

455

456

<para>

457

Here are some other tips that you might find useful:

458

459

<listitem><para>When adding new packages, it is worth watching for

460

undesirable items making their way into compiler command lines.

461

For example, you do not want references to local system files like

462

<filename>/usr/lib/</filename> or <filename>/usr/include/</filename>.

463

</para></listitem>

464

<listitem><para>If you want to remove the <filename>psplash</filename>

465

boot splashscreen,

466

add <filename>psplash=false</filename> to the kernel command line.

467

Doing so prevents <filename>psplash</filename> from loading

468

and thus allows you to see the console.

469

It is also possible to switch out of the splashscreen by

470

switching the virtual console (e.g. Fn+Left or Fn+Right on a Zaurus).

</para></listitem>

</itemizedlist>

</para>

</section>

</section>

<title>Maintaining Build Output Quality</title>

479

480

<para>

481

Many factors can influence the quality of a build.

482

For example, if you upgrade a recipe to use a new version of an upstream software

483

package or you experiment with some new configuration options, subtle changes

484

can occur that you might not detect until later.

485

Consider the case where your recipe is using a newer version of an upstream package.

486

In this case, a new version of a piece of software might introduce an optional

487

dependency on another library, which is auto-detected.

488

If that library has already been built when the software is building,

489

the software will link to the built library and that library will be pulled

490

into your image along with the new software even if you did not want the

library.

</para>

<para>

The

class exists to help you maintain

498

the quality of your build output.

499

You can use the class to highlight unexpected and possibly unwanted

500

changes in the build output.

501

When you enable build history, it records information about the contents of

502

each package and image and then commits that information to a local Git

503

repository where you can examine the information.

</para>

<para>

The remainder of this section describes the following:

508

509

<listitem><para>How you can enable and disable

510

build history</para></listitem>

511

<listitem><para>How to understand what the build history contains

512

</para></listitem>

513

<listitem><para>How to limit the information used for build history

514

</para></listitem>

515

<listitem><para>How to examine the build history from both a

516

command-line and web interface</para></listitem>

</itemizedlist>

</para>

<title>Enabling and Disabling Build History</title>

522

523

<para>

524

Build history is disabled by default.

525

To enable it, add the following <filename>INHERIT</filename>

526

statement and set the

527

528

variable to "1" at the end of your

529

<filename>conf/local.conf</filename> file found in the

530

<ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>:

531

532

INHERIT += "buildhistory"

533

BUILDHISTORY_COMMIT = "1"

534

</literallayout>

535

Enabling build history as previously described

536

causes the build process to collect build

537

output information and commit it to a local

538

<ulink url='&YOCTO_DOCS_DEV_URL;#git'>Git</ulink> repository.

539

<note>

540

Enabling build history increases your build times slightly,

541

particularly for images, and increases the amount of disk

542

space used during the build.

</note>

</para>

<para>

You can disable build history by removing the previous statements

548

from your <filename>conf/local.conf</filename> file.

</para>

</section>

<title>Understanding What the Build History Contains</title>

554

555

<para>

556

Build history information is kept in

557

<filename>${</filename><link linkend='var-TOPDIR'><filename>TOPDIR</filename></link><filename>}/buildhistory</filename>

558

in the Build Directory as defined by the

559

560

variable.

561

The following is an example abbreviated listing:

</para>

<para>

At the top level, there is a <filename>metadata-revs</filename> file

567

that lists the revisions of the repositories for the layers enabled

568

when the build was produced.

569

The rest of the data splits into separate

570

<filename>packages</filename>, <filename>images</filename> and

571

<filename>sdk</filename> directories, the contents of which are

described below.

</para>

<title>Build History Package Information</title>

577

578

<para>

579

The history for each package contains a text file that has

580

name-value pairs with information about the package.

581

For example, <filename>buildhistory/packages/i586-poky-linux/busybox/busybox/latest</filename>

582

contains the following:

PV = 1.22.1

PR = r32

RPROVIDES =

RDEPENDS = glibc (>= 2.20) update-alternatives-opkg

588

RRECOMMENDS = busybox-syslog busybox-udhcpc update-rc.d

589

PKGSIZE = 540168

590

FILES = /usr/bin/* /usr/sbin/* /usr/lib/busybox/* /usr/lib/lib*.so.* \

591

/etc /com /var /bin/* /sbin/* /lib/*.so.* /lib/udev/rules.d \

592

/usr/lib/udev/rules.d /usr/share/busybox /usr/lib/busybox/* \

593

/usr/share/pixmaps /usr/share/applications /usr/share/idl \

594

/usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers

595

FILELIST = /bin/busybox /bin/busybox.nosuid /bin/busybox.suid /bin/sh \

596

/etc/busybox.links.nosuid /etc/busybox.links.suid

597

</literallayout>

598

Most of these name-value pairs correspond to variables used

599

to produce the package.

600

The exceptions are <filename>FILELIST</filename>, which is the

601

actual list of files in the package, and

602

<filename>PKGSIZE</filename>, which is the total size of files

603

in the package in bytes.

</para>

<para>

There is also a file corresponding to the recipe from which the

608

package came (e.g.

609

<filename>buildhistory/packages/i586-poky-linux/busybox/latest</filename>):

PV = 1.22.1

PR = r32

DEPENDS = initscripts kern-tools-native update-rc.d-native \

614

virtual/i586-poky-linux-compilerlibs virtual/i586-poky-linux-gcc \

615

virtual/libc virtual/update-alternatives

616

PACKAGES = busybox-ptest busybox-httpd busybox-udhcpd busybox-udhcpc \

617

busybox-syslog busybox-mdev busybox-hwclock busybox-dbg \

618

busybox-staticdev busybox-dev busybox-doc busybox-locale busybox

</literallayout>

</para>

<para>

Finally, for those recipes fetched from a version control

624

system (e.g., Git), a file exists that lists source revisions

625

that are specified in the recipe and lists the actual revisions

626

used during the build.

627

Listed and actual revisions might differ when

628

629

is set to

630

<filename>${<link linkend='var-AUTOREV'>AUTOREV</link>}</filename>.

631

Here is an example assuming

632

<filename>buildhistory/packages/qemux86-poky-linux/linux-yocto/latest_srcrev</filename>):

633

634

# SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"

635

SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"

636

# SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f"

637

SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f"

638

</literallayout>

639

You can use the <filename>buildhistory-collect-srcrevs</filename>

640

command with the <filename>-a</filename> option to

641

collect the stored <filename>SRCREV</filename> values

642

from build history and report them in a format suitable for

643

use in global configuration (e.g.,

644

<filename>local.conf</filename> or a distro include file) to

645

override floating <filename>AUTOREV</filename> values to a

646

fixed set of revisions.

647

Here is some example output from this command:

648

649

$ buildhistory-collect-srcrevs -a

650

# i586-poky-linux

651

SRCREV_pn-glibc = "b8079dd0d360648e4e8de48656c5c38972621072"

652

SRCREV_pn-glibc-initial = "b8079dd0d360648e4e8de48656c5c38972621072"

653

SRCREV_pn-opkg-utils = "53274f087565fd45d8452c5367997ba6a682a37a"

654

SRCREV_pn-kmod = "fd56638aed3fe147015bfa10ed4a5f7491303cb4"

655

# x86_64-linux

656

SRCREV_pn-gtk-doc-stub-native = "1dea266593edb766d6d898c79451ef193eb17cfa"

657

SRCREV_pn-dtc-native = "65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf"

658

SRCREV_pn-update-rc.d-native = "eca680ddf28d024954895f59a241a622dd575c11"

659

SRCREV_glibc_pn-cross-localedef-native = "b8079dd0d360648e4e8de48656c5c38972621072"

660

SRCREV_localedef_pn-cross-localedef-native = "c833367348d39dad7ba018990bfdaffaec8e9ed3"

661

SRCREV_pn-prelink-native = "faa069deec99bf61418d0bab831c83d7c1b797ca"

662

SRCREV_pn-opkg-utils-native = "53274f087565fd45d8452c5367997ba6a682a37a"

663

SRCREV_pn-kern-tools-native = "23345b8846fe4bd167efdf1bd8a1224b2ba9a5ff"

664

SRCREV_pn-kmod-native = "fd56638aed3fe147015bfa10ed4a5f7491303cb4"

665

# qemux86-poky-linux

666

SRCREV_machine_pn-linux-yocto = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"

667

SRCREV_meta_pn-linux-yocto = "a227f20eff056e511d504b2e490f3774ab260d6f"

668

# all-poky-linux

669

SRCREV_pn-update-rc.d = "eca680ddf28d024954895f59a241a622dd575c11"

670

</literallayout>

671

<note>

672

Here are some notes on using the

673

<filename>buildhistory-collect-srcrevs</filename> command:

674

675

<listitem><para>By default, only values where the

676

<filename>SRCREV</filename> was

677

not hardcoded (usually when <filename>AUTOREV</filename>

678

was used) are reported.

679

Use the <filename>-a</filename> option to see all

680

<filename>SRCREV</filename> values.

681

</para></listitem>

682

<listitem><para>The output statements might not have any effect

683

if overrides are applied elsewhere in the build system

684

configuration.

685

Use the <filename>-f</filename> option to add the

686

<filename>forcevariable</filename> override to each output line

687

if you need to work around this restriction.

688

</para></listitem>

689

<listitem><para>The script does apply special handling when

690

building for multiple machines.

691

However, the script does place a

692

comment before each set of values that specifies

693

which triplet to which they belong as shown above

694

(e.g., <filename>i586-poky-linux</filename>).

</para></listitem>

</itemizedlist>

</note>

</para>

</section>

<title>Build History Image Information</title>

703

704

<para>

705

The files produced for each image are as follows:

706

707

<listitem><para><filename>image-files:</filename>

708

A directory containing selected files from the root

709

filesystem.

710

The files are defined by

711

712

</para></listitem>

713

<listitem><para><filename>build-id.txt:</filename>

714

Human-readable information about the build configuration

715

and metadata source revisions.

716

This file contains the full build header as printed

717

by BitBake.</para></listitem>

718

719

Dependency graphs for the image that are

720

compatible with <filename>graphviz</filename>.

721

</para></listitem>

722

<listitem><para><filename>files-in-image.txt:</filename>

723

A list of files in the image with permissions,

724

owner, group, size, and symlink information.

725

</para></listitem>

726

<listitem><para><filename>image-info.txt:</filename>

727

A text file containing name-value pairs with information

728

about the image.

729

See the following listing example for more information.

730

</para></listitem>

731

<listitem><para><filename>installed-package-names.txt:</filename>

732

A list of installed packages by name only.</para></listitem>

733

<listitem><para><filename>installed-package-sizes.txt:</filename>

734

A list of installed packages ordered by size.

735

</para></listitem>

736

<listitem><para><filename>installed-packages.txt:</filename>

737

A list of installed packages with full package

738

filenames.</para></listitem>

739

</itemizedlist>

740

<note>

741

Installed package information is able to be gathered and

742

produced even if package management is disabled for the final

image.

</note>

</para>

<para>

Here is an example of <filename>image-info.txt</filename>:

DISTRO = poky

DISTRO_VERSION = 1.7

USER_CLASSES = buildstats image-mklibs image-prelink

753

IMAGE_CLASSES = image_types

754

IMAGE_FEATURES = debug-tweaks

755

IMAGE_LINGUAS =

756

IMAGE_INSTALL = packagegroup-core-boot run-postinsts

757

BAD_RECOMMENDATIONS =

758

NO_RECOMMENDATIONS =

759

PACKAGE_EXCLUDE =

760

ROOTFS_POSTPROCESS_COMMAND = write_package_manifest; license_create_manifest; \

761

write_image_manifest ; buildhistory_list_installed_image ; \

762

buildhistory_get_image_installed ; ssh_allow_empty_password; \

763

postinst_enable_logging; rootfs_update_timestamp ; ssh_disable_dns_lookup ;

764

IMAGE_POSTPROCESS_COMMAND = buildhistory_get_imageinfo ;

765

IMAGESIZE = 6900

766

</literallayout>

767

Other than <filename>IMAGESIZE</filename>, which is the

768

total size of the files in the image in Kbytes, the

769

name-value pairs are variables that may have influenced the

770

content of the image.

771

This information is often useful when you are trying to determine

772

why a change in the package or file listings has occurred.

</para>

</section>

<title>Using Build History to Gather Image Information Only</title>

778

779

<para>

780

As you can see, build history produces image information,

781

including dependency graphs, so you can see why something

782

was pulled into the image.

783

If you are just interested in this information and not

784

interested in collecting specific package or SDK information,

785

you can enable writing only image information without

786

any history by adding the following to your

787

<filename>conf/local.conf</filename> file found in the

788

<ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>:

789

790

INHERIT += "buildhistory"

791

BUILDHISTORY_COMMIT = "0"

792

BUILDHISTORY_FEATURES = "image"

</literallayout>

Here, you set the

variable to use the image feature only.

</para>

</section>

<title>Build History SDK Information</title>

Patrick Williams

d8c66bc

2016-06-20 12:57:21 -0500

[diff] [blame]

802

Patrick Williams

c124f4f

2015-09-15 14:41:29 -0500

[diff] [blame]

803

<para>

804

Build history collects similar information on the contents

Patrick Williams

d8c66bc

2016-06-20 12:57:21 -0500

[diff] [blame]

805

of SDKs

806

(e.g. <filename>bitbake -c populate_sdk imagename</filename>)

Patrick Williams

c124f4f

2015-09-15 14:41:29 -0500

[diff] [blame]

807

as compared to information it collects for images.

Patrick Williams

d8c66bc

2016-06-20 12:57:21 -0500

[diff] [blame]

808

Furthermore, this information differs depending on whether an

809

extensible or standard SDK is being produced.

</para>

<para>

The following list shows the files produced for SDKs:

Patrick Williams

c124f4f

2015-09-15 14:41:29 -0500

[diff] [blame]

814

815

<listitem><para><filename>files-in-sdk.txt:</filename>

816

A list of files in the SDK with permissions,

817

owner, group, size, and symlink information.

818

This list includes both the host and target parts

of the SDK.

</para></listitem>

A text file containing name-value pairs with information

823

about the SDK.

824

See the following listing example for more information.

825

</para></listitem>

Patrick Williams

d8c66bc

2016-06-20 12:57:21 -0500

[diff] [blame]

826

<listitem><para><filename>sstate-task-sizes.txt:</filename>

827

A text file containing name-value pairs with information

828

about task group sizes

829

(e.g. <filename>do_populate_sysroot</filename> tasks

830

have a total size).

831

The <filename>sstate-task-sizes.txt</filename> file

832

exists only when an extensible SDK is created.

833

</para></listitem>

834

<listitem><para><filename>sstate-package-sizes.txt:</filename>

835

A text file containing name-value pairs with information

836

for the shared-state packages and sizes in the SDK.

837

The <filename>sstate-package-sizes.txt</filename> file

838

exists only when an extensible SDK is created.

839

</para></listitem>

840

<listitem><para><filename>sdk-files:</filename>

841

A folder that contains copies of the files mentioned in

842

<filename>BUILDHISTORY_SDK_FILES</filename> if the

843

files are present in the output.

844

Additionally, the default value of

845

<filename>BUILDHISTORY_SDK_FILES</filename> is specific

846

to the extensible SDK although you can set it

847

differently if you would like to pull in specific files

848

from the standard SDK.</para>

849

<para>The default files are

850

<filename>conf/local.conf</filename>,

851

<filename>conf/bblayers.conf</filename>,

852

<filename>conf/auto.conf</filename>,

853

<filename>conf/locked-sigs.inc</filename>, and

854

<filename>conf/devtool.conf</filename>.

855

Thus, for an extensible SDK, these files get copied

856

into the <filename>sdk-files</filename> directory.

857

</para></listitem>

Patrick Williams

c124f4f

2015-09-15 14:41:29 -0500

[diff] [blame]

858

<listitem><para>The following information appears under

859

each of the <filename>host</filename>

860

and <filename>target</filename> directories

861

for the portions of the SDK that run on the host and

862

on the target, respectively:

Patrick Williams

d8c66bc

2016-06-20 12:57:21 -0500

[diff] [blame]

863

<note>

864

The following files for the most part are empty

865

when producing an extensible SDK because this

866

type of SDK is not constructed from packages as is

867

the standard SDK.

868

</note>

Patrick Williams

c124f4f

2015-09-15 14:41:29 -0500

[diff] [blame]

869

870

<listitem><para><filename>depends.dot:</filename>

871

Dependency graph for the SDK that is

872

compatible with <filename>graphviz</filename>.

873

</para></listitem>

874

<listitem><para><filename>installed-package-names.txt:</filename>

875

A list of installed packages by name only.

876

</para></listitem>

877

<listitem><para><filename>installed-package-sizes.txt:</filename>

878

A list of installed packages ordered by size.

879

</para></listitem>

880

<listitem><para><filename>installed-packages.txt:</filename>

881

A list of installed packages with full package

882

filenames.</para></listitem>

</itemizedlist>

</para></listitem>

</itemizedlist>

</para>

<para>

Here is an example of <filename>sdk-info.txt</filename>:

890

891

DISTRO = poky

892

DISTRO_VERSION = 1.3+snapshot-20130327

893

SDK_NAME = poky-glibc-i686-arm

894

SDK_VERSION = 1.3+snapshot

895

SDKMACHINE =

896

SDKIMAGE_FEATURES = dev-pkgs dbg-pkgs

897

BAD_RECOMMENDATIONS =

898

SDKSIZE = 352712

899

</literallayout>

900

Other than <filename>SDKSIZE</filename>, which is the

901

total size of the files in the SDK in Kbytes, the

902

name-value pairs are variables that might have influenced the

903

content of the SDK.

904

This information is often useful when you are trying to

905

determine why a change in the package or file listings

has occurred.

</para>

</section>

<title>Examining Build History Information</title>

912

913

<para>

914

You can examine build history output from the command line or

915

from a web interface.

</para>

<para>

To see any changes that have occurred (assuming you have

920

921

you can simply

922

use any Git command that allows you to view the history of

a repository.

Here is one method:

$ git log -p

</literallayout>

You need to realize, however, that this method does show

929

changes that are not significant (e.g. a package's size

930

changing by a few bytes).

</para>

<para>

A command-line tool called <filename>buildhistory-diff</filename>

935

does exist, though, that queries the Git repository and prints just

936

the differences that might be significant in human-readable form.

937

Here is an example:

938

939

$ ~/poky/poky/scripts/buildhistory-diff . HEAD^

940

Changes to images/qemux86_64/glibc/core-image-minimal (files-in-image.txt):

941

/etc/anotherpkg.conf was added

942

/sbin/anotherpkg was added

943

* (installed-package-names.txt):

944

* anotherpkg was added

945

Changes to images/qemux86_64/glibc/core-image-minimal (installed-package-names.txt):

946

anotherpkg was added

947

packages/qemux86_64-poky-linux/v86d: PACKAGES: added "v86d-extras"

948

* PR changed from "r0" to "r1"

949

* PV changed from "0.1.10" to "0.1.12"

950

packages/qemux86_64-poky-linux/v86d/v86d: PKGSIZE changed from 110579 to 144381 (+30%)

951

* PR changed from "r0" to "r1"

952

* PV changed from "0.1.10" to "0.1.12"

</literallayout>

</para>

<para>

To see changes to the build history using a web interface, follow

958

the instruction in the <filename>README</filename> file here.

959

<ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/buildhistory-web/'></ulink>.

</para>

<para>

Here is a sample screenshot of the interface:

</para>

</section>

</section>

</section>

<title>Speeding Up the Build</title>

972

973

<para>

974

Build time can be an issue.

975

By default, the build system uses simple controls to try and maximize

976

build efficiency.

977

In general, the default settings for all the following variables

978

result in the most efficient build times when dealing with single

979

socket systems (i.e. a single CPU).

980

If you have multiple CPUs, you might try increasing the default

981

values to gain more speed.

982

See the descriptions in the glossary for each variable for more

information:

The maximum number of threads BitBake simultaneously executes.

988

</para></listitem>

989

990

<ulink url='&YOCTO_DOCS_BB_URL;#var-BB_NUMBER_PARSE_THREADS'><filename>BB_NUMBER_PARSE_THREADS</filename>:</ulink>

991

The number of threads BitBake uses during parsing.

</para></listitem>

Extra options passed to the <filename>make</filename> command

996

during the

997

998

task in order to specify parallel compilation on the

local build host.

</para></listitem>

Extra options passed to the <filename>make</filename> command

1004

during the

1005

1006

task in order to specify parallel installation on the

local build host.

</para></listitem>

</itemizedlist>

As mentioned, these variables all scale to the number of processor

1011

cores available on the build system.

1012

For single socket systems, this auto-scaling ensures that the build

1013

system fundamentally takes advantage of potential parallel operations

1014

during the build based on the build machine's capabilities.

</para>

<para>

Following are additional factors that can affect build speed:

File system type:

The file system type that the build is being performed on can

1023

also influence performance.

1024

Using <filename>ext4</filename> is recommended as compared

1025

to <filename>ext2</filename> and <filename>ext3</filename>

1026

due to <filename>ext4</filename> improved features

such as extents.

</para></listitem>

Disabling the updating of access time using

1031

<filename>noatime</filename>:

1032

The <filename>noatime</filename> mount option prevents the

1033

build system from updating file and directory access times.

1034

</para></listitem>

1035

1036

Setting a longer commit:

1037

Using the "commit=" mount option increases the interval

1038

in seconds between disk cache writes.

1039

Changing this interval from the five second default to

1040

something longer increases the risk of data loss but decreases

1041

the need to write to the disk, thus increasing the build

performance.

</para></listitem>

Choosing the packaging backend:

1046

Of the available packaging backends, IPK is the fastest.

1047

Additionally, selecting a singular packaging backend also

helps.

</para></listitem>

Using <filename>tmpfs</filename> for

1052

1053

as a temporary file system:

1054

While this can help speed up the build, the benefits are

1055

limited due to the compiler using

1056

<filename>-pipe</filename>.

1057

The build system goes to some lengths to avoid

1058

<filename>sync()</filename> calls into the

1059

file system on the principle that if there was a significant

1060

failure, the

1061

<ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>

1062

contents could easily be rebuilt.

</para></listitem>

Inheriting the

class:

Inheriting this class has shown to speed up builds due to

1069

significantly lower amounts of data stored in the data

1070

cache as well as on disk.

1071

Inheriting this class also makes cleanup of

1072

1073

faster, at the expense of being easily able to dive into the

1074

source code.

1075

File system maintainers have recommended that the fastest way

1076

to clean up large numbers of files is to reformat partitions

1077

rather than delete files due to the linear nature of partitions.

1078

This, of course, assumes you structure the disk partitions and

1079

file systems in a way that this is practical.

1080

</para></listitem>

1081

</itemizedlist>

1082

Aside from the previous list, you should keep some trade offs in

1083

mind that can help you speed up the build:

Remove items from

that you might not need.

1089

</para></listitem>

1090

1091

Exclude debug symbols and other debug information:

1092

If you do not need these symbols and other debug information,

1093

disabling the <filename>*-dbg</filename> package generation

1094

can speed up the build.

1095

You can disable this generation by setting the

variable to "1".

</para></listitem>

Disable static library generation for recipes derived from

1101

<filename>autoconf</filename> or <filename>libtool</filename>:

1102

Following is an example showing how to disable static

1103

libraries and still provide an override to handle exceptions:

1104

1105

STATICLIBCONF = "--disable-static"

1106

STATICLIBCONF_sqlite3-native = ""

1107

EXTRA_OECONF += "${STATICLIBCONF}"

1108

</literallayout>

1109

<note><title>Notes</title>

1110

1111

1112

Some recipes need static libraries in order to work

1113

correctly (e.g. <filename>pseudo-native</filename>

1114

needs <filename>sqlite3-native</filename>).

1115

Overrides, as in the previous example, account for

1116

these kinds of exceptions.

1117

</para></listitem>

1118

1119

Some packages have packaging code that assumes the

1120

presence of the static libraries.

1121

If so, you might need to exclude them as well.

</para></listitem>

</itemizedlist>

</note>

</para></listitem>

</itemizedlist>

</para>

</section>

</chapter>

<!--

vim: expandtab tw=80 ts=4

1132

-->