Blame - import-layers/yocto-poky/documentation/ref-manual/ref-development-environment.xml - stefanberger/openbmc

"<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Product's Lifecycle</ulink>"

820

section in the Yocto Project Development Tasks Manual.

821

</para>

</section>

<title>Recipe Syntax</title>

826

827

<para>

828

Understanding recipe file syntax is important for

829

writing recipes.

830

The following list overviews the basic items that make up a

831

BitBake recipe file.

832

For more complete BitBake syntax descriptions, see the

833

"<ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata'>Syntax and Operators</ulink>"

834

chapter of the BitBake User Manual.

835

836

<listitem><para><emphasis>Variable Assignments and Manipulations:</emphasis>

837

Variable assignments allow a value to be assigned to a

838

variable.

839

The assignment can be static text or might include

840

the contents of other variables.

841

In addition to the assignment, appending and prepending

842

operations are also supported.</para>

843

<para>The following example shows some of the ways

844

you can use variables in recipes:

845

846

S = "${WORKDIR}/postfix-${PV}"

847

CFLAGS += "-DNO_ASM"

848

SRC_URI_append = " file://fixup.patch"

849

</literallayout>

850

</para></listitem>

851

<listitem><para><emphasis>Functions:</emphasis>

852

Functions provide a series of actions to be performed.

853

You usually use functions to override the default

854

implementation of a task function or to complement

855

a default function (i.e. append or prepend to an

856

existing function).

857

Standard functions use <filename>sh</filename> shell

858

syntax, although access to OpenEmbedded variables and

859

internal methods are also available.</para>

860

<para>The following is an example function from the

861

<filename>sed</filename> recipe:

do_install () {

autotools_do_install

install -d ${D}${base_bindir}

866

mv ${D}${bindir}/sed ${D}${base_bindir}/sed

rmdir ${D}${bindir}/

}

</literallayout>

It is also possible to implement new functions that

871

are called between existing tasks as long as the

872

new functions are not replacing or complementing the

873

default functions.

874

You can implement functions in Python

875

instead of shell.

876

Both of these options are not seen in the majority of

877

recipes.</para></listitem>

878

<listitem><para><emphasis>Keywords:</emphasis>

879

BitBake recipes use only a few keywords.

880

You use keywords to include common

881

functions (<filename>inherit</filename>), load parts

882

of a recipe from other files

883

(<filename>include</filename> and

884

<filename>require</filename>) and export variables

885

to the environment (<filename>export</filename>).</para>

886

<para>The following example shows the use of some of

887

these keywords:

888

889

export POSTCONF = "${STAGING_BINDIR}/postconf"

890

inherit autoconf

891

require otherfile.inc

892

</literallayout>

893

</para></listitem>

894

<listitem><para><emphasis>Comments:</emphasis>

895

Any lines that begin with the hash character

896

(<filename>#</filename>) are treated as comment lines

and are ignored:

# This is a comment

</literallayout>

</para></listitem>

</itemizedlist>

</para>

<para>

This next list summarizes the most important and most commonly

907

used parts of the recipe syntax.

908

For more information on these parts of the syntax, you can

909

reference the

910

<ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata'>Syntax and Operators</ulink>

911

chapter in the BitBake User Manual.

912

913

<listitem><para><emphasis>Line Continuation: <filename>\</filename></emphasis> -

914

Use the backward slash (<filename>\</filename>)

915

character to split a statement over multiple lines.

916

Place the slash character at the end of the line that

917

is to be continued on the next line:

918

919

VAR = "A really long \

line"

</literallayout>

<note>

You cannot have any characters including spaces

924

or tabs after the slash character.

</note>

</para></listitem>

<emphasis>Using Variables: <filename>${...}</filename></emphasis> -

929

Use the <filename>${<replaceable>VARNAME</replaceable>}</filename> syntax to

930

access the contents of a variable:

931

932

SRC_URI = "${SOURCEFORGE_MIRROR}/libpng/zlib-${PV}.tar.gz"

933

</literallayout>

934

<note>

935

It is important to understand that the value of a

936

variable expressed in this form does not get

937

substituted automatically.

938

The expansion of these expressions happens

939

on-demand later (e.g. usually when a function that

940

makes reference to the variable executes).

941

This behavior ensures that the values are most

942

appropriate for the context in which they are

943

finally used.

944

On the rare occasion that you do need the variable

945

expression to be expanded immediately, you can use

946

the <filename>:=</filename> operator instead of

947

<filename>=</filename> when you make the

948

assignment, but this is not generally needed.

949

</note>

950

</para></listitem>

951

<listitem><para><emphasis>Quote All Assignments: <filename>"<replaceable>value</replaceable>"</filename></emphasis> -

952

Use double quotes around the value in all variable

assignments.

VAR1 = "${OTHERVAR}"

VAR2 = "The version is ${PV}"

957

</literallayout>

958

</para></listitem>

959

<listitem><para><emphasis>Conditional Assignment: <filename>?=</filename></emphasis> -

960

Conditional assignment is used to assign a value to

961

a variable, but only when the variable is currently

962

unset.

963

Use the question mark followed by the equal sign

964

(<filename>?=</filename>) to make a "soft" assignment

965

used for conditional assignment.

966

Typically, "soft" assignments are used in the

967

<filename>local.conf</filename> file for variables

968

that are allowed to come through from the external

969

environment.

970

</para>

971

<para>Here is an example where

972

<filename>VAR1</filename> is set to "New value" if

973

it is currently empty.

974

However, if <filename>VAR1</filename> has already been

975

set, it remains unchanged:

VAR1 ?= "New value"

</literallayout>

In this next example, <filename>VAR1</filename>

980

is left with the value "Original value":

981

982

VAR1 = "Original value"

VAR1 ?= "New value"

</literallayout>

</para></listitem>

<listitem><para><emphasis>Appending: <filename>+=</filename></emphasis> -

987

Use the plus character followed by the equals sign

988

(<filename>+=</filename>) to append values to existing

989

variables.

990

<note>

991

This operator adds a space between the existing

992

content of the variable and the new content.

993

</note></para>

994

<para>Here is an example:

995

996

SRC_URI += "file://fix-makefile.patch"

997

</literallayout>

998

</para></listitem>

999

<listitem><para><emphasis>Prepending: <filename>=+</filename></emphasis> -

1000

Use the equals sign followed by the plus character

1001

(<filename>=+</filename>) to prepend values to existing

1002

variables.

1003

<note>

1004

This operator adds a space between the new content

1005

and the existing content of the variable.

1006

</note></para>

1007

<para>Here is an example:

VAR =+ "Starts"

</literallayout>

</para></listitem>

<listitem><para><emphasis>Appending: <filename>_append</filename></emphasis> -

1013

Use the <filename>_append</filename> operator to

1014

append values to existing variables.

1015

This operator does not add any additional space.

1016

Also, the operator is applied after all the

1017

<filename>+=</filename>, and

1018

<filename>=+</filename> operators have been applied and

1019

after all <filename>=</filename> assignments have

1020

occurred.

1021

</para>

1022

<para>The following example shows the space being

1023

explicitly added to the start to ensure the appended

1024

value is not merged with the existing value:

1025

1026

SRC_URI_append = " file://fix-makefile.patch"

1027

</literallayout>

1028

You can also use the <filename>_append</filename>

1029

operator with overrides, which results in the actions

1030

only being performed for the specified target or

1031

machine:

1032

1033

SRC_URI_append_sh4 = " file://fix-makefile.patch"

1034

</literallayout>

1035

</para></listitem>

1036

<listitem><para><emphasis>Prepending: <filename>_prepend</filename></emphasis> -

1037

Use the <filename>_prepend</filename> operator to

1038

prepend values to existing variables.

1039

This operator does not add any additional space.

1040

Also, the operator is applied after all the

1041

<filename>+=</filename>, and

1042

<filename>=+</filename> operators have been applied and

1043

after all <filename>=</filename> assignments have

1044

occurred.

1045

</para>

1046

<para>The following example shows the space being

1047

explicitly added to the end to ensure the prepended

1048

value is not merged with the existing value:

1049

1050

CFLAGS_prepend = "-I${S}/myincludes "

1051

</literallayout>

1052

You can also use the <filename>_prepend</filename>

1053

operator with overrides, which results in the actions

1054

only being performed for the specified target or

1055

machine:

1056

1057

CFLAGS_prepend_sh4 = "-I${S}/myincludes "

1058

</literallayout>

1059

</para></listitem>

1060

<listitem><para><emphasis>Overrides:</emphasis> -

1061

You can use overrides to set a value conditionally,

1062

typically based on how the recipe is being built.

1063

For example, to set the

1064

1065

variable's value to "standard/base" for any target

1066

1067

except for qemuarm where it should be set to

1068

"standard/arm-versatile-926ejs", you would do the

1069

following:

1070

1071

KBRANCH = "standard/base"

1072

KBRANCH_qemuarm = "standard/arm-versatile-926ejs"

1073

</literallayout>

1074

Overrides are also used to separate alternate values

1075

of a variable in other situations.

1076

For example, when setting variables such as

and

that are specific to individual packages produced by

1081

a recipe, you should always use an override that

1082

specifies the name of the package.

1083

</para></listitem>

1084

<listitem><para><emphasis>Indentation:</emphasis>

1085

Use spaces for indentation rather than than tabs.

1086

For shell functions, both currently work.

1087

However, it is a policy decision of the Yocto Project

1088

to use tabs in shell functions.

1089

Realize that some layers have a policy to use spaces

1090

for all indentation.

1091

</para></listitem>

1092

<listitem><para><emphasis>Using Python for Complex Operations: <filename>${@<replaceable>python_code</replaceable>}</filename></emphasis> -

1093

For more advanced processing, it is possible to use

1094

Python code during variable assignments (e.g.

1095

search and replacement on a variable).</para>

1096

<para>You indicate Python code using the

1097

<filename>${@<replaceable>python_code</replaceable>}</filename>

1098

syntax for the variable assignment:

1099

1100

SRC_URI = "ftp://ftp.info-zip.org/pub/infozip/src/zip${@d.getVar('PV',1).replace('.', '')}.tgz

1101

</literallayout>

1102

</para></listitem>

1103

<listitem><para><emphasis>Shell Function Syntax:</emphasis>

1104

Write shell functions as if you were writing a shell

1105

script when you describe a list of actions to take.

1106

You should ensure that your script works with a generic

1107

<filename>sh</filename> and that it does not require

1108

any <filename>bash</filename> or other shell-specific

1109

functionality.

1110

The same considerations apply to various system

1111

utilities (e.g. <filename>sed</filename>,

1112

<filename>grep</filename>, <filename>awk</filename>,

1113

and so forth) that you might wish to use.

1114

If in doubt, you should check with multiple

1115

implementations - including those from BusyBox.

</para></listitem>

</itemizedlist>

</para>

</section>

<title>Development Concepts</title>

1123

1124

<para>

1125

This section takes a more detailed look inside the development

1126

process.

1127

The following diagram represents development at a high level.

1128

The remainder of this chapter expands on the fundamental input, output,

1129

process, and

1130

1131

that make up development in the Yocto Project environment.

</para>

</para>

<para>

In general, development consists of several functional areas:

1140

1141

<listitem><para><emphasis>User Configuration:</emphasis>

1142

Metadata you can use to control the build process.

1143

</para></listitem>

1144

<listitem><para><emphasis>Metadata Layers:</emphasis>

1145

Various layers that provide software, machine, and

1146

distro Metadata.</para></listitem>

1147

<listitem><para><emphasis>Source Files:</emphasis>

1148

Upstream releases, local projects, and SCMs.</para></listitem>

1149

<listitem><para><emphasis>Build System:</emphasis>

1150

Processes under the control of

1151

1152

This block expands on how BitBake fetches source, applies

1153

patches, completes compilation, analyzes output for package

1154

generation, creates and tests packages, generates images, and

1155

generates cross-development tools.</para></listitem>

1156

<listitem><para><emphasis>Package Feeds:</emphasis>

1157

Directories containing output packages (RPM, DEB or IPK),

1158

which are subsequently used in the construction of an image or

1159

SDK, produced by the build system.

1160

These feeds can also be copied and shared using a web server or

1161

other means to facilitate extending or updating existing

1162

images on devices at runtime if runtime package management is

1163

enabled.</para></listitem>

1164

<listitem><para><emphasis>Images:</emphasis>

1165

Images produced by the development process.

1166

</para></listitem>

1167

<listitem><para><emphasis>Application Development SDK:</emphasis>

1168

Cross-development tools that are produced along with an image

1169

or separately with BitBake.</para></listitem>

</itemizedlist>

</para>

<title>User Configuration</title>

1175

1176

<para>

1177

User configuration helps define the build.

1178

Through user configuration, you can tell BitBake the

1179

target architecture for which you are building the image,

1180

where to store downloaded source, and other build properties.

</para>

<para>

The following figure shows an expanded representation of the

1185

"User Configuration" box of the

</para>

<para>

</para>

<para>

BitBake needs some basic configuration files in order to complete

1195

a build.

1196

These files are <filename>*.conf</filename> files.

1197

The minimally necessary ones reside as example files in the

1198

1199

For simplicity, this section refers to the Source Directory as

1200

the "Poky Directory."

</para>

<para>

When you clone the <filename>poky</filename> Git repository or you

1205

download and unpack a Yocto Project release, you can set up the

1206

Source Directory to be named anything you want.

1207

For this discussion, the cloned repository uses the default

1208

name <filename>poky</filename>.

1209

<note>

1210

The Poky repository is primarily an aggregation of existing

1211

repositories.

1212

It is not a canonical upstream source.

</note>

</para>

<para>

The <filename>meta-poky</filename> layer inside Poky contains

1218

a <filename>conf</filename> directory that has example

1219

configuration files.

1220

These example files are used as a basis for creating actual

1221

configuration files when you source the build environment

script

(i.e.

</para>

<para>

Sourcing the build environment script creates a

1229

1230

if one does not already exist.

1231

BitBake uses the Build Directory for all its work during builds.

1232

The Build Directory has a <filename>conf</filename> directory that

1233

contains default versions of your <filename>local.conf</filename>

1234

and <filename>bblayers.conf</filename> configuration files.

1235

These default configuration files are created only if versions

1236

do not already exist in the Build Directory at the time you

1237

source the build environment setup script.

</para>

<para>

Because the Poky repository is fundamentally an aggregation of

1242

existing repositories, some users might be familiar with running

1243

the <filename>&OE_INIT_FILE;</filename> script in the context

1244

of separate OpenEmbedded-Core and BitBake repositories rather than a

1245

single Poky repository.

1246

This discussion assumes the script is executed from within a cloned

1247

or unpacked version of Poky.

</para>

<para>

Depending on where the script is sourced, different sub-scripts

1252

are called to set up the Build Directory (Yocto or OpenEmbedded).

1253

Specifically, the script

1254

<filename>scripts/oe-setup-builddir</filename> inside the

1255

poky directory sets up the Build Directory and seeds the directory

1256

(if necessary) with configuration files appropriate for the

1257

Yocto Project development environment.

1258

<note>

1259

The <filename>scripts/oe-setup-builddir</filename> script

1260

uses the <filename>$TEMPLATECONF</filename> variable to

1261

determine which sample configuration files to locate.

</note>

</para>

<para>

The <filename>local.conf</filename> file provides many

1267

basic variables that define a build environment.

1268

Here is a list of a few.

1269

To see the default configurations in a <filename>local.conf</filename>

1270

file created by the build environment script, see the

1271

<filename>local.conf.sample</filename> in the

1272

<filename>meta-poky</filename> layer:

1273

1274

<listitem><para><emphasis>Parallelism Options:</emphasis>

Controlled by the

and

<ulink url='&YOCTO_DOCS_BB_URL;#var-BB_NUMBER_PARSE_THREADS'><filename>BB_NUMBER_PARSE_THREADS</filename></ulink>

1280

variables.</para></listitem>

1281

<listitem><para><emphasis>Target Machine Selection:</emphasis>

1282

Controlled by the

1283

1284

variable.</para></listitem>

1285

<listitem><para><emphasis>Download Directory:</emphasis>

1286

Controlled by the

1287

1288

variable.</para></listitem>

1289

<listitem><para><emphasis>Shared State Directory:</emphasis>

1290

Controlled by the

1291

1292

variable.</para></listitem>

1293

<listitem><para><emphasis>Build Output:</emphasis>

1294

Controlled by the

1295

1296

variable.</para></listitem>

1297

</itemizedlist>

1298

<note>

1299

Configurations set in the <filename>conf/local.conf</filename>

1300

file can also be set in the

1301

<filename>conf/site.conf</filename> and

1302

<filename>conf/auto.conf</filename> configuration files.

</note>

</para>

<para>

The <filename>bblayers.conf</filename> file tells BitBake what

1308

layers you want considered during the build.

1309

By default, the layers listed in this file include layers

1310

minimally needed by the build system.

1311

However, you must manually add any custom layers you have created.

1312

You can find more information on working with the

1313

<filename>bblayers.conf</filename> file in the

1314

"<ulink url='&YOCTO_DOCS_DEV_URL;#enabling-your-layer'>Enabling Your Layer</ulink>"

1315

section in the Yocto Project Development Tasks Manual.

</para>

<para>

The files <filename>site.conf</filename> and

1320

<filename>auto.conf</filename> are not created by the environment

1321

initialization script.

1322

If you want the <filename>site.conf</filename> file, you need to

1323

create that yourself.

1324

The <filename>auto.conf</filename> file is typically created by

an autobuilder:

You can use the <filename>conf/site.conf</filename>

1329

configuration file to configure multiple build directories.

1330

For example, suppose you had several build environments and

1331

they shared some common features.

1332

You can set these default build properties here.

1333

A good example is perhaps the packaging format to use

through the

variable.</para>

<para>One useful scenario for using the

1338

<filename>conf/site.conf</filename> file is to extend your

1339

1340

variable to include the path to a

1341

<filename>conf/site.conf</filename>.

1342

Then, when BitBake looks for Metadata using

1343

<filename>BBPATH</filename>, it finds the

1344

<filename>conf/site.conf</filename> file and applies your

1345

common configurations found in the file.

1346

To override configurations in a particular build directory,

1347

alter the similar configurations within that build

1348

directory's <filename>conf/local.conf</filename> file.

1349

</para></listitem>

1350

1351

The file is usually created and written to by

1352

an autobuilder.

1353

The settings put into the file are typically the same as

1354

you would find in the <filename>conf/local.conf</filename>

1355

or the <filename>conf/site.conf</filename> files.

</para></listitem>

</itemizedlist>

</para>

<para>

You can edit all configuration files to further define

1362

any particular build environment.

1363

This process is represented by the "User Configuration Edits"

box in the figure.

</para>

<para>

When you launch your build with the

1369

<filename>bitbake <replaceable>target</replaceable></filename>

1370

command, BitBake sorts out the configurations to ultimately

1371

define your build environment.

1372

It is important to understand that the OpenEmbedded build system

1373

reads the configuration files in a specific order:

1374

<filename>site.conf</filename>, <filename>auto.conf</filename>,

1375

and <filename>local.conf</filename>.

1376

And, the build system applies the normal assignment statement

1377

rules.

1378

Because the files are parsed in a specific order, variable

1379

assignments for the same variable could be affected.

1380

For example, if the <filename>auto.conf</filename> file and

1381

the <filename>local.conf</filename> set

1382

<replaceable>variable1</replaceable> to different values, because

1383

the build system parses <filename>local.conf</filename> after

1384

<filename>auto.conf</filename>,

1385

<replaceable>variable1</replaceable> is assigned the value from

1386

the <filename>local.conf</filename> file.

</para>

</section>

<title>Metadata, Machine Configuration, and Policy Configuration</title>

1392

1393

<para>

1394

The previous section described the user configurations that

1395

define BitBake's global behavior.

1396

This section takes a closer look at the layers the build system

1397

uses to further control the build.

1398

These layers provide Metadata for the software, machine, and

policy.

</para>

<para>

In general, three types of layer input exist:

1404

1405

<listitem><para><emphasis>Policy Configuration:</emphasis>

1406

Distribution Layers provide top-level or general

1407

policies for the image or SDK being built.

1408

For example, this layer would dictate whether BitBake

1409

produces RPM or IPK packages.</para></listitem>

1410

<listitem><para><emphasis>Machine Configuration:</emphasis>

1411

Board Support Package (BSP) layers provide machine

1412

configurations.

1413

This type of information is specific to a particular

1414

target architecture.</para></listitem>

1415

<listitem><para><emphasis>Metadata:</emphasis>

1416

Software layers contain user-supplied recipe files,

1417

patches, and append files.

</para></listitem>

</itemizedlist>

</para>

<para>

The following figure shows an expanded representation of the

1424

Metadata, Machine Configuration, and Policy Configuration input

1425

(layers) boxes of the

</para>

<para>

</para>

<para>

In general, all layers have a similar structure.

1435

They all contain a licensing file

1436

(e.g. <filename>COPYING</filename>) if the layer is to be

1437

distributed, a <filename>README</filename> file as good practice

1438

and especially if the layer is to be distributed, a

1439

configuration directory, and recipe directories.

</para>

<para>

The Yocto Project has many layers that can be used.

1444

You can see a web-interface listing of them on the

1445

<ulink url="http://git.yoctoproject.org/">Source Repositories</ulink>

1446

page.

1447

The layers are shown at the bottom categorized under

1448

"Yocto Metadata Layers."

1449

These layers are fundamentally a subset of the

1450

<ulink url="http://layers.openembedded.org/layerindex/layers/">OpenEmbedded Metadata Index</ulink>,

1451

which lists all layers provided by the OpenEmbedded community.

1452

<note>

1453

Layers exist in the Yocto Project Source Repositories that

1454

cannot be found in the OpenEmbedded Metadata Index.

1455

These layers are either deprecated or experimental in nature.

</note>

</para>

<para>

BitBake uses the <filename>conf/bblayers.conf</filename> file,

1461

which is part of the user configuration, to find what layers it

1462

should be using as part of the build.

</para>

<para>

For more information on layers, see the

1467

"<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and Creating Layers</ulink>"

1468

section in the Yocto Project Development Tasks Manual.

</para>

<title>Distro Layer</title>

1473

1474

<para>

1475

The distribution layer provides policy configurations for your

1476

distribution.

1477

Best practices dictate that you isolate these types of

1478

configurations into their own layer.

1479

Settings you provide in

1480

<filename>conf/distro/<replaceable>distro</replaceable>.conf</filename> override

1481

similar

1482

settings that BitBake finds in your

1483

<filename>conf/local.conf</filename> file in the Build

Directory.

</para>

<para>

The following list provides some explanation and references

1489

for what you typically find in the distribution layer:

1490

1491

<listitem><para><emphasis>classes:</emphasis>

1492

Class files (<filename>.bbclass</filename>) hold

1493

common functionality that can be shared among

1494

recipes in the distribution.

1495

When your recipes inherit a class, they take on the

1496

settings and functions for that class.

1497

You can read more about class files in the

1498

"<link linkend='ref-classes'>Classes</link>" section.

1499

</para></listitem>

1500

1501

This area holds configuration files for the

1502

layer (<filename>conf/layer.conf</filename>),

1503

the distribution

1504

(<filename>conf/distro/<replaceable>distro</replaceable>.conf</filename>),

1505

and any distribution-wide include files.

1506

</para></listitem>

1507

<listitem><para><emphasis>recipes-*:</emphasis>

1508

Recipes and append files that affect common

1509

functionality across the distribution.

1510

This area could include recipes and append files

1511

to add distribution-specific configuration,

1512

initialization scripts, custom image recipes,

1513

and so forth.</para></listitem>

</itemizedlist>

</para>

</section>

<title>BSP Layer</title>

1520

1521

<para>

1522

The BSP Layer provides machine configurations.

1523

Everything in this layer is specific to the machine for which

1524

you are building the image or the SDK.

1525

A common structure or form is defined for BSP layers.

1526

You can learn more about this structure in the

1527

<ulink url='&YOCTO_DOCS_BSP_URL;'>Yocto Project Board Support Package (BSP) Developer's Guide</ulink>.

1528

<note>

1529

In order for a BSP layer to be considered compliant with the

1530

Yocto Project, it must meet some structural requirements.

</note>

</para>

<para>

The BSP Layer's configuration directory contains

1536

configuration files for the machine

1537

(<filename>conf/machine/<replaceable>machine</replaceable>.conf</filename>) and,

1538

of course, the layer (<filename>conf/layer.conf</filename>).

</para>

<para>

The remainder of the layer is dedicated to specific recipes

1543

by function: <filename>recipes-bsp</filename>,

1544

<filename>recipes-core</filename>,

1545

<filename>recipes-graphics</filename>, and

1546

<filename>recipes-kernel</filename>.

1547

Metadata can exist for multiple formfactors, graphics

1548

support systems, and so forth.

1549

<note>

1550

While the figure shows several <filename>recipes-*</filename>

1551

directories, not all these directories appear in all

BSP layers.

</note>

</para>

</section>

<title>Software Layer</title>

1559

1560

<para>

1561

The software layer provides the Metadata for additional

1562

software packages used during the build.

1563

This layer does not include Metadata that is specific to the

1564

distribution or the machine, which are found in their

respective layers.

</para>

<para>

This layer contains any new recipes that your project needs

1570

in the form of recipe files.

</para>

</section>

</section>

<title>Sources</title>

1577

1578

<para>

1579

In order for the OpenEmbedded build system to create an image or

1580

any target, it must be able to access source files.

1581

The

1582

1583

represents source files using the "Upstream Project Releases",

1584

"Local Projects", and "SCMs (optional)" boxes.

1585

The figure represents mirrors, which also play a role in locating

1586

source files, with the "Source Mirror(s)" box.

</para>

<para>

The method by which source files are ultimately organized is

1591

a function of the project.

1592

For example, for released software, projects tend to use tarballs

1593

or other archived files that can capture the state of a release

1594

guaranteeing that it is statically represented.

1595

On the other hand, for a project that is more dynamic or

1596

experimental in nature, a project might keep source files in a

1597

repository controlled by a Source Control Manager (SCM) such as

1598

Git.

1599

Pulling source from a repository allows you to control

1600

the point in the repository (the revision) from which you want to

1601

build software.

1602

Finally, a combination of the two might exist, which would give the

1603

consumer a choice when deciding where to get source files.

</para>

<para>

BitBake uses the

variable to point to source files regardless of their location.

1610

Each recipe must have a <filename>SRC_URI</filename> variable

1611

that points to the source.

</para>

<para>

Another area that plays a significant role in where source files

1616

come from is pointed to by the

1617

1618

variable.

1619

This area is a cache that can hold previously downloaded source.

1620

You can also instruct the OpenEmbedded build system to create

1621

tarballs from Git repositories, which is not the default behavior,

1622

and store them in the <filename>DL_DIR</filename> by using the

variable.

</para>

<para>

Judicious use of a <filename>DL_DIR</filename> directory can

1629

save the build system a trip across the Internet when looking

1630

for files.

1631

A good method for using a download directory is to have

1632

<filename>DL_DIR</filename> point to an area outside of your

1633

Build Directory.

1634

Doing so allows you to safely delete the Build Directory

1635

if needed without fear of removing any downloaded source file.

</para>

<para>

The remainder of this section provides a deeper look into the

1640

source files and the mirrors.

1641

Here is a more detailed look at the source file area of the

base figure:

</para>

<title>Upstream Project Releases</title>

1648

1649

<para>

1650

Upstream project releases exist anywhere in the form of an

1651

archived file (e.g. tarball or zip file).

1652

These files correspond to individual recipes.

1653

For example, the figure uses specific releases each for

1654

BusyBox, Qt, and Dbus.

1655

An archive file can be for any released product that can be

1656

built using a recipe.

</para>

</section>

<title>Local Projects</title>

1662

1663

<para>

1664

Local projects are custom bits of software the user provides.

1665

These bits reside somewhere local to a project - perhaps

1666

a directory into which the user checks in items (e.g.

1667

a local directory containing a development source tree

used by the group).

</para>

<para>

The canonical method through which to include a local project

1673

is to use the

1674

1675

class to include that local project.

1676

You use either the <filename>local.conf</filename> or a

1677

recipe's append file to override or set the

1678

recipe to point to the local directory on your disk to pull

1679

in the whole source tree.

</para>

<para>

For information on how to use the

1684

<filename>externalsrc</filename> class, see the

1685

"<link linkend='ref-classes-externalsrc'><filename>externalsrc.bbclass</filename></link>"

section.

</para>

</section>

<title>Source Control Managers (Optional)</title>

1692

1693

<para>

1694

Another place the build system can get source files from is

1695

through an SCM such as Git or Subversion.

1696

In this case, a repository is cloned or checked out.

1697

The

1698

1699

task inside BitBake uses

1700

the <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>

1701

variable and the argument's prefix to determine the correct

fetcher module.

</para>

<note>

For information on how to have the OpenEmbedded build system

1707

generate tarballs for Git repositories and place them in the

directory, see the

variable.

</note>

<para>

When fetching a repository, BitBake uses the

1716

1717

variable to determine the specific revision from which to

build.

</para>

</section>

<title>Source Mirror(s)</title>

1724

1725

<para>

1726

Two kinds of mirrors exist: pre-mirrors and regular mirrors.

1727

The <link linkend='var-PREMIRRORS'><filename>PREMIRRORS</filename></link>

1728

and

1729

1730

variables point to these, respectively.

1731

BitBake checks pre-mirrors before looking upstream for any

1732

source files.

1733

Pre-mirrors are appropriate when you have a shared directory

1734

that is not a directory defined by the

1735

1736

variable.

1737

A Pre-mirror typically points to a shared directory that is

1738

local to your organization.

</para>

<para>

Regular mirrors can be any site across the Internet that is

1743

used as an alternative location for source code should the

1744

primary site not be functioning for some reason or another.

</para>

</section>

</section>

<title>Package Feeds</title>

1751

1752

<para>

1753

When the OpenEmbedded build system generates an image or an SDK,

1754

it gets the packages from a package feed area located in the

The

shows this package feeds area in the upper-right corner.

</para>

<para>

This section looks a little closer into the package feeds area used

1763

by the build system.

1764

Here is a more detailed look at the area:

</para>

<para>

Package feeds are an intermediary step in the build process.

1770

The OpenEmbedded build system provides classes to generate

1771

different package types, and you specify which classes to enable

through the

variable.

Before placing the packages into package feeds,

1776

the build process validates them with generated output quality

1777

assurance checks through the

class.

</para>

<para>

The package feed area resides in the Build Directory.

1784

The directory the build system uses to temporarily store packages

1785

is determined by a combination of variables and the particular

1786

package manager in use.

1787

See the "Package Feeds" box in the illustration and note the

1788

information to the right of that area.

1789

In particular, the following defines where package files are

1790

kept:

1791

1792

<listitem><para><link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>:

1793

Defined as <filename>tmp/deploy</filename> in the Build

1794

Directory.

1795

</para></listitem>

1796

<listitem><para><filename>DEPLOY_DIR_*</filename>:

1797

Depending on the package manager used, the package type

1798

sub-folder.

1799

Given RPM, IPK, or DEB packaging and tarball creation, the

or

variables are used, respectively.

1806

</para></listitem>

1807

<listitem><para><link linkend='var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></link>:

1808

Defines architecture-specific sub-folders.

1809

For example, packages could exist for the i586 or qemux86

architectures.

</para></listitem>

</itemizedlist>

</para>

<para>

BitBake uses the <filename>do_package_write_*</filename> tasks to

1817

generate packages and place them into the package holding area (e.g.

1818

<filename>do_package_write_ipk</filename> for IPK packages).

1819

See the

1820

"<link linkend='ref-tasks-package_write_deb'><filename>do_package_write_deb</filename></link>",

1821

"<link linkend='ref-tasks-package_write_ipk'><filename>do_package_write_ipk</filename></link>",

1822

"<link linkend='ref-tasks-package_write_rpm'><filename>do_package_write_rpm</filename></link>",

1823

and

1824

"<link linkend='ref-tasks-package_write_tar'><filename>do_package_write_tar</filename></link>"

1825

sections for additional information.

1826

As an example, consider a scenario where an IPK packaging manager

1827

is being used and package architecture support for both i586

1828

and qemux86 exist.

1829

Packages for the i586 architecture are placed in

1830

<filename>build/tmp/deploy/ipk/i586</filename>, while packages for

1831

the qemux86 architecture are placed in

1832

<filename>build/tmp/deploy/ipk/qemux86</filename>.

</para>

</section>

<title>BitBake</title>

1838

1839

<para>

1840

The OpenEmbedded build system uses

to produce images.

You can see from the

the BitBake area consists of several functional areas.

1846

This section takes a closer look at each of those areas.

</para>

<para>

Separate documentation exists for the BitBake tool.

1851

See the

1852

<ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual'>BitBake User Manual</ulink>

1853

for reference material on BitBake.

</para>

<title>Source Fetching</title>

1858

1859

<para>

1860

The first stages of building a recipe are to fetch and unpack

the source code:

</para>

<para>

The

and

tasks fetch the source files and unpack them into the work

1871

directory.

1872

<note>

1873

For every local file (e.g. <filename>file://</filename>)

1874

that is part of a recipe's

1875

1876

statement, the OpenEmbedded build system takes a checksum

1877

of the file for the recipe and inserts the checksum into

1878

the signature for the <filename>do_fetch</filename>.

1879

If any local file has been modified, the

1880

<filename>do_fetch</filename> task and all tasks that

1881

depend on it are re-executed.

1882

</note>

1883

By default, everything is accomplished in the

1884

1885

which has a defined structure.

1886

For additional general information on the Build Directory,

1887

see the

1888

"<link linkend='structure-core-build'><filename>build/</filename></link>"

section.

</para>

<para>

Unpacked source files are pointed to by the

1894

1895

Each recipe has an area in the Build Directory where the

1896

unpacked source code resides.

1897

The name of that directory for any given recipe is defined from

1898

several different variables.

1899

You can see the variables that define these directories

1900

by looking at the figure:

1901

1902

<listitem><para><link linkend='var-TMPDIR'><filename>TMPDIR</filename></link> -

1903

The base directory where the OpenEmbedded build system

1904

performs all its work during the build.

1905

</para></listitem>

1906

<listitem><para><link linkend='var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></link> -

1907

The architecture of the built package or packages.

1908

</para></listitem>

1909

<listitem><para><link linkend='var-TARGET_OS'><filename>TARGET_OS</filename></link> -

1910

The operating system of the target device.

1911

</para></listitem>

1912

<listitem><para><link linkend='var-PN'><filename>PN</filename></link> -

1913

The name of the built package.

1914

</para></listitem>

1915

<listitem><para><link linkend='var-PV'><filename>PV</filename></link> -

1916

The version of the recipe used to build the package.

1917

</para></listitem>

1918

<listitem><para><link linkend='var-PR'><filename>PR</filename></link> -

1919

The revision of the recipe used to build the package.

1920

</para></listitem>

1921

<listitem><para><link linkend='var-WORKDIR'><filename>WORKDIR</filename></link> -

1922

The location within <filename>TMPDIR</filename> where

1923

a specific package is built.

1924

</para></listitem>

1925

<listitem><para><link linkend='var-S'><filename>S</filename></link> -

1926

Contains the unpacked source files for a given recipe.

</para></listitem>

</itemizedlist>

</para>

</section>

<title>Patching</title>

1934

1935

<para>

1936

Once source code is fetched and unpacked, BitBake locates

1937

patch files and applies them to the source files:

</para>

<para>

The

task processes recipes by

1945

using the

1946

1947

variable to locate applicable patch files, which by default

1948

are <filename>*.patch</filename> or

1949

<filename>*.diff</filename> files, or any file if

1950

"apply=yes" is specified for the file in

1951

<filename>SRC_URI</filename>.

</para>

<para>

BitBake finds and applies multiple patches for a single recipe

1956

in the order in which it finds the patches.

1957

Patches are applied to the recipe's source files located in the

</para>

<para>

For more information on how the source directories are

1963

created, see the

1964

"<link linkend='source-fetching-dev-environment'>Source Fetching</link>"

section.

</para>

</section>

<title>Configuration and Compilation</title>

1971

1972

<para>

1973

After source code is patched, BitBake executes tasks that

1974

configure and compile the source code:

</para>

<para>

This step in the build process consists of three tasks:

1980

1981

1982

<emphasis><link linkend='ref-tasks-prepare_recipe_sysroot'><filename>do_prepare_recipe_sysroot</filename></link>:</emphasis>

1983

This task sets up the two sysroots in

1984

<filename>${</filename><link linkend='var-WORKDIR'><filename>WORKDIR</filename></link><filename>}</filename>

1985

(i.e. <filename>recipe-sysroot</filename> and

1986

<filename>recipe-sysroot-native</filename>) so that

1987

the sysroots contain the contents of the

1988

1989

tasks of the recipes on which the recipe

1990

containing the tasks depends.

1991

A sysroot exists for both the target and for the native

1992

binaries, which run on the host system.

1993

</para></listitem>

1994

<listitem><para><emphasis><filename>do_configure</filename>:</emphasis>

1995

This task configures the source by enabling and

1996

disabling any build-time and configuration options for

1997

the software being built.

1998

Configurations can come from the recipe itself as well

1999

as from an inherited class.

2000

Additionally, the software itself might configure itself

2001

depending on the target for which it is being built.

2002

</para>

2003

2004

<para>The configurations handled by the

2005

2006

task are specific

2007

to source code configuration for the source code

2008

being built by the recipe.</para>

2009

2010

<para>If you are using the

2011

2012

class,

2013

you can add additional configuration options by using

2014

the <link linkend='var-EXTRA_OECONF'><filename>EXTRA_OECONF</filename></link>

or

variables.

For information on how this variable works within

2019

that class, see the

2020

<filename>meta/classes/autotools.bbclass</filename> file.

2021

</para></listitem>

2022

<listitem><para><emphasis><filename>do_compile</filename>:</emphasis>

2023

Once a configuration task has been satisfied, BitBake

2024

compiles the source using the

2025

2026

task.

2027

Compilation occurs in the directory pointed to by the

2028

2029

variable.

2030

Realize that the <filename>B</filename> directory is, by

2031

default, the same as the

2032

2033

directory.</para></listitem>

2034

<listitem><para><emphasis><filename>do_install</filename>:</emphasis>

2035

Once compilation is done, BitBake executes the

2036

2037

task.

2038

This task copies files from the <filename>B</filename>

2039

directory and places them in a holding area pointed to

2040

by the

2041

2042

variable.</para></listitem>

</itemizedlist>

</para>

</section>

<title>Package Splitting</title>

2049

2050

<para>

2051

After source code is configured and compiled, the

2052

OpenEmbedded build system analyzes

2053

the results and splits the output into packages:

</para>

<para>

The

and

tasks combine to analyze

2063

the files found in the

2064

2065

and split them into subsets based on available packages and

2066

files.

2067

The analyzing process involves the following as well as other

2068

items: splitting out debugging symbols,

2069

looking at shared library dependencies between packages,

2070

and looking at package relationships.

2071

The <filename>do_packagedata</filename> task creates package

2072

metadata based on the analysis such that the

2073

OpenEmbedded build system can generate the final packages.

2074

Working, staged, and intermediate results of the analysis

2075

and package splitting process use these areas:

2076

2077

<listitem><para><link linkend='var-PKGD'><filename>PKGD</filename></link> -

2078

The destination directory for packages before they are

2079

split.

2080

</para></listitem>

2081

<listitem><para><link linkend='var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></link> -

2082

A shared, global-state directory that holds data

2083

generated during the packaging process.

2084

</para></listitem>

2085

<listitem><para><link linkend='var-PKGDESTWORK'><filename>PKGDESTWORK</filename></link> -

2086

A temporary work area used by the

2087

<filename>do_package</filename> task.

2088

</para></listitem>

2089

<listitem><para><link linkend='var-PKGDEST'><filename>PKGDEST</filename></link> -

2090

The parent directory for packages after they have

been split.

</para></listitem>

</itemizedlist>

The <link linkend='var-FILES'><filename>FILES</filename></link>

2095

variable defines the files that go into each package in

2096

2097

If you want details on how this is accomplished, you can

look at the

class.

</para>

<para>

Depending on the type of packages being created (RPM, DEB, or

2105

IPK), the <filename>do_package_write_*</filename> task

2106

creates the actual packages and places them in the

2107

Package Feed area, which is

2108

<filename>${TMPDIR}/deploy</filename>.

2109

You can see the

2110

"<link linkend='package-feeds-dev-environment'>Package Feeds</link>"

2111

section for more detail on that part of the build process.

2112

<note>

2113

Support for creating feeds directly from the

2114

<filename>deploy/*</filename> directories does not exist.

2115

Creating such feeds usually requires some kind of feed

2116

maintenance mechanism that would upload the new packages

2117

into an official package feed (e.g. the

2118

Ångström distribution).

2119

This functionality is highly distribution-specific

2120

and thus is not provided out of the box.

</note>

</para>

</section>

<title>Image Generation</title>

2127

2128

<para>

2129

Once packages are split and stored in the Package Feeds area,

2130

the OpenEmbedded build system uses BitBake to generate the

2131

root filesystem image:

</para>

<para>

The image generation process consists of several stages and

2137

depends on several tasks and variables.

2138

The

2139

2140

task creates the root filesystem (file and directory structure)

2141

for an image.

2142

This task uses several key variables to help create the list

2143

of packages to actually install:

2144

2145

<listitem><para><link linkend='var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></link>:

2146

Lists out the base set of packages to install from

2147

the Package Feeds area.</para></listitem>

2148

<listitem><para><link linkend='var-PACKAGE_EXCLUDE'><filename>PACKAGE_EXCLUDE</filename></link>:

2149

Specifies packages that should not be installed.

2150

</para></listitem>

2151

<listitem><para><link linkend='var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></link>:

2152

Specifies features to include in the image.

2153

Most of these features map to additional packages for

2154

installation.</para></listitem>

2155

<listitem><para><link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link>:

2156

Specifies the package backend to use and consequently

2157

helps determine where to locate packages within the

2158

Package Feeds area.</para></listitem>

2159

<listitem><para><link linkend='var-IMAGE_LINGUAS'><filename>IMAGE_LINGUAS</filename></link>:

2160

Determines the language(s) for which additional

2161

language support packages are installed.

2162

</para></listitem>

2163

<listitem><para><link linkend='var-PACKAGE_INSTALL'><filename>PACKAGE_INSTALL</filename></link>:

2164

The final list of packages passed to the package manager

2165

for installation into the image.

</para></listitem>

</itemizedlist>

</para>

<para>

With

pointing to the location of the filesystem under construction and

2174

the <filename>PACKAGE_INSTALL</filename> variable providing the

2175

final list of packages to install, the root file system is

created.

</para>

<para>

Package installation is under control of the package manager

2181

(e.g. dnf/rpm, opkg, or apt/dpkg) regardless of whether or

2182

not package management is enabled for the target.

2183

At the end of the process, if package management is not

2184

enabled for the target, the package manager's data files

2185

are deleted from the root filesystem.

2186

As part of the final stage of package installation, postinstall

2187

scripts that are part of the packages are run.

2188

Any scripts that fail to run

2189

on the build host are run on the target when the target system

2190

is first booted.

2191

If you are using a

2192

<ulink url='&YOCTO_DOCS_DEV_URL;#creating-a-read-only-root-filesystem'>read-only root filesystem</ulink>,

2193

all the post installation scripts must succeed during the

2194

package installation phase since the root filesystem is

read-only.

</para>

<para>

The final stages of the <filename>do_rootfs</filename> task

2200

handle post processing.

2201

Post processing includes creation of a manifest file and

optimizations.

</para>

<para>

The manifest file (<filename>.manifest</filename>) resides

2207

in the same directory as the root filesystem image.

2208

This file lists out, line-by-line, the installed packages.

2209

The manifest file is useful for the

2210

2211

class, for example, to determine whether or not to run

specific tests.

See the

variable for additional information.

</para>

<para>

Optimizing processes run across the image include

2220

<filename>mklibs</filename>, <filename>prelink</filename>,

2221

and any other post-processing commands as defined by the

2222

2223

variable.

2224

The <filename>mklibs</filename> process optimizes the size

2225

of the libraries, while the

2226

<filename>prelink</filename> process optimizes the dynamic

2227

linking of shared libraries to reduce start up time of

executables.

</para>

<para>

After the root filesystem is built, processing begins on

2233

the image through the

2234

2235

task.

2236

The build system runs any pre-processing commands as defined

by the

variable.

This variable specifies a list of functions to call before

2241

the OpenEmbedded build system creates the final image output

files.

</para>

<para>

The OpenEmbedded build system dynamically creates

2247

<filename>do_image_*</filename> tasks as needed, based

2248

on the image types specified in the

2249

2250

variable.

2251

The process turns everything into an image file or a set of

2252

image files and compresses the root filesystem image to reduce

2253

the overall size of the image.

2254

The formats used for the root filesystem depend on the

2255

<filename>IMAGE_FSTYPES</filename> variable.

</para>

<para>

As an example, a dynamically created task when creating a

2260

particular image <replaceable>type</replaceable> would take the

2261

following form:

2262

2263

do_image_<replaceable>type</replaceable>[depends]

2264

</literallayout>

2265

So, if the <replaceable>type</replaceable> as specified by the

2266

<filename>IMAGE_FSTYPES</filename> were

2267

<filename>ext4</filename>, the dynamically generated task

2268

would be as follows:

2269

2270

do_image_ext4[depends]

</literallayout>

</para>

<para>

The final task involved in image creation is the

2276

2277

task.

2278

This task completes the image by applying any image

2279

post processing as defined through the

2280

2281

variable.

2282

The variable specifies a list of functions to call once the

2283

OpenEmbedded build system has created the final image output

files.

</para>

<note>

The entire image generation process is run under Pseudo.

2289

Running under Pseudo ensures that the files in the root

2290

filesystem have correct ownership.

</note>

</section>

<title>SDK Generation</title>

2296

2297

<para>

2298

The OpenEmbedded build system uses BitBake to generate the

2299

Software Development Kit (SDK) installer script for both the

2300

standard and extensible SDKs:

</para>

<note>

For more information on the cross-development toolchain

2306

generation, see the

2307

"<link linkend='cross-development-toolchain-generation'>Cross-Development Toolchain Generation</link>"

2308

section.

2309

For information on advantages gained when building a

2310

cross-development toolchain using the

2311

2312

task, see the

2313

"<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-building-an-sdk-installer'>Building an SDK Installer</ulink>"

2314

section in the Yocto Project Application Development and the

2315

Extensible Software Development Kit (SDK) manual.

</note>

<para>

Like image generation, the SDK script process consists of

2320

several stages and depends on many variables.

2321

The <filename>do_populate_sdk</filename> and

2322

<filename>do_populate_sdk_ext</filename> tasks use these

2323

key variables to help create the list of packages to actually

2324

install.

2325

For information on the variables listed in the figure, see the

2326

"<link linkend='sdk-dev-environment'>Application Development SDK</link>"

section.

</para>

<para>

The <filename>do_populate_sdk</filename> task helps create

2332

the standard SDK and handles two parts: a target part and a

2333

host part.

2334

The target part is the part built for the target hardware and

2335

includes libraries and headers.

2336

The host part is the part of the SDK that runs on the

</para>

<para>

The <filename>do_populate_sdk_ext</filename> task helps create

2342

the extensible SDK and handles host and target parts

2343

differently than its counter part does for the standard SDK.

2344

For the extensible SDK, the task encapsulates the build system,

2345

which includes everything needed (host and target) for the SDK.

</para>

<para>

Regardless of the type of SDK being constructed, the

2350

tasks perform some cleanup after which a cross-development

2351

environment setup script and any needed configuration files

2352

are created.

2353

The final output is the Cross-development

2354

toolchain installation script (<filename>.sh</filename> file),

2355

which includes the environment setup script.

</para>

</section>

<title>Stamp Files and the Rerunning of Tasks</title>

2361

2362

<para>

2363

For each task that completes successfully, BitBake writes a

stamp file into the

directory.

The beginning of the stamp file's filename is determined by the

2368

2369

variable, and the end of the name consists of the task's name

2370

and current

2371

<ulink url='&YOCTO_DOCS_BB_URL;#checksums'>input checksum</ulink>.

2372

<note>

2373

This naming scheme assumes that

2374

<ulink url='&YOCTO_DOCS_BB_URL;#var-BB_SIGNATURE_HANDLER'><filename>BB_SIGNATURE_HANDLER</filename></ulink>

2375

is "OEBasicHash", which is almost always the case in

2376

current OpenEmbedded.

2377

</note>

2378

To determine if a task needs to be rerun, BitBake checks if a

2379

stamp file with a matching input checksum exists for the task.

2380

If such a stamp file exists, the task's output is assumed to

2381

exist and still be valid.

2382

If the file does not exist, the task is rerun.

2383

<note>

2384

<para>The stamp mechanism is more general than the shared

2385

state (sstate) cache mechanism described in the

2386

"<link linkend='setscene-tasks-and-shared-state'>Setscene Tasks and Shared State</link>"

2387

section.

2388

BitBake avoids rerunning any task that has a valid

2389

stamp file, not just tasks that can be accelerated through

2390

the sstate cache.</para>

2391

<para>However, you should realize that stamp files only

2392

serve as a marker that some work has been done and that

2393

these files do not record task output.

2394

The actual task output would usually be somewhere in

2395

2396

(e.g. in some recipe's

2397

2398

What the sstate cache mechanism adds is a way to cache task

2399

output that can then be shared between build machines.

2400

</para>

2401

</note>

2402

Since <filename>STAMPS_DIR</filename> is usually a subdirectory

2403

of <filename>TMPDIR</filename>, removing

2404

<filename>TMPDIR</filename> will also remove

2405

<filename>STAMPS_DIR</filename>, which means tasks will

2406

properly be rerun to repopulate <filename>TMPDIR</filename>.

</para>

<para>

If you want some task to always be considered "out of date",

2411

you can mark it with the

2412

<ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'><filename>nostamp</filename></ulink>

2413

varflag.

2414

If some other task depends on such a task, then that task will

2415

also always be considered out of date, which might not be what

you want.

</para>

<para>

For details on how to view information about a task's

2421

signature, see the

2422

"<link linkend='usingpoky-viewing-task-variable-dependencies'>Viewing Task Variable Dependencies</link>"

section.

</para>

</section>

<title>Setscene Tasks and Shared State</title>

2429

2430

<para>

2431

The description of tasks so far assumes that BitBake needs to

2432

build everything and there are no prebuilt objects available.

2433

BitBake does support skipping tasks if prebuilt objects are

2434

available.

2435

These objects are usually made available in the form of a

2436

shared state (sstate) cache.

2437

<note>

2438

For information on variables affecting sstate, see the

and

variables.

</note>

</para>

<para>

The idea of a setscene task (i.e

2448

<filename>do_</filename><replaceable>taskname</replaceable><filename>_setscene</filename>)

2449

is a version of the task where

2450

instead of building something, BitBake can skip to the end

2451

result and simply place a set of files into specific locations

2452

as needed.

2453

In some cases, it makes sense to have a setscene task variant

2454

(e.g. generating package files in the

2455

<filename>do_package_write_*</filename> task).

2456

In other cases, it does not make sense, (e.g. a

task or

task) since the work involved would be equal to or greater than

the underlying task.

</para>

<para>

In the OpenEmbedded build system, the common tasks that have

2466

setscene variants are <link linkend='ref-tasks-package'><filename>do_package</filename></link>,

2467

<filename>do_package_write_*</filename>,

and

Notice that these are most of the tasks whose output is an

end result.

</para>

<para>

The OpenEmbedded build system has knowledge of the relationship

2478

between these tasks and other tasks that precede them.

2479

For example, if BitBake runs

2480

<filename>do_populate_sysroot_setscene</filename> for

2481

something, there is little point in running any of the

2482

<filename>do_fetch</filename>, <filename>do_unpack</filename>,

2483

<filename>do_patch</filename>,

2484

<filename>do_configure</filename>,

2485

<filename>do_compile</filename>, and

2486

<filename>do_install</filename> tasks.

2487

However, if <filename>do_package</filename> needs to be run,

2488

BitBake would need to run those other tasks.

</para>

<para>

It becomes more complicated if everything can come from an

2493

sstate cache because some objects are simply not required at

2494

all.

2495

For example, you do not need a compiler or native tools, such

2496

as quilt, if there is nothing to compile or patch.

2497

If the <filename>do_package_write_*</filename> packages are

2498

available from sstate, BitBake does not need the

2499

<filename>do_package</filename> task data.

</para>

<para>

To handle all these complexities, BitBake runs in two phases.

2504

The first is the "setscene" stage.

2505

During this stage, BitBake first checks the sstate cache for

2506

any targets it is planning to build.

2507

BitBake does a fast check to see if the object exists rather

2508

than a complete download.

2509

If nothing exists, the second phase, which is the setscene

2510

stage, completes and the main build proceeds.

</para>

<para>

If objects are found in the sstate cache, the OpenEmbedded

2515

build system works backwards from the end targets specified

2516

by the user.

2517

For example, if an image is being built, the OpenEmbedded build

2518

system first looks for the packages needed for that image and

2519

the tools needed to construct an image.

2520

If those are available, the compiler is not needed.

2521

Thus, the compiler is not even downloaded.

2522

If something was found to be unavailable, or the download or

2523

setscene task fails, the OpenEmbedded build system then tries

2524

to install dependencies, such as the compiler, from the cache.

</para>

<para>

The availability of objects in the sstate cache is handled by

2529

the function specified by the

2530

<ulink url='&YOCTO_DOCS_BB_URL;#var-BB_HASHCHECK_FUNCTION'><filename>BB_HASHCHECK_FUNCTION</filename></ulink>

2531

variable and returns a list of the objects that are available.

2532

The function specified by the

2533

<ulink url='&YOCTO_DOCS_BB_URL;#var-BB_SETSCENE_DEPVALID'><filename>BB_SETSCENE_DEPVALID</filename></ulink>

2534

variable is the function that determines whether a given

2535

dependency needs to be followed, and whether for any given

2536

relationship the function needs to be passed.

2537

The function returns a True or False value.

</para>

</section>

</section>

<title>Images</title>

2544

2545

<para>

2546

The images produced by the OpenEmbedded build system

2547

are compressed forms of the

2548

root filesystem that are ready to boot on a target device.

2549

You can see from the

2550

2551

that BitBake output, in part, consists of images.

2552

This section is going to look more closely at this output:

</para>

<para>

For a list of example images that the Yocto Project provides,

2558

see the

2559

"<link linkend='ref-images'>Images</link>" chapter.

</para>

<para>

Images are written out to the

2564

2565

inside the <filename>tmp/deploy/images/<replaceable>machine</replaceable>/</filename>

2566

folder as shown in the figure.

2567

This folder contains any files expected to be loaded on the

target device.

The

variable points to the <filename>deploy</filename> directory,

2572

while the

2573

2574

variable points to the appropriate directory containing images for

2575

the current configuration.

2576

2577

<listitem><para><filename><replaceable>kernel-image</replaceable></filename>:

2578

A kernel binary file.

2579

The <link linkend='var-KERNEL_IMAGETYPE'><filename>KERNEL_IMAGETYPE</filename></link>

2580

variable setting determines the naming scheme for the

2581

kernel image file.

2582

Depending on that variable, the file could begin with

2583

a variety of naming strings.

2584

The <filename>deploy/images/<replaceable>machine</replaceable></filename>

2585

directory can contain multiple image files for the

2586

machine.</para></listitem>

2587

<listitem><para><filename><replaceable>root-filesystem-image</replaceable></filename>:

2588

Root filesystems for the target device (e.g.

2589

2590

files).

2591

The <link linkend='var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></link>

2592

variable setting determines the root filesystem image

2593

type.

2594

The <filename>deploy/images/<replaceable>machine</replaceable></filename>

2595

directory can contain multiple root filesystems for the

2596

machine.</para></listitem>

2597

<listitem><para><filename><replaceable>kernel-modules</replaceable></filename>:

2598

Tarballs that contain all the modules built for the kernel.

2599

Kernel module tarballs exist for legacy purposes and

2600

can be suppressed by setting the

2601

2602

variable to "0".

2603

The <filename>deploy/images/<replaceable>machine</replaceable></filename>

2604

directory can contain multiple kernel module tarballs

2605

for the machine.</para></listitem>

2606

<listitem><para><filename><replaceable>bootloaders</replaceable></filename>:

2607

Bootloaders supporting the image, if applicable to the

2608

target machine.

2609

The <filename>deploy/images/<replaceable>machine</replaceable></filename>

2610

directory can contain multiple bootloaders for the

2611

machine.</para></listitem>

2612

<listitem><para><filename><replaceable>symlinks</replaceable></filename>:

2613

The <filename>deploy/images/<replaceable>machine</replaceable></filename>

2614

folder contains

2615

a symbolic link that points to the most recently built file

2616

for each machine.

2617

These links might be useful for external scripts that

2618

need to obtain the latest version of each file.

</para></listitem>

</itemizedlist>

</para>

</section>

<title>Application Development SDK</title>

<para>

In the

the output labeled "Application Development SDK" represents an

2631

SDK.

2632

The SDK generation process differs depending on whether you build

2633

a standard SDK

2634

(e.g. <filename>bitbake -c populate_sdk</filename> <replaceable>imagename</replaceable>)

2635

or an extensible SDK

2636

(e.g. <filename>bitbake -c populate_sdk_ext</filename> <replaceable>imagename</replaceable>).

2637

This section is going to take a closer look at this output:

</para>

<para>

The specific form of this output is a self-extracting

2643

SDK installer (<filename>*.sh</filename>) that, when run,

2644

installs the SDK, which consists of a cross-development

2645

toolchain, a set of libraries and headers, and an SDK

2646

environment setup script.

2647

Running this installer essentially sets up your

2648

cross-development environment.

2649

You can think of the cross-toolchain as the "host"

2650

part because it runs on the SDK machine.

2651

You can think of the libraries and headers as the "target"

2652

part because they are built for the target hardware.

2653

The environment setup script is added so that you can initialize

2654

the environment before using the tools.

2655

</para>

2656

2657

<note><title>Notes</title>

2658

2659

2660

The Yocto Project supports several methods by which you can

2661

set up this cross-development environment.

2662

These methods include downloading pre-built SDK installers

2663

or building and installing your own SDK installer.

2664

</para></listitem>

2665

2666

For background information on cross-development toolchains

2667

in the Yocto Project development environment, see the

2668

"<link linkend='cross-development-toolchain-generation'>Cross-Development Toolchain Generation</link>"

section.

</para></listitem>

For information on setting up a cross-development

2673

environment, see the

2674

<ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>.

</para></listitem>

</itemizedlist>

</note>

<para>

Once built, the SDK installers are written out to the

2680

<filename>deploy/sdk</filename> folder inside the

2681

2682

as shown in the figure at the beginning of this section.

2683

Depending on the type of SDK, several variables exist that help

2684

configure these files.

2685

The following list shows the variables associated with a standard

2686

SDK:

2687

2688

<listitem><para><link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>:

2689

Points to the <filename>deploy</filename>

2690

directory.</para></listitem>

2691

<listitem><para><link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>:

2692

Specifies the architecture of the machine

2693

on which the cross-development tools are run to

2694

create packages for the target hardware.

2695

</para></listitem>

2696

<listitem><para><link linkend='var-SDKIMAGE_FEATURES'><filename>SDKIMAGE_FEATURES</filename></link>:

2697

Lists the features to include in the "target" part

2698

of the SDK.

2699

</para></listitem>

2700

<listitem><para><link linkend='var-TOOLCHAIN_HOST_TASK'><filename>TOOLCHAIN_HOST_TASK</filename></link>:

2701

Lists packages that make up the host

2702

part of the SDK (i.e. the part that runs on

2703

the <filename>SDKMACHINE</filename>).

2704

When you use

2705

<filename>bitbake -c populate_sdk <replaceable>imagename</replaceable></filename>

2706

to create the SDK, a set of default packages

2707

apply.

2708

This variable allows you to add more packages.

2709

</para></listitem>

2710

<listitem><para><link linkend='var-TOOLCHAIN_TARGET_TASK'><filename>TOOLCHAIN_TARGET_TASK</filename></link>:

2711

Lists packages that make up the target part

2712

of the SDK (i.e. the part built for the

2713

target hardware).

2714

</para></listitem>

2715

<listitem><para><link linkend='var-SDKPATH'><filename>SDKPATH</filename></link>:

2716

Defines the default SDK installation path offered by the

installation script.

</para></listitem>

</itemizedlist>

This next list, shows the variables associated with an extensible

2721

SDK:

2722

2723

<listitem><para><link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>:

2724

Points to the <filename>deploy</filename> directory.

2725

</para></listitem>

2726

<listitem><para><link linkend='var-SDK_EXT_TYPE'><filename>SDK_EXT_TYPE</filename></link>:

2727

Controls whether or not shared state artifacts are copied

2728

into the extensible SDK.

2729

By default, all required shared state artifacts are copied

2730

into the SDK.

2731

</para></listitem>

2732

<listitem><para><link linkend='var-SDK_INCLUDE_PKGDATA'><filename>SDK_INCLUDE_PKGDATA</filename></link>:

2733

Specifies whether or not packagedata will be included in

2734

the extensible SDK for all recipes in the "world" target.

2735

</para></listitem>

2736

<listitem><para><link linkend='var-SDK_INCLUDE_TOOLCHAIN'><filename>SDK_INCLUDE_TOOLCHAIN</filename></link>:

2737

Specifies whether or not the toolchain will be included

2738

when building the extensible SDK.

2739

</para></listitem>

2740

<listitem><para><link linkend='var-SDK_LOCAL_CONF_WHITELIST'><filename>SDK_LOCAL_CONF_WHITELIST</filename></link>:

2741

A list of variables allowed through from the build system

2742

configuration into the extensible SDK configuration.

2743

</para></listitem>

2744

<listitem><para><link linkend='var-SDK_LOCAL_CONF_BLACKLIST'><filename>SDK_LOCAL_CONF_BLACKLIST</filename></link>:

2745

A list of variables not allowed through from the build

2746

system configuration into the extensible SDK configuration.

2747

</para></listitem>

2748

<listitem><para><link linkend='var-SDK_INHERIT_BLACKLIST'><filename>SDK_INHERIT_BLACKLIST</filename></link>:

2749

A list of classes to remove from the

2750

2751

value globally within the extensible SDK configuration.

</para></listitem>

</itemizedlist>

</para>

</section>

</section>

</chapter>

<!--

vim: expandtab tw=80 ts=4

2761

-->