blob: 7440580e7cec1a06a44e69cdb67b2aa3cb45ee72 [file] [log] [blame]
Patrick Williamsc124f4f2015-09-15 14:41:29 -05001<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4
5<chapter id='toaster-manual-reference'>
6
7<title>Concepts and Reference</title>
8
9 <para>
10 In order to configure and use Toaster, you should understand some
11 concepts and have some basic command reference material available.
12 This final chapter provides conceptual information on layer sources,
13 releases, and JSON configuration files.
14 Also provided is a quick look at some useful
15 <filename>manage.py</filename> commands that are Toaster-specific.
16 Information on <filename>manage.py</filename> commands does exist
17 across the Web and the information in this manual by no means
18 attempts to provide a command comprehensive reference.
19 </para>
20
21 <section id='layer-source'>
22 <title>Layer Source</title>
23
24 <para>
25 In general, a "layer source" is a source of information about
26 existing layers.
27 In particular, we are concerned with layers that you can use
28 with the Yocto Project and Toaster.
29 This chapter describes a particular type of layer source called
30 a "layer index."
31 </para>
32
33 <para>
34 A layer index is a web application that contains information
35 about a set of custom layers.
36 A good example of an existing layer index is the
Brad Bishop316dfdd2018-06-25 12:45:53 -040037 OpenEmbedded Layer Index.
Patrick Williamsc124f4f2015-09-15 14:41:29 -050038 A public instance of this layer index exists at
39 <ulink url='http://layers.openembedded.org'></ulink>.
40 You can find the code for this layer index's web application at
41 <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/layerindex-web/'></ulink>.
42 </para>
43
44 <para>
45 When you tie a layer source into Toaster, it can query the layer
46 source through a
47 <ulink url='http://en.wikipedia.org/wiki/Representational_state_transfer'>REST</ulink>
48 API, store the information about the layers in the Toaster
49 database, and then show the information to users.
50 Users are then able to view that information and build layers
51 from Toaster itself without worrying about cloning or editing
52 the BitBake layers configuration file
53 <filename>bblayers.conf</filename>.
54 </para>
55
56 <para>
57 Tying a layer source into Toaster is convenient when you have
58 many custom layers that need to be built on a regular basis by
59 a community of developers.
60 In fact, Toaster comes pre-configured with the OpenEmbedded
61 Metadata Index.
62 <note>
63 You do not have to use a layer source to use Toaster.
64 Tying into a layer source is optional.
65 </note>
66 </para>
67
68 <section id='layer-source-using-with-toaster'>
69 <title>Setting Up and Using a Layer Source</title>
70
71 <para>
72 To use your own layer source, you need to set up the layer
73 source and then tie it into Toaster.
74 This section describes how to tie into a layer index in a manner
75 similar to the way Toaster ties into the OpenEmbedded Metadata
76 Index.
77 </para>
78
79 <section id='understanding-your-layers'>
80 <title>Understanding Your Layers</title>
81
82 <para>
83 The obvious first step for using a layer index is to have
84 several custom layers that developers build and access using
85 the Yocto Project on a regular basis.
86 This set of layers needs to exist and you need to be
87 familiar with where they reside.
88 You will need that information when you set up the
89 code for the web application that "hooks" into your set of
90 layers.
91 </para>
92
93 <para>
94 For general information on layers, see the
Brad Bishop316dfdd2018-06-25 12:45:53 -040095 "<ulink url='&YOCTO_DOCS_OM_URL;#the-yocto-project-layer-model'>The Yocto Project Layer Model</ulink>"
96 section in the Yocto Project Overview and Concepts Manual.
97 For information on how to create layers, see the
98 "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and Creating Layers</ulink>"
99 section in the Yocto Project Development Tasks Manual.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500100 </para>
101 </section>
102
103 <section id='configuring-toaster-to-hook-into-your-layer-source'>
104 <title>Configuring Toaster to Hook Into Your Layer Index</title>
105
106 <para>
107 If you want Toaster to use your layer index, you must host
108 the web application in a server to which Toaster can
109 connect.
110 You also need to give Toaster the information about your
111 layer index.
112 In other words, you have to configure Toaster to use your
113 layer index.
114 This section describes two methods by which you can
115 configure and use your layer index.
116 </para>
117
118 <para>
119 In the previous section, the code for the OpenEmbedded
120 Metadata Index (i.e.
121 <ulink url='http://layers.openembedded.org'></ulink>) was
122 referenced.
123 You can use this code, which is at
124 <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/layerindex-web/'></ulink>,
125 as a base to create your own layer index.
126 </para>
127
128 <section id='use-the-administration-interface'>
129 <title>Use the Administration Interface</title>
130
131 <para>
132 Access the administration interface through a
133 browser by entering the URL of your Toaster instance and
134 adding "<filename>/admin</filename>" to the end of the
135 URL.
136 As an example, if you are running Toaster locally, use
137 the following URL:
138 <literallayout class='monospaced'>
139 http://127.0.0.1:8000/admin
140 </literallayout>
141 </para>
142
143 <para>
144 The administration interface has a "Layer sources"
145 section that includes an "Add layer source" button.
146 Click that button and provide the required information.
147 Make sure you select "layerindex" as the layer source type.
148 </para>
149 </section>
150
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500151 <section id='use-the-fixture-feature'>
152 <title>Use the Fixture Feature</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500153
154 <para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500155 The Django fixture feature overrides the default layer
156 server when you use it to specify a custom URL. To use
157 the fixture feature, create (or edit) the file
158 <filename>bitbake/lib/toaster.orm/fixtures/custom.xml</filename>,
159 and then set the following Toaster setting to your
160 custom URL:
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500161 <literallayout class='monospaced'>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500162 &lt;?xml version="1.0" ?&gt;
163 &lt;django-objects version="1.0"&gt;
164 &lt;object model="orm.toastersetting" pk="100"&gt;
165 &lt;field name="name" type="CharField"&gt;CUSTOM_LAYERINDEX_SERVER&lt;/field&gt;
166 &lt;field name="value" type="CharField"&gt;https://layers.my_organization.org/layerindex/branch/master/layers/&lt;/field&gt;
167 &lt;/object&gt;
168 &lt;django-objects&gt;
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500169 </literallayout>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500170 When you start Toaster for the first time, or if you
171 delete the file <filename>toaster.sqlite</filename> and restart,
172 the database will populate cleanly from this layer index server.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500173 </para>
174
175 <para>
176 Once the information has been updated, verify the new layer
177 information is available by using the Toaster web interface.
178 To do that, visit the "All compatible layers" page inside a
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500179 Toaster project. The layers from your layer source should be
180 listed there.
181 </para>
182
183 <para>
184 If you change the information in your layer index server,
185 refresh the Toaster database by running the following command:
186 <literallayout class='monospaced'>
187 $ bitbake/lib/toaster/manage.py lsupdates
188 </literallayout>
189 If Toaster can reach the API URL, you should see a message
190 telling you that Toaster is updating the layer source information.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500191 </para>
192 </section>
193 </section>
194 </section>
195 </section>
196
197 <section id='toaster-releases'>
198 <title>Releases</title>
199
200 <para>
201 When you create a Toaster project using the web interface,
202 you are asked to choose a "Release."
203 In the context of Toaster, the term "Release" refers to a set of
204 layers and a BitBake version the OpenEmbedded build system uses
205 to build something.
206 As shipped, Toaster is pre-configured with releases that
207 correspond to Yocto Project release branches.
208 However, you can modify, delete, and create new releases
209 according to your needs.
210 This section provides some background information on releases.
211 </para>
212
213 <section id='toaster-releases-supported'>
214 <title>Pre-Configured Releases</title>
215
216 <para>
217 As shipped, Toaster is configured to use a specific set of
218 releases.
219 Of course, you can always configure Toaster to use any
220 release.
221 For example, you might want your project to build against a
222 specific commit of any of the "out-of-the-box" releases.
223 Or, you might want your project to build against different
224 revisions of OpenEmbedded and BitBake.
225 </para>
226
227 <para>
228 As shipped, Toaster is configured to work with the following
229 releases:
230 <itemizedlist>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500231 <listitem><para><emphasis>
232 Yocto Project &DISTRO; "&DISTRO_NAME;" or OpenEmbedded "&DISTRO_NAME;":</emphasis>
233 This release causes your Toaster projects to build
234 against the head of the &DISTRO_NAME_NO_CAP; branch at
235 <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/log/?h=rocko'></ulink>
236 or <ulink url='http://git.openembedded.org/openembedded-core/commit/?h=rocko'></ulink>.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500237 </para></listitem>
238 <listitem><para><emphasis>Yocto Project "Master" or OpenEmbedded "Master":</emphasis>
239 This release causes your Toaster Projects to
240 build against the head of the master branch, which is
241 where active development takes place, at
242 <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/log/'></ulink>
243 or
244 <ulink url='http://git.openembedded.org/openembedded-core/log/'></ulink>.
245 </para></listitem>
246 <listitem><para><emphasis>Local Yocto Project or Local OpenEmbedded:</emphasis>
247 This release causes your Toaster Projects to
248 build against the head of the <filename>poky</filename>
249 or <filename>openembedded-core</filename> clone you
250 have local to the machine running Toaster.
251 </para></listitem>
252 </itemizedlist>
253 </para>
254 </section>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500255 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500256
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500257 <section id='configuring-toaster'>
258 <title>Configuring Toaster</title>
259
260 <para>
261 In order to use Toaster, you must configure the database with the
262 default content. The following subsections describe various aspects
263 of Toaster configuration.
264 </para>
265
266 <section id='configuring-the-workflow'>
267 <title>Configuring the Workflow</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500268
269 <para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500270 The
271 <filename>bldcontrol/management/commands/checksettings.py</filename>
272 file controls workflow configuration.
273 The following steps outline the process to initially populate
274 this database.
275 <orderedlist>
276 <listitem><para>
277 The default project settings are set from
278 <filename>orm/fixtures/settings.xml</filename>.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500279 </para></listitem>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500280 <listitem><para>
281 The default project distro and layers are added
282 from <filename>orm/fixtures/poky.xml</filename> if poky
283 is installed.
284 If poky is not installed, they are added
285 from <filename>orm/fixtures/oe-core.xml</filename>.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500286 </para></listitem>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500287 <listitem><para>
288 If the <filename>orm/fixtures/custom.xml</filename> file
289 exists, then its values are added.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500290 </para></listitem>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500291 <listitem><para>
292 The layer index is then scanned and added to the database.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500293 </para></listitem>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500294 </orderedlist>
295 Once these steps complete, Toaster is set up and ready to use.
296 </para>
297 </section>
298
299 <section id='customizing-pre-set-data'>
300 <title>Customizing Pre-Set Data</title>
301
302 <para>
303 The pre-set data for Toaster is easily customizable. You can
304 create the <filename>orm/fixtures/custom.xml</filename> file
305 to customize the values that go into to the database.
306 Customization is additive,
307 and can either extend or completely replace the existing values.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500308 </para>
309
310 <para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500311 You use the <filename>orm/fixtures/custom.xml</filename> file
312 to change the default project settings for the machine, distro,
313 file images, and layers.
314 When creating a new project, you can use the file to define
315 the offered alternate project release selections.
316 For example, you can add one or more additional selections that
317 present custom layer sets or distros, and any other local or proprietary
318 content.
319 </para>
320
321 <para>
322 Additionally, you can completely disable the content from the
323 <filename>oe-core.xml</filename> and <filename>poky.xml</filename>
324 files by defining the section shown below in the
325 <filename>settings.xml</filename> file.
326 For example, this option is particularly useful if your custom
327 configuration defines fewer releases or layers than the default
328 fixture files.
329 </para>
330
331 <para>
332 The following example sets "name" to "CUSTOM_XML_ONLY" and its value
333 to "True".
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500334 <literallayout class='monospaced'>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500335 &lt;object model="orm.toastersetting" pk="99"&gt;
336 &lt;field type="CharField" name="name"&gt;CUSTOM_XML_ONLY&lt;/field&gt;
337 &lt;field type="CharField" name="value"&gt;True&lt;/field&gt;
338 &lt;/object&gt;
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500339 </literallayout>
340 </para>
341 </section>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500342
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500343 <section id='understanding-fixture-file-format'>
344 <title>Understanding Fixture File Format</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500345
346 <para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500347 The following is an overview of the file format used by the
348 <filename>oe-core.xml</filename>, <filename>poky.xml</filename>,
349 and <filename>custom.xml</filename> files.
350 </para>
351
352 <para>
353 The following subsections describe each of the sections in the
354 fixture files, and outline an example section of the XML code.
355 you can use to help understand this information and create a local
356 <filename>custom.xml</filename> file.
357 </para>
358
359 <section id='defining-the-default-distro-and-other-values'>
360 <title>Defining the Default Distro and Other Values</title>
361
362 <para>
363 This section defines the default distro value for new projects.
364 By default, it reserves the first Toaster Setting record "1".
365 The following demonstrates how to set the project default value
366 for
367 <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink>:
368 <literallayout class='monospaced'>
369 &lt;!-- Set the project default value for DISTRO --&gt;
370 &lt;object model="orm.toastersetting" pk="1"&gt;
371 &lt;field type="CharField" name="name"&gt;DEFCONF_DISTRO&lt;/field&gt;
372 &lt;field type="CharField" name="value"&gt;poky&lt;/field&gt;
373 &lt;/object&gt;
374 </literallayout>
375 You can override other default project values by adding
376 additional Toaster Setting sections such as any of the
377 settings coming from the <filename>settings.xml</filename>
378 file.
379 Also, you can add custom values that are included in the
380 BitBake environment.
381 The "pk" values must be unique.
382 By convention, values that set default project values have a
383 "DEFCONF" prefix.
384 </para>
385 </section>
386
387 <section id='defining-bitbake-version'>
388 <title>Defining BitBake Version</title>
389
390 <para>
391 The following defines which version of BitBake is used
392 for the following release selection:
393 <literallayout class='monospaced'>
394 &lt;!-- Bitbake versions which correspond to the metadata release --&gt;
395 &lt;object model="orm.bitbakeversion" pk="1"&gt;
396 &lt;field type="CharField" name="name"&gt;rocko&lt;/field&gt;
397 &lt;field type="CharField" name="giturl"&gt;git://git.yoctoproject.org/poky&lt;/field&gt;
398 &lt;field type="CharField" name="branch"&gt;rocko&lt;/field&gt;
399 &lt;field type="CharField" name="dirpath"&gt;bitbake&lt;/field&gt;
400 &lt;/object&gt;
401 </literallayout>
402 </para>
403 </section>
404
405 <section id='defining-releases'>
406 <title>Defining Release</title>
407
408 <para>
409 The following defines the releases when you create a new
410 project.
411 <literallayout class='monospaced'>
412 &lt;!-- Releases available --&gt;
413 &lt;object model="orm.release" pk="1"&gt;
414 &lt;field type="CharField" name="name"&gt;rocko&lt;/field&gt;
415 &lt;field type="CharField" name="description"&gt;Yocto Project 2.4 "Rocko"&lt;/field&gt;
416 &lt;field rel="ManyToOneRel" to="orm.bitbakeversion" name="bitbake_version"&gt;1&lt;/field&gt;
417 &lt;field type="CharField" name="branch_name"&gt;rocko&lt;/field&gt;
418 &lt;field type="TextField" name="helptext"&gt;Toaster will run your builds using the tip of the &lt;a href="http://git.yoctoproject.org/cgit/cgit.cgi/poky/log/?h=rocko"&gt;Yocto Project Rocko branch&lt;/a&gt;.&lt;/field&gt;
419 &lt;/object&gt;
420 </literallayout>
421 The "pk" value must match the above respective BitBake
422 version record.
423 </para>
424 </section>
425
426 <section id='defining-the-release-default-layer-names'>
427 <title>Defining the Release Default Layer Names</title>
428
429 <para>
430 The following defines the default layers for each release:
431 <literallayout class='monospaced'>
432 &lt;!-- Default project layers for each release --&gt;
433 &lt;object model="orm.releasedefaultlayer" pk="1"&gt;
434 &lt;field rel="ManyToOneRel" to="orm.release" name="release"&gt;1&lt;/field&gt;
435 &lt;field type="CharField" name="layer_name"&gt;openembedded-core&lt;/field&gt;
436 &lt;/object&gt;
437 </literallayout>
438 The 'pk' values in the example above should start at "1" and increment
439 uniquely.
440 You can use the same layer name in multiple releases.
441 </para>
442 </section>
443
444 <section id='defining-layer-definitions'>
445 <title>Defining Layer Definitions</title>
446
447 <para>
448 Layer definitions are the most complex.
449 The following defines each of the layers, and then defines the exact layer
450 version of the layer used for each respective release.
451 You must have one <filename>orm.layer</filename>
452 entry for each layer.
453 Then, with each entry you need a set of
454 <filename>orm.layer_version</filename> entries that connects
455 the layer with each release that includes the layer.
456 In general all releases include the layer.
457 <literallayout class='monospaced'>
458 &lt;object model="orm.layer" pk="1"&gt;
459 &lt;field type="CharField" name="name"&gt;openembedded-core&lt;/field&gt;
460 &lt;field type="CharField" name="layer_index_url"&gt;&lt;/field&gt;
461 &lt;field type="CharField" name="vcs_url"&gt;git://git.yoctoproject.org/poky&lt;/field&gt;
462 &lt;field type="CharField" name="vcs_web_url"&gt;http://git.yoctoproject.org/cgit/cgit.cgi/poky&lt;/field&gt;
463 &lt;field type="CharField" name="vcs_web_tree_base_url"&gt;http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/%path%?h=%branch%&lt;/field&gt;
464 &lt;field type="CharField" name="vcs_web_file_base_url"&gt;http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/%path%?h=%branch%&lt;/field&gt;
465 &lt;/object&gt;
466 &lt;object model="orm.layer_version" pk="1"&gt;
467 &lt;field rel="ManyToOneRel" to="orm.layer" name="layer"&gt;1&lt;/field&gt;
468 &lt;field type="IntegerField" name="layer_source"&gt;0&lt;/field&gt;
469 &lt;field rel="ManyToOneRel" to="orm.release" name="release"&gt;1&lt;/field&gt;
470 &lt;field type="CharField" name="branch"&gt;rocko&lt;/field&gt;
471 &lt;field type="CharField" name="dirpath"&gt;meta&lt;/field&gt;
472 &lt;/object&gt;
473 &lt;object model="orm.layer_version" pk="2"&gt;
474 &lt;field rel="ManyToOneRel" to="orm.layer" name="layer"&gt;1&lt;/field&gt;
475 &lt;field type="IntegerField" name="layer_source"&gt;0&lt;/field&gt;
476 &lt;field rel="ManyToOneRel" to="orm.release" name="release"&gt;2&lt;/field&gt;
477 &lt;field type="CharField" name="branch"&gt;HEAD&lt;/field&gt;
478 &lt;field type="CharField" name="commit"&gt;HEAD&lt;/field&gt;
479 &lt;field type="CharField" name="dirpath"&gt;meta&lt;/field&gt;
480 &lt;/object&gt;
481 &lt;object model="orm.layer_version" pk="3"&gt;
482 &lt;field rel="ManyToOneRel" to="orm.layer" name="layer"&gt;1&lt;/field&gt;
483 &lt;field type="IntegerField" name="layer_source"&gt;0&lt;/field&gt;
484 &lt;field rel="ManyToOneRel" to="orm.release" name="release"&gt;3&lt;/field&gt;
485
486 &lt;field type="CharField" name="branch"&gt;master&lt;/field&gt;
487 &lt;field type="CharField" name="dirpath"&gt;meta&lt;/field&gt;
488 &lt;/object&gt;
489 </literallayout>
490 The layer "pk" values above must be unique, and typically start at "1".
491 The layer version "pk" values must also be unique across all layers,
492 and typically start at "1".
493 </para>
494 </section>
495 </section>
496 </section>
497
498 <section id='remote-toaster-monitoring'>
499 <title>Remote Toaster Monitoring</title>
500
501 <para>
502 Toaster has an API that allows remote management applications to
503 directly query the state of the Toaster server and its builds
504 in a machine-to-machine manner.
505 This API uses the
506 <ulink url='http://en.wikipedia.org/wiki/Representational_state_transfer'>REST</ulink>
507 interface and the transfer of JSON files.
508 For example, you might
509 monitor a build inside a container through well supported
510 known HTTP ports in order to easily access a Toaster server
511 inside the container.
512 In this example, when you use this direct JSON API, you avoid
513 having web page parsing against the display the user sees.
514 </para>
515
516 <section id='checking-health'>
517 <title>Checking Health</title>
518
519 <para>
520 Before you use remote Toaster monitoring, you should do
521 a health check.
522 To do this, ping the Toaster server using the following call
523 to see if it is still alive:
524 <literallayout class='monospaced'>
525 http://<replaceable>host</replaceable>:<replaceable>port</replaceable>/health
526 </literallayout>
527 Be sure to provide values for <replaceable>host</replaceable>
528 and <replaceable>port</replaceable>.
529 If the server is alive, you will get the response HTML:
530 <literallayout class='monospaced'>
531 &lt;!DOCTYPE html&gt;
532 &lt;html lang="en"&gt;
533 &lt;head&gt;&lt;title&gt;Toaster Health&lt;/title&gt;&lt;/head&gt;
534 &lt;body&gt;Ok&lt;/body&gt;
535 &lt;/html&gt;
536 </literallayout>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500537 </para>
538 </section>
539
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500540 <section id='determining-status-of-builds-in-progress'>
541 <title>Determining Status of Builds in Progress</title>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500542
543 <para>
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500544 Sometimes it is useful to determine the status of a build
545 in progress.
546 To get the status of pending builds, use the following call:
547 <literallayout class='monospaced'>
548 http://<replaceable>host</replaceable>:<replaceable>port</replaceable>/toastergui/api/building
549 </literallayout>
550 Be sure to provide values for <replaceable>host</replaceable>
551 and <replaceable>port</replaceable>.
552 The output is a JSON file that itemizes all builds in
553 progress.
554 This file includes the time in seconds since each
555 respective build started as well as the progress of the
556 cloning, parsing, and task execution.
557 The following is sample output for a build in progress:
558 <literallayout class='monospaced'>
559 {"count": 1,
560 "building": [
561 {"machine": "beaglebone",
562 "seconds": "463.869",
563 "task": "927:2384",
564 "distro": "poky",
565 "clone": "1:1",
566 "id": 2,
567 "start": "2017-09-22T09:31:44.887Z",
568 "name": "20170922093200",
569 "parse": "818:818",
570 "project": "my_rocko",
571 "target": "core-image-minimal"
572 }]
573 }
574 </literallayout>
575 The JSON data for this query is returned in a single line.
576 In the previous example the line has been artificially split for readability.
577 </para>
578 </section>
579
580 <section id='checking-status-of-builds-completed'>
581 <title>Checking Status of Builds Completed</title>
582
583 <para>
584 Once a build is completed, you get the status when you use
585 the following call:
586 <literallayout class='monospaced'>
587 http://<replaceable>host</replaceable>:<replaceable>port</replaceable>/toastergui/api/builds
588 </literallayout>
589 Be sure to provide values for <replaceable>host</replaceable>
590 and <replaceable>port</replaceable>.
591 The output is a JSON file that itemizes all complete builds,
592 and includes build summary information.
593 The following is sample output for a completed build:
594 <literallayout class='monospaced'>
595 {"count": 1,
596 "builds": [
597 {"distro": "poky",
598 "errors": 0,
599 "machine":
600 "beaglebone",
601 "project": "my_rocko",
602 "stop": "2017-09-22T09:26:36.017Z",
603 "target": "quilt-native",
604 "seconds": "78.193",
605 "outcome": "Succeeded",
606 "id": 1,
607 "start": "2017-09-22T09:25:17.824Z",
608 "warnings": 1,
609 "name": "20170922092618"
610 }]
611 }
612 </literallayout>
613 The JSON data for this query is returned in a single line.
614 In the previous example the line has been artificially split for readability.
615 </para>
616 </section>
617
618 <section id='determining-status-of-a-specific-build'>
619 <title>Determining Status of a Specific Build</title>
620
621 <para>
622 Sometimes it is useful to determine the status of a specific
623 build.
624 To get the status of a specific build, use the following
625 call:
626 <literallayout class='monospaced'>
627 http://<replaceable>host</replaceable>:<replaceable>port</replaceable>/toastergui/api/build/<replaceable>ID</replaceable>
628 </literallayout>
629 Be sure to provide values for <replaceable>host</replaceable>,
630 <replaceable>port</replaceable>, and <replaceable>ID</replaceable>.
631 You can find the value for <replaceable>ID</replaceable> from the
632 Builds Completed query. See the
633 "<link linkend='checking-status-of-builds-completed'>Checking Status of Builds Completed</link>"
634 section for more information.
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500635 </para>
636
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500637 <para>
638 The output is a JSON file that itemizes the specific build
639 and includes build summary information.
640 The following is sample output for a specific build:
641 <literallayout class='monospaced'>
642 {"build":
643 {"distro": "poky",
644 "errors": 0,
645 "machine": "beaglebone",
646 "project": "my_rocko",
647 "stop": "2017-09-22T09:26:36.017Z",
648 "target": "quilt-native",
649 "seconds": "78.193",
650 "outcome": "Succeeded",
651 "id": 1,
652 "start": "2017-09-22T09:25:17.824Z",
653 "warnings": 1,
654 "name": "20170922092618",
655 "cooker_log": "/opt/user/poky/build-toaster-2/tmp/log/cooker/beaglebone/build_20170922_022607.991.log"
Patrick Williamsf1e5d692016-03-30 15:21:19 -0500656 }
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500657 }
658 </literallayout>
659 The JSON data for this query is returned in a single line.
660 In the previous example the line has been artificially split for readability.
661 </para>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500662 </section>
663 </section>
664
665 <section id='toaster-useful-commands'>
666 <title>Useful Commands</title>
667
668 <para>
669 In addition to the web user interface and the scripts that start
670 and stop Toaster, command-line commands exist through the
671 <filename>manage.py</filename> management script.
672 You can find general documentation on
673 <filename>manage.py</filename> at the
674 <ulink url='https://docs.djangoproject.com/en/1.7/topics/settings/'>Django</ulink>
675 site.
676 However, several <filename>manage.py</filename> commands have been
677 created that are specific to Toaster and are used to control
678 configuration and back-end tasks.
679 You can locate these commands in the
Brad Bishopd7bf8c12018-02-25 22:55:05 -0500680 <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500681 (e.g. <filename>poky</filename>) at
682 <filename>bitbake/lib/manage.py</filename>.
683 This section documents those commands.
Brad Bishop316dfdd2018-06-25 12:45:53 -0400684 <note><title>Notes</title>
685 <itemizedlist>
686 <listitem><para>
687 When using <filename>manage.py</filename> commands given
688 a default configuration, you must be sure that your
689 working directory is set to the
690 <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>.
691 Using <filename>manage.py</filename> commands from the
692 Build Directory allows Toaster to find the
693 <filename>toaster.sqlite</filename> file, which is located
694 in the Build Directory.
695 </para></listitem>
696 <listitem><para>
697 For non-default database configurations, it is possible
698 that you can use <filename>manage.py</filename> commands
699 from a directory other than the Build Directory.
700 To do so, the
701 <filename>toastermain/settings.py</filename> file must be
702 configured to point to the correct database backend.
703 </para></listitem>
704 </itemizedlist>
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500705 </note>
706 </para>
707
708 <section id='toaster-command-buildslist'>
709 <title><filename>buildslist</filename></title>
710
711 <para>
712 The <filename>buildslist</filename> command lists all builds
713 that Toaster has recorded.
714 Access the command as follows:
715 <literallayout class='monospaced'>
716 $ bitbake/lib/toaster/manage.py buildslist
717 </literallayout>
718 The command returns a list, which includes numeric
719 identifications, of the builds that Toaster has recorded in the
720 current database.
721 </para>
722
723 <para>
724 You need to run the <filename>buildslist</filename> command
725 first to identify existing builds in the database before
726 using the
727 <link linkend='toaster-command-builddelete'><filename>builddelete</filename></link>
728 command.
729 Here is an example that assumes default repository and build
730 directory names:
731 <literallayout class='monospaced'>
732 $ cd ~/poky/build
733 $ python ../bitbake/lib/toaster/manage.py buildslist
734 </literallayout>
735 If your Toaster database had only one build, the above
736 <filename>buildslist</filename> command would return something
737 like the following:
738 <literallayout class='monospaced'>
739 1: qemux86 poky core-image-minimal
740 </literallayout>
741 </para>
742 </section>
743
744 <section id='toaster-command-builddelete'>
745 <title><filename>builddelete</filename></title>
746
747 <para>
748 The <filename>builddelete</filename> command deletes data
749 associated with a build.
750 Access the command as follows:
751 <literallayout class='monospaced'>
752 $ bitbake/lib/toaster/manage.py builddelete <replaceable>build_id</replaceable>
753 </literallayout>
754 The command deletes all the build data for the specified
755 <replaceable>build_id</replaceable>.
756 This command is useful for removing old and unused data from
757 the database.
758 </para>
759
760 <para>
761 Prior to running the <filename>builddelete</filename>
762 command, you need to get the ID associated with builds
763 by using the
764 <link linkend='toaster-command-buildslist'><filename>buildslist</filename></link>
765 command.
766 </para>
767 </section>
768
769 <section id='toaster-command-perf'>
770 <title><filename>perf</filename></title>
771
772 <para>
773 The <filename>perf</filename> command measures Toaster
774 performance.
775 Access the command as follows:
776 <literallayout class='monospaced'>
777 $ bitbake/lib/toaster/manage.py perf
778 </literallayout>
779 The command is a sanity check that returns page loading
780 times in order to identify performance problems.
781 </para>
782 </section>
783
784 <section id='toaster-command-checksettings'>
785 <title><filename>checksettings</filename></title>
786
787 <para>
788 The <filename>checksettings</filename> command verifies
789 existing Toaster settings.
790 Access the command as follows:
791 <literallayout class='monospaced'>
792 $ bitbake/lib/toaster/manage.py checksettings
793 </literallayout>
Patrick Williamsd8c66bc2016-06-20 12:57:21 -0500794 Toaster uses settings that are based on the
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500795 database to configure the building tasks.
796 The <filename>checksettings</filename> command verifies that
797 the database settings are valid in the sense that they have
798 the minimal information needed to start a build.
799 </para>
800
801 <para>
802 In order for the <filename>checksettings</filename> command
803 to work, the database must be correctly set up and not have
804 existing data.
805 To be sure the database is ready, you can run the following:
806 <literallayout class='monospaced'>
807 $ bitbake/lib/toaster/mana​ge.py syncdb
808 $ bitbake/lib/toaster/mana​ge.py migrate orm
809 $ bitbake/lib/toaster/mana​ge.py migrate bldcontrol
810 </literallayout>
811 After running these commands, you can run the
812 <filename>checksettings</filename> command.
813 </para>
814 </section>
815
Patrick Williamsc124f4f2015-09-15 14:41:29 -0500816 <section id='toaster-command-runbuilds'>
817 <title><filename>runbuilds</filename></title>
818
819 <para>
820 The <filename>runbuilds</filename> command launches
821 scheduled builds.
822 Access the command as follows:
823 <literallayout class='monospaced'>
824 $ bitbake/lib/toaster/manage.py runbuilds
825 </literallayout>
826 The <filename>runbuilds</filename> command checks if
827 scheduled builds exist in the database and then launches them
828 per schedule.
829 The command returns after the builds start but before they
830 complete.
831 The Toaster Logging Interface records and updates the database
832 when the builds complete.
833 </para>
834 </section>
835 </section>
836</chapter>