Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 1 | <!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" |
| 2 | "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" |
| 3 | [<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > |
| 4 | |
| 5 | <chapter id='toaster-manual-reference'> |
| 6 | |
| 7 | <title>Concepts and Reference</title> |
| 8 | |
| 9 | <para> |
| 10 | In order to configure and use Toaster, you should understand some |
| 11 | concepts and have some basic command reference material available. |
| 12 | This final chapter provides conceptual information on layer sources, |
| 13 | releases, and JSON configuration files. |
| 14 | Also provided is a quick look at some useful |
| 15 | <filename>manage.py</filename> commands that are Toaster-specific. |
| 16 | Information on <filename>manage.py</filename> commands does exist |
| 17 | across the Web and the information in this manual by no means |
| 18 | attempts to provide a command comprehensive reference. |
| 19 | </para> |
| 20 | |
| 21 | <section id='layer-source'> |
| 22 | <title>Layer Source</title> |
| 23 | |
| 24 | <para> |
| 25 | In general, a "layer source" is a source of information about |
| 26 | existing layers. |
| 27 | In particular, we are concerned with layers that you can use |
| 28 | with the Yocto Project and Toaster. |
| 29 | This chapter describes a particular type of layer source called |
| 30 | a "layer index." |
| 31 | </para> |
| 32 | |
| 33 | <para> |
| 34 | A layer index is a web application that contains information |
| 35 | about a set of custom layers. |
| 36 | A good example of an existing layer index is the |
| 37 | OpenEmbedded Metadata Index. |
| 38 | A public instance of this layer index exists at |
| 39 | <ulink url='http://layers.openembedded.org'></ulink>. |
| 40 | You can find the code for this layer index's web application at |
| 41 | <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/layerindex-web/'></ulink>. |
| 42 | </para> |
| 43 | |
| 44 | <para> |
| 45 | When you tie a layer source into Toaster, it can query the layer |
| 46 | source through a |
| 47 | <ulink url='http://en.wikipedia.org/wiki/Representational_state_transfer'>REST</ulink> |
| 48 | API, store the information about the layers in the Toaster |
| 49 | database, and then show the information to users. |
| 50 | Users are then able to view that information and build layers |
| 51 | from Toaster itself without worrying about cloning or editing |
| 52 | the BitBake layers configuration file |
| 53 | <filename>bblayers.conf</filename>. |
| 54 | </para> |
| 55 | |
| 56 | <para> |
| 57 | Tying a layer source into Toaster is convenient when you have |
| 58 | many custom layers that need to be built on a regular basis by |
| 59 | a community of developers. |
| 60 | In fact, Toaster comes pre-configured with the OpenEmbedded |
| 61 | Metadata Index. |
| 62 | <note> |
| 63 | You do not have to use a layer source to use Toaster. |
| 64 | Tying into a layer source is optional. |
| 65 | </note> |
| 66 | </para> |
| 67 | |
| 68 | <section id='layer-source-using-with-toaster'> |
| 69 | <title>Setting Up and Using a Layer Source</title> |
| 70 | |
| 71 | <para> |
| 72 | To use your own layer source, you need to set up the layer |
| 73 | source and then tie it into Toaster. |
| 74 | This section describes how to tie into a layer index in a manner |
| 75 | similar to the way Toaster ties into the OpenEmbedded Metadata |
| 76 | Index. |
| 77 | </para> |
| 78 | |
| 79 | <section id='understanding-your-layers'> |
| 80 | <title>Understanding Your Layers</title> |
| 81 | |
| 82 | <para> |
| 83 | The obvious first step for using a layer index is to have |
| 84 | several custom layers that developers build and access using |
| 85 | the Yocto Project on a regular basis. |
| 86 | This set of layers needs to exist and you need to be |
| 87 | familiar with where they reside. |
| 88 | You will need that information when you set up the |
| 89 | code for the web application that "hooks" into your set of |
| 90 | layers. |
| 91 | </para> |
| 92 | |
| 93 | <para> |
| 94 | For general information on layers, see the |
| 95 | "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>" |
| 96 | and |
| 97 | "<ulink url='&YOCTO_DOCS_BSP_URL;#using-the-yocto-projects-bsp-tools'>Using the Yocto Project's BSP Tools</ulink>" |
| 98 | sections in the Yocto Project Board Support Package (BSP) |
| 99 | Developer's Guide. |
| 100 | </para> |
| 101 | </section> |
| 102 | |
| 103 | <section id='configuring-toaster-to-hook-into-your-layer-source'> |
| 104 | <title>Configuring Toaster to Hook Into Your Layer Index</title> |
| 105 | |
| 106 | <para> |
| 107 | If you want Toaster to use your layer index, you must host |
| 108 | the web application in a server to which Toaster can |
| 109 | connect. |
| 110 | You also need to give Toaster the information about your |
| 111 | layer index. |
| 112 | In other words, you have to configure Toaster to use your |
| 113 | layer index. |
| 114 | This section describes two methods by which you can |
| 115 | configure and use your layer index. |
| 116 | </para> |
| 117 | |
| 118 | <para> |
| 119 | In the previous section, the code for the OpenEmbedded |
| 120 | Metadata Index (i.e. |
| 121 | <ulink url='http://layers.openembedded.org'></ulink>) was |
| 122 | referenced. |
| 123 | You can use this code, which is at |
| 124 | <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/layerindex-web/'></ulink>, |
| 125 | as a base to create your own layer index. |
| 126 | </para> |
| 127 | |
| 128 | <section id='use-the-administration-interface'> |
| 129 | <title>Use the Administration Interface</title> |
| 130 | |
| 131 | <para> |
| 132 | Access the administration interface through a |
| 133 | browser by entering the URL of your Toaster instance and |
| 134 | adding "<filename>/admin</filename>" to the end of the |
| 135 | URL. |
| 136 | As an example, if you are running Toaster locally, use |
| 137 | the following URL: |
| 138 | <literallayout class='monospaced'> |
| 139 | http://127.0.0.1:8000/admin |
| 140 | </literallayout> |
| 141 | </para> |
| 142 | |
| 143 | <para> |
| 144 | The administration interface has a "Layer sources" |
| 145 | section that includes an "Add layer source" button. |
| 146 | Click that button and provide the required information. |
| 147 | Make sure you select "layerindex" as the layer source type. |
| 148 | </para> |
| 149 | </section> |
| 150 | |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 151 | <section id='use-the-fixture-feature'> |
| 152 | <title>Use the Fixture Feature</title> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 153 | |
| 154 | <para> |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 155 | The Django fixture feature overrides the default layer |
| 156 | server when you use it to specify a custom URL. To use |
| 157 | the fixture feature, create (or edit) the file |
| 158 | <filename>bitbake/lib/toaster.orm/fixtures/custom.xml</filename>, |
| 159 | and then set the following Toaster setting to your |
| 160 | custom URL: |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 161 | <literallayout class='monospaced'> |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 162 | <?xml version="1.0" ?> |
| 163 | <django-objects version="1.0"> |
| 164 | <object model="orm.toastersetting" pk="100"> |
| 165 | <field name="name" type="CharField">CUSTOM_LAYERINDEX_SERVER</field> |
| 166 | <field name="value" type="CharField">https://layers.my_organization.org/layerindex/branch/master/layers/</field> |
| 167 | </object> |
| 168 | <django-objects> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 169 | </literallayout> |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 170 | When you start Toaster for the first time, or if you |
| 171 | delete the file <filename>toaster.sqlite</filename> and restart, |
| 172 | the database will populate cleanly from this layer index server. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 173 | </para> |
| 174 | |
| 175 | <para> |
| 176 | Once the information has been updated, verify the new layer |
| 177 | information is available by using the Toaster web interface. |
| 178 | To do that, visit the "All compatible layers" page inside a |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 179 | Toaster project. The layers from your layer source should be |
| 180 | listed there. |
| 181 | </para> |
| 182 | |
| 183 | <para> |
| 184 | If you change the information in your layer index server, |
| 185 | refresh the Toaster database by running the following command: |
| 186 | <literallayout class='monospaced'> |
| 187 | $ bitbake/lib/toaster/manage.py lsupdates |
| 188 | </literallayout> |
| 189 | If Toaster can reach the API URL, you should see a message |
| 190 | telling you that Toaster is updating the layer source information. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 191 | </para> |
| 192 | </section> |
| 193 | </section> |
| 194 | </section> |
| 195 | </section> |
| 196 | |
| 197 | <section id='toaster-releases'> |
| 198 | <title>Releases</title> |
| 199 | |
| 200 | <para> |
| 201 | When you create a Toaster project using the web interface, |
| 202 | you are asked to choose a "Release." |
| 203 | In the context of Toaster, the term "Release" refers to a set of |
| 204 | layers and a BitBake version the OpenEmbedded build system uses |
| 205 | to build something. |
| 206 | As shipped, Toaster is pre-configured with releases that |
| 207 | correspond to Yocto Project release branches. |
| 208 | However, you can modify, delete, and create new releases |
| 209 | according to your needs. |
| 210 | This section provides some background information on releases. |
| 211 | </para> |
| 212 | |
| 213 | <section id='toaster-releases-supported'> |
| 214 | <title>Pre-Configured Releases</title> |
| 215 | |
| 216 | <para> |
| 217 | As shipped, Toaster is configured to use a specific set of |
| 218 | releases. |
| 219 | Of course, you can always configure Toaster to use any |
| 220 | release. |
| 221 | For example, you might want your project to build against a |
| 222 | specific commit of any of the "out-of-the-box" releases. |
| 223 | Or, you might want your project to build against different |
| 224 | revisions of OpenEmbedded and BitBake. |
| 225 | </para> |
| 226 | |
| 227 | <para> |
| 228 | As shipped, Toaster is configured to work with the following |
| 229 | releases: |
| 230 | <itemizedlist> |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 231 | <listitem><para><emphasis> |
| 232 | Yocto Project &DISTRO; "&DISTRO_NAME;" or OpenEmbedded "&DISTRO_NAME;":</emphasis> |
| 233 | This release causes your Toaster projects to build |
| 234 | against the head of the &DISTRO_NAME_NO_CAP; branch at |
| 235 | <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/log/?h=rocko'></ulink> |
| 236 | or <ulink url='http://git.openembedded.org/openembedded-core/commit/?h=rocko'></ulink>. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 237 | </para></listitem> |
| 238 | <listitem><para><emphasis>Yocto Project "Master" or OpenEmbedded "Master":</emphasis> |
| 239 | This release causes your Toaster Projects to |
| 240 | build against the head of the master branch, which is |
| 241 | where active development takes place, at |
| 242 | <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/log/'></ulink> |
| 243 | or |
| 244 | <ulink url='http://git.openembedded.org/openembedded-core/log/'></ulink>. |
| 245 | </para></listitem> |
| 246 | <listitem><para><emphasis>Local Yocto Project or Local OpenEmbedded:</emphasis> |
| 247 | This release causes your Toaster Projects to |
| 248 | build against the head of the <filename>poky</filename> |
| 249 | or <filename>openembedded-core</filename> clone you |
| 250 | have local to the machine running Toaster. |
| 251 | </para></listitem> |
| 252 | </itemizedlist> |
| 253 | </para> |
| 254 | </section> |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 255 | </section> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 256 | |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 257 | <section id='configuring-toaster'> |
| 258 | <title>Configuring Toaster</title> |
| 259 | |
| 260 | <para> |
| 261 | In order to use Toaster, you must configure the database with the |
| 262 | default content. The following subsections describe various aspects |
| 263 | of Toaster configuration. |
| 264 | </para> |
| 265 | |
| 266 | <section id='configuring-the-workflow'> |
| 267 | <title>Configuring the Workflow</title> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 268 | |
| 269 | <para> |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 270 | The |
| 271 | <filename>bldcontrol/management/commands/checksettings.py</filename> |
| 272 | file controls workflow configuration. |
| 273 | The following steps outline the process to initially populate |
| 274 | this database. |
| 275 | <orderedlist> |
| 276 | <listitem><para> |
| 277 | The default project settings are set from |
| 278 | <filename>orm/fixtures/settings.xml</filename>. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 279 | </para></listitem> |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 280 | <listitem><para> |
| 281 | The default project distro and layers are added |
| 282 | from <filename>orm/fixtures/poky.xml</filename> if poky |
| 283 | is installed. |
| 284 | If poky is not installed, they are added |
| 285 | from <filename>orm/fixtures/oe-core.xml</filename>. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 286 | </para></listitem> |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 287 | <listitem><para> |
| 288 | If the <filename>orm/fixtures/custom.xml</filename> file |
| 289 | exists, then its values are added. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 290 | </para></listitem> |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 291 | <listitem><para> |
| 292 | The layer index is then scanned and added to the database. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 293 | </para></listitem> |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 294 | </orderedlist> |
| 295 | Once these steps complete, Toaster is set up and ready to use. |
| 296 | </para> |
| 297 | </section> |
| 298 | |
| 299 | <section id='customizing-pre-set-data'> |
| 300 | <title>Customizing Pre-Set Data</title> |
| 301 | |
| 302 | <para> |
| 303 | The pre-set data for Toaster is easily customizable. You can |
| 304 | create the <filename>orm/fixtures/custom.xml</filename> file |
| 305 | to customize the values that go into to the database. |
| 306 | Customization is additive, |
| 307 | and can either extend or completely replace the existing values. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 308 | </para> |
| 309 | |
| 310 | <para> |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 311 | You use the <filename>orm/fixtures/custom.xml</filename> file |
| 312 | to change the default project settings for the machine, distro, |
| 313 | file images, and layers. |
| 314 | When creating a new project, you can use the file to define |
| 315 | the offered alternate project release selections. |
| 316 | For example, you can add one or more additional selections that |
| 317 | present custom layer sets or distros, and any other local or proprietary |
| 318 | content. |
| 319 | </para> |
| 320 | |
| 321 | <para> |
| 322 | Additionally, you can completely disable the content from the |
| 323 | <filename>oe-core.xml</filename> and <filename>poky.xml</filename> |
| 324 | files by defining the section shown below in the |
| 325 | <filename>settings.xml</filename> file. |
| 326 | For example, this option is particularly useful if your custom |
| 327 | configuration defines fewer releases or layers than the default |
| 328 | fixture files. |
| 329 | </para> |
| 330 | |
| 331 | <para> |
| 332 | The following example sets "name" to "CUSTOM_XML_ONLY" and its value |
| 333 | to "True". |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 334 | <literallayout class='monospaced'> |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 335 | <object model="orm.toastersetting" pk="99"> |
| 336 | <field type="CharField" name="name">CUSTOM_XML_ONLY</field> |
| 337 | <field type="CharField" name="value">True</field> |
| 338 | </object> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 339 | </literallayout> |
| 340 | </para> |
| 341 | </section> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 342 | |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 343 | <section id='understanding-fixture-file-format'> |
| 344 | <title>Understanding Fixture File Format</title> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 345 | |
| 346 | <para> |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 347 | The following is an overview of the file format used by the |
| 348 | <filename>oe-core.xml</filename>, <filename>poky.xml</filename>, |
| 349 | and <filename>custom.xml</filename> files. |
| 350 | </para> |
| 351 | |
| 352 | <para> |
| 353 | The following subsections describe each of the sections in the |
| 354 | fixture files, and outline an example section of the XML code. |
| 355 | you can use to help understand this information and create a local |
| 356 | <filename>custom.xml</filename> file. |
| 357 | </para> |
| 358 | |
| 359 | <section id='defining-the-default-distro-and-other-values'> |
| 360 | <title>Defining the Default Distro and Other Values</title> |
| 361 | |
| 362 | <para> |
| 363 | This section defines the default distro value for new projects. |
| 364 | By default, it reserves the first Toaster Setting record "1". |
| 365 | The following demonstrates how to set the project default value |
| 366 | for |
| 367 | <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink>: |
| 368 | <literallayout class='monospaced'> |
| 369 | <!-- Set the project default value for DISTRO --> |
| 370 | <object model="orm.toastersetting" pk="1"> |
| 371 | <field type="CharField" name="name">DEFCONF_DISTRO</field> |
| 372 | <field type="CharField" name="value">poky</field> |
| 373 | </object> |
| 374 | </literallayout> |
| 375 | You can override other default project values by adding |
| 376 | additional Toaster Setting sections such as any of the |
| 377 | settings coming from the <filename>settings.xml</filename> |
| 378 | file. |
| 379 | Also, you can add custom values that are included in the |
| 380 | BitBake environment. |
| 381 | The "pk" values must be unique. |
| 382 | By convention, values that set default project values have a |
| 383 | "DEFCONF" prefix. |
| 384 | </para> |
| 385 | </section> |
| 386 | |
| 387 | <section id='defining-bitbake-version'> |
| 388 | <title>Defining BitBake Version</title> |
| 389 | |
| 390 | <para> |
| 391 | The following defines which version of BitBake is used |
| 392 | for the following release selection: |
| 393 | <literallayout class='monospaced'> |
| 394 | <!-- Bitbake versions which correspond to the metadata release --> |
| 395 | <object model="orm.bitbakeversion" pk="1"> |
| 396 | <field type="CharField" name="name">rocko</field> |
| 397 | <field type="CharField" name="giturl">git://git.yoctoproject.org/poky</field> |
| 398 | <field type="CharField" name="branch">rocko</field> |
| 399 | <field type="CharField" name="dirpath">bitbake</field> |
| 400 | </object> |
| 401 | </literallayout> |
| 402 | </para> |
| 403 | </section> |
| 404 | |
| 405 | <section id='defining-releases'> |
| 406 | <title>Defining Release</title> |
| 407 | |
| 408 | <para> |
| 409 | The following defines the releases when you create a new |
| 410 | project. |
| 411 | <literallayout class='monospaced'> |
| 412 | <!-- Releases available --> |
| 413 | <object model="orm.release" pk="1"> |
| 414 | <field type="CharField" name="name">rocko</field> |
| 415 | <field type="CharField" name="description">Yocto Project 2.4 "Rocko"</field> |
| 416 | <field rel="ManyToOneRel" to="orm.bitbakeversion" name="bitbake_version">1</field> |
| 417 | <field type="CharField" name="branch_name">rocko</field> |
| 418 | <field type="TextField" name="helptext">Toaster will run your builds using the tip of the <a href="http://git.yoctoproject.org/cgit/cgit.cgi/poky/log/?h=rocko">Yocto Project Rocko branch</a>.</field> |
| 419 | </object> |
| 420 | </literallayout> |
| 421 | The "pk" value must match the above respective BitBake |
| 422 | version record. |
| 423 | </para> |
| 424 | </section> |
| 425 | |
| 426 | <section id='defining-the-release-default-layer-names'> |
| 427 | <title>Defining the Release Default Layer Names</title> |
| 428 | |
| 429 | <para> |
| 430 | The following defines the default layers for each release: |
| 431 | <literallayout class='monospaced'> |
| 432 | <!-- Default project layers for each release --> |
| 433 | <object model="orm.releasedefaultlayer" pk="1"> |
| 434 | <field rel="ManyToOneRel" to="orm.release" name="release">1</field> |
| 435 | <field type="CharField" name="layer_name">openembedded-core</field> |
| 436 | </object> |
| 437 | </literallayout> |
| 438 | The 'pk' values in the example above should start at "1" and increment |
| 439 | uniquely. |
| 440 | You can use the same layer name in multiple releases. |
| 441 | </para> |
| 442 | </section> |
| 443 | |
| 444 | <section id='defining-layer-definitions'> |
| 445 | <title>Defining Layer Definitions</title> |
| 446 | |
| 447 | <para> |
| 448 | Layer definitions are the most complex. |
| 449 | The following defines each of the layers, and then defines the exact layer |
| 450 | version of the layer used for each respective release. |
| 451 | You must have one <filename>orm.layer</filename> |
| 452 | entry for each layer. |
| 453 | Then, with each entry you need a set of |
| 454 | <filename>orm.layer_version</filename> entries that connects |
| 455 | the layer with each release that includes the layer. |
| 456 | In general all releases include the layer. |
| 457 | <literallayout class='monospaced'> |
| 458 | <object model="orm.layer" pk="1"> |
| 459 | <field type="CharField" name="name">openembedded-core</field> |
| 460 | <field type="CharField" name="layer_index_url"></field> |
| 461 | <field type="CharField" name="vcs_url">git://git.yoctoproject.org/poky</field> |
| 462 | <field type="CharField" name="vcs_web_url">http://git.yoctoproject.org/cgit/cgit.cgi/poky</field> |
| 463 | <field type="CharField" name="vcs_web_tree_base_url">http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/%path%?h=%branch%</field> |
| 464 | <field type="CharField" name="vcs_web_file_base_url">http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/%path%?h=%branch%</field> |
| 465 | </object> |
| 466 | <object model="orm.layer_version" pk="1"> |
| 467 | <field rel="ManyToOneRel" to="orm.layer" name="layer">1</field> |
| 468 | <field type="IntegerField" name="layer_source">0</field> |
| 469 | <field rel="ManyToOneRel" to="orm.release" name="release">1</field> |
| 470 | <field type="CharField" name="branch">rocko</field> |
| 471 | <field type="CharField" name="dirpath">meta</field> |
| 472 | </object> |
| 473 | <object model="orm.layer_version" pk="2"> |
| 474 | <field rel="ManyToOneRel" to="orm.layer" name="layer">1</field> |
| 475 | <field type="IntegerField" name="layer_source">0</field> |
| 476 | <field rel="ManyToOneRel" to="orm.release" name="release">2</field> |
| 477 | <field type="CharField" name="branch">HEAD</field> |
| 478 | <field type="CharField" name="commit">HEAD</field> |
| 479 | <field type="CharField" name="dirpath">meta</field> |
| 480 | </object> |
| 481 | <object model="orm.layer_version" pk="3"> |
| 482 | <field rel="ManyToOneRel" to="orm.layer" name="layer">1</field> |
| 483 | <field type="IntegerField" name="layer_source">0</field> |
| 484 | <field rel="ManyToOneRel" to="orm.release" name="release">3</field> |
| 485 | |
| 486 | <field type="CharField" name="branch">master</field> |
| 487 | <field type="CharField" name="dirpath">meta</field> |
| 488 | </object> |
| 489 | </literallayout> |
| 490 | The layer "pk" values above must be unique, and typically start at "1". |
| 491 | The layer version "pk" values must also be unique across all layers, |
| 492 | and typically start at "1". |
| 493 | </para> |
| 494 | </section> |
| 495 | </section> |
| 496 | </section> |
| 497 | |
| 498 | <section id='remote-toaster-monitoring'> |
| 499 | <title>Remote Toaster Monitoring</title> |
| 500 | |
| 501 | <para> |
| 502 | Toaster has an API that allows remote management applications to |
| 503 | directly query the state of the Toaster server and its builds |
| 504 | in a machine-to-machine manner. |
| 505 | This API uses the |
| 506 | <ulink url='http://en.wikipedia.org/wiki/Representational_state_transfer'>REST</ulink> |
| 507 | interface and the transfer of JSON files. |
| 508 | For example, you might |
| 509 | monitor a build inside a container through well supported |
| 510 | known HTTP ports in order to easily access a Toaster server |
| 511 | inside the container. |
| 512 | In this example, when you use this direct JSON API, you avoid |
| 513 | having web page parsing against the display the user sees. |
| 514 | </para> |
| 515 | |
| 516 | <section id='checking-health'> |
| 517 | <title>Checking Health</title> |
| 518 | |
| 519 | <para> |
| 520 | Before you use remote Toaster monitoring, you should do |
| 521 | a health check. |
| 522 | To do this, ping the Toaster server using the following call |
| 523 | to see if it is still alive: |
| 524 | <literallayout class='monospaced'> |
| 525 | http://<replaceable>host</replaceable>:<replaceable>port</replaceable>/health |
| 526 | </literallayout> |
| 527 | Be sure to provide values for <replaceable>host</replaceable> |
| 528 | and <replaceable>port</replaceable>. |
| 529 | If the server is alive, you will get the response HTML: |
| 530 | <literallayout class='monospaced'> |
| 531 | <!DOCTYPE html> |
| 532 | <html lang="en"> |
| 533 | <head><title>Toaster Health</title></head> |
| 534 | <body>Ok</body> |
| 535 | </html> |
| 536 | </literallayout> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 537 | </para> |
| 538 | </section> |
| 539 | |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 540 | <section id='determining-status-of-builds-in-progress'> |
| 541 | <title>Determining Status of Builds in Progress</title> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 542 | |
| 543 | <para> |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 544 | Sometimes it is useful to determine the status of a build |
| 545 | in progress. |
| 546 | To get the status of pending builds, use the following call: |
| 547 | <literallayout class='monospaced'> |
| 548 | http://<replaceable>host</replaceable>:<replaceable>port</replaceable>/toastergui/api/building |
| 549 | </literallayout> |
| 550 | Be sure to provide values for <replaceable>host</replaceable> |
| 551 | and <replaceable>port</replaceable>. |
| 552 | The output is a JSON file that itemizes all builds in |
| 553 | progress. |
| 554 | This file includes the time in seconds since each |
| 555 | respective build started as well as the progress of the |
| 556 | cloning, parsing, and task execution. |
| 557 | The following is sample output for a build in progress: |
| 558 | <literallayout class='monospaced'> |
| 559 | {"count": 1, |
| 560 | "building": [ |
| 561 | {"machine": "beaglebone", |
| 562 | "seconds": "463.869", |
| 563 | "task": "927:2384", |
| 564 | "distro": "poky", |
| 565 | "clone": "1:1", |
| 566 | "id": 2, |
| 567 | "start": "2017-09-22T09:31:44.887Z", |
| 568 | "name": "20170922093200", |
| 569 | "parse": "818:818", |
| 570 | "project": "my_rocko", |
| 571 | "target": "core-image-minimal" |
| 572 | }] |
| 573 | } |
| 574 | </literallayout> |
| 575 | The JSON data for this query is returned in a single line. |
| 576 | In the previous example the line has been artificially split for readability. |
| 577 | </para> |
| 578 | </section> |
| 579 | |
| 580 | <section id='checking-status-of-builds-completed'> |
| 581 | <title>Checking Status of Builds Completed</title> |
| 582 | |
| 583 | <para> |
| 584 | Once a build is completed, you get the status when you use |
| 585 | the following call: |
| 586 | <literallayout class='monospaced'> |
| 587 | http://<replaceable>host</replaceable>:<replaceable>port</replaceable>/toastergui/api/builds |
| 588 | </literallayout> |
| 589 | Be sure to provide values for <replaceable>host</replaceable> |
| 590 | and <replaceable>port</replaceable>. |
| 591 | The output is a JSON file that itemizes all complete builds, |
| 592 | and includes build summary information. |
| 593 | The following is sample output for a completed build: |
| 594 | <literallayout class='monospaced'> |
| 595 | {"count": 1, |
| 596 | "builds": [ |
| 597 | {"distro": "poky", |
| 598 | "errors": 0, |
| 599 | "machine": |
| 600 | "beaglebone", |
| 601 | "project": "my_rocko", |
| 602 | "stop": "2017-09-22T09:26:36.017Z", |
| 603 | "target": "quilt-native", |
| 604 | "seconds": "78.193", |
| 605 | "outcome": "Succeeded", |
| 606 | "id": 1, |
| 607 | "start": "2017-09-22T09:25:17.824Z", |
| 608 | "warnings": 1, |
| 609 | "name": "20170922092618" |
| 610 | }] |
| 611 | } |
| 612 | </literallayout> |
| 613 | The JSON data for this query is returned in a single line. |
| 614 | In the previous example the line has been artificially split for readability. |
| 615 | </para> |
| 616 | </section> |
| 617 | |
| 618 | <section id='determining-status-of-a-specific-build'> |
| 619 | <title>Determining Status of a Specific Build</title> |
| 620 | |
| 621 | <para> |
| 622 | Sometimes it is useful to determine the status of a specific |
| 623 | build. |
| 624 | To get the status of a specific build, use the following |
| 625 | call: |
| 626 | <literallayout class='monospaced'> |
| 627 | http://<replaceable>host</replaceable>:<replaceable>port</replaceable>/toastergui/api/build/<replaceable>ID</replaceable> |
| 628 | </literallayout> |
| 629 | Be sure to provide values for <replaceable>host</replaceable>, |
| 630 | <replaceable>port</replaceable>, and <replaceable>ID</replaceable>. |
| 631 | You can find the value for <replaceable>ID</replaceable> from the |
| 632 | Builds Completed query. See the |
| 633 | "<link linkend='checking-status-of-builds-completed'>Checking Status of Builds Completed</link>" |
| 634 | section for more information. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 635 | </para> |
| 636 | |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 637 | <para> |
| 638 | The output is a JSON file that itemizes the specific build |
| 639 | and includes build summary information. |
| 640 | The following is sample output for a specific build: |
| 641 | <literallayout class='monospaced'> |
| 642 | {"build": |
| 643 | {"distro": "poky", |
| 644 | "errors": 0, |
| 645 | "machine": "beaglebone", |
| 646 | "project": "my_rocko", |
| 647 | "stop": "2017-09-22T09:26:36.017Z", |
| 648 | "target": "quilt-native", |
| 649 | "seconds": "78.193", |
| 650 | "outcome": "Succeeded", |
| 651 | "id": 1, |
| 652 | "start": "2017-09-22T09:25:17.824Z", |
| 653 | "warnings": 1, |
| 654 | "name": "20170922092618", |
| 655 | "cooker_log": "/opt/user/poky/build-toaster-2/tmp/log/cooker/beaglebone/build_20170922_022607.991.log" |
Patrick Williams | f1e5d69 | 2016-03-30 15:21:19 -0500 | [diff] [blame] | 656 | } |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 657 | } |
| 658 | </literallayout> |
| 659 | The JSON data for this query is returned in a single line. |
| 660 | In the previous example the line has been artificially split for readability. |
| 661 | </para> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 662 | </section> |
| 663 | </section> |
| 664 | |
| 665 | <section id='toaster-useful-commands'> |
| 666 | <title>Useful Commands</title> |
| 667 | |
| 668 | <para> |
| 669 | In addition to the web user interface and the scripts that start |
| 670 | and stop Toaster, command-line commands exist through the |
| 671 | <filename>manage.py</filename> management script. |
| 672 | You can find general documentation on |
| 673 | <filename>manage.py</filename> at the |
| 674 | <ulink url='https://docs.djangoproject.com/en/1.7/topics/settings/'>Django</ulink> |
| 675 | site. |
| 676 | However, several <filename>manage.py</filename> commands have been |
| 677 | created that are specific to Toaster and are used to control |
| 678 | configuration and back-end tasks. |
| 679 | You can locate these commands in the |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 680 | <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 681 | (e.g. <filename>poky</filename>) at |
| 682 | <filename>bitbake/lib/manage.py</filename>. |
| 683 | This section documents those commands. |
| 684 | <note> |
| 685 | <para> |
| 686 | When using <filename>manage.py</filename> commands given |
| 687 | a default configuration, you must be sure that your |
| 688 | working directory is set to the |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame^] | 689 | <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>. |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 690 | Using <filename>manage.py</filename> commands from the |
| 691 | Build Directory allows Toaster to find the |
| 692 | <filename>toaster.sqlite</filename> file, which is located |
| 693 | in the Build Directory. |
| 694 | </para> |
| 695 | |
| 696 | <para> |
| 697 | For non-default database configurations, it is possible |
| 698 | that you can use <filename>manage.py</filename> commands |
| 699 | from a directory other than the Build directory. |
| 700 | To do so, the |
| 701 | <filename>toastermain/settings.py</filename> file must be |
| 702 | configured to point to the correct database backend. |
| 703 | </para> |
| 704 | </note> |
| 705 | </para> |
| 706 | |
| 707 | <section id='toaster-command-buildslist'> |
| 708 | <title><filename>buildslist</filename></title> |
| 709 | |
| 710 | <para> |
| 711 | The <filename>buildslist</filename> command lists all builds |
| 712 | that Toaster has recorded. |
| 713 | Access the command as follows: |
| 714 | <literallayout class='monospaced'> |
| 715 | $ bitbake/lib/toaster/manage.py buildslist |
| 716 | </literallayout> |
| 717 | The command returns a list, which includes numeric |
| 718 | identifications, of the builds that Toaster has recorded in the |
| 719 | current database. |
| 720 | </para> |
| 721 | |
| 722 | <para> |
| 723 | You need to run the <filename>buildslist</filename> command |
| 724 | first to identify existing builds in the database before |
| 725 | using the |
| 726 | <link linkend='toaster-command-builddelete'><filename>builddelete</filename></link> |
| 727 | command. |
| 728 | Here is an example that assumes default repository and build |
| 729 | directory names: |
| 730 | <literallayout class='monospaced'> |
| 731 | $ cd ~/poky/build |
| 732 | $ python ../bitbake/lib/toaster/manage.py buildslist |
| 733 | </literallayout> |
| 734 | If your Toaster database had only one build, the above |
| 735 | <filename>buildslist</filename> command would return something |
| 736 | like the following: |
| 737 | <literallayout class='monospaced'> |
| 738 | 1: qemux86 poky core-image-minimal |
| 739 | </literallayout> |
| 740 | </para> |
| 741 | </section> |
| 742 | |
| 743 | <section id='toaster-command-builddelete'> |
| 744 | <title><filename>builddelete</filename></title> |
| 745 | |
| 746 | <para> |
| 747 | The <filename>builddelete</filename> command deletes data |
| 748 | associated with a build. |
| 749 | Access the command as follows: |
| 750 | <literallayout class='monospaced'> |
| 751 | $ bitbake/lib/toaster/manage.py builddelete <replaceable>build_id</replaceable> |
| 752 | </literallayout> |
| 753 | The command deletes all the build data for the specified |
| 754 | <replaceable>build_id</replaceable>. |
| 755 | This command is useful for removing old and unused data from |
| 756 | the database. |
| 757 | </para> |
| 758 | |
| 759 | <para> |
| 760 | Prior to running the <filename>builddelete</filename> |
| 761 | command, you need to get the ID associated with builds |
| 762 | by using the |
| 763 | <link linkend='toaster-command-buildslist'><filename>buildslist</filename></link> |
| 764 | command. |
| 765 | </para> |
| 766 | </section> |
| 767 | |
| 768 | <section id='toaster-command-perf'> |
| 769 | <title><filename>perf</filename></title> |
| 770 | |
| 771 | <para> |
| 772 | The <filename>perf</filename> command measures Toaster |
| 773 | performance. |
| 774 | Access the command as follows: |
| 775 | <literallayout class='monospaced'> |
| 776 | $ bitbake/lib/toaster/manage.py perf |
| 777 | </literallayout> |
| 778 | The command is a sanity check that returns page loading |
| 779 | times in order to identify performance problems. |
| 780 | </para> |
| 781 | </section> |
| 782 | |
| 783 | <section id='toaster-command-checksettings'> |
| 784 | <title><filename>checksettings</filename></title> |
| 785 | |
| 786 | <para> |
| 787 | The <filename>checksettings</filename> command verifies |
| 788 | existing Toaster settings. |
| 789 | Access the command as follows: |
| 790 | <literallayout class='monospaced'> |
| 791 | $ bitbake/lib/toaster/manage.py checksettings |
| 792 | </literallayout> |
Patrick Williams | d8c66bc | 2016-06-20 12:57:21 -0500 | [diff] [blame] | 793 | Toaster uses settings that are based on the |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 794 | database to configure the building tasks. |
| 795 | The <filename>checksettings</filename> command verifies that |
| 796 | the database settings are valid in the sense that they have |
| 797 | the minimal information needed to start a build. |
| 798 | </para> |
| 799 | |
| 800 | <para> |
| 801 | In order for the <filename>checksettings</filename> command |
| 802 | to work, the database must be correctly set up and not have |
| 803 | existing data. |
| 804 | To be sure the database is ready, you can run the following: |
| 805 | <literallayout class='monospaced'> |
| 806 | $ bitbake/lib/toaster/manage.py syncdb |
| 807 | $ bitbake/lib/toaster/manage.py migrate orm |
| 808 | $ bitbake/lib/toaster/manage.py migrate bldcontrol |
| 809 | </literallayout> |
| 810 | After running these commands, you can run the |
| 811 | <filename>checksettings</filename> command. |
| 812 | </para> |
| 813 | </section> |
| 814 | |
Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 815 | <section id='toaster-command-runbuilds'> |
| 816 | <title><filename>runbuilds</filename></title> |
| 817 | |
| 818 | <para> |
| 819 | The <filename>runbuilds</filename> command launches |
| 820 | scheduled builds. |
| 821 | Access the command as follows: |
| 822 | <literallayout class='monospaced'> |
| 823 | $ bitbake/lib/toaster/manage.py runbuilds |
| 824 | </literallayout> |
| 825 | The <filename>runbuilds</filename> command checks if |
| 826 | scheduled builds exist in the database and then launches them |
| 827 | per schedule. |
| 828 | The command returns after the builds start but before they |
| 829 | complete. |
| 830 | The Toaster Logging Interface records and updates the database |
| 831 | when the builds complete. |
| 832 | </para> |
| 833 | </section> |
| 834 | </section> |
| 835 | </chapter> |