Patrick Williams | c124f4f | 2015-09-15 14:41:29 -0500 | [diff] [blame] | 1 | <!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" |
| 2 | "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" |
| 3 | [<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > |
| 4 | |
| 5 | <chapter id='dev-manual-newbie'> |
| 6 | |
| 7 | <title>The Yocto Project Open Source Development Environment</title> |
| 8 | |
| 9 | <para> |
| 10 | This chapter helps you understand the Yocto Project as an open source development project. |
| 11 | In general, working in an open source environment is very different from working in a |
| 12 | closed, proprietary environment. |
| 13 | Additionally, the Yocto Project uses specific tools and constructs as part of its development |
| 14 | environment. |
| 15 | This chapter specifically addresses open source philosophy, using the |
| 16 | Yocto Project in a team environment, source repositories, Yocto Project |
| 17 | terms, licensing, the open source distributed version control system Git, |
| 18 | workflows, bug tracking, and how to submit changes. |
| 19 | </para> |
| 20 | |
| 21 | <section id='open-source-philosophy'> |
| 22 | <title>Open Source Philosophy</title> |
| 23 | |
| 24 | <para> |
| 25 | Open source philosophy is characterized by software development directed by peer production |
| 26 | and collaboration through an active community of developers. |
| 27 | Contrast this to the more standard centralized development models used by commercial software |
| 28 | companies where a finite set of developers produces a product for sale using a defined set |
| 29 | of procedures that ultimately result in an end product whose architecture and source material |
| 30 | are closed to the public. |
| 31 | </para> |
| 32 | |
| 33 | <para> |
| 34 | Open source projects conceptually have differing concurrent agendas, approaches, and production. |
| 35 | These facets of the development process can come from anyone in the public (community) that has a |
| 36 | stake in the software project. |
| 37 | The open source environment contains new copyright, licensing, domain, and consumer issues |
| 38 | that differ from the more traditional development environment. |
| 39 | In an open source environment, the end product, source material, and documentation are |
| 40 | all available to the public at no cost. |
| 41 | </para> |
| 42 | |
| 43 | <para> |
| 44 | A benchmark example of an open source project is the Linux kernel, which was initially conceived |
| 45 | and created by Finnish computer science student Linus Torvalds in 1991. |
| 46 | Conversely, a good example of a non-open source project is the |
| 47 | <trademark class='registered'>Windows</trademark> family of operating |
| 48 | systems developed by <trademark class='registered'>Microsoft</trademark> Corporation. |
| 49 | </para> |
| 50 | |
| 51 | <para> |
| 52 | Wikipedia has a good historical description of the Open Source Philosophy |
| 53 | <ulink url='http://en.wikipedia.org/wiki/Open_source'>here</ulink>. |
| 54 | You can also find helpful information on how to participate in the Linux Community |
| 55 | <ulink url='http://ldn.linuxfoundation.org/book/how-participate-linux-community'>here</ulink>. |
| 56 | </para> |
| 57 | </section> |
| 58 | |
| 59 | <section id="usingpoky-changes-collaborate"> |
| 60 | <title>Using the Yocto Project in a Team Environment</title> |
| 61 | |
| 62 | <para> |
| 63 | It might not be immediately clear how you can use the Yocto |
| 64 | Project in a team environment, or scale it for a large team of |
| 65 | developers. |
| 66 | One of the strengths of the Yocto Project is that it is extremely |
| 67 | flexible. |
| 68 | Thus, you can adapt it to many different use cases and scenarios. |
| 69 | However, these characteristics can cause a struggle if you are trying |
| 70 | to create a working setup that scales across a large team. |
| 71 | </para> |
| 72 | |
| 73 | <para> |
| 74 | To help with these types of situations, this section presents |
| 75 | some of the project's most successful experiences, |
| 76 | practices, solutions, and available technologies that work well. |
| 77 | Keep in mind, the information here is a starting point. |
| 78 | You can build off it and customize it to fit any |
| 79 | particular working environment and set of practices. |
| 80 | </para> |
| 81 | |
| 82 | <section id='best-practices-system-configurations'> |
| 83 | <title>System Configurations</title> |
| 84 | |
| 85 | <para> |
| 86 | Systems across a large team should meet the needs of |
| 87 | two types of developers: those working on the contents of the |
| 88 | operating system image itself and those developing applications. |
| 89 | Regardless of the type of developer, their workstations must |
| 90 | be both reasonably powerful and run Linux. |
| 91 | </para> |
| 92 | |
| 93 | <section id='best-practices-application-development'> |
| 94 | <title>Application Development</title> |
| 95 | |
| 96 | <para> |
| 97 | For developers who mainly do application level work |
| 98 | on top of an existing software stack, |
| 99 | here are some practices that work best: |
| 100 | <itemizedlist> |
| 101 | <listitem><para>Use a pre-built toolchain that |
| 102 | contains the software stack itself. |
| 103 | Then, develop the application code on top of the |
| 104 | stack. |
| 105 | This method works well for small numbers of relatively |
| 106 | isolated applications.</para></listitem> |
| 107 | <listitem><para>When possible, use the Yocto Project |
| 108 | plug-in for the <trademark class='trade'>Eclipse</trademark> IDE |
| 109 | and other pieces of Application Development |
| 110 | Technology (ADT). |
| 111 | For more information, see the |
| 112 | "<link linkend='application-development-workflow'>Application |
| 113 | Development Workflow</link>" section as well as the |
| 114 | <ulink url='&YOCTO_DOCS_ADT_URL;'>Yocto Project Application Developer's Guide</ulink>. |
| 115 | </para></listitem> |
| 116 | <listitem><para>Keep your cross-development toolchains |
| 117 | updated. |
| 118 | You can do this through provisioning either as new |
| 119 | toolchain downloads or as updates through a package |
| 120 | update mechanism using <filename>opkg</filename> |
| 121 | to provide updates to an existing toolchain. |
| 122 | The exact mechanics of how and when to do this are a |
| 123 | question for local policy.</para></listitem> |
| 124 | <listitem><para>Use multiple toolchains installed locally |
| 125 | into different locations to allow development across |
| 126 | versions.</para></listitem> |
| 127 | </itemizedlist> |
| 128 | </para> |
| 129 | </section> |
| 130 | |
| 131 | <section id='best-practices-core-system-development'> |
| 132 | <title>Core System Development</title> |
| 133 | |
| 134 | <para> |
| 135 | For core system development, it is often best to have the |
| 136 | build system itself available on the developer workstations |
| 137 | so developers can run their own builds and directly |
| 138 | rebuild the software stack. |
| 139 | You should keep the core system unchanged as much as |
| 140 | possible and do your work in layers on top of the core system. |
| 141 | Doing so gives you a greater level of portability when |
| 142 | upgrading to new versions of the core system or Board |
| 143 | Support Packages (BSPs). |
| 144 | You can share layers amongst the developers of a particular |
| 145 | project and contain the policy configuration that defines |
| 146 | the project. |
| 147 | </para> |
| 148 | |
| 149 | <para> |
| 150 | Aside from the previous best practices, there exists a number |
| 151 | of tips and tricks that can help speed up core development |
| 152 | projects: |
| 153 | <itemizedlist> |
| 154 | <listitem><para>Use a |
| 155 | <ulink url='&YOCTO_DOCS_REF_URL;#shared-state-cache'>Shared State Cache</ulink> |
| 156 | (sstate) among groups of developers who are on a |
| 157 | fast network. |
| 158 | The best way to share sstate is through a |
| 159 | Network File System (NFS) share. |
| 160 | The first user to build a given component for the |
| 161 | first time contributes that object to the sstate, |
| 162 | while subsequent builds from other developers then |
| 163 | reuse the object rather than rebuild it themselves. |
| 164 | </para> |
| 165 | <para>Although it is possible to use other protocols for the |
| 166 | sstate such as HTTP and FTP, you should avoid these. |
| 167 | Using HTTP limits the sstate to read-only and |
| 168 | FTP provides poor performance. |
| 169 | </para></listitem> |
| 170 | <listitem><para>Have autobuilders contribute to the sstate |
| 171 | pool similarly to how the developer workstations |
| 172 | contribute. |
| 173 | For information, see the |
| 174 | "<link linkend='best-practices-autobuilders'>Autobuilders</link>" |
| 175 | section.</para></listitem> |
| 176 | <listitem><para>Build stand-alone tarballs that contain |
| 177 | "missing" system requirements if for some reason |
| 178 | developer workstations do not meet minimum system |
| 179 | requirements such as latest Python versions, |
| 180 | <filename>chrpath</filename>, or other tools. |
| 181 | You can install and relocate the tarball exactly as you |
| 182 | would the usual cross-development toolchain so that |
| 183 | all developers can meet minimum version requirements |
| 184 | on most distributions.</para></listitem> |
| 185 | <listitem><para>Use a small number of shared, |
| 186 | high performance systems for testing purposes |
| 187 | (e.g. dual, six-core Xeons with 24 Gbytes of RAM |
| 188 | and plenty of disk space). |
| 189 | Developers can use these systems for wider, more |
| 190 | extensive testing while they continue to develop |
| 191 | locally using their primary development system. |
| 192 | </para></listitem> |
| 193 | <listitem><para>Enable the PR Service when package feeds |
| 194 | need to be incremental with continually increasing |
| 195 | <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'>PR</ulink> |
| 196 | values. |
| 197 | Typically, this situation occurs when you use or |
| 198 | publish package feeds and use a shared state. |
| 199 | You should enable the PR Service for all users who |
| 200 | use the shared state pool. |
| 201 | For more information on the PR Service, see the |
| 202 | "<link linkend='working-with-a-pr-service'>Working With a PR Service</link>". |
| 203 | </para></listitem> |
| 204 | </itemizedlist> |
| 205 | </para> |
| 206 | </section> |
| 207 | </section> |
| 208 | |
| 209 | <section id='best-practices-source-control-management'> |
| 210 | <title>Source Control Management (SCM)</title> |
| 211 | |
| 212 | <para> |
| 213 | Keeping your |
| 214 | <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink> |
| 215 | and any software you are developing under the |
| 216 | control of an SCM system that is compatible |
| 217 | with the OpenEmbedded build system is advisable. |
| 218 | Of the SCMs BitBake supports, the |
| 219 | Yocto Project team strongly recommends using |
| 220 | <link linkend='git'>Git</link>. |
| 221 | Git is a distributed system that is easy to backup, |
| 222 | allows you to work remotely, and then connects back to the |
| 223 | infrastructure. |
| 224 | <note> |
| 225 | For information about BitBake, see the |
| 226 | <ulink url='&YOCTO_DOCS_BB_URL;'>BitBake User Manual</ulink>. |
| 227 | </note> |
| 228 | </para> |
| 229 | |
| 230 | <para> |
| 231 | It is relatively easy to set up Git services and create |
| 232 | infrastructure like |
| 233 | <ulink url='&YOCTO_GIT_URL;'>http://git.yoctoproject.org</ulink>, |
| 234 | which is based on server software called |
| 235 | <filename>gitolite</filename> with <filename>cgit</filename> |
| 236 | being used to generate the web interface that lets you view the |
| 237 | repositories. |
| 238 | The <filename>gitolite</filename> software identifies users |
| 239 | using SSH keys and allows branch-based |
| 240 | access controls to repositories that you can control as little |
| 241 | or as much as necessary. |
| 242 | </para> |
| 243 | |
| 244 | <note> |
| 245 | The setup of these services is beyond the scope of this manual. |
| 246 | However, sites such as these exist that describe how to perform |
| 247 | setup: |
| 248 | <itemizedlist> |
| 249 | <listitem><para><ulink url='http://git-scm.com/book/ch4-8.html'>Git documentation</ulink>: |
| 250 | Describes how to install <filename>gitolite</filename> |
| 251 | on the server.</para></listitem> |
| 252 | <listitem><para><ulink url='http://sitaramc.github.com/gitolite/master-toc.html'>The <filename>gitolite</filename> master index</ulink>: |
| 253 | All topics for <filename>gitolite</filename>. |
| 254 | </para></listitem> |
| 255 | <listitem><para><ulink url='https://git.wiki.kernel.org/index.php/Interfaces,_frontends,_and_tools'>Interfaces, frontends, and tools</ulink>: |
| 256 | Documentation on how to create interfaces and frontends |
| 257 | for Git.</para></listitem> |
| 258 | </itemizedlist> |
| 259 | </note> |
| 260 | </section> |
| 261 | |
| 262 | <section id='best-practices-autobuilders'> |
| 263 | <title>Autobuilders</title> |
| 264 | |
| 265 | <para> |
| 266 | Autobuilders are often the core of a development project. |
| 267 | It is here that changes from individual developers are brought |
| 268 | together and centrally tested and subsequent decisions about |
| 269 | releases can be made. |
| 270 | Autobuilders also allow for "continuous integration" style |
| 271 | testing of software components and regression identification |
| 272 | and tracking. |
| 273 | </para> |
| 274 | |
| 275 | <para> |
| 276 | See "<ulink url='http://autobuilder.yoctoproject.org'>Yocto Project Autobuilder</ulink>" |
| 277 | for more information and links to buildbot. |
| 278 | The Yocto Project team has found this implementation |
| 279 | works well in this role. |
| 280 | A public example of this is the Yocto Project |
| 281 | Autobuilders, which we use to test the overall health of the |
| 282 | project. |
| 283 | </para> |
| 284 | |
| 285 | <para> |
| 286 | The features of this system are: |
| 287 | <itemizedlist> |
| 288 | <listitem><para>Highlights when commits break the build. |
| 289 | </para></listitem> |
| 290 | <listitem><para>Populates an sstate cache from which |
| 291 | developers can pull rather than requiring local |
| 292 | builds.</para></listitem> |
| 293 | <listitem><para>Allows commit hook triggers, |
| 294 | which trigger builds when commits are made. |
| 295 | </para></listitem> |
| 296 | <listitem><para>Allows triggering of automated image booting |
| 297 | and testing under the QuickEMUlator (QEMU). |
| 298 | </para></listitem> |
| 299 | <listitem><para>Supports incremental build testing and |
| 300 | from-scratch builds.</para></listitem> |
| 301 | <listitem><para>Shares output that allows developer |
| 302 | testing and historical regression investigation. |
| 303 | </para></listitem> |
| 304 | <listitem><para>Creates output that can be used for releases. |
| 305 | </para></listitem> |
| 306 | <listitem><para>Allows scheduling of builds so that resources |
| 307 | can be used efficiently.</para></listitem> |
| 308 | </itemizedlist> |
| 309 | </para> |
| 310 | </section> |
| 311 | |
| 312 | <section id='best-practices-policies-and-change-flow'> |
| 313 | <title>Policies and Change Flow</title> |
| 314 | |
| 315 | <para> |
| 316 | The Yocto Project itself uses a hierarchical structure and a |
| 317 | pull model. |
| 318 | Scripts exist to create and send pull requests |
| 319 | (i.e. <filename>create-pull-request</filename> and |
| 320 | <filename>send-pull-request</filename>). |
| 321 | This model is in line with other open source projects where |
| 322 | maintainers are responsible for specific areas of the project |
| 323 | and a single maintainer handles the final "top-of-tree" merges. |
| 324 | </para> |
| 325 | |
| 326 | <note> |
| 327 | You can also use a more collective push model. |
| 328 | The <filename>gitolite</filename> software supports both the |
| 329 | push and pull models quite easily. |
| 330 | </note> |
| 331 | |
| 332 | <para> |
| 333 | As with any development environment, it is important |
| 334 | to document the policy used as well as any main project |
| 335 | guidelines so they are understood by everyone. |
| 336 | It is also a good idea to have well structured |
| 337 | commit messages, which are usually a part of a project's |
| 338 | guidelines. |
| 339 | Good commit messages are essential when looking back in time and |
| 340 | trying to understand why changes were made. |
| 341 | </para> |
| 342 | |
| 343 | <para> |
| 344 | If you discover that changes are needed to the core layer of the |
| 345 | project, it is worth sharing those with the community as soon |
| 346 | as possible. |
| 347 | Chances are if you have discovered the need for changes, someone |
| 348 | else in the community needs them also. |
| 349 | </para> |
| 350 | </section> |
| 351 | |
| 352 | <section id='best-practices-summary'> |
| 353 | <title>Summary</title> |
| 354 | |
| 355 | <para> |
| 356 | This section summarizes the key recommendations described in the |
| 357 | previous sections: |
| 358 | <itemizedlist> |
| 359 | <listitem><para>Use <link linkend='git'>Git</link> |
| 360 | as the source control system.</para></listitem> |
| 361 | <listitem><para>Maintain your Metadata in layers that make sense |
| 362 | for your situation. |
| 363 | See the "<link linkend='understanding-and-creating-layers'>Understanding |
| 364 | and Creating Layers</link>" section for more information on |
| 365 | layers.</para></listitem> |
| 366 | <listitem><para> |
| 367 | Separate the project's Metadata and code by using |
| 368 | separate Git repositories. |
| 369 | See the |
| 370 | "<link linkend='yocto-project-repositories'>Yocto Project Source Repositories</link>" |
| 371 | section for information on these repositories. |
| 372 | See the |
| 373 | "<link linkend='getting-setup'>Getting Set Up</link>" |
| 374 | section for information on how to set up local Git |
| 375 | repositories for related upstream Yocto Project |
| 376 | Git repositories. |
| 377 | </para></listitem> |
| 378 | <listitem><para>Set up the directory for the shared state cache |
| 379 | (<ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink>) |
| 380 | where it makes sense. |
| 381 | For example, set up the sstate cache on a system used |
| 382 | by developers in the same organization and share the |
| 383 | same source directories on their machines. |
| 384 | </para></listitem> |
| 385 | <listitem><para>Set up an Autobuilder and have it populate the |
| 386 | sstate cache and source directories.</para></listitem> |
| 387 | <listitem><para>The Yocto Project community encourages you |
| 388 | to send patches to the project to fix bugs or add features. |
| 389 | If you do submit patches, follow the project commit |
| 390 | guidelines for writing good commit messages. |
| 391 | See the "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" |
| 392 | section.</para></listitem> |
| 393 | <listitem><para>Send changes to the core sooner than later |
| 394 | as others are likely to run into the same issues. |
| 395 | For some guidance on mailing lists to use, see the list in the |
| 396 | "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" |
| 397 | section. |
| 398 | For a description of the available mailing lists, see the |
| 399 | "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>" |
| 400 | section in the Yocto Project Reference Manual. |
| 401 | </para></listitem> |
| 402 | </itemizedlist> |
| 403 | </para> |
| 404 | </section> |
| 405 | </section> |
| 406 | |
| 407 | <section id='yocto-project-repositories'> |
| 408 | <title>Yocto Project Source Repositories</title> |
| 409 | |
| 410 | <para> |
| 411 | The Yocto Project team maintains complete source repositories for all |
| 412 | Yocto Project files at |
| 413 | <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'></ulink>. |
| 414 | This web-based source code browser is organized into categories by |
| 415 | function such as IDE Plugins, Matchbox, Poky, Yocto Linux Kernel, and |
| 416 | so forth. |
| 417 | From the interface, you can click on any particular item in the "Name" |
| 418 | column and see the URL at the bottom of the page that you need to clone |
| 419 | a Git repository for that particular item. |
| 420 | Having a local Git repository of the |
| 421 | <link linkend='source-directory'>Source Directory</link>, which is |
| 422 | usually named "poky", allows |
| 423 | you to make changes, contribute to the history, and ultimately enhance |
| 424 | the Yocto Project's tools, Board Support Packages, and so forth. |
| 425 | </para> |
| 426 | |
| 427 | <para> |
| 428 | For any supported release of Yocto Project, you can also go to the |
| 429 | <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink> and |
| 430 | select the "Downloads" tab and get a released tarball of the |
| 431 | <filename>poky</filename> repository or any supported BSP tarballs. |
| 432 | Unpacking these tarballs gives you a snapshot of the released |
| 433 | files. |
| 434 | <note><title>Notes</title> |
| 435 | <itemizedlist> |
| 436 | <listitem><para> |
| 437 | The recommended method for setting up the Yocto Project |
| 438 | <link linkend='source-directory'>Source Directory</link> |
| 439 | and the files for supported BSPs |
| 440 | (e.g., <filename>meta-intel</filename>) is to use |
| 441 | <link linkend='git'>Git</link> to create a local copy of |
| 442 | the upstream repositories. |
| 443 | </para></listitem> |
| 444 | <listitem><para> |
| 445 | Be sure to always work in matching branches for both |
| 446 | the selected BSP repository and the |
| 447 | <link linkend='source-directory'>Source Directory</link> |
| 448 | (i.e. <filename>poky</filename>) repository. |
| 449 | For example, if you have checked out the "master" branch |
| 450 | of <filename>poky</filename> and you are going to use |
| 451 | <filename>meta-intel</filename>, be sure to checkout the |
| 452 | "master" branch of <filename>meta-intel</filename>. |
| 453 | </para></listitem> |
| 454 | </itemizedlist> |
| 455 | </note> |
| 456 | </para> |
| 457 | |
| 458 | <para> |
| 459 | In summary, here is where you can get the project files needed for development: |
| 460 | <itemizedlist> |
| 461 | <listitem><para id='source-repositories'><emphasis><ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'>Source Repositories:</ulink></emphasis> |
| 462 | This area contains IDE Plugins, Matchbox, Poky, Poky Support, Tools, Yocto Linux Kernel, and Yocto |
| 463 | Metadata Layers. |
| 464 | You can create local copies of Git repositories for each of these areas.</para> |
| 465 | <para> |
| 466 | <imagedata fileref="figures/source-repos.png" align="center" width="6in" depth="4in" /> |
| 467 | </para></listitem> |
| 468 | <listitem><para><anchor id='index-downloads' /><emphasis><ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink></emphasis> |
| 469 | This is an index of releases such as |
| 470 | the <trademark class='trade'>Eclipse</trademark> |
| 471 | Yocto Plug-in, miscellaneous support, Poky, Pseudo, installers for cross-development toolchains, |
| 472 | and all released versions of Yocto Project in the form of images or tarballs. |
| 473 | Downloading and extracting these files does not produce a local copy of the |
| 474 | Git repository but rather a snapshot of a particular release or image.</para> |
| 475 | <para> |
| 476 | <imagedata fileref="figures/index-downloads.png" align="center" width="6in" depth="3.5in" /> |
| 477 | </para></listitem> |
| 478 | <listitem><para><emphasis>"Downloads" page for the |
| 479 | <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>:</emphasis> |
| 480 | Access this page by going to the website and then selecting |
| 481 | the "Downloads" tab. |
| 482 | This page allows you to download any Yocto Project |
| 483 | release or Board Support Package (BSP) in tarball form. |
| 484 | The tarballs are similar to those found in the |
| 485 | <ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink> area.</para> |
| 486 | <para> |
| 487 | <imagedata fileref="figures/yp-download.png" align="center" width="6in" depth="4in" /> |
| 488 | </para></listitem> |
| 489 | </itemizedlist> |
| 490 | </para> |
| 491 | </section> |
| 492 | |
| 493 | <section id='yocto-project-terms'> |
| 494 | <title>Yocto Project Terms</title> |
| 495 | |
| 496 | <para> |
| 497 | Following is a list of terms and definitions users new to the Yocto Project development |
| 498 | environment might find helpful. |
| 499 | While some of these terms are universal, the list includes them just in case: |
| 500 | <itemizedlist> |
| 501 | <listitem><para><emphasis>Append Files:</emphasis> Files that append build information to |
| 502 | a recipe file. |
| 503 | Append files are known as BitBake append files and <filename>.bbappend</filename> files. |
| 504 | The OpenEmbedded build system expects every append file to have a corresponding |
| 505 | recipe (<filename>.bb</filename>) file. |
| 506 | Furthermore, the append file and corresponding recipe file |
| 507 | must use the same root filename. |
| 508 | The filenames can differ only in the file type suffix used (e.g. |
| 509 | <filename>formfactor_0.0.bb</filename> and <filename>formfactor_0.0.bbappend</filename>). |
| 510 | </para> |
| 511 | <para>Information in append files extends or overrides the |
| 512 | information in the similarly-named recipe file. |
| 513 | For an example of an append file in use, see the |
| 514 | "<link linkend='using-bbappend-files'>Using .bbappend Files</link>" section. |
| 515 | <note> |
| 516 | Append files can also use wildcard patterns in their version numbers |
| 517 | so they can be applied to more than one version of the underlying recipe file. |
| 518 | </note> |
| 519 | </para></listitem> |
| 520 | <listitem><para id='bitbake-term'><emphasis>BitBake:</emphasis> |
| 521 | The task executor and scheduler used by the OpenEmbedded build |
| 522 | system to build images. |
| 523 | For more information on BitBake, see the |
| 524 | <ulink url='&YOCTO_DOCS_BB_URL;'>BitBake User Manual</ulink>. |
| 525 | </para></listitem> |
| 526 | <listitem> |
| 527 | <para id='build-directory'><emphasis>Build Directory:</emphasis> |
| 528 | This term refers to the area used by the OpenEmbedded build |
| 529 | system for builds. |
| 530 | The area is created when you <filename>source</filename> the |
| 531 | setup environment script that is found in the Source Directory |
| 532 | (i.e. <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink> |
| 533 | or |
| 534 | <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>). |
| 535 | The <ulink url='&YOCTO_DOCS_REF_URL;#var-TOPDIR'><filename>TOPDIR</filename></ulink> |
| 536 | variable points to the Build Directory.</para> |
| 537 | |
| 538 | <para> |
| 539 | You have a lot of flexibility when creating the Build |
| 540 | Directory. |
| 541 | Following are some examples that show how to create the |
| 542 | directory. |
| 543 | The examples assume your |
| 544 | <link linkend='source-directory'>Source Directory</link> is |
| 545 | named <filename>poky</filename>: |
| 546 | <itemizedlist> |
| 547 | <listitem><para>Create the Build Directory inside your |
| 548 | Source Directory and let the name of the Build |
| 549 | Directory default to <filename>build</filename>: |
| 550 | <literallayout class='monospaced'> |
| 551 | $ cd $HOME/poky |
| 552 | $ source &OE_INIT_FILE; |
| 553 | </literallayout></para></listitem> |
| 554 | <listitem><para>Create the Build Directory inside your |
| 555 | home directory and specifically name it |
| 556 | <filename>test-builds</filename>: |
| 557 | <literallayout class='monospaced'> |
| 558 | $ cd $HOME |
| 559 | $ source poky/&OE_INIT_FILE; test-builds |
| 560 | </literallayout></para></listitem> |
| 561 | <listitem><para> |
| 562 | Provide a directory path and |
| 563 | specifically name the Build Directory. |
| 564 | Any intermediate folders in the pathname must |
| 565 | exist. |
| 566 | This next example creates a Build Directory named |
| 567 | <filename>YP-&POKYVERSION;</filename> |
| 568 | in your home directory within the existing |
| 569 | directory <filename>mybuilds</filename>: |
| 570 | <literallayout class='monospaced'> |
| 571 | $cd $HOME |
| 572 | $ source $HOME/poky/&OE_INIT_FILE; $HOME/mybuilds/YP-&POKYVERSION; |
| 573 | </literallayout></para></listitem> |
| 574 | </itemizedlist> |
| 575 | <note> |
| 576 | By default, the Build Directory contains |
| 577 | <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>, |
| 578 | which is a temporary directory the build system uses for |
| 579 | its work. |
| 580 | <filename>TMPDIR</filename> cannot be under NFS. |
| 581 | Thus, by default, the Build Directory cannot be under NFS. |
| 582 | However, if you need the Build Directory to be under NFS, |
| 583 | you can set this up by setting <filename>TMPDIR</filename> |
| 584 | in your <filename>local.conf</filename> file |
| 585 | to use a local drive. |
| 586 | Doing so effectively separates <filename>TMPDIR</filename> |
| 587 | from <filename>TOPDIR</filename>, which is the Build |
| 588 | Directory. |
| 589 | </note> |
| 590 | </para></listitem> |
| 591 | <listitem><para><emphasis>Classes:</emphasis> Files that provide for logic encapsulation |
| 592 | and inheritance so that commonly used patterns can be defined once and then easily used |
| 593 | in multiple recipes. |
| 594 | For reference information on the Yocto Project classes, see the |
| 595 | "<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes'>Classes</ulink>" chapter of the |
| 596 | Yocto Project Reference Manual. |
| 597 | Class files end with the <filename>.bbclass</filename> filename extension. |
| 598 | </para></listitem> |
| 599 | <listitem><para><emphasis>Configuration File:</emphasis> |
| 600 | Configuration information in various <filename>.conf</filename> |
| 601 | files provides global definitions of variables. |
| 602 | The <filename>conf/local.conf</filename> configuration file in |
| 603 | the |
| 604 | <link linkend='build-directory'>Build Directory</link> |
| 605 | contains user-defined variables that affect every build. |
| 606 | The <filename>meta-yocto/conf/distro/poky.conf</filename> |
| 607 | configuration file defines Yocto "distro" configuration |
| 608 | variables used only when building with this policy. |
| 609 | Machine configuration files, which |
| 610 | are located throughout the |
| 611 | <link linkend='source-directory'>Source Directory</link>, define |
| 612 | variables for specific hardware and are only used when building |
| 613 | for that target (e.g. the |
| 614 | <filename>machine/beaglebone.conf</filename> configuration |
| 615 | file defines variables for the Texas Instruments ARM Cortex-A8 |
| 616 | development board). |
| 617 | Configuration files end with a <filename>.conf</filename> |
| 618 | filename extension. |
| 619 | </para></listitem> |
| 620 | <listitem><para id='cross-development-toolchain'> |
| 621 | <emphasis>Cross-Development Toolchain:</emphasis> |
| 622 | In general, a cross-development toolchain is a collection of |
| 623 | software development tools and utilities that run on one |
| 624 | architecture and allow you to develop software for a |
| 625 | different, or targeted, architecture. |
| 626 | These toolchains contain cross-compilers, linkers, and |
| 627 | debuggers that are specific to the target architecture. |
| 628 | </para> |
| 629 | |
| 630 | <para>The Yocto Project supports two different cross-development |
| 631 | toolchains: |
| 632 | <itemizedlist> |
| 633 | <listitem><para>A toolchain only used by and within |
| 634 | BitBake when building an image for a target |
| 635 | architecture.</para></listitem> |
| 636 | <listitem><para>A relocatable toolchain used outside of |
| 637 | BitBake by developers when developing applications |
| 638 | that will run on a targeted device. |
| 639 | Sometimes this relocatable cross-development |
| 640 | toolchain is referred to as the meta-toolchain. |
| 641 | </para></listitem> |
| 642 | </itemizedlist> |
| 643 | </para> |
| 644 | |
| 645 | <para> |
| 646 | Creation of these toolchains is simple and automated. |
| 647 | For information on toolchain concepts as they apply to the |
| 648 | Yocto Project, see the |
| 649 | "<ulink url='&YOCTO_DOCS_REF_URL;#cross-development-toolchain-generation'>Cross-Development Toolchain Generation</ulink>" |
| 650 | section in the Yocto Project Reference Manual. |
| 651 | You can also find more information on using the |
| 652 | relocatable toolchain in the |
| 653 | <ulink url='&YOCTO_DOCS_ADT_URL;'>Yocto Project |
| 654 | Application Developer's Guide</ulink>. |
| 655 | </para></listitem> |
| 656 | <listitem><para><emphasis>Image:</emphasis> |
| 657 | An image is an artifact of the BitBake build process given |
| 658 | a collection of recipes and related Metadata. |
| 659 | Images are the binary output that run on specific hardware or |
| 660 | QEMU and are used for specific use-cases. |
| 661 | For a list of the supported image types that the Yocto Project provides, see the |
| 662 | "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>" |
| 663 | chapter in the Yocto Project Reference Manual.</para></listitem> |
| 664 | <listitem><para id='layer'><emphasis>Layer:</emphasis> A collection of recipes representing the core, |
| 665 | a BSP, or an application stack. |
| 666 | For a discussion specifically on BSP Layers, see the |
| 667 | "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>" |
| 668 | section in the Yocto Project Board Support Packages (BSP) |
| 669 | Developer's Guide.</para></listitem> |
| 670 | <listitem><para id='meta-toolchain'><emphasis>Meta-Toolchain:</emphasis> |
| 671 | A term sometimes used for |
| 672 | <link linkend='cross-development-toolchain'>Cross-Development Toolchain</link>. |
| 673 | </para></listitem> |
| 674 | <listitem><para id='metadata'><emphasis>Metadata:</emphasis> |
| 675 | The files that BitBake parses when building an image. |
| 676 | In general, Metadata includes recipes, classes, and |
| 677 | configuration files. |
| 678 | In the context of the kernel ("kernel Metadata"), |
| 679 | it refers to Metadata in the <filename>meta</filename> |
| 680 | branches of the kernel source Git repositories. |
| 681 | </para></listitem> |
| 682 | <listitem><para id='oe-core'><emphasis>OE-Core:</emphasis> A core set of Metadata originating |
| 683 | with OpenEmbedded (OE) that is shared between OE and the Yocto Project. |
| 684 | This Metadata is found in the <filename>meta</filename> directory of the |
| 685 | <link linkend='source-directory'>Source Directory</link>.</para></listitem> |
| 686 | <listitem><para id='build-system-term'><emphasis>OpenEmbedded Build System:</emphasis> |
| 687 | The build system specific to the Yocto Project. |
| 688 | The OpenEmbedded build system is based on another project known |
| 689 | as "Poky", which uses |
| 690 | <link linkend='bitbake-term'>BitBake</link> as the task |
| 691 | executor. |
| 692 | Throughout the Yocto Project documentation set, the |
| 693 | OpenEmbedded build system is sometimes referred to simply |
| 694 | as "the build system". |
| 695 | If other build systems, such as a host or target build system |
| 696 | are referenced, the documentation clearly states the |
| 697 | difference. |
| 698 | <note> |
| 699 | For some historical information about Poky, see the |
| 700 | <link linkend='poky'>Poky</link> term. |
| 701 | </note> |
| 702 | </para></listitem> |
| 703 | <listitem><para><emphasis>Package:</emphasis> |
| 704 | In the context of the Yocto Project, this term refers to a |
| 705 | recipe's packaged output produced by BitBake (i.e. a |
| 706 | "baked recipe"). |
| 707 | A package is generally the compiled binaries produced from the |
| 708 | recipe's sources. |
| 709 | You "bake" something by running it through BitBake.</para> |
| 710 | <para>It is worth noting that the term "package" can, in general, have subtle |
| 711 | meanings. For example, the packages referred to in the |
| 712 | "<ulink url='&YOCTO_DOCS_QS_URL;#packages'>The Build Host Packages</ulink>" section are |
| 713 | compiled binaries that, when installed, add functionality to your Linux |
| 714 | distribution.</para> |
| 715 | <para>Another point worth noting is that historically within the Yocto Project, |
| 716 | recipes were referred to as packages - thus, the existence of several BitBake |
| 717 | variables that are seemingly mis-named, |
| 718 | (e.g. <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>, |
| 719 | <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>, and |
| 720 | <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>). |
| 721 | </para></listitem> |
| 722 | <listitem><para><emphasis>Package Groups:</emphasis> |
| 723 | Arbitrary groups of software Recipes. |
| 724 | You use package groups to hold recipes that, when built, |
| 725 | usually accomplish a single task. |
| 726 | For example, a package group could contain the recipes for a |
| 727 | company’s proprietary or value-add software. |
| 728 | Or, the package group could contain the recipes that enable |
| 729 | graphics. |
| 730 | A package group is really just another recipe. |
| 731 | Because package group files are recipes, they end with the |
| 732 | <filename>.bb</filename> filename extension.</para></listitem> |
| 733 | <listitem><para id='poky'><emphasis>Poky:</emphasis> |
| 734 | The term "poky" can mean several things. |
| 735 | In its most general sense, it is an open-source |
| 736 | project that was initially developed by OpenedHand. |
| 737 | With OpenedHand, poky was developed off of the existing |
| 738 | OpenEmbedded build system becoming a commercially |
| 739 | supportable build system for embedded Linux. |
| 740 | After Intel Corporation acquired OpenedHand, the |
| 741 | project poky became the basis for the Yocto Project's |
| 742 | build system.</para> |
| 743 | <para>Within the Yocto Project source repositories, |
| 744 | <filename>poky</filename> exists as a separate Git |
| 745 | repository you can clone to yield a local copy on your |
| 746 | host system. |
| 747 | Thus, "poky" can refer to the local copy of the Source |
| 748 | Directory used for development within the Yocto |
| 749 | Project.</para> |
| 750 | <para>Finally, "poky" can refer to the default |
| 751 | <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink> |
| 752 | (i.e. distribution) created when you use the Yocto |
| 753 | Project in conjunction with the |
| 754 | <filename>poky</filename> repository to build an image. |
| 755 | </para></listitem> |
| 756 | <listitem><para><emphasis>Recipe:</emphasis> |
| 757 | A set of instructions for building packages. |
| 758 | A recipe describes where you get source code, which patches |
| 759 | to apply, how to configure the source, how to compile it and so on. |
| 760 | Recipes also describe dependencies for libraries or for other |
| 761 | recipes. |
| 762 | Recipes represent the logical unit of execution, the software |
| 763 | to build, the images to build, and use the |
| 764 | <filename>.bb</filename> file extension. |
| 765 | </para></listitem> |
| 766 | <listitem> |
| 767 | <para id='source-directory'><emphasis>Source Directory:</emphasis> |
| 768 | This term refers to the directory structure created as a result |
| 769 | of creating a local copy of the <filename>poky</filename> Git |
| 770 | repository <filename>git://git.yoctoproject.org/poky</filename> |
| 771 | or expanding a released <filename>poky</filename> tarball. |
| 772 | <note> |
| 773 | Creating a local copy of the <filename>poky</filename> |
| 774 | Git repository is the recommended method for setting up |
| 775 | your Source Directory. |
| 776 | </note> |
| 777 | Sometimes you might hear the term "poky directory" used to refer |
| 778 | to this directory structure. |
| 779 | <note> |
| 780 | The OpenEmbedded build system does not support file or |
| 781 | directory names that contain spaces. |
| 782 | Be sure that the Source Directory you use does not contain |
| 783 | these types of names. |
| 784 | </note></para> |
| 785 | |
| 786 | <para>The Source Directory contains BitBake, Documentation, |
| 787 | Metadata and other files that all support the Yocto Project. |
| 788 | Consequently, you must have the Source Directory in place on |
| 789 | your development system in order to do any development using |
| 790 | the Yocto Project.</para> |
| 791 | |
| 792 | <para>When you create a local copy of the Git repository, you |
| 793 | can name the repository anything you like. |
| 794 | Throughout much of the documentation, "poky" |
| 795 | is used as the name of the top-level folder of the local copy of |
| 796 | the poky Git repository. |
| 797 | So, for example, cloning the <filename>poky</filename> Git |
| 798 | repository results in a local Git repository whose top-level |
| 799 | folder is also named "poky".</para> |
| 800 | |
| 801 | <para>While it is not recommended that you use tarball expansion |
| 802 | to set up the Source Directory, if you do, the top-level |
| 803 | directory name of the Source Directory is derived from the |
| 804 | Yocto Project release tarball. |
| 805 | For example, downloading and unpacking |
| 806 | <filename>&YOCTO_POKY_TARBALL;</filename> results in a |
| 807 | Source Directory whose root folder is named |
| 808 | <filename>&YOCTO_POKY;</filename>.</para> |
| 809 | |
| 810 | <para>It is important to understand the differences between the |
| 811 | Source Directory created by unpacking a released tarball as |
| 812 | compared to cloning |
| 813 | <filename>git://git.yoctoproject.org/poky</filename>. |
| 814 | When you unpack a tarball, you have an exact copy of the files |
| 815 | based on the time of release - a fixed release point. |
| 816 | Any changes you make to your local files in the Source Directory |
| 817 | are on top of the release and will remain local only. |
| 818 | On the other hand, when you clone the <filename>poky</filename> |
| 819 | Git repository, you have an active development repository with |
| 820 | access to the upstream repository's branches and tags. |
| 821 | In this case, any local changes you make to the local |
| 822 | Source Directory can be later applied to active development |
| 823 | branches of the upstream <filename>poky</filename> Git |
| 824 | repository.</para> |
| 825 | |
| 826 | <para>For more information on concepts related to Git |
| 827 | repositories, branches, and tags, see the |
| 828 | "<link linkend='repositories-tags-and-branches'>Repositories, Tags, and Branches</link>" |
| 829 | section.</para></listitem> |
| 830 | <listitem><para><emphasis>Task:</emphasis> |
| 831 | A unit of execution for BitBake (e.g. |
| 832 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink>, |
| 833 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-fetch'><filename>do_fetch</filename></ulink>, |
| 834 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink>, |
| 835 | and so forth). |
| 836 | </para></listitem> |
| 837 | <listitem><para><emphasis>Upstream:</emphasis> A reference to source code or repositories |
| 838 | that are not local to the development system but located in a master area that is controlled |
| 839 | by the maintainer of the source code. |
| 840 | For example, in order for a developer to work on a particular piece of code, they need to |
| 841 | first get a copy of it from an "upstream" source.</para></listitem> |
| 842 | </itemizedlist> |
| 843 | </para> |
| 844 | </section> |
| 845 | |
| 846 | <section id='licensing'> |
| 847 | <title>Licensing</title> |
| 848 | |
| 849 | <para> |
| 850 | Because open source projects are open to the public, they have different licensing structures in place. |
| 851 | License evolution for both Open Source and Free Software has an interesting history. |
| 852 | If you are interested in this history, you can find basic information here: |
| 853 | <itemizedlist> |
| 854 | <listitem><para><ulink url='http://en.wikipedia.org/wiki/Open-source_license'>Open source license history</ulink> |
| 855 | </para></listitem> |
| 856 | <listitem><para><ulink url='http://en.wikipedia.org/wiki/Free_software_license'>Free software license |
| 857 | history</ulink></para></listitem> |
| 858 | </itemizedlist> |
| 859 | </para> |
| 860 | |
| 861 | <para> |
| 862 | In general, the Yocto Project is broadly licensed under the Massachusetts Institute of Technology |
| 863 | (MIT) License. |
| 864 | MIT licensing permits the reuse of software within proprietary software as long as the |
| 865 | license is distributed with that software. |
| 866 | MIT is also compatible with the GNU General Public License (GPL). |
| 867 | Patches to the Yocto Project follow the upstream licensing scheme. |
| 868 | You can find information on the MIT license |
| 869 | <ulink url='http://www.opensource.org/licenses/mit-license.php'>here</ulink>. |
| 870 | You can find information on the GNU GPL <ulink url='http://www.opensource.org/licenses/LGPL-3.0'> |
| 871 | here</ulink>. |
| 872 | </para> |
| 873 | |
| 874 | <para> |
| 875 | When you build an image using the Yocto Project, the build process uses a |
| 876 | known list of licenses to ensure compliance. |
| 877 | You can find this list in the |
| 878 | <link linkend='source-directory'>Source Directory</link> at |
| 879 | <filename>meta/files/common-licenses</filename>. |
| 880 | Once the build completes, the list of all licenses found and used during that build are |
| 881 | kept in the |
| 882 | <link linkend='build-directory'>Build Directory</link> at |
| 883 | <filename>tmp/deploy/licenses</filename>. |
| 884 | </para> |
| 885 | |
| 886 | <para> |
| 887 | If a module requires a license that is not in the base list, the build process |
| 888 | generates a warning during the build. |
| 889 | These tools make it easier for a developer to be certain of the licenses with which |
| 890 | their shipped products must comply. |
| 891 | However, even with these tools it is still up to the developer to resolve potential licensing issues. |
| 892 | </para> |
| 893 | |
| 894 | <para> |
| 895 | The base list of licenses used by the build process is a combination of the Software Package |
| 896 | Data Exchange (SPDX) list and the Open Source Initiative (OSI) projects. |
| 897 | <ulink url='http://spdx.org'>SPDX Group</ulink> is a working group of the Linux Foundation |
| 898 | that maintains a specification |
| 899 | for a standard format for communicating the components, licenses, and copyrights |
| 900 | associated with a software package. |
| 901 | <ulink url='http://opensource.org'>OSI</ulink> is a corporation dedicated to the Open Source |
| 902 | Definition and the effort for reviewing and approving licenses that |
| 903 | conform to the Open Source Definition (OSD). |
| 904 | </para> |
| 905 | |
| 906 | <para> |
| 907 | You can find a list of the combined SPDX and OSI licenses that the |
| 908 | Yocto Project uses in the |
| 909 | <filename>meta/files/common-licenses</filename> directory in your |
| 910 | <link linkend='source-directory'>Source Directory</link>. |
| 911 | </para> |
| 912 | |
| 913 | <para> |
| 914 | For information that can help you maintain compliance with various |
| 915 | open source licensing during the lifecycle of a product created using |
| 916 | the Yocto Project, see the |
| 917 | "<link linkend='maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Product's Lifecycle</link>" |
| 918 | section. |
| 919 | </para> |
| 920 | </section> |
| 921 | |
| 922 | <section id='git'> |
| 923 | <title>Git</title> |
| 924 | |
| 925 | <para> |
| 926 | The Yocto Project makes extensive use of Git, |
| 927 | which is a free, open source distributed version control system. |
| 928 | Git supports distributed development, non-linear development, and can handle large projects. |
| 929 | It is best that you have some fundamental understanding of how Git tracks projects and |
| 930 | how to work with Git if you are going to use the Yocto Project for development. |
| 931 | This section provides a quick overview of how Git works and provides you with a summary |
| 932 | of some essential Git commands. |
| 933 | </para> |
| 934 | |
| 935 | <para> |
| 936 | For more information on Git, see |
| 937 | <ulink url='http://git-scm.com/documentation'></ulink>. |
| 938 | If you need to download Git, go to <ulink url='http://git-scm.com/download'></ulink>. |
| 939 | </para> |
| 940 | |
| 941 | <section id='repositories-tags-and-branches'> |
| 942 | <title>Repositories, Tags, and Branches</title> |
| 943 | |
| 944 | <para> |
| 945 | As mentioned earlier in the section |
| 946 | "<link linkend='yocto-project-repositories'>Yocto Project Source Repositories</link>", |
| 947 | the Yocto Project maintains source repositories at |
| 948 | <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink>. |
| 949 | If you look at this web-interface of the repositories, each item is a separate |
| 950 | Git repository. |
| 951 | </para> |
| 952 | |
| 953 | <para> |
| 954 | Git repositories use branching techniques that track content change (not files) |
| 955 | within a project (e.g. a new feature or updated documentation). |
| 956 | Creating a tree-like structure based on project divergence allows for excellent historical |
| 957 | information over the life of a project. |
| 958 | This methodology also allows for an environment from which you can do lots of |
| 959 | local experimentation on projects as you develop changes or new features. |
| 960 | </para> |
| 961 | |
| 962 | <para> |
| 963 | A Git repository represents all development efforts for a given project. |
| 964 | For example, the Git repository <filename>poky</filename> contains all changes |
| 965 | and developments for Poky over the course of its entire life. |
| 966 | That means that all changes that make up all releases are captured. |
| 967 | The repository maintains a complete history of changes. |
| 968 | </para> |
| 969 | |
| 970 | <para> |
| 971 | You can create a local copy of any repository by "cloning" it with the Git |
| 972 | <filename>clone</filename> command. |
| 973 | When you clone a Git repository, you end up with an identical copy of the |
| 974 | repository on your development system. |
| 975 | Once you have a local copy of a repository, you can take steps to develop locally. |
| 976 | For examples on how to clone Git repositories, see the |
| 977 | "<link linkend='getting-setup'>Getting Set Up</link>" section. |
| 978 | </para> |
| 979 | |
| 980 | <para> |
| 981 | It is important to understand that Git tracks content change and |
| 982 | not files. |
| 983 | Git uses "branches" to organize different development efforts. |
| 984 | For example, the <filename>poky</filename> repository has |
| 985 | several branches that include the current |
| 986 | <filename>&DISTRO_NAME;</filename> branch, the |
| 987 | <filename>master</filename> branch, and many branches for past |
| 988 | Yocto Project releases. |
| 989 | You can see all the branches by going to |
| 990 | <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and |
| 991 | clicking on the |
| 992 | <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/heads'>[...]</ulink></filename> |
| 993 | link beneath the "Branch" heading. |
| 994 | </para> |
| 995 | |
| 996 | <para> |
| 997 | Each of these branches represents a specific area of development. |
| 998 | The <filename>master</filename> branch represents the current or most recent |
| 999 | development. |
| 1000 | All other branches represent offshoots of the <filename>master</filename> |
| 1001 | branch. |
| 1002 | </para> |
| 1003 | |
| 1004 | <para> |
| 1005 | When you create a local copy of a Git repository, the copy has the same set |
| 1006 | of branches as the original. |
| 1007 | This means you can use Git to create a local working area (also called a branch) |
| 1008 | that tracks a specific development branch from the source Git repository. |
| 1009 | in other words, you can define your local Git environment to work on any development |
| 1010 | branch in the repository. |
| 1011 | To help illustrate, here is a set of commands that creates a local copy of the |
| 1012 | <filename>poky</filename> Git repository and then creates and checks out a local |
| 1013 | Git branch that tracks the Yocto Project &DISTRO; Release (&DISTRO_NAME;) development: |
| 1014 | <literallayout class='monospaced'> |
| 1015 | $ cd ~ |
| 1016 | $ git clone git://git.yoctoproject.org/poky |
| 1017 | $ cd poky |
| 1018 | $ git checkout -b &DISTRO_NAME; origin/&DISTRO_NAME; |
| 1019 | </literallayout> |
| 1020 | In this example, the name of the top-level directory of your local |
| 1021 | <link linkend='source-directory'>Source Directory</link> |
| 1022 | is "poky" and the name of that local working area (local branch) |
| 1023 | you just created and checked out is "&DISTRO_NAME;". |
| 1024 | The files in your local repository now reflect the same files that |
| 1025 | are in the "&DISTRO_NAME;" development branch of the |
| 1026 | Yocto Project's "poky" upstream repository. |
| 1027 | It is important to understand that when you create and checkout a |
| 1028 | local working branch based on a branch name, |
| 1029 | your local environment matches the "tip" of that development branch |
| 1030 | at the time you created your local branch, which could be |
| 1031 | different from the files at the time of a similarly named release. |
| 1032 | In other words, creating and checking out a local branch based on |
| 1033 | the "&DISTRO_NAME;" branch name is not the same as |
| 1034 | cloning and checking out the "master" branch. |
| 1035 | Keep reading to see how you create a local snapshot of a Yocto |
| 1036 | Project Release. |
| 1037 | </para> |
| 1038 | |
| 1039 | <para> |
| 1040 | Git uses "tags" to mark specific changes in a repository. |
| 1041 | Typically, a tag is used to mark a special point such as the final |
| 1042 | change before a project is released. |
| 1043 | You can see the tags used with the <filename>poky</filename> Git |
| 1044 | repository by going to |
| 1045 | <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and |
| 1046 | clicking on the |
| 1047 | <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/tags'>[...]</ulink></filename> |
| 1048 | link beneath the "Tag" heading. |
| 1049 | </para> |
| 1050 | |
| 1051 | <para> |
| 1052 | Some key tags are <filename>dylan-9.0.4</filename>, |
| 1053 | <filename>dora-10.0.4</filename>, <filename>daisy-11.0.2</filename>, |
| 1054 | <filename>dizzy-12.0.0</filename>, and |
| 1055 | <filename>&DISTRO_NAME;-&POKYVERSION;</filename>. |
| 1056 | These tags represent Yocto Project releases. |
| 1057 | </para> |
| 1058 | |
| 1059 | <para> |
| 1060 | When you create a local copy of the Git repository, you also have access to all the |
| 1061 | tags. |
| 1062 | Similar to branches, you can create and checkout a local working Git branch based |
| 1063 | on a tag name. |
| 1064 | When you do this, you get a snapshot of the Git repository that reflects |
| 1065 | the state of the files when the change was made associated with that tag. |
| 1066 | The most common use is to checkout a working branch that matches a specific |
| 1067 | Yocto Project release. |
| 1068 | Here is an example: |
| 1069 | <literallayout class='monospaced'> |
| 1070 | $ cd ~ |
| 1071 | $ git clone git://git.yoctoproject.org/poky |
| 1072 | $ cd poky |
| 1073 | $ git checkout -b my-&DISTRO_NAME;-&POKYVERSION; &DISTRO_NAME;-&POKYVERSION; |
| 1074 | </literallayout> |
| 1075 | In this example, the name of the top-level directory of your local Yocto Project |
| 1076 | Files Git repository is <filename>poky</filename>. |
| 1077 | And, the name of the local branch you have created and checked out is |
| 1078 | <filename>my-&DISTRO_NAME;-&POKYVERSION;</filename>. |
| 1079 | The files in your repository now exactly match the Yocto Project &DISTRO; |
| 1080 | Release tag (<filename>&DISTRO_NAME;-&POKYVERSION;</filename>). |
| 1081 | It is important to understand that when you create and checkout a local |
| 1082 | working branch based on a tag, your environment matches a specific point |
| 1083 | in time and not the entire development branch. |
| 1084 | </para> |
| 1085 | </section> |
| 1086 | |
| 1087 | <section id='basic-commands'> |
| 1088 | <title>Basic Commands</title> |
| 1089 | |
| 1090 | <para> |
| 1091 | Git has an extensive set of commands that lets you manage changes and perform |
| 1092 | collaboration over the life of a project. |
| 1093 | Conveniently though, you can manage with a small set of basic operations and workflows |
| 1094 | once you understand the basic philosophy behind Git. |
| 1095 | You do not have to be an expert in Git to be functional. |
| 1096 | A good place to look for instruction on a minimal set of Git commands is |
| 1097 | <ulink url='http://git-scm.com/documentation'>here</ulink>. |
| 1098 | If you need to download Git, you can do so |
| 1099 | <ulink url='http://git-scm.com/download'>here</ulink>, although |
| 1100 | any reasonably current Linux distribution should already have an |
| 1101 | installable package for Git. |
| 1102 | </para> |
| 1103 | |
| 1104 | <para> |
| 1105 | If you do not know much about Git, you should educate |
| 1106 | yourself by visiting the links previously mentioned. |
| 1107 | </para> |
| 1108 | |
| 1109 | <para> |
| 1110 | The following list briefly describes some basic Git operations as a way to get started. |
| 1111 | As with any set of commands, this list (in most cases) simply shows the base command and |
| 1112 | omits the many arguments they support. |
| 1113 | See the Git documentation for complete descriptions and strategies on how to use these commands: |
| 1114 | <itemizedlist> |
| 1115 | <listitem><para><emphasis><filename>git init</filename>:</emphasis> Initializes an empty Git repository. |
| 1116 | You cannot use Git commands unless you have a <filename>.git</filename> repository.</para></listitem> |
| 1117 | <listitem><para><emphasis><filename>git clone</filename>:</emphasis> |
| 1118 | Creates a local clone of a Git repository. |
| 1119 | During collaboration, this command allows you to create a |
| 1120 | local Git repository that is on equal footing with a fellow |
| 1121 | developer’s Git repository. |
| 1122 | </para></listitem> |
| 1123 | <listitem><para><emphasis><filename>git add</filename>:</emphasis> Stages updated file contents |
| 1124 | to the index that |
| 1125 | Git uses to track changes. |
| 1126 | You must stage all files that have changed before you can commit them.</para></listitem> |
| 1127 | <listitem><para><emphasis><filename>git commit</filename>:</emphasis> Creates a "commit" that documents |
| 1128 | the changes you made. |
| 1129 | Commits are used for historical purposes, for determining if a maintainer of a project |
| 1130 | will allow the change, and for ultimately pushing the change from your local Git repository |
| 1131 | into the project’s upstream (or master) repository.</para></listitem> |
| 1132 | <listitem><para><emphasis><filename>git status</filename>:</emphasis> Reports any modified files that |
| 1133 | possibly need to be staged and committed.</para></listitem> |
| 1134 | <listitem><para><emphasis><filename>git checkout <branch-name></filename>:</emphasis> Changes |
| 1135 | your working branch. |
| 1136 | This command is analogous to "cd".</para></listitem> |
| 1137 | <listitem><para><emphasis><filename>git checkout –b <working-branch></filename>:</emphasis> Creates |
| 1138 | a working branch on your local machine where you can isolate work. |
| 1139 | It is a good idea to use local branches when adding specific features or changes. |
| 1140 | This way if you do not like what you have done you can easily get rid of the work.</para></listitem> |
| 1141 | <listitem><para><emphasis><filename>git branch</filename>:</emphasis> Reports |
| 1142 | existing local branches and |
| 1143 | tells you the branch in which you are currently working.</para></listitem> |
| 1144 | <listitem><para><emphasis><filename>git branch -D <branch-name></filename>:</emphasis> |
| 1145 | Deletes an existing local branch. |
| 1146 | You need to be in a local branch other than the one you are deleting |
| 1147 | in order to delete <filename><branch-name></filename>.</para></listitem> |
| 1148 | <listitem><para><emphasis><filename>git pull</filename>:</emphasis> Retrieves information |
| 1149 | from an upstream Git |
| 1150 | repository and places it in your local Git repository. |
| 1151 | You use this command to make sure you are synchronized with the repository |
| 1152 | from which you are basing changes (.e.g. the master branch).</para></listitem> |
| 1153 | <listitem><para><emphasis><filename>git push</filename>:</emphasis> |
| 1154 | Sends all your committed local changes to an upstream Git |
| 1155 | repository (e.g. a contribution repository). |
| 1156 | The maintainer of the project draws from these repositories |
| 1157 | when adding changes to the project’s master repository or |
| 1158 | other development branch. |
| 1159 | </para></listitem> |
| 1160 | <listitem><para><emphasis><filename>git merge</filename>:</emphasis> Combines or adds changes from one |
| 1161 | local branch of your repository with another branch. |
| 1162 | When you create a local Git repository, the default branch is named "master". |
| 1163 | A typical workflow is to create a temporary branch for isolated work, make and commit your |
| 1164 | changes, switch to your local master branch, merge the changes from the temporary branch into the |
| 1165 | local master branch, and then delete the temporary branch.</para></listitem> |
| 1166 | <listitem><para><emphasis><filename>git cherry-pick</filename>:</emphasis> Choose and apply specific |
| 1167 | commits from one branch into another branch. |
| 1168 | There are times when you might not be able to merge all the changes in one branch with |
| 1169 | another but need to pick out certain ones.</para></listitem> |
| 1170 | <listitem><para><emphasis><filename>gitk</filename>:</emphasis> Provides a GUI view of the branches |
| 1171 | and changes in your local Git repository. |
| 1172 | This command is a good way to graphically see where things have diverged in your |
| 1173 | local repository.</para></listitem> |
| 1174 | <listitem><para><emphasis><filename>git log</filename>:</emphasis> Reports a history of your changes to the |
| 1175 | repository.</para></listitem> |
| 1176 | <listitem><para><emphasis><filename>git diff</filename>:</emphasis> Displays line-by-line differences |
| 1177 | between your local working files and the same files in the upstream Git repository that your |
| 1178 | branch currently tracks.</para></listitem> |
| 1179 | </itemizedlist> |
| 1180 | </para> |
| 1181 | </section> |
| 1182 | </section> |
| 1183 | |
| 1184 | <section id='workflows'> |
| 1185 | <title>Workflows</title> |
| 1186 | |
| 1187 | <para> |
| 1188 | This section provides some overview on workflows using Git. |
| 1189 | In particular, the information covers basic practices that describe roles and actions in a |
| 1190 | collaborative development environment. |
| 1191 | Again, if you are familiar with this type of development environment, you might want to just |
| 1192 | skip this section. |
| 1193 | </para> |
| 1194 | |
| 1195 | <para> |
| 1196 | The Yocto Project files are maintained using Git in a "master" branch whose Git history |
| 1197 | tracks every change and whose structure provides branches for all diverging functionality. |
| 1198 | Although there is no need to use Git, many open source projects do so. |
| 1199 | For the Yocto Project, a key individual called the "maintainer" is responsible for the "master" |
| 1200 | branch of a given Git repository. |
| 1201 | The "master" branch is the “upstream” repository where the final builds of the project occur. |
| 1202 | The maintainer is responsible for accepting changes from other developers and for |
| 1203 | organizing the underlying branch structure to reflect release strategies and so forth. |
| 1204 | <note>For information on finding out who is responsible for (maintains) |
| 1205 | a particular area of code, see the |
| 1206 | "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" |
| 1207 | section. |
| 1208 | </note> |
| 1209 | </para> |
| 1210 | |
| 1211 | <para> |
| 1212 | The project also has an upstream contribution Git repository named |
| 1213 | <filename>poky-contrib</filename>. |
| 1214 | You can see all the branches in this repository using the web interface |
| 1215 | of the |
| 1216 | <ulink url='&YOCTO_GIT_URL;'>Source Repositories</ulink> organized |
| 1217 | within the "Poky Support" area. |
| 1218 | These branches temporarily hold changes to the project that have been |
| 1219 | submitted or committed by the Yocto Project development team and by |
| 1220 | community members who contribute to the project. |
| 1221 | The maintainer determines if the changes are qualified to be moved |
| 1222 | from the "contrib" branches into the "master" branch of the Git |
| 1223 | repository. |
| 1224 | </para> |
| 1225 | |
| 1226 | <para> |
| 1227 | Developers (including contributing community members) create and maintain cloned repositories |
| 1228 | of the upstream "master" branch. |
| 1229 | These repositories are local to their development platforms and are used to develop changes. |
| 1230 | When a developer is satisfied with a particular feature or change, they "push" the changes |
| 1231 | to the appropriate "contrib" repository. |
| 1232 | </para> |
| 1233 | |
| 1234 | <para> |
| 1235 | Developers are responsible for keeping their local repository up-to-date with "master". |
| 1236 | They are also responsible for straightening out any conflicts that might arise within files |
| 1237 | that are being worked on simultaneously by more than one person. |
| 1238 | All this work is done locally on the developer’s machines before anything is pushed to a |
| 1239 | "contrib" area and examined at the maintainer’s level. |
| 1240 | </para> |
| 1241 | |
| 1242 | <para> |
| 1243 | A somewhat formal method exists by which developers commit changes and push them into the |
| 1244 | "contrib" area and subsequently request that the maintainer include them into "master" |
| 1245 | This process is called “submitting a patch” or "submitting a change." |
| 1246 | For information on submitting patches and changes, see the |
| 1247 | "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" section. |
| 1248 | </para> |
| 1249 | |
| 1250 | <para> |
| 1251 | To summarize the environment: a single point of entry exists for |
| 1252 | changes into the project’s "master" branch of the Git repository, |
| 1253 | which is controlled by the project’s maintainer. |
| 1254 | And, a set of developers exist who independently develop, test, and |
| 1255 | submit changes to "contrib" areas for the maintainer to examine. |
| 1256 | The maintainer then chooses which changes are going to become a |
| 1257 | permanent part of the project. |
| 1258 | </para> |
| 1259 | |
| 1260 | <para> |
| 1261 | <imagedata fileref="figures/git-workflow.png" width="6in" depth="3in" align="left" scalefit="1" /> |
| 1262 | </para> |
| 1263 | |
| 1264 | <para> |
| 1265 | While each development environment is unique, there are some best practices or methods |
| 1266 | that help development run smoothly. |
| 1267 | The following list describes some of these practices. |
| 1268 | For more information about Git workflows, see the workflow topics in the |
| 1269 | <ulink url='http://book.git-scm.com'>Git Community Book</ulink>. |
| 1270 | <itemizedlist> |
| 1271 | <listitem><para><emphasis>Make Small Changes:</emphasis> It is best to keep the changes you commit |
| 1272 | small as compared to bundling many disparate changes into a single commit. |
| 1273 | This practice not only keeps things manageable but also allows the maintainer |
| 1274 | to more easily include or refuse changes.</para> |
| 1275 | <para>It is also good practice to leave the repository in a state that allows you to |
| 1276 | still successfully build your project. In other words, do not commit half of a feature, |
| 1277 | then add the other half as a separate, later commit. |
| 1278 | Each commit should take you from one buildable project state to another |
| 1279 | buildable state.</para></listitem> |
| 1280 | <listitem><para><emphasis>Use Branches Liberally:</emphasis> It is very easy to create, use, and |
| 1281 | delete local branches in your working Git repository. |
| 1282 | You can name these branches anything you like. |
| 1283 | It is helpful to give them names associated with the particular feature or change |
| 1284 | on which you are working. |
| 1285 | Once you are done with a feature or change and have merged it |
| 1286 | into your local master branch, simply discard the temporary |
| 1287 | branch.</para></listitem> |
| 1288 | <listitem><para><emphasis>Merge Changes:</emphasis> The <filename>git merge</filename> |
| 1289 | command allows you to take the |
| 1290 | changes from one branch and fold them into another branch. |
| 1291 | This process is especially helpful when more than a single developer might be working |
| 1292 | on different parts of the same feature. |
| 1293 | Merging changes also automatically identifies any collisions or "conflicts" |
| 1294 | that might happen as a result of the same lines of code being altered by two different |
| 1295 | developers.</para></listitem> |
| 1296 | <listitem><para><emphasis>Manage Branches:</emphasis> Because branches are easy to use, you should |
| 1297 | use a system where branches indicate varying levels of code readiness. |
| 1298 | For example, you can have a "work" branch to develop in, a "test" branch where the code or |
| 1299 | change is tested, a "stage" branch where changes are ready to be committed, and so forth. |
| 1300 | As your project develops, you can merge code across the branches to reflect ever-increasing |
| 1301 | stable states of the development.</para></listitem> |
| 1302 | <listitem><para><emphasis>Use Push and Pull:</emphasis> The push-pull workflow is based on the |
| 1303 | concept of developers "pushing" local commits to a remote repository, which is |
| 1304 | usually a contribution repository. |
| 1305 | This workflow is also based on developers "pulling" known states of the project down into their |
| 1306 | local development repositories. |
| 1307 | The workflow easily allows you to pull changes submitted by other developers from the |
| 1308 | upstream repository into your work area ensuring that you have the most recent software |
| 1309 | on which to develop. |
| 1310 | The Yocto Project has two scripts named <filename>create-pull-request</filename> and |
| 1311 | <filename>send-pull-request</filename> that ship with the release to facilitate this |
| 1312 | workflow. |
| 1313 | You can find these scripts in the <filename>scripts</filename> |
| 1314 | folder of the |
| 1315 | <link linkend='source-directory'>Source Directory</link>. |
| 1316 | For information on how to use these scripts, see the |
| 1317 | "<link linkend='pushing-a-change-upstream'>Using Scripts to Push a Change Upstream and Request a Pull</link>" section. |
| 1318 | </para></listitem> |
| 1319 | <listitem><para><emphasis>Patch Workflow:</emphasis> This workflow allows you to notify the |
| 1320 | maintainer through an email that you have a change (or patch) you would like considered |
| 1321 | for the "master" branch of the Git repository. |
| 1322 | To send this type of change, you format the patch and then send the email using the Git commands |
| 1323 | <filename>git format-patch</filename> and <filename>git send-email</filename>. |
| 1324 | For information on how to use these scripts, see the |
| 1325 | "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" |
| 1326 | section. |
| 1327 | </para></listitem> |
| 1328 | </itemizedlist> |
| 1329 | </para> |
| 1330 | </section> |
| 1331 | |
| 1332 | <section id='tracking-bugs'> |
| 1333 | <title>Tracking Bugs</title> |
| 1334 | |
| 1335 | <para> |
| 1336 | The Yocto Project uses its own implementation of |
| 1337 | <ulink url='http://www.bugzilla.org/about/'>Bugzilla</ulink> to track bugs. |
| 1338 | Implementations of Bugzilla work well for group development because they track bugs and code |
| 1339 | changes, can be used to communicate changes and problems with developers, can be used to |
| 1340 | submit and review patches, and can be used to manage quality assurance. |
| 1341 | The home page for the Yocto Project implementation of Bugzilla is |
| 1342 | <ulink url='&YOCTO_BUGZILLA_URL;'>&YOCTO_BUGZILLA_URL;</ulink>. |
| 1343 | </para> |
| 1344 | |
| 1345 | <para> |
| 1346 | Sometimes it is helpful to submit, investigate, or track a bug against the Yocto Project itself |
| 1347 | such as when discovering an issue with some component of the build system that acts contrary |
| 1348 | to the documentation or your expectations. |
| 1349 | Following is the general procedure for submitting a new bug using the Yocto Project |
| 1350 | Bugzilla. |
| 1351 | You can find more information on defect management, bug tracking, and feature request |
| 1352 | processes all accomplished through the Yocto Project Bugzilla on the |
| 1353 | <ulink url='&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking'>wiki page</ulink>. |
| 1354 | <orderedlist> |
| 1355 | <listitem><para>Always use the Yocto Project implementation of Bugzilla to submit |
| 1356 | a bug.</para></listitem> |
| 1357 | <listitem><para>When submitting a new bug, be sure to choose the appropriate |
| 1358 | Classification, Product, and Component for which the issue was found. |
| 1359 | Defects for the Yocto Project fall into one of seven classifications: |
| 1360 | Yocto Project Components, Infrastructure, Build System & Metadata, |
| 1361 | Documentation, QA/Testing, Runtime and Hardware. |
| 1362 | Each of these Classifications break down into multiple Products and, in some |
| 1363 | cases, multiple Components.</para></listitem> |
| 1364 | <listitem><para>Use the bug form to choose the correct Hardware and Architecture |
| 1365 | for which the bug applies.</para></listitem> |
| 1366 | <listitem><para>Indicate the Yocto Project version you were using when the issue |
| 1367 | occurred.</para></listitem> |
| 1368 | <listitem><para>Be sure to indicate the Severity of the bug. |
| 1369 | Severity communicates how the bug impacted your work.</para></listitem> |
| 1370 | <listitem><para>Select the appropriate "Documentation change" item |
| 1371 | for the bug. |
| 1372 | Fixing a bug may or may not affect the Yocto Project |
| 1373 | documentation.</para></listitem> |
| 1374 | <listitem><para>Provide a brief summary of the issue. |
| 1375 | Try to limit your summary to just a line or two and be sure to capture the |
| 1376 | essence of the issue.</para></listitem> |
| 1377 | <listitem><para>Provide a detailed description of the issue. |
| 1378 | You should provide as much detail as you can about the context, behavior, output, |
| 1379 | and so forth that surrounds the issue. |
| 1380 | You can even attach supporting files for output from logs by |
| 1381 | using the "Add an attachment" button.</para></listitem> |
| 1382 | <listitem><para>Be sure to copy the appropriate people in the |
| 1383 | "CC List" for the bug. |
| 1384 | See the "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" |
| 1385 | section for information about finding out who is responsible |
| 1386 | for code.</para></listitem> |
| 1387 | <listitem><para>Submit the bug by clicking the "Submit Bug" button.</para></listitem> |
| 1388 | </orderedlist> |
| 1389 | </para> |
| 1390 | </section> |
| 1391 | |
| 1392 | <section id='how-to-submit-a-change'> |
| 1393 | <title>How to Submit a Change</title> |
| 1394 | |
| 1395 | <para> |
| 1396 | Contributions to the Yocto Project and OpenEmbedded are very welcome. |
| 1397 | Because the system is extremely configurable and flexible, we recognize that developers |
| 1398 | will want to extend, configure or optimize it for their specific uses. |
| 1399 | You should send patches to the appropriate mailing list so that they |
| 1400 | can be reviewed and merged by the appropriate maintainer. |
| 1401 | </para> |
| 1402 | |
| 1403 | <para> |
| 1404 | Before submitting any change, be sure to find out who you should be |
| 1405 | notifying. |
| 1406 | Several methods exist through which you find out who you should be copying |
| 1407 | or notifying: |
| 1408 | <itemizedlist> |
| 1409 | <listitem><para><emphasis>Maintenance File:</emphasis> |
| 1410 | Examine the <filename>maintainers.inc</filename> file, which is |
| 1411 | located in the |
| 1412 | <link linkend='source-directory'>Source Directory</link> |
| 1413 | at <filename>meta-yocto/conf/distro/include</filename>, to |
| 1414 | see who is responsible for code. |
| 1415 | </para></listitem> |
| 1416 | <listitem><para><emphasis>Board Support Package (BSP) README Files:</emphasis> |
| 1417 | For BSP maintainers of supported BSPs, you can examine |
| 1418 | individual BSP <filename>README</filename> files. |
| 1419 | In addition, some layers (such as the <filename>meta-intel</filename> layer), |
| 1420 | include a <filename>MAINTAINERS</filename> file which contains |
| 1421 | a list of all supported BSP maintainers for that layer. |
| 1422 | </para></listitem> |
| 1423 | <listitem><para><emphasis>Search by File:</emphasis> |
| 1424 | Using <link linkend='git'>Git</link>, you can enter the |
| 1425 | following command to bring up a short list of all commits |
| 1426 | against a specific file: |
| 1427 | <literallayout class='monospaced'> |
| 1428 | git shortlog -- <replaceable>filename</replaceable> |
| 1429 | </literallayout> |
| 1430 | Just provide the name of the file for which you are interested. |
| 1431 | The information returned is not ordered by history but does |
| 1432 | include a list of all committers grouped by name. |
| 1433 | From the list, you can see who is responsible for the bulk of |
| 1434 | the changes against the file. |
| 1435 | </para></listitem> |
| 1436 | </itemizedlist> |
| 1437 | </para> |
| 1438 | |
| 1439 | <para> |
| 1440 | For a list of the Yocto Project and related mailing lists, see the |
| 1441 | "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing lists</ulink>" section in |
| 1442 | the Yocto Project Reference Manual. |
| 1443 | </para> |
| 1444 | |
| 1445 | <para> |
| 1446 | Here is some guidance on which mailing list to use for what type of change: |
| 1447 | <itemizedlist> |
| 1448 | <listitem><para>For changes to the core |
| 1449 | <link linkend='metadata'>Metadata</link>, send your patch to the |
| 1450 | <ulink url='&OE_LISTS_URL;/listinfo/openembedded-core'>openembedded-core</ulink> mailing list. |
| 1451 | For example, a change to anything under the <filename>meta</filename> or |
| 1452 | <filename>scripts</filename> directories |
| 1453 | should be sent to this mailing list.</para></listitem> |
| 1454 | <listitem><para>For changes to BitBake (anything under the <filename>bitbake</filename> |
| 1455 | directory), send your patch to the |
| 1456 | <ulink url='&OE_LISTS_URL;/listinfo/bitbake-devel'>bitbake-devel</ulink> mailing list.</para></listitem> |
| 1457 | <listitem><para>For changes to <filename>meta-yocto</filename>, send your patch to the |
| 1458 | <ulink url='&YOCTO_LISTS_URL;/listinfo/poky'>poky</ulink> mailing list.</para></listitem> |
| 1459 | <listitem><para>For changes to other layers hosted on |
| 1460 | <filename>yoctoproject.org</filename> (unless the |
| 1461 | layer's documentation specifies otherwise), tools, and Yocto Project |
| 1462 | documentation, use the |
| 1463 | <ulink url='&YOCTO_LISTS_URL;/listinfo/yocto'>yocto</ulink> mailing list.</para></listitem> |
| 1464 | <listitem><para>For additional recipes that do not fit into the core Metadata, |
| 1465 | you should determine which layer the recipe should go into and submit the |
| 1466 | change in the manner recommended by the documentation (e.g. README) supplied |
| 1467 | with the layer. If in doubt, please ask on the |
| 1468 | <ulink url='&YOCTO_LISTS_URL;/listinfo/yocto'>yocto</ulink> or |
| 1469 | <ulink url='&OE_LISTS_URL;/listinfo/openembedded-devel'>openembedded-devel</ulink> |
| 1470 | mailing lists.</para></listitem> |
| 1471 | </itemizedlist> |
| 1472 | </para> |
| 1473 | |
| 1474 | <para> |
| 1475 | When you send a patch, be sure to include a "Signed-off-by:" |
| 1476 | line in the same style as required by the Linux kernel. |
| 1477 | Adding this line signifies that you, the submitter, have agreed to the Developer's Certificate of Origin 1.1 |
| 1478 | as follows: |
| 1479 | <literallayout class='monospaced'> |
| 1480 | Developer's Certificate of Origin 1.1 |
| 1481 | |
| 1482 | By making a contribution to this project, I certify that: |
| 1483 | |
| 1484 | (a) The contribution was created in whole or in part by me and I |
| 1485 | have the right to submit it under the open source license |
| 1486 | indicated in the file; or |
| 1487 | |
| 1488 | (b) The contribution is based upon previous work that, to the best |
| 1489 | of my knowledge, is covered under an appropriate open source |
| 1490 | license and I have the right under that license to submit that |
| 1491 | work with modifications, whether created in whole or in part |
| 1492 | by me, under the same open source license (unless I am |
| 1493 | permitted to submit under a different license), as indicated |
| 1494 | in the file; or |
| 1495 | |
| 1496 | (c) The contribution was provided directly to me by some other |
| 1497 | person who certified (a), (b) or (c) and I have not modified |
| 1498 | it. |
| 1499 | |
| 1500 | (d) I understand and agree that this project and the contribution |
| 1501 | are public and that a record of the contribution (including all |
| 1502 | personal information I submit with it, including my sign-off) is |
| 1503 | maintained indefinitely and may be redistributed consistent with |
| 1504 | this project or the open source license(s) involved. |
| 1505 | </literallayout> |
| 1506 | </para> |
| 1507 | |
| 1508 | <para> |
| 1509 | In a collaborative environment, it is necessary to have some sort of standard |
| 1510 | or method through which you submit changes. |
| 1511 | Otherwise, things could get quite chaotic. |
| 1512 | One general practice to follow is to make small, controlled changes. |
| 1513 | Keeping changes small and isolated aids review, makes merging/rebasing easier |
| 1514 | and keeps the change history clean when anyone needs to refer to it in future. |
| 1515 | </para> |
| 1516 | |
| 1517 | <para> |
| 1518 | When you make a commit, you must follow certain standards established by the |
| 1519 | OpenEmbedded and Yocto Project development teams. |
| 1520 | For each commit, you must provide a single-line summary of the change and you |
| 1521 | should almost always provide a more detailed description of what you did (i.e. |
| 1522 | the body of the commit message). |
| 1523 | The only exceptions for not providing a detailed description would be if your |
| 1524 | change is a simple, self-explanatory change that needs no further description |
| 1525 | beyond the summary. |
| 1526 | Here are the guidelines for composing a commit message: |
| 1527 | <itemizedlist> |
| 1528 | <listitem><para>Provide a single-line, short summary of the change. |
| 1529 | This summary is typically viewable in the "shortlist" of changes. |
| 1530 | Thus, providing something short and descriptive that gives the reader |
| 1531 | a summary of the change is useful when viewing a list of many commits. |
| 1532 | This short description should be prefixed by the recipe name (if changing a recipe), or |
| 1533 | else the short form path to the file being changed. |
| 1534 | </para></listitem> |
| 1535 | <listitem><para>For the body of the commit message, provide detailed information |
| 1536 | that describes what you changed, why you made the change, and the approach |
| 1537 | you used. It may also be helpful if you mention how you tested the change. |
| 1538 | Provide as much detail as you can in the body of the commit message. |
| 1539 | </para></listitem> |
| 1540 | <listitem><para> |
| 1541 | If the change addresses a specific bug or issue that is |
| 1542 | associated with a bug-tracking ID, include a reference to that |
| 1543 | ID in your detailed description. |
| 1544 | For example, the Yocto Project uses a specific convention for |
| 1545 | bug references - any commit that addresses a specific bug should |
| 1546 | use the following form for the detailed description: |
| 1547 | <literallayout class='monospaced'> |
| 1548 | Fixes [YOCTO #<replaceable>bug-id</replaceable>] |
| 1549 | |
| 1550 | <replaceable>detailed description of change</replaceable> |
| 1551 | </literallayout></para></listitem> |
| 1552 | Where <replaceable>bug-id</replaceable> is replaced with the |
| 1553 | specific bug ID from the Yocto Project Bugzilla instance. |
| 1554 | </itemizedlist> |
| 1555 | </para> |
| 1556 | |
| 1557 | <para> |
| 1558 | You can find more guidance on creating well-formed commit messages at this OpenEmbedded |
| 1559 | wiki page: |
| 1560 | <ulink url='&OE_HOME_URL;/wiki/Commit_Patch_Message_Guidelines'></ulink>. |
| 1561 | </para> |
| 1562 | |
| 1563 | <para> |
| 1564 | The next two sections describe general instructions for both pushing |
| 1565 | changes upstream and for submitting changes as patches. |
| 1566 | </para> |
| 1567 | |
| 1568 | <section id='pushing-a-change-upstream'> |
| 1569 | <title>Using Scripts to Push a Change Upstream and Request a Pull</title> |
| 1570 | |
| 1571 | <para> |
| 1572 | The basic flow for pushing a change to an upstream "contrib" Git repository is as follows: |
| 1573 | <itemizedlist> |
| 1574 | <listitem><para>Make your changes in your local Git repository.</para></listitem> |
| 1575 | <listitem><para>Stage your changes by using the <filename>git add</filename> |
| 1576 | command on each file you changed.</para></listitem> |
| 1577 | <listitem><para> |
| 1578 | Commit the change by using the |
| 1579 | <filename>git commit</filename> command. |
| 1580 | Be sure to provide a commit message that follows the |
| 1581 | project’s commit message standards as described earlier. |
| 1582 | </para></listitem> |
| 1583 | <listitem><para> |
| 1584 | Push the change to the upstream "contrib" repository by |
| 1585 | using the <filename>git push</filename> command. |
| 1586 | </para></listitem> |
| 1587 | <listitem><para>Notify the maintainer that you have pushed a change by making a pull |
| 1588 | request. |
| 1589 | The Yocto Project provides two scripts that conveniently let you generate and send |
| 1590 | pull requests to the Yocto Project. |
| 1591 | These scripts are <filename>create-pull-request</filename> and |
| 1592 | <filename>send-pull-request</filename>. |
| 1593 | You can find these scripts in the <filename>scripts</filename> directory |
| 1594 | within the <link linkend='source-directory'>Source Directory</link>.</para> |
| 1595 | <para>Using these scripts correctly formats the requests without introducing any |
| 1596 | whitespace or HTML formatting. |
| 1597 | The maintainer that receives your patches needs to be able to save and apply them |
| 1598 | directly from your emails. |
| 1599 | Using these scripts is the preferred method for sending patches.</para> |
| 1600 | <para>For help on using these scripts, simply provide the |
| 1601 | <filename>-h</filename> argument as follows: |
| 1602 | <literallayout class='monospaced'> |
| 1603 | $ poky/scripts/create-pull-request -h |
| 1604 | $ poky/scripts/send-pull-request -h |
| 1605 | </literallayout></para></listitem> |
| 1606 | </itemizedlist> |
| 1607 | </para> |
| 1608 | |
| 1609 | <para> |
| 1610 | You can find general Git information on how to push a change upstream in the |
| 1611 | <ulink url='http://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows'>Git Community Book</ulink>. |
| 1612 | </para> |
| 1613 | </section> |
| 1614 | |
| 1615 | <section id='submitting-a-patch'> |
| 1616 | <title>Using Email to Submit a Patch</title> |
| 1617 | |
| 1618 | <para> |
| 1619 | You can submit patches without using the <filename>create-pull-request</filename> and |
| 1620 | <filename>send-pull-request</filename> scripts described in the previous section. |
| 1621 | However, keep in mind, the preferred method is to use the scripts. |
| 1622 | </para> |
| 1623 | |
| 1624 | <para> |
| 1625 | Depending on the components changed, you need to submit the email to a specific |
| 1626 | mailing list. |
| 1627 | For some guidance on which mailing list to use, see the list in the |
| 1628 | "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" |
| 1629 | section. |
| 1630 | For a description of the available mailing lists, see the |
| 1631 | "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>" |
| 1632 | section in the Yocto Project Reference Manual. |
| 1633 | </para> |
| 1634 | |
| 1635 | <para> |
| 1636 | Here is the general procedure on how to submit a patch through email without using the |
| 1637 | scripts: |
| 1638 | <itemizedlist> |
| 1639 | <listitem><para>Make your changes in your local Git repository.</para></listitem> |
| 1640 | <listitem><para>Stage your changes by using the <filename>git add</filename> |
| 1641 | command on each file you changed.</para></listitem> |
| 1642 | <listitem><para>Commit the change by using the |
| 1643 | <filename>git commit --signoff</filename> command. |
| 1644 | Using the <filename>--signoff</filename> option identifies you as the person |
| 1645 | making the change and also satisfies the Developer's Certificate of |
| 1646 | Origin (DCO) shown earlier.</para> |
| 1647 | <para>When you form a commit, you must follow certain standards established by the |
| 1648 | Yocto Project development team. |
| 1649 | See the earlier section |
| 1650 | "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" |
| 1651 | for Yocto Project commit message standards.</para></listitem> |
| 1652 | <listitem><para>Format the commit into an email message. |
| 1653 | To format commits, use the <filename>git format-patch</filename> command. |
| 1654 | When you provide the command, you must include a revision list or a number of patches |
| 1655 | as part of the command. |
| 1656 | For example, either of these two commands takes your most |
| 1657 | recent single commit and formats it as an email message in |
| 1658 | the current directory: |
| 1659 | <literallayout class='monospaced'> |
| 1660 | $ git format-patch -1 |
| 1661 | </literallayout> |
| 1662 | or |
| 1663 | <literallayout class='monospaced'> |
| 1664 | $ git format-patch HEAD~ |
| 1665 | </literallayout></para> |
| 1666 | <para>After the command is run, the current directory contains a |
| 1667 | numbered <filename>.patch</filename> file for the commit.</para> |
| 1668 | <para>If you provide several commits as part of the command, |
| 1669 | the <filename>git format-patch</filename> command produces a |
| 1670 | series of numbered files in the current directory – one for each commit. |
| 1671 | If you have more than one patch, you should also use the |
| 1672 | <filename>--cover</filename> option with the command, which generates a |
| 1673 | cover letter as the first "patch" in the series. |
| 1674 | You can then edit the cover letter to provide a description for |
| 1675 | the series of patches. |
| 1676 | For information on the <filename>git format-patch</filename> command, |
| 1677 | see <filename>GIT_FORMAT_PATCH(1)</filename> displayed using the |
| 1678 | <filename>man git-format-patch</filename> command.</para> |
| 1679 | <note>If you are or will be a frequent contributor to the Yocto Project |
| 1680 | or to OpenEmbedded, you might consider requesting a contrib area and the |
| 1681 | necessary associated rights.</note></listitem> |
| 1682 | <listitem><para>Import the files into your mail client by using the |
| 1683 | <filename>git send-email</filename> command. |
| 1684 | <note>In order to use <filename>git send-email</filename>, you must have the |
| 1685 | the proper Git packages installed. |
| 1686 | For Ubuntu, Debian, and Fedora the package is <filename>git-email</filename>.</note></para> |
| 1687 | <para>The <filename>git send-email</filename> command sends email by using a local |
| 1688 | or remote Mail Transport Agent (MTA) such as |
| 1689 | <filename>msmtp</filename>, <filename>sendmail</filename>, or through a direct |
| 1690 | <filename>smtp</filename> configuration in your Git <filename>config</filename> |
| 1691 | file. |
| 1692 | If you are submitting patches through email only, it is very important |
| 1693 | that you submit them without any whitespace or HTML formatting that |
| 1694 | either you or your mailer introduces. |
| 1695 | The maintainer that receives your patches needs to be able to save and |
| 1696 | apply them directly from your emails. |
| 1697 | A good way to verify that what you are sending will be applicable by the |
| 1698 | maintainer is to do a dry run and send them to yourself and then |
| 1699 | save and apply them as the maintainer would.</para> |
| 1700 | <para>The <filename>git send-email</filename> command is the preferred method |
| 1701 | for sending your patches since there is no risk of compromising whitespace |
| 1702 | in the body of the message, which can occur when you use your own mail client. |
| 1703 | The command also has several options that let you |
| 1704 | specify recipients and perform further editing of the email message. |
| 1705 | For information on how to use the <filename>git send-email</filename> command, |
| 1706 | see <filename>GIT-SEND-EMAIL(1)</filename> displayed using |
| 1707 | the <filename>man git-send-email</filename> command. |
| 1708 | </para></listitem> |
| 1709 | </itemizedlist> |
| 1710 | </para> |
| 1711 | </section> |
| 1712 | </section> |
| 1713 | </chapter> |
| 1714 | <!-- |
| 1715 | vim: expandtab tw=80 ts=4 |
| 1716 | --> |