Brad Bishop | 316dfdd | 2018-06-25 12:45:53 -0400 | [diff] [blame] | 1 | <!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" |
| 2 | "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" |
| 3 | [<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > |
| 4 | |
| 5 | <chapter id='overview-development-environment'> |
| 6 | <title>The Yocto Project Development Environment</title> |
| 7 | |
| 8 | <para> |
| 9 | This chapter takes a look at the Yocto Project development |
| 10 | environment. |
| 11 | The chapter provides Yocto Project Development environment concepts that |
| 12 | help you understand how work is accomplished in an open source environment, |
| 13 | which is very different as compared to work accomplished in a closed, |
| 14 | proprietary environment. |
| 15 | </para> |
| 16 | |
| 17 | <para> |
| 18 | Specifically, this chapter addresses open source philosophy, source |
| 19 | repositories, workflows, Git, and licensing. |
| 20 | </para> |
| 21 | |
| 22 | <section id='open-source-philosophy'> |
| 23 | <title>Open Source Philosophy</title> |
| 24 | |
| 25 | <para> |
| 26 | Open source philosophy is characterized by software development |
| 27 | directed by peer production and collaboration through an active |
| 28 | community of developers. |
| 29 | Contrast this to the more standard centralized development models |
| 30 | used by commercial software companies where a finite set of developers |
| 31 | produces a product for sale using a defined set of procedures that |
| 32 | ultimately result in an end product whose architecture and source |
| 33 | material are closed to the public. |
| 34 | </para> |
| 35 | |
| 36 | <para> |
| 37 | Open source projects conceptually have differing concurrent agendas, |
| 38 | approaches, and production. |
| 39 | These facets of the development process can come from anyone in the |
| 40 | public (community) who has a stake in the software project. |
| 41 | The open source environment contains new copyright, licensing, domain, |
| 42 | and consumer issues that differ from the more traditional development |
| 43 | environment. |
| 44 | In an open source environment, the end product, source material, |
| 45 | and documentation are all available to the public at no cost. |
| 46 | </para> |
| 47 | |
| 48 | <para> |
| 49 | A benchmark example of an open source project is the Linux kernel, |
| 50 | which was initially conceived and created by Finnish computer science |
| 51 | student Linus Torvalds in 1991. |
| 52 | Conversely, a good example of a non-open source project is the |
| 53 | <trademark class='registered'>Windows</trademark> family of operating |
| 54 | systems developed by |
| 55 | <trademark class='registered'>Microsoft</trademark> Corporation. |
| 56 | </para> |
| 57 | |
| 58 | <para> |
| 59 | Wikipedia has a good historical description of the Open Source |
| 60 | Philosophy |
| 61 | <ulink url='http://en.wikipedia.org/wiki/Open_source'>here</ulink>. |
| 62 | You can also find helpful information on how to participate in the |
| 63 | Linux Community |
| 64 | <ulink url='http://ldn.linuxfoundation.org/book/how-participate-linux-community'>here</ulink>. |
| 65 | </para> |
| 66 | </section> |
| 67 | |
| 68 | <section id='gs-the-development-host'> |
| 69 | <title>The Development Host</title> |
| 70 | |
| 71 | <para> |
| 72 | A development host or |
| 73 | <ulink url='&YOCTO_DOCS_REF_URL;#hardware-build-system-term'>build host</ulink> |
| 74 | is key to using the Yocto Project. |
| 75 | Because the goal of the Yocto Project is to develop images or |
| 76 | applications that run on embedded hardware, development of those |
| 77 | images and applications generally takes place on a system not |
| 78 | intended to run the software - the development host. |
| 79 | </para> |
| 80 | |
| 81 | <para> |
| 82 | You need to set up a development host in order to use it with the |
| 83 | Yocto Project. |
| 84 | Most find that it is best to have a native Linux machine function as |
| 85 | the development host. |
| 86 | However, it is possible to use a system that does not run Linux |
| 87 | as its operating system as your development host. |
| 88 | When you have a Mac or Windows-based system, you can set it up |
| 89 | as the development host by using |
| 90 | <ulink url='https://git.yoctoproject.org/cgit/cgit.cgi/crops/about/'>CROPS</ulink>, |
| 91 | which leverages |
| 92 | <ulink url='https://www.docker.com/'>Docker Containers</ulink>. |
| 93 | Once you take the steps to set up a CROPS machine, you effectively |
| 94 | have access to a shell environment that is similar to what you see |
| 95 | when using a Linux-based development host. |
| 96 | For the steps needed to set up a system using CROPS, see the |
| 97 | "<ulink url='&YOCTO_DOCS_DEV_URL;#setting-up-to-use-crops'>Setting Up to Use CROss PlatformS (CROPS)</ulink>" |
| 98 | section in the Yocto Project Development Tasks Manual. |
| 99 | </para> |
| 100 | |
| 101 | <para> |
| 102 | If your development host is going to be a system that runs a Linux |
| 103 | distribution, steps still exist that you must take to prepare the |
| 104 | system for use with the Yocto Project. |
| 105 | You need to be sure that the Linux distribution on the system is |
| 106 | one that supports the Yocto Project. |
| 107 | You also need to be sure that the correct set of host packages are |
| 108 | installed that allow development using the Yocto Project. |
| 109 | For the steps needed to set up a development host that runs Linux, |
| 110 | see the |
| 111 | "<ulink url='&YOCTO_DOCS_DEV_URL;#setting-up-a-native-linux-host'>Setting Up a Native Linux Host</ulink>" |
| 112 | section in the Yocto Project Development Tasks Manual. |
| 113 | </para> |
| 114 | |
| 115 | <para> |
| 116 | Once your development host is set up to use the Yocto Project, |
| 117 | several methods exist for you to do work in the Yocto Project |
| 118 | environment: |
| 119 | <itemizedlist> |
| 120 | <listitem><para> |
| 121 | <emphasis>Command Lines, BitBake, and Shells:</emphasis> |
| 122 | Traditional development in the Yocto Project involves using the |
| 123 | <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>OpenEmbedded build system</ulink>, |
| 124 | which uses BitBake, in a command-line environment from a shell |
| 125 | on your development host. |
| 126 | You can accomplish this from a host that is a native Linux |
| 127 | machine or from a host that has been set up with CROPS. |
| 128 | Either way, you create, modify, and build images and |
| 129 | applications all within a shell-based environment using |
| 130 | components and tools available through your Linux distribution |
| 131 | and the Yocto Project.</para> |
| 132 | |
| 133 | <para>For a general flow of the build procedures, see the |
| 134 | "<ulink url='&YOCTO_DOCS_DEV_URL;#dev-building-a-simple-image'>Building a Simple Image</ulink>" |
| 135 | section in the Yocto Project Development Tasks Manual. |
| 136 | </para></listitem> |
| 137 | <listitem><para> |
| 138 | <emphasis>Board Support Package (BSP) Development:</emphasis> |
| 139 | Development of BSPs involves using the Yocto Project to |
| 140 | create and test layers that allow easy development of |
| 141 | images and applications targeted for specific hardware. |
| 142 | To development BSPs, you need to take some additional steps |
| 143 | beyond what was described in setting up a development host. |
| 144 | </para> |
| 145 | |
| 146 | <para>The |
| 147 | <ulink url='&YOCTO_DOCS_BSP_URL;'>Yocto Project Board Support Package (BSP) Developer's Guide</ulink> |
| 148 | provides BSP-related development information. |
| 149 | For specifics on development host preparation, see the |
| 150 | "<ulink url='&YOCTO_DOCS_BSP_URL;#preparing-your-build-host-to-work-with-bsp-layers'>Preparing Your Build Host to Work With BSP Layers</ulink>" |
| 151 | section in the Yocto Project Board Support Package (BSP) |
| 152 | Developer's Guide. |
| 153 | </para></listitem> |
| 154 | <listitem><para> |
| 155 | <emphasis>Kernel Development:</emphasis> |
| 156 | If you are going to be developing kernels using the Yocto |
| 157 | Project you likely will be using <filename>devtool</filename>. |
| 158 | A workflow using <filename>devtool</filename> makes kernel |
| 159 | development quicker by reducing iteration cycle times.</para> |
| 160 | |
| 161 | <para>The |
| 162 | <ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;'>Yocto Project Linux Kernel Development Manual</ulink> |
| 163 | provides kernel-related development information. |
| 164 | For specifics on development host preparation, see the |
| 165 | "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#preparing-the-build-host-to-work-on-the-kernel'>Preparing the Build Host to Work on the Kernel</ulink>" |
| 166 | section in the Yocto Project Linux Kernel Development Manual. |
| 167 | </para></listitem> |
| 168 | <listitem><para> |
| 169 | <emphasis>Using the <trademark class='trade'>Eclipse</trademark> IDE:</emphasis> |
| 170 | One of two Yocto Project development methods that involves an |
| 171 | interface that effectively puts the Yocto Project into the |
| 172 | background is the popular Eclipse IDE. |
| 173 | This method of development is advantageous if you are already |
| 174 | familiar with working within Eclipse. |
| 175 | Development is supported through a plugin that you install |
| 176 | onto your development host.</para> |
| 177 | |
| 178 | <para>For steps that show you how to set up your development |
| 179 | host to use the Eclipse Yocto Project plugin, see the |
| 180 | "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-eclipse-project'>Developing Applications Using <trademark class='trade'>Eclipse</trademark></ulink>" |
| 181 | Chapter in the Yocto Project Application Development and the |
| 182 | Extensible Software Development Kit (eSDK) manual. |
| 183 | </para></listitem> |
| 184 | <listitem><para> |
| 185 | <emphasis>Using Toaster:</emphasis> |
| 186 | The other Yocto Project development method that involves an |
| 187 | interface that effectively puts the Yocto Project into the |
| 188 | background is Toaster. |
| 189 | Toaster provides an interface to the OpenEmbedded build system. |
| 190 | The interface enables you to configure and run your builds. |
| 191 | Information about builds is collected and stored in a database. |
| 192 | You can use Toaster to configure and start builds on multiple |
| 193 | remote build servers.</para> |
| 194 | |
| 195 | <para>For steps that show you how to set up your development |
| 196 | host to use Toaster and on how to use Toaster in general, |
| 197 | see the |
| 198 | <ulink url='&YOCTO_DOCS_TOAST_URL;'>Toaster User Manual</ulink>. |
| 199 | </para></listitem> |
| 200 | </itemizedlist> |
| 201 | </para> |
| 202 | </section> |
| 203 | |
| 204 | <section id='yocto-project-repositories'> |
| 205 | <title>Yocto Project Source Repositories</title> |
| 206 | |
| 207 | <para> |
| 208 | The Yocto Project team maintains complete source repositories for all |
| 209 | Yocto Project files at |
| 210 | <ulink url='&YOCTO_GIT_URL;'></ulink>. |
| 211 | This web-based source code browser is organized into categories by |
| 212 | function such as IDE Plugins, Matchbox, Poky, Yocto Linux Kernel, and |
| 213 | so forth. |
| 214 | From the interface, you can click on any particular item in the "Name" |
| 215 | column and see the URL at the bottom of the page that you need to clone |
| 216 | a Git repository for that particular item. |
| 217 | Having a local Git repository of the |
| 218 | <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>, |
| 219 | which is usually named "poky", allows |
| 220 | you to make changes, contribute to the history, and ultimately enhance |
| 221 | the Yocto Project's tools, Board Support Packages, and so forth. |
| 222 | </para> |
| 223 | |
| 224 | <para> |
| 225 | For any supported release of Yocto Project, you can also go to the |
| 226 | <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink> and |
| 227 | select the "DOWNLOADS" item from the "SOFTWARE" menu and get a |
| 228 | released tarball of the <filename>poky</filename> repository, any |
| 229 | supported BSP tarball, or Yocto Project tools. |
| 230 | Unpacking these tarballs gives you a snapshot of the released |
| 231 | files. |
| 232 | <note><title>Notes</title> |
| 233 | <itemizedlist> |
| 234 | <listitem><para> |
| 235 | The recommended method for setting up the Yocto Project |
| 236 | <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> |
| 237 | and the files for supported BSPs |
| 238 | (e.g., <filename>meta-intel</filename>) is to use |
| 239 | <link linkend='git'>Git</link> to create a local copy of |
| 240 | the upstream repositories. |
| 241 | </para></listitem> |
| 242 | <listitem><para> |
| 243 | Be sure to always work in matching branches for both |
| 244 | the selected BSP repository and the Source Directory |
| 245 | (i.e. <filename>poky</filename>) repository. |
| 246 | For example, if you have checked out the "master" branch |
| 247 | of <filename>poky</filename> and you are going to use |
| 248 | <filename>meta-intel</filename>, be sure to checkout the |
| 249 | "master" branch of <filename>meta-intel</filename>. |
| 250 | </para></listitem> |
| 251 | </itemizedlist> |
| 252 | </note> |
| 253 | </para> |
| 254 | |
| 255 | <para> |
| 256 | In summary, here is where you can get the project files needed for |
| 257 | development: |
| 258 | <itemizedlist> |
| 259 | <listitem><para id='source-repositories'> |
| 260 | <emphasis> |
| 261 | <ulink url='&YOCTO_GIT_URL;'>Source Repositories:</ulink> |
| 262 | </emphasis> |
| 263 | This area contains IDE Plugins, Matchbox, Poky, Poky Support, |
| 264 | Tools, Yocto Linux Kernel, and Yocto Metadata Layers. |
| 265 | You can create local copies of Git repositories for each of |
| 266 | these areas.</para> |
| 267 | |
| 268 | <para> |
| 269 | <imagedata fileref="figures/source-repos.png" align="center" width="6in" depth="4in" /> |
| 270 | For steps on how to view and access these upstream Git |
| 271 | repositories, see the |
| 272 | "<ulink url='&YOCTO_DOCS_DEV_URL;#accessing-source-repositories'>Accessing Source Repositories</ulink>" |
| 273 | Section in the Yocto Project Development Tasks Manual. |
| 274 | </para></listitem> |
| 275 | <listitem><para><anchor id='index-downloads' /> |
| 276 | <emphasis> |
| 277 | <ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink> |
| 278 | </emphasis> |
| 279 | This is an index of releases such as |
| 280 | the <trademark class='trade'>Eclipse</trademark> |
| 281 | Yocto Plug-in, miscellaneous support, Poky, Pseudo, installers |
| 282 | for cross-development toolchains, and all released versions of |
| 283 | Yocto Project in the form of images or tarballs. |
| 284 | Downloading and extracting these files does not produce a local |
| 285 | copy of the Git repository but rather a snapshot of a |
| 286 | particular release or image.</para> |
| 287 | |
| 288 | <para> |
| 289 | <imagedata fileref="figures/index-downloads.png" align="center" width="6in" depth="3.5in" /> |
| 290 | For steps on how to view and access these files, see the |
| 291 | "<ulink url='&YOCTO_DOCS_DEV_URL;#accessing-index-of-releases'>Accessing Index of Releases</ulink>" |
| 292 | section in the Yocto Project Development Tasks Manual. |
| 293 | </para></listitem> |
| 294 | <listitem><para id='downloads-page'> |
| 295 | <emphasis>"DOWNLOADS" page for the |
| 296 | <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>: |
| 297 | </emphasis></para> |
| 298 | |
| 299 | <para>The Yocto Project website includes a "DOWNLOADS" page |
| 300 | accessible through the "SOFTWARE" menu that allows you to |
| 301 | download any Yocto Project release, tool, and Board Support |
| 302 | Package (BSP) in tarball form. |
| 303 | The tarballs are similar to those found in the |
| 304 | <ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink> |
| 305 | area.</para> |
| 306 | |
| 307 | <para> |
| 308 | <imagedata fileref="figures/yp-download.png" align="center" width="6in" depth="4in" /> |
| 309 | For steps on how to use the "DOWNLOADS" page, see the |
| 310 | "<ulink url='&YOCTO_DOCS_DEV_URL;#using-the-downloads-page'>Using the Downloads Page</ulink>" |
| 311 | section in the Yocto Project Development Tasks Manual. |
| 312 | </para></listitem> |
| 313 | </itemizedlist> |
| 314 | </para> |
| 315 | </section> |
| 316 | |
| 317 | <section id='gs-git-workflows-and-the-yocto-project'> |
| 318 | <title>Git Workflows and the Yocto Project</title> |
| 319 | |
| 320 | <para> |
| 321 | Developing using the Yocto Project likely requires the use of |
| 322 | <link linkend='git'>Git</link>. |
| 323 | Git is a free, open source distributed version control system |
| 324 | used as part of many collaborative design environments. |
| 325 | This section provides workflow concepts using the Yocto Project and |
| 326 | Git. |
| 327 | In particular, the information covers basic practices that describe |
| 328 | roles and actions in a collaborative development environment. |
| 329 | <note> |
| 330 | If you are familiar with this type of development environment, you |
| 331 | might not want to read this section. |
| 332 | </note> |
| 333 | </para> |
| 334 | |
| 335 | <para> |
| 336 | The Yocto Project files are maintained using Git in "branches" |
| 337 | whose Git histories track every change and whose structures |
| 338 | provide branches for all diverging functionality. |
| 339 | Although there is no need to use Git, many open source projects do so. |
| 340 | <para> |
| 341 | |
| 342 | </para> |
| 343 | For the Yocto Project, a key individual called the "maintainer" is |
| 344 | responsible for the integrity of the "master" branch of a given Git |
| 345 | repository. |
| 346 | The "master" branch is the “upstream” repository from which final or |
| 347 | most recent builds of a project occur. |
| 348 | The maintainer is responsible for accepting changes from other |
| 349 | developers and for organizing the underlying branch structure to |
| 350 | reflect release strategies and so forth. |
| 351 | <note> |
| 352 | For information on finding out who is responsible for (maintains) |
| 353 | a particular area of code in the Yocto Project, see the |
| 354 | "<ulink url='&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change'>Submitting a Change to the Yocto Project</ulink>" |
| 355 | section of the Yocto Project Development Tasks Manual. |
| 356 | </note> |
| 357 | </para> |
| 358 | |
| 359 | <para> |
| 360 | The Yocto Project <filename>poky</filename> Git repository also has an |
| 361 | upstream contribution Git repository named |
| 362 | <filename>poky-contrib</filename>. |
| 363 | You can see all the branches in this repository using the web interface |
| 364 | of the |
| 365 | <ulink url='&YOCTO_GIT_URL;'>Source Repositories</ulink> organized |
| 366 | within the "Poky Support" area. |
| 367 | These branches hold changes (commits) to the project that have been |
| 368 | submitted or committed by the Yocto Project development team and by |
| 369 | community members who contribute to the project. |
| 370 | The maintainer determines if the changes are qualified to be moved |
| 371 | from the "contrib" branches into the "master" branch of the Git |
| 372 | repository. |
| 373 | </para> |
| 374 | |
| 375 | <para> |
| 376 | Developers (including contributing community members) create and |
| 377 | maintain cloned repositories of upstream branches. |
| 378 | The cloned repositories are local to their development platforms and |
| 379 | are used to develop changes. |
| 380 | When a developer is satisfied with a particular feature or change, |
| 381 | they "push" the change to the appropriate "contrib" repository. |
| 382 | </para> |
| 383 | |
| 384 | <para> |
| 385 | Developers are responsible for keeping their local repository |
| 386 | up-to-date with whatever upstream branch they are working against. |
| 387 | They are also responsible for straightening out any conflicts that |
| 388 | might arise within files that are being worked on simultaneously by |
| 389 | more than one person. |
| 390 | All this work is done locally on the development host before |
| 391 | anything is pushed to a "contrib" area and examined at the maintainer’s |
| 392 | level. |
| 393 | </para> |
| 394 | |
| 395 | <para> |
| 396 | A somewhat formal method exists by which developers commit changes |
| 397 | and push them into the "contrib" area and subsequently request that |
| 398 | the maintainer include them into an upstream branch. |
| 399 | This process is called “submitting a patch” or "submitting a change." |
| 400 | For information on submitting patches and changes, see the |
| 401 | "<ulink url='&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change'>Submitting a Change to the Yocto Project</ulink>" |
| 402 | section in the Yocto Project Development Tasks Manual. |
| 403 | </para> |
| 404 | |
| 405 | <para> |
| 406 | In summary, a single point of entry |
| 407 | exists for changes into a "master" or development branch of the |
| 408 | Git repository, which is controlled by the project’s maintainer. |
| 409 | And, a set of developers exist who independently develop, test, and |
| 410 | submit changes to "contrib" areas for the maintainer to examine. |
| 411 | The maintainer then chooses which changes are going to become a |
| 412 | permanent part of the project. |
| 413 | </para> |
| 414 | |
| 415 | <para> |
| 416 | <imagedata fileref="figures/git-workflow.png" width="6in" depth="3in" align="left" scalefit="1" /> |
| 417 | </para> |
| 418 | |
| 419 | <para> |
| 420 | While each development environment is unique, there are some best |
| 421 | practices or methods that help development run smoothly. |
| 422 | The following list describes some of these practices. |
| 423 | For more information about Git workflows, see the workflow topics in |
| 424 | the |
| 425 | <ulink url='http://book.git-scm.com'>Git Community Book</ulink>. |
| 426 | <itemizedlist> |
| 427 | <listitem><para> |
| 428 | <emphasis>Make Small Changes:</emphasis> |
| 429 | It is best to keep the changes you commit small as compared to |
| 430 | bundling many disparate changes into a single commit. |
| 431 | This practice not only keeps things manageable but also allows |
| 432 | the maintainer to more easily include or refuse changes. |
| 433 | </para></listitem> |
| 434 | <listitem><para> |
| 435 | <emphasis>Make Complete Changes:</emphasis> |
| 436 | It is also good practice to leave the repository in a |
| 437 | state that allows you to still successfully build your project. |
| 438 | In other words, do not commit half of a feature, |
| 439 | then add the other half as a separate, later commit. |
| 440 | Each commit should take you from one buildable project state |
| 441 | to another buildable state. |
| 442 | </para></listitem> |
| 443 | <listitem><para> |
| 444 | <emphasis>Use Branches Liberally:</emphasis> |
| 445 | It is very easy to create, use, and delete local branches in |
| 446 | your working Git repository on the development host. |
| 447 | You can name these branches anything you like. |
| 448 | It is helpful to give them names associated with the particular |
| 449 | feature or change on which you are working. |
| 450 | Once you are done with a feature or change and have merged it |
| 451 | into your local master branch, simply discard the temporary |
| 452 | branch. |
| 453 | </para></listitem> |
| 454 | <listitem><para> |
| 455 | <emphasis>Merge Changes:</emphasis> |
| 456 | The <filename>git merge</filename> command allows you to take |
| 457 | the changes from one branch and fold them into another branch. |
| 458 | This process is especially helpful when more than a single |
| 459 | developer might be working on different parts of the same |
| 460 | feature. |
| 461 | Merging changes also automatically identifies any collisions |
| 462 | or "conflicts" that might happen as a result of the same lines |
| 463 | of code being altered by two different developers. |
| 464 | </para></listitem> |
| 465 | <listitem><para> |
| 466 | <emphasis>Manage Branches:</emphasis> |
| 467 | Because branches are easy to use, you should use a system |
| 468 | where branches indicate varying levels of code readiness. |
| 469 | For example, you can have a "work" branch to develop in, a |
| 470 | "test" branch where the code or change is tested, a "stage" |
| 471 | branch where changes are ready to be committed, and so forth. |
| 472 | As your project develops, you can merge code across the |
| 473 | branches to reflect ever-increasing stable states of the |
| 474 | development. |
| 475 | </para></listitem> |
| 476 | <listitem><para> |
| 477 | <emphasis>Use Push and Pull:</emphasis> |
| 478 | The push-pull workflow is based on the concept of developers |
| 479 | "pushing" local commits to a remote repository, which is |
| 480 | usually a contribution repository. |
| 481 | This workflow is also based on developers "pulling" known |
| 482 | states of the project down into their local development |
| 483 | repositories. |
| 484 | The workflow easily allows you to pull changes submitted by |
| 485 | other developers from the upstream repository into your |
| 486 | work area ensuring that you have the most recent software |
| 487 | on which to develop. |
| 488 | The Yocto Project has two scripts named |
| 489 | <filename>create-pull-request</filename> and |
| 490 | <filename>send-pull-request</filename> that ship with the |
| 491 | release to facilitate this workflow. |
| 492 | You can find these scripts in the <filename>scripts</filename> |
| 493 | folder of the |
| 494 | <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>. |
| 495 | For information on how to use these scripts, see the |
| 496 | "<ulink url='&YOCTO_DOCS_DEV_URL;#pushing-a-change-upstream'>Using Scripts to Push a Change Upstream and Request a Pull</ulink>" |
| 497 | section in the Yocto Project Development Tasks Manual. |
| 498 | </para></listitem> |
| 499 | <listitem><para> |
| 500 | <emphasis>Patch Workflow:</emphasis> |
| 501 | This workflow allows you to notify the maintainer through an |
| 502 | email that you have a change (or patch) you would like |
| 503 | considered for the "master" branch of the Git repository. |
| 504 | To send this type of change, you format the patch and then |
| 505 | send the email using the Git commands |
| 506 | <filename>git format-patch</filename> and |
| 507 | <filename>git send-email</filename>. |
| 508 | For information on how to use these scripts, see the |
| 509 | "<ulink url='&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change'>Submitting a Change to the Yocto Project</ulink>" |
| 510 | section in the Yocto Project Development Tasks Manual. |
| 511 | </para></listitem> |
| 512 | </itemizedlist> |
| 513 | </para> |
| 514 | </section> |
| 515 | |
| 516 | <section id='git'> |
| 517 | <title>Git</title> |
| 518 | |
| 519 | <para> |
| 520 | The Yocto Project makes extensive use of Git, which is a |
| 521 | free, open source distributed version control system. |
| 522 | Git supports distributed development, non-linear development, |
| 523 | and can handle large projects. |
| 524 | It is best that you have some fundamental understanding |
| 525 | of how Git tracks projects and how to work with Git if |
| 526 | you are going to use the Yocto Project for development. |
| 527 | This section provides a quick overview of how Git works and |
| 528 | provides you with a summary of some essential Git commands. |
| 529 | <note><title>Notes</title> |
| 530 | <itemizedlist> |
| 531 | <listitem><para> |
| 532 | For more information on Git, see |
| 533 | <ulink url='http://git-scm.com/documentation'></ulink>. |
| 534 | </para></listitem> |
| 535 | <listitem><para> |
| 536 | If you need to download Git, it is recommended that you add |
| 537 | Git to your system through your distribution's "software |
| 538 | store" (e.g. for Ubuntu, use the Ubuntu Software feature). |
| 539 | For the Git download page, see |
| 540 | <ulink url='http://git-scm.com/download'></ulink>. |
| 541 | </para></listitem> |
| 542 | <listitem><para> |
| 543 | For information beyond the introductory nature in this |
| 544 | section, see the |
| 545 | "<ulink url='&YOCTO_DOCS_DEV_URL;#locating-yocto-project-source-files'>Locating Yocto Project Source Files</ulink>" |
| 546 | section in the Yocto Project Development Tasks Manual. |
| 547 | </para></listitem> |
| 548 | </itemizedlist> |
| 549 | </note> |
| 550 | </para> |
| 551 | |
| 552 | <section id='repositories-tags-and-branches'> |
| 553 | <title>Repositories, Tags, and Branches</title> |
| 554 | |
| 555 | <para> |
| 556 | As mentioned briefly in the previous section and also in the |
| 557 | "<link linkend='gs-git-workflows-and-the-yocto-project'>Git Workflows and the Yocto Project</link>" |
| 558 | section, the Yocto Project maintains source repositories at |
| 559 | <ulink url='&YOCTO_GIT_URL;'></ulink>. |
| 560 | If you look at this web-interface of the repositories, each item |
| 561 | is a separate Git repository. |
| 562 | </para> |
| 563 | |
| 564 | <para> |
| 565 | Git repositories use branching techniques that track content |
| 566 | change (not files) within a project (e.g. a new feature or updated |
| 567 | documentation). |
| 568 | Creating a tree-like structure based on project divergence allows |
| 569 | for excellent historical information over the life of a project. |
| 570 | This methodology also allows for an environment from which you can |
| 571 | do lots of local experimentation on projects as you develop |
| 572 | changes or new features. |
| 573 | </para> |
| 574 | |
| 575 | <para> |
| 576 | A Git repository represents all development efforts for a given |
| 577 | project. |
| 578 | For example, the Git repository <filename>poky</filename> contains |
| 579 | all changes and developments for that repository over the course |
| 580 | of its entire life. |
| 581 | That means that all changes that make up all releases are captured. |
| 582 | The repository maintains a complete history of changes. |
| 583 | </para> |
| 584 | |
| 585 | <para> |
| 586 | You can create a local copy of any repository by "cloning" it |
| 587 | with the <filename>git clone</filename> command. |
| 588 | When you clone a Git repository, you end up with an identical |
| 589 | copy of the repository on your development system. |
| 590 | Once you have a local copy of a repository, you can take steps to |
| 591 | develop locally. |
| 592 | For examples on how to clone Git repositories, see the |
| 593 | "<ulink url='&YOCTO_DOCS_DEV_URL;#locating-yocto-project-source-files'>Locating Yocto Project Source Files</ulink>" |
| 594 | section in the Yocto Project Development Tasks Manual. |
| 595 | </para> |
| 596 | |
| 597 | <para> |
| 598 | It is important to understand that Git tracks content change and |
| 599 | not files. |
| 600 | Git uses "branches" to organize different development efforts. |
| 601 | For example, the <filename>poky</filename> repository has |
| 602 | several branches that include the current "&DISTRO_NAME_NO_CAP;" |
| 603 | branch, the "master" branch, and many branches for past |
| 604 | Yocto Project releases. |
| 605 | You can see all the branches by going to |
| 606 | <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and |
| 607 | clicking on the |
| 608 | <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/heads'>[...]</ulink></filename> |
| 609 | link beneath the "Branch" heading. |
| 610 | </para> |
| 611 | |
| 612 | <para> |
| 613 | Each of these branches represents a specific area of development. |
| 614 | The "master" branch represents the current or most recent |
| 615 | development. |
| 616 | All other branches represent offshoots of the "master" branch. |
| 617 | </para> |
| 618 | |
| 619 | <para> |
| 620 | When you create a local copy of a Git repository, the copy has |
| 621 | the same set of branches as the original. |
| 622 | This means you can use Git to create a local working area |
| 623 | (also called a branch) that tracks a specific development branch |
| 624 | from the upstream source Git repository. |
| 625 | in other words, you can define your local Git environment to |
| 626 | work on any development branch in the repository. |
| 627 | To help illustrate, consider the following example Git commands: |
| 628 | <literallayout class='monospaced'> |
| 629 | $ cd ~ |
| 630 | $ git clone git://git.yoctoproject.org/poky |
| 631 | $ cd poky |
| 632 | $ git checkout -b &DISTRO_NAME_NO_CAP; origin/&DISTRO_NAME_NO_CAP; |
| 633 | </literallayout> |
| 634 | In the previous example after moving to the home directory, the |
| 635 | <filename>git clone</filename> command creates a |
| 636 | local copy of the upstream <filename>poky</filename> Git repository. |
| 637 | By default, Git checks out the "master" branch for your work. |
| 638 | After changing the working directory to the new local repository |
| 639 | (i.e. <filename>poky</filename>), the |
| 640 | <filename>git checkout</filename> command creates |
| 641 | and checks out a local branch named "&DISTRO_NAME_NO_CAP;", which |
| 642 | tracks the upstream "origin/&DISTRO_NAME_NO_CAP;" branch. |
| 643 | Changes you make while in this branch would ultimately affect |
| 644 | the upstream "&DISTRO_NAME_NO_CAP;" branch of the |
| 645 | <filename>poky</filename> repository. |
| 646 | </para> |
| 647 | |
| 648 | <para> |
| 649 | It is important to understand that when you create and checkout a |
| 650 | local working branch based on a branch name, |
| 651 | your local environment matches the "tip" of that particular |
| 652 | development branch at the time you created your local branch, |
| 653 | which could be different from the files in the "master" branch |
| 654 | of the upstream repository. |
| 655 | In other words, creating and checking out a local branch based on |
| 656 | the "&DISTRO_NAME_NO_CAP;" branch name is not the same as |
| 657 | checking out the "master" branch in the repository. |
| 658 | Keep reading to see how you create a local snapshot of a Yocto |
| 659 | Project Release. |
| 660 | </para> |
| 661 | |
| 662 | <para> |
| 663 | Git uses "tags" to mark specific changes in a repository branch |
| 664 | structure. |
| 665 | Typically, a tag is used to mark a special point such as the final |
| 666 | change (or commit) before a project is released. |
| 667 | You can see the tags used with the <filename>poky</filename> Git |
| 668 | repository by going to |
| 669 | <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and |
| 670 | clicking on the |
| 671 | <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/tags'>[...]</ulink></filename> |
| 672 | link beneath the "Tag" heading. |
| 673 | </para> |
| 674 | |
| 675 | <para> |
| 676 | Some key tags for the <filename>poky</filename> repository are |
| 677 | <filename>jethro-14.0.3</filename>, |
| 678 | <filename>morty-16.0.1</filename>, |
| 679 | <filename>pyro-17.0.0</filename>, and |
| 680 | <filename>&DISTRO_NAME_NO_CAP;-&POKYVERSION;</filename>. |
| 681 | These tags represent Yocto Project releases. |
| 682 | </para> |
| 683 | |
| 684 | <para> |
| 685 | When you create a local copy of the Git repository, you also |
| 686 | have access to all the tags in the upstream repository. |
| 687 | Similar to branches, you can create and checkout a local working |
| 688 | Git branch based on a tag name. |
| 689 | When you do this, you get a snapshot of the Git repository that |
| 690 | reflects the state of the files when the change was made associated |
| 691 | with that tag. |
| 692 | The most common use is to checkout a working branch that matches |
| 693 | a specific Yocto Project release. |
| 694 | Here is an example: |
| 695 | <literallayout class='monospaced'> |
| 696 | $ cd ~ |
| 697 | $ git clone git://git.yoctoproject.org/poky |
| 698 | $ cd poky |
| 699 | $ git fetch --tags |
| 700 | $ git checkout tags/rocko-18.0.0 -b my_rocko-18.0.0 |
| 701 | </literallayout> |
| 702 | In this example, the name of the top-level directory of your |
| 703 | local Yocto Project repository is <filename>poky</filename>. |
| 704 | After moving to the <filename>poky</filename> directory, the |
| 705 | <filename>git fetch</filename> command makes all the upstream |
| 706 | tags available locally in your repository. |
| 707 | Finally, the <filename>git checkout</filename> command |
| 708 | creates and checks out a branch named "my-rocko-18.0.0" that is |
| 709 | based on the upstream branch whose "HEAD" matches the |
| 710 | commit in the repository associated with the "rocko-18.0.0" tag. |
| 711 | The files in your repository now exactly match that particular |
| 712 | Yocto Project release as it is tagged in the upstream Git |
| 713 | repository. |
| 714 | It is important to understand that when you create and |
| 715 | checkout a local working branch based on a tag, your environment |
| 716 | matches a specific point in time and not the entire development |
| 717 | branch (i.e. from the "tip" of the branch backwards). |
| 718 | </para> |
| 719 | </section> |
| 720 | |
| 721 | <section id='basic-commands'> |
| 722 | <title>Basic Commands</title> |
| 723 | |
| 724 | <para> |
| 725 | Git has an extensive set of commands that lets you manage changes |
| 726 | and perform collaboration over the life of a project. |
| 727 | Conveniently though, you can manage with a small set of basic |
| 728 | operations and workflows once you understand the basic |
| 729 | philosophy behind Git. |
| 730 | You do not have to be an expert in Git to be functional. |
| 731 | A good place to look for instruction on a minimal set of Git |
| 732 | commands is |
| 733 | <ulink url='http://git-scm.com/documentation'>here</ulink>. |
| 734 | </para> |
| 735 | |
| 736 | <para> |
| 737 | The following list of Git commands briefly describes some basic |
| 738 | Git operations as a way to get started. |
| 739 | As with any set of commands, this list (in most cases) simply shows |
| 740 | the base command and omits the many arguments it supports. |
| 741 | See the Git documentation for complete descriptions and strategies |
| 742 | on how to use these commands: |
| 743 | <itemizedlist> |
| 744 | <listitem><para> |
| 745 | <emphasis><filename>git init</filename>:</emphasis> |
| 746 | Initializes an empty Git repository. |
| 747 | You cannot use Git commands unless you have a |
| 748 | <filename>.git</filename> repository. |
| 749 | </para></listitem> |
| 750 | <listitem><para id='git-commands-clone'> |
| 751 | <emphasis><filename>git clone</filename>:</emphasis> |
| 752 | Creates a local clone of a Git repository that is on |
| 753 | equal footing with a fellow developer’s Git repository |
| 754 | or an upstream repository. |
| 755 | </para></listitem> |
| 756 | <listitem><para> |
| 757 | <emphasis><filename>git add</filename>:</emphasis> |
| 758 | Locally stages updated file contents to the index that |
| 759 | Git uses to track changes. |
| 760 | You must stage all files that have changed before you |
| 761 | can commit them. |
| 762 | </para></listitem> |
| 763 | <listitem><para> |
| 764 | <emphasis><filename>git commit</filename>:</emphasis> |
| 765 | Creates a local "commit" that documents the changes you |
| 766 | made. |
| 767 | Only changes that have been staged can be committed. |
| 768 | Commits are used for historical purposes, for determining |
| 769 | if a maintainer of a project will allow the change, |
| 770 | and for ultimately pushing the change from your local |
| 771 | Git repository into the project’s upstream repository. |
| 772 | </para></listitem> |
| 773 | <listitem><para> |
| 774 | <emphasis><filename>git status</filename>:</emphasis> |
| 775 | Reports any modified files that possibly need to be |
| 776 | staged and gives you a status of where you stand regarding |
| 777 | local commits as compared to the upstream repository. |
| 778 | </para></listitem> |
| 779 | <listitem><para> |
| 780 | <emphasis><filename>git checkout</filename> <replaceable>branch-name</replaceable>:</emphasis> |
| 781 | Changes your local working branch and in this form |
| 782 | assumes the local branch already exists. |
| 783 | This command is analogous to "cd". |
| 784 | </para></listitem> |
| 785 | <listitem><para> |
| 786 | <emphasis><filename>git checkout –b</filename> <replaceable>working-branch</replaceable> <replaceable>upstream-branch</replaceable>:</emphasis> |
| 787 | Creates and checks out a working branch on your local |
| 788 | machine. |
| 789 | The local branch tracks the upstream branch. |
| 790 | You can use your local branch to isolate your work. |
| 791 | It is a good idea to use local branches when adding |
| 792 | specific features or changes. |
| 793 | Using isolated branches facilitates easy removal of |
| 794 | changes if they do not work out. |
| 795 | </para></listitem> |
| 796 | <listitem><para><emphasis><filename>git branch</filename>:</emphasis> |
| 797 | Displays the existing local branches associated with your |
| 798 | local repository. |
| 799 | The branch that you have currently checked out is noted |
| 800 | with an asterisk character. |
| 801 | </para></listitem> |
| 802 | <listitem><para> |
| 803 | <emphasis><filename>git branch -D</filename> <replaceable>branch-name</replaceable>:</emphasis> |
| 804 | Deletes an existing local branch. |
| 805 | You need to be in a local branch other than the one you |
| 806 | are deleting in order to delete |
| 807 | <replaceable>branch-name</replaceable>. |
| 808 | </para></listitem> |
| 809 | <listitem><para> |
| 810 | <emphasis><filename>git pull --rebase</filename>:</emphasis> |
| 811 | Retrieves information from an upstream Git repository |
| 812 | and places it in your local Git repository. |
| 813 | You use this command to make sure you are synchronized with |
| 814 | the repository from which you are basing changes |
| 815 | (.e.g. the "master" branch). |
| 816 | The "--rebase" option ensures that any local commits you |
| 817 | have in your branch are preserved at the top of your |
| 818 | local branch. |
| 819 | </para></listitem> |
| 820 | <listitem><para> |
| 821 | <emphasis><filename>git push</filename> <replaceable>repo-name</replaceable> <replaceable>local-branch</replaceable><filename>:</filename><replaceable>upstream-branch</replaceable>:</emphasis> |
| 822 | Sends all your committed local changes to the upstream Git |
| 823 | repository that your local repository is tracking |
| 824 | (e.g. a contribution repository). |
| 825 | The maintainer of the project draws from these repositories |
| 826 | to merge changes (commits) into the appropriate branch |
| 827 | of project's upstream repository. |
| 828 | </para></listitem> |
| 829 | <listitem><para> |
| 830 | <emphasis><filename>git merge</filename>:</emphasis> |
| 831 | Combines or adds changes from one |
| 832 | local branch of your repository with another branch. |
| 833 | When you create a local Git repository, the default branch |
| 834 | is named "master". |
| 835 | A typical workflow is to create a temporary branch that is |
| 836 | based off "master" that you would use for isolated work. |
| 837 | You would make your changes in that isolated branch, |
| 838 | stage and commit them locally, switch to the "master" |
| 839 | branch, and then use the <filename>git merge</filename> |
| 840 | command to apply the changes from your isolated branch |
| 841 | into the currently checked out branch (e.g. "master"). |
| 842 | After the merge is complete and if you are done with |
| 843 | working in that isolated branch, you can safely delete |
| 844 | the isolated branch. |
| 845 | </para></listitem> |
| 846 | <listitem><para> |
| 847 | <emphasis><filename>git cherry-pick</filename> <replaceable>commits</replaceable>:</emphasis> |
| 848 | Choose and apply specific commits from one branch |
| 849 | into another branch. |
| 850 | There are times when you might not be able to merge |
| 851 | all the changes in one branch with |
| 852 | another but need to pick out certain ones. |
| 853 | </para></listitem> |
| 854 | <listitem><para> |
| 855 | <emphasis><filename>gitk</filename>:</emphasis> |
| 856 | Provides a GUI view of the branches and changes in your |
| 857 | local Git repository. |
| 858 | This command is a good way to graphically see where things |
| 859 | have diverged in your local repository. |
| 860 | <note> |
| 861 | You need to install the <filename>gitk</filename> |
| 862 | package on your development system to use this |
| 863 | command. |
| 864 | </note> |
| 865 | </para></listitem> |
| 866 | <listitem><para> |
| 867 | <emphasis><filename>git log</filename>:</emphasis> |
| 868 | Reports a history of your commits to the repository. |
| 869 | This report lists all commits regardless of whether you |
| 870 | have pushed them upstream or not. |
| 871 | </para></listitem> |
| 872 | <listitem><para> |
| 873 | <emphasis><filename>git diff</filename>:</emphasis> |
| 874 | Displays line-by-line differences between a local |
| 875 | working file and the same file as understood by Git. |
| 876 | This command is useful to see what you have changed |
| 877 | in any given file. |
| 878 | </para></listitem> |
| 879 | </itemizedlist> |
| 880 | </para> |
| 881 | </section> |
| 882 | </section> |
| 883 | |
| 884 | <section id='licensing'> |
| 885 | <title>Licensing</title> |
| 886 | |
| 887 | <para> |
| 888 | Because open source projects are open to the public, they have |
| 889 | different licensing structures in place. |
| 890 | License evolution for both Open Source and Free Software has an |
| 891 | interesting history. |
| 892 | If you are interested in this history, you can find basic information |
| 893 | here: |
| 894 | <itemizedlist> |
| 895 | <listitem><para> |
| 896 | <ulink url='http://en.wikipedia.org/wiki/Open-source_license'>Open source license history</ulink> |
| 897 | </para></listitem> |
| 898 | <listitem><para> |
| 899 | <ulink url='http://en.wikipedia.org/wiki/Free_software_license'>Free software license history</ulink> |
| 900 | </para></listitem> |
| 901 | </itemizedlist> |
| 902 | </para> |
| 903 | |
| 904 | <para> |
| 905 | In general, the Yocto Project is broadly licensed under the |
| 906 | Massachusetts Institute of Technology (MIT) License. |
| 907 | MIT licensing permits the reuse of software within proprietary |
| 908 | software as long as the license is distributed with that software. |
| 909 | MIT is also compatible with the GNU General Public License (GPL). |
| 910 | Patches to the Yocto Project follow the upstream licensing scheme. |
| 911 | You can find information on the MIT license |
| 912 | <ulink url='http://www.opensource.org/licenses/mit-license.php'>here</ulink>. |
| 913 | You can find information on the GNU GPL |
| 914 | <ulink url='http://www.opensource.org/licenses/LGPL-3.0'>here</ulink>. |
| 915 | </para> |
| 916 | |
| 917 | <para> |
| 918 | When you build an image using the Yocto Project, the build process |
| 919 | uses a known list of licenses to ensure compliance. |
| 920 | You can find this list in the |
| 921 | <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> |
| 922 | at <filename>meta/files/common-licenses</filename>. |
| 923 | Once the build completes, the list of all licenses found and used |
| 924 | during that build are kept in the |
| 925 | <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink> |
| 926 | at <filename>tmp/deploy/licenses</filename>. |
| 927 | </para> |
| 928 | |
| 929 | <para> |
| 930 | If a module requires a license that is not in the base list, the |
| 931 | build process generates a warning during the build. |
| 932 | These tools make it easier for a developer to be certain of the |
| 933 | licenses with which their shipped products must comply. |
| 934 | However, even with these tools it is still up to the developer to |
| 935 | resolve potential licensing issues. |
| 936 | </para> |
| 937 | |
| 938 | <para> |
| 939 | The base list of licenses used by the build process is a combination |
| 940 | of the Software Package Data Exchange (SPDX) list and the Open |
| 941 | Source Initiative (OSI) projects. |
| 942 | <ulink url='http://spdx.org'>SPDX Group</ulink> is a working group of |
| 943 | the Linux Foundation that maintains a specification for a standard |
| 944 | format for communicating the components, licenses, and copyrights |
| 945 | associated with a software package. |
| 946 | <ulink url='http://opensource.org'>OSI</ulink> is a corporation |
| 947 | dedicated to the Open Source Definition and the effort for reviewing |
| 948 | and approving licenses that conform to the Open Source Definition |
| 949 | (OSD). |
| 950 | </para> |
| 951 | |
| 952 | <para> |
| 953 | You can find a list of the combined SPDX and OSI licenses that the |
| 954 | Yocto Project uses in the |
| 955 | <filename>meta/files/common-licenses</filename> directory in your |
| 956 | <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>. |
| 957 | </para> |
| 958 | |
| 959 | <para> |
| 960 | For information that can help you maintain compliance with various |
| 961 | open source licensing during the lifecycle of a product created using |
| 962 | the Yocto Project, see the |
| 963 | "<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Product's Lifecycle</ulink>" |
| 964 | section in the Yocto Project Development Tasks Manual. |
| 965 | </para> |
| 966 | </section> |
| 967 | </chapter> |
| 968 | <!-- |
| 969 | vim: expandtab tw=80 ts=4 |
| 970 | --> |