Andrew Geissler | f034379 | 2020-11-18 10:42:21 -0600 | [diff] [blame] | 1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 2 | |
| 3 | ***************************************** |
| 4 | The Yocto Project Development Environment |
| 5 | ***************************************** |
| 6 | |
| 7 | This chapter takes a look at the Yocto Project development environment. |
| 8 | The chapter provides Yocto Project Development environment concepts that |
| 9 | help you understand how work is accomplished in an open source |
| 10 | environment, which is very different as compared to work accomplished in |
| 11 | a closed, proprietary environment. |
| 12 | |
| 13 | Specifically, this chapter addresses open source philosophy, source |
| 14 | repositories, workflows, Git, and licensing. |
| 15 | |
| 16 | Open Source Philosophy |
| 17 | ====================== |
| 18 | |
| 19 | Open source philosophy is characterized by software development directed |
| 20 | by peer production and collaboration through an active community of |
| 21 | developers. Contrast this to the more standard centralized development |
| 22 | models used by commercial software companies where a finite set of |
| 23 | developers produces a product for sale using a defined set of procedures |
| 24 | that ultimately result in an end product whose architecture and source |
| 25 | material are closed to the public. |
| 26 | |
| 27 | Open source projects conceptually have differing concurrent agendas, |
| 28 | approaches, and production. These facets of the development process can |
| 29 | come from anyone in the public (community) who has a stake in the |
| 30 | software project. The open source environment contains new copyright, |
| 31 | licensing, domain, and consumer issues that differ from the more |
| 32 | traditional development environment. In an open source environment, the |
| 33 | end product, source material, and documentation are all available to the |
| 34 | public at no cost. |
| 35 | |
| 36 | A benchmark example of an open source project is the Linux kernel, which |
| 37 | was initially conceived and created by Finnish computer science student |
| 38 | Linus Torvalds in 1991. Conversely, a good example of a non-open source |
| 39 | project is the Windows family of operating systems developed by |
| 40 | Microsoft Corporation. |
| 41 | |
| 42 | Wikipedia has a good historical description of the Open Source |
Andrew Geissler | d1e8949 | 2021-02-12 15:35:20 -0600 | [diff] [blame] | 43 | Philosophy `here <https://en.wikipedia.org/wiki/Open_source>`__. You can |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 44 | also find helpful information on how to participate in the Linux |
| 45 | Community |
Andrew Geissler | c3d88e4 | 2020-10-02 09:45:00 -0500 | [diff] [blame] | 46 | `here <https://www.kernel.org/doc/html/latest/process/index.html>`__. |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 47 | |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 48 | The Development Host |
| 49 | ==================== |
| 50 | |
| 51 | A development host or :term:`Build Host` is key to |
| 52 | using the Yocto Project. Because the goal of the Yocto Project is to |
| 53 | develop images or applications that run on embedded hardware, |
| 54 | development of those images and applications generally takes place on a |
Andrew Geissler | 615f2f1 | 2022-07-15 14:00:58 -0500 | [diff] [blame] | 55 | system not intended to run the software --- the development host. |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 56 | |
| 57 | You need to set up a development host in order to use it with the Yocto |
| 58 | Project. Most find that it is best to have a native Linux machine |
| 59 | function as the development host. However, it is possible to use a |
| 60 | system that does not run Linux as its operating system as your |
| 61 | development host. When you have a Mac or Windows-based system, you can |
| 62 | set it up as the development host by using |
| 63 | `CROPS <https://github.com/crops/poky-container>`__, which leverages |
| 64 | `Docker Containers <https://www.docker.com/>`__. Once you take the steps |
| 65 | to set up a CROPS machine, you effectively have access to a shell |
| 66 | environment that is similar to what you see when using a Linux-based |
| 67 | development host. For the steps needed to set up a system using CROPS, |
| 68 | see the |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 69 | ":ref:`dev-manual/start:setting up to use cross platforms (crops)`" |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 70 | section in |
| 71 | the Yocto Project Development Tasks Manual. |
| 72 | |
| 73 | If your development host is going to be a system that runs a Linux |
William A. Kennington III | ac69b48 | 2021-06-02 12:28:27 -0700 | [diff] [blame] | 74 | distribution, you must still take steps to prepare the system |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 75 | for use with the Yocto Project. You need to be sure that the Linux |
| 76 | distribution on the system is one that supports the Yocto Project. You |
| 77 | also need to be sure that the correct set of host packages are installed |
| 78 | that allow development using the Yocto Project. For the steps needed to |
| 79 | set up a development host that runs Linux, see the |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 80 | ":ref:`dev-manual/start:setting up a native linux host`" |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 81 | section in the Yocto Project Development Tasks Manual. |
| 82 | |
William A. Kennington III | ac69b48 | 2021-06-02 12:28:27 -0700 | [diff] [blame] | 83 | Once your development host is set up to use the Yocto Project, there |
| 84 | are several ways of working in the Yocto Project environment: |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 85 | |
| 86 | - *Command Lines, BitBake, and Shells:* Traditional development in the |
| 87 | Yocto Project involves using the :term:`OpenEmbedded Build System`, |
| 88 | which uses |
| 89 | BitBake, in a command-line environment from a shell on your |
| 90 | development host. You can accomplish this from a host that is a |
| 91 | native Linux machine or from a host that has been set up with CROPS. |
| 92 | Either way, you create, modify, and build images and applications all |
| 93 | within a shell-based environment using components and tools available |
| 94 | through your Linux distribution and the Yocto Project. |
| 95 | |
| 96 | For a general flow of the build procedures, see the |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 97 | ":ref:`dev-manual/common-tasks:building a simple image`" |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 98 | section in the Yocto Project Development Tasks Manual. |
| 99 | |
| 100 | - *Board Support Package (BSP) Development:* Development of BSPs |
| 101 | involves using the Yocto Project to create and test layers that allow |
| 102 | easy development of images and applications targeted for specific |
| 103 | hardware. To development BSPs, you need to take some additional steps |
| 104 | beyond what was described in setting up a development host. |
| 105 | |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 106 | The :doc:`/bsp-guide/index` provides BSP-related development |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 107 | information. For specifics on development host preparation, see the |
| 108 | ":ref:`bsp-guide/bsp:preparing your build host to work with bsp layers`" |
| 109 | section in the Yocto Project Board Support Package (BSP) Developer's |
| 110 | Guide. |
| 111 | |
| 112 | - *Kernel Development:* If you are going to be developing kernels using |
| 113 | the Yocto Project you likely will be using ``devtool``. A workflow |
| 114 | using ``devtool`` makes kernel development quicker by reducing |
| 115 | iteration cycle times. |
| 116 | |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 117 | The :doc:`/kernel-dev/index` provides kernel-related |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 118 | development information. For specifics on development host |
| 119 | preparation, see the |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 120 | ":ref:`kernel-dev/common:preparing the build host to work on the kernel`" |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 121 | section in the Yocto Project Linux Kernel Development Manual. |
| 122 | |
| 123 | - *Using Toaster:* The other Yocto Project development method that |
| 124 | involves an interface that effectively puts the Yocto Project into |
| 125 | the background is Toaster. Toaster provides an interface to the |
| 126 | OpenEmbedded build system. The interface enables you to configure and |
| 127 | run your builds. Information about builds is collected and stored in |
| 128 | a database. You can use Toaster to configure and start builds on |
| 129 | multiple remote build servers. |
| 130 | |
| 131 | For steps that show you how to set up your development host to use |
| 132 | Toaster and on how to use Toaster in general, see the |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 133 | :doc:`/toaster-manual/index`. |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 134 | |
| 135 | Yocto Project Source Repositories |
| 136 | ================================= |
| 137 | |
| 138 | The Yocto Project team maintains complete source repositories for all |
| 139 | Yocto Project files at :yocto_git:`/`. This web-based source |
| 140 | code browser is organized into categories by function such as IDE |
| 141 | Plugins, Matchbox, Poky, Yocto Linux Kernel, and so forth. From the |
| 142 | interface, you can click on any particular item in the "Name" column and |
| 143 | see the URL at the bottom of the page that you need to clone a Git |
| 144 | repository for that particular item. Having a local Git repository of |
| 145 | the :term:`Source Directory`, which |
| 146 | is usually named "poky", allows you to make changes, contribute to the |
| 147 | history, and ultimately enhance the Yocto Project's tools, Board Support |
| 148 | Packages, and so forth. |
| 149 | |
| 150 | For any supported release of Yocto Project, you can also go to the |
| 151 | :yocto_home:`Yocto Project Website <>` and select the "DOWNLOADS" |
| 152 | item from the "SOFTWARE" menu and get a released tarball of the ``poky`` |
| 153 | repository, any supported BSP tarball, or Yocto Project tools. Unpacking |
| 154 | these tarballs gives you a snapshot of the released files. |
| 155 | |
| 156 | .. note:: |
| 157 | |
| 158 | - The recommended method for setting up the Yocto Project |
| 159 | :term:`Source Directory` and the files |
Andrew Geissler | 3b8a17c | 2021-04-15 15:55:55 -0500 | [diff] [blame] | 160 | for supported BSPs (e.g., ``meta-intel``) is to use |
| 161 | :ref:`overview-manual/development-environment:git` |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 162 | to create a local copy of the upstream repositories. |
| 163 | |
| 164 | - Be sure to always work in matching branches for both the selected |
| 165 | BSP repository and the Source Directory (i.e. ``poky``) |
Andrew Geissler | 595f630 | 2022-01-24 19:11:47 +0000 | [diff] [blame] | 166 | repository. For example, if you have checked out the "&DISTRO_NAME_NO_CAP;" |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 167 | branch of ``poky`` and you are going to use ``meta-intel``, be |
Andrew Geissler | 595f630 | 2022-01-24 19:11:47 +0000 | [diff] [blame] | 168 | sure to checkout the "&DISTRO_NAME_NO_CAP;" branch of ``meta-intel``. |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 169 | |
| 170 | In summary, here is where you can get the project files needed for |
| 171 | development: |
| 172 | |
| 173 | - :yocto_git:`Source Repositories: <>` This area contains IDE |
| 174 | Plugins, Matchbox, Poky, Poky Support, Tools, Yocto Linux Kernel, and |
| 175 | Yocto Metadata Layers. You can create local copies of Git |
| 176 | repositories for each of these areas. |
| 177 | |
| 178 | .. image:: figures/source-repos.png |
Andrew Geissler | d583833 | 2022-05-27 11:33:10 -0500 | [diff] [blame] | 179 | :width: 100% |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 180 | |
| 181 | For steps on how to view and access these upstream Git repositories, |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 182 | see the ":ref:`dev-manual/start:accessing source repositories`" |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 183 | Section in the Yocto Project Development Tasks Manual. |
| 184 | |
Andrew Geissler | c3d88e4 | 2020-10-02 09:45:00 -0500 | [diff] [blame] | 185 | - :yocto_dl:`Index of /releases: </releases>` This is an index |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 186 | of releases such as Poky, Pseudo, installers for cross-development |
| 187 | toolchains, miscellaneous support and all released versions of Yocto |
| 188 | Project in the form of images or tarballs. Downloading and extracting |
| 189 | these files does not produce a local copy of the Git repository but |
| 190 | rather a snapshot of a particular release or image. |
| 191 | |
| 192 | .. image:: figures/index-downloads.png |
| 193 | :align: center |
Andrew Geissler | d583833 | 2022-05-27 11:33:10 -0500 | [diff] [blame] | 194 | :width: 50% |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 195 | |
| 196 | For steps on how to view and access these files, see the |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 197 | ":ref:`dev-manual/start:accessing index of releases`" |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 198 | section in the Yocto Project Development Tasks Manual. |
| 199 | |
| 200 | - *"DOWNLOADS" page for the* :yocto_home:`Yocto Project Website <>` *:* |
| 201 | |
| 202 | The Yocto Project website includes a "DOWNLOADS" page accessible |
| 203 | through the "SOFTWARE" menu that allows you to download any Yocto |
| 204 | Project release, tool, and Board Support Package (BSP) in tarball |
| 205 | form. The tarballs are similar to those found in the |
Andrew Geissler | c3d88e4 | 2020-10-02 09:45:00 -0500 | [diff] [blame] | 206 | :yocto_dl:`Index of /releases: </releases>` area. |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 207 | |
| 208 | .. image:: figures/yp-download.png |
Andrew Geissler | d583833 | 2022-05-27 11:33:10 -0500 | [diff] [blame] | 209 | :width: 100% |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 210 | |
| 211 | For steps on how to use the "DOWNLOADS" page, see the |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 212 | ":ref:`dev-manual/start:using the downloads page`" |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 213 | section in the Yocto Project Development Tasks Manual. |
| 214 | |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 215 | Git Workflows and the Yocto Project |
| 216 | =================================== |
| 217 | |
| 218 | Developing using the Yocto Project likely requires the use of |
Andrew Geissler | 3b8a17c | 2021-04-15 15:55:55 -0500 | [diff] [blame] | 219 | :ref:`overview-manual/development-environment:git`. |
| 220 | Git is a free, open source distributed version control |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 221 | system used as part of many collaborative design environments. This |
| 222 | section provides workflow concepts using the Yocto Project and Git. In |
| 223 | particular, the information covers basic practices that describe roles |
| 224 | and actions in a collaborative development environment. |
| 225 | |
| 226 | .. note:: |
| 227 | |
| 228 | If you are familiar with this type of development environment, you |
| 229 | might not want to read this section. |
| 230 | |
| 231 | The Yocto Project files are maintained using Git in "branches" whose Git |
| 232 | histories track every change and whose structures provide branches for |
| 233 | all diverging functionality. Although there is no need to use Git, many |
| 234 | open source projects do so. |
| 235 | |
| 236 | For the Yocto Project, a key individual called the "maintainer" is |
Andrew Geissler | 595f630 | 2022-01-24 19:11:47 +0000 | [diff] [blame] | 237 | responsible for the integrity of the development branch of a given Git |
| 238 | repository. The development branch is the "upstream" repository from which |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 239 | final or most recent builds of a project occur. The maintainer is |
| 240 | responsible for accepting changes from other developers and for |
| 241 | organizing the underlying branch structure to reflect release strategies |
| 242 | and so forth. |
| 243 | |
| 244 | .. note:: |
| 245 | |
| 246 | For information on finding out who is responsible for (maintains) a |
| 247 | particular area of code in the Yocto Project, see the |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 248 | ":ref:`dev-manual/common-tasks:submitting a change to the yocto project`" |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 249 | section of the Yocto Project Development Tasks Manual. |
| 250 | |
| 251 | The Yocto Project ``poky`` Git repository also has an upstream |
| 252 | contribution Git repository named ``poky-contrib``. You can see all the |
| 253 | branches in this repository using the web interface of the |
| 254 | :yocto_git:`Source Repositories <>` organized within the "Poky Support" |
| 255 | area. These branches hold changes (commits) to the project that have |
| 256 | been submitted or committed by the Yocto Project development team and by |
| 257 | community members who contribute to the project. The maintainer |
| 258 | determines if the changes are qualified to be moved from the "contrib" |
| 259 | branches into the "master" branch of the Git repository. |
| 260 | |
| 261 | Developers (including contributing community members) create and |
| 262 | maintain cloned repositories of upstream branches. The cloned |
| 263 | repositories are local to their development platforms and are used to |
| 264 | develop changes. When a developer is satisfied with a particular feature |
| 265 | or change, they "push" the change to the appropriate "contrib" |
| 266 | repository. |
| 267 | |
| 268 | Developers are responsible for keeping their local repository up-to-date |
| 269 | with whatever upstream branch they are working against. They are also |
| 270 | responsible for straightening out any conflicts that might arise within |
| 271 | files that are being worked on simultaneously by more than one person. |
| 272 | All this work is done locally on the development host before anything is |
| 273 | pushed to a "contrib" area and examined at the maintainer's level. |
| 274 | |
William A. Kennington III | ac69b48 | 2021-06-02 12:28:27 -0700 | [diff] [blame] | 275 | There is a somewhat formal method by which developers commit changes and |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 276 | push them into the "contrib" area and subsequently request that the |
| 277 | maintainer include them into an upstream branch. This process is called |
| 278 | "submitting a patch" or "submitting a change." For information on |
| 279 | submitting patches and changes, see the |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 280 | ":ref:`dev-manual/common-tasks:submitting a change to the yocto project`" |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 281 | section in the Yocto Project Development Tasks Manual. |
| 282 | |
Andrew Geissler | 595f630 | 2022-01-24 19:11:47 +0000 | [diff] [blame] | 283 | In summary, there is a single point of entry for changes into the |
| 284 | development branch of the Git repository, which is controlled by the |
William A. Kennington III | ac69b48 | 2021-06-02 12:28:27 -0700 | [diff] [blame] | 285 | project's maintainer. A set of developers independently |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 286 | develop, test, and submit changes to "contrib" areas for the maintainer |
| 287 | to examine. The maintainer then chooses which changes are going to |
| 288 | become a permanent part of the project. |
| 289 | |
Andrew Geissler | eff2747 | 2021-10-29 15:35:00 -0500 | [diff] [blame] | 290 | .. image:: svg/git-workflow.* |
| 291 | :width: 100% |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 292 | |
| 293 | While each development environment is unique, there are some best |
| 294 | practices or methods that help development run smoothly. The following |
| 295 | list describes some of these practices. For more information about Git |
| 296 | workflows, see the workflow topics in the `Git Community |
Andrew Geissler | d1e8949 | 2021-02-12 15:35:20 -0600 | [diff] [blame] | 297 | Book <https://book.git-scm.com>`__. |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 298 | |
| 299 | - *Make Small Changes:* It is best to keep the changes you commit small |
| 300 | as compared to bundling many disparate changes into a single commit. |
| 301 | This practice not only keeps things manageable but also allows the |
| 302 | maintainer to more easily include or refuse changes. |
| 303 | |
| 304 | - *Make Complete Changes:* It is also good practice to leave the |
| 305 | repository in a state that allows you to still successfully build |
| 306 | your project. In other words, do not commit half of a feature, then |
| 307 | add the other half as a separate, later commit. Each commit should |
| 308 | take you from one buildable project state to another buildable state. |
| 309 | |
| 310 | - *Use Branches Liberally:* It is very easy to create, use, and delete |
| 311 | local branches in your working Git repository on the development |
| 312 | host. You can name these branches anything you like. It is helpful to |
| 313 | give them names associated with the particular feature or change on |
| 314 | which you are working. Once you are done with a feature or change and |
Andrew Geissler | 595f630 | 2022-01-24 19:11:47 +0000 | [diff] [blame] | 315 | have merged it into your local development branch, simply discard the |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 316 | temporary branch. |
| 317 | |
| 318 | - *Merge Changes:* The ``git merge`` command allows you to take the |
| 319 | changes from one branch and fold them into another branch. This |
| 320 | process is especially helpful when more than a single developer might |
| 321 | be working on different parts of the same feature. Merging changes |
| 322 | also automatically identifies any collisions or "conflicts" that |
| 323 | might happen as a result of the same lines of code being altered by |
| 324 | two different developers. |
| 325 | |
| 326 | - *Manage Branches:* Because branches are easy to use, you should use a |
| 327 | system where branches indicate varying levels of code readiness. For |
| 328 | example, you can have a "work" branch to develop in, a "test" branch |
| 329 | where the code or change is tested, a "stage" branch where changes |
| 330 | are ready to be committed, and so forth. As your project develops, |
| 331 | you can merge code across the branches to reflect ever-increasing |
| 332 | stable states of the development. |
| 333 | |
| 334 | - *Use Push and Pull:* The push-pull workflow is based on the concept |
| 335 | of developers "pushing" local commits to a remote repository, which |
| 336 | is usually a contribution repository. This workflow is also based on |
| 337 | developers "pulling" known states of the project down into their |
| 338 | local development repositories. The workflow easily allows you to |
| 339 | pull changes submitted by other developers from the upstream |
| 340 | repository into your work area ensuring that you have the most recent |
| 341 | software on which to develop. The Yocto Project has two scripts named |
| 342 | ``create-pull-request`` and ``send-pull-request`` that ship with the |
| 343 | release to facilitate this workflow. You can find these scripts in |
| 344 | the ``scripts`` folder of the |
| 345 | :term:`Source Directory`. For information |
| 346 | on how to use these scripts, see the |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 347 | ":ref:`dev-manual/common-tasks:using scripts to push a change upstream and request a pull`" |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 348 | section in the Yocto Project Development Tasks Manual. |
| 349 | |
| 350 | - *Patch Workflow:* This workflow allows you to notify the maintainer |
| 351 | through an email that you have a change (or patch) you would like |
Andrew Geissler | 595f630 | 2022-01-24 19:11:47 +0000 | [diff] [blame] | 352 | considered for the development branch of the Git repository. To send |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 353 | this type of change, you format the patch and then send the email |
| 354 | using the Git commands ``git format-patch`` and ``git send-email``. |
| 355 | For information on how to use these scripts, see the |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 356 | ":ref:`dev-manual/common-tasks:submitting a change to the yocto project`" |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 357 | section in the Yocto Project Development Tasks Manual. |
| 358 | |
| 359 | Git |
| 360 | === |
| 361 | |
| 362 | The Yocto Project makes extensive use of Git, which is a free, open |
| 363 | source distributed version control system. Git supports distributed |
| 364 | development, non-linear development, and can handle large projects. It |
| 365 | is best that you have some fundamental understanding of how Git tracks |
| 366 | projects and how to work with Git if you are going to use the Yocto |
| 367 | Project for development. This section provides a quick overview of how |
| 368 | Git works and provides you with a summary of some essential Git |
| 369 | commands. |
| 370 | |
| 371 | .. note:: |
| 372 | |
| 373 | - For more information on Git, see |
Andrew Geissler | d1e8949 | 2021-02-12 15:35:20 -0600 | [diff] [blame] | 374 | https://git-scm.com/documentation. |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 375 | |
| 376 | - If you need to download Git, it is recommended that you add Git to |
| 377 | your system through your distribution's "software store" (e.g. for |
| 378 | Ubuntu, use the Ubuntu Software feature). For the Git download |
Andrew Geissler | d1e8949 | 2021-02-12 15:35:20 -0600 | [diff] [blame] | 379 | page, see https://git-scm.com/download. |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 380 | |
| 381 | - For information beyond the introductory nature in this section, |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 382 | see the ":ref:`dev-manual/start:locating yocto project source files`" |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 383 | section in the Yocto Project Development Tasks Manual. |
| 384 | |
| 385 | Repositories, Tags, and Branches |
| 386 | -------------------------------- |
| 387 | |
Andrew Geissler | 3b8a17c | 2021-04-15 15:55:55 -0500 | [diff] [blame] | 388 | As mentioned briefly in the previous section and also in the |
| 389 | ":ref:`overview-manual/development-environment:git workflows and the yocto project`" |
| 390 | section, the Yocto Project maintains source repositories at :yocto_git:`/`. |
| 391 | If you look at this web-interface of the repositories, each item is a separate |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 392 | Git repository. |
| 393 | |
| 394 | Git repositories use branching techniques that track content change (not |
| 395 | files) within a project (e.g. a new feature or updated documentation). |
| 396 | Creating a tree-like structure based on project divergence allows for |
| 397 | excellent historical information over the life of a project. This |
| 398 | methodology also allows for an environment from which you can do lots of |
| 399 | local experimentation on projects as you develop changes or new |
| 400 | features. |
| 401 | |
| 402 | A Git repository represents all development efforts for a given project. |
| 403 | For example, the Git repository ``poky`` contains all changes and |
| 404 | developments for that repository over the course of its entire life. |
| 405 | That means that all changes that make up all releases are captured. The |
| 406 | repository maintains a complete history of changes. |
| 407 | |
| 408 | You can create a local copy of any repository by "cloning" it with the |
| 409 | ``git clone`` command. When you clone a Git repository, you end up with |
| 410 | an identical copy of the repository on your development system. Once you |
| 411 | have a local copy of a repository, you can take steps to develop |
| 412 | locally. For examples on how to clone Git repositories, see the |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 413 | ":ref:`dev-manual/start:locating yocto project source files`" |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 414 | section in the Yocto Project Development Tasks Manual. |
| 415 | |
| 416 | It is important to understand that Git tracks content change and not |
| 417 | files. Git uses "branches" to organize different development efforts. |
| 418 | For example, the ``poky`` repository has several branches that include |
| 419 | the current "&DISTRO_NAME_NO_CAP;" branch, the "master" branch, and many |
| 420 | branches for past Yocto Project releases. You can see all the branches |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 421 | by going to :yocto_git:`/poky/` and clicking on the |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 422 | ``[...]`` link beneath the "Branch" heading. |
| 423 | |
| 424 | Each of these branches represents a specific area of development. The |
| 425 | "master" branch represents the current or most recent development. All |
| 426 | other branches represent offshoots of the "master" branch. |
| 427 | |
| 428 | When you create a local copy of a Git repository, the copy has the same |
| 429 | set of branches as the original. This means you can use Git to create a |
| 430 | local working area (also called a branch) that tracks a specific |
Patrick Williams | 4585273 | 2022-04-02 08:58:32 -0500 | [diff] [blame] | 431 | development branch from the upstream source Git repository. In other |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 432 | words, you can define your local Git environment to work on any |
| 433 | development branch in the repository. To help illustrate, consider the |
Andrew Geissler | c926e17 | 2021-05-07 16:11:35 -0500 | [diff] [blame] | 434 | following example Git commands:: |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 435 | |
| 436 | $ cd ~ |
Andrew Geissler | 595f630 | 2022-01-24 19:11:47 +0000 | [diff] [blame] | 437 | $ git clone git://git.yoctoproject.org/poky -b &DISTRO_NAME_NO_CAP; |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 438 | |
| 439 | In the previous example |
| 440 | after moving to the home directory, the ``git clone`` command creates a |
Andrew Geissler | 595f630 | 2022-01-24 19:11:47 +0000 | [diff] [blame] | 441 | local copy of the upstream ``poky`` Git repository and checks out a |
| 442 | local branch named "&DISTRO_NAME_NO_CAP;", which tracks the upstream |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 443 | "origin/&DISTRO_NAME_NO_CAP;" branch. Changes you make while in this |
| 444 | branch would ultimately affect the upstream "&DISTRO_NAME_NO_CAP;" branch |
| 445 | of the ``poky`` repository. |
| 446 | |
| 447 | It is important to understand that when you create and checkout a local |
| 448 | working branch based on a branch name, your local environment matches |
| 449 | the "tip" of that particular development branch at the time you created |
| 450 | your local branch, which could be different from the files in the |
| 451 | "master" branch of the upstream repository. In other words, creating and |
| 452 | checking out a local branch based on the "&DISTRO_NAME_NO_CAP;" branch |
| 453 | name is not the same as checking out the "master" branch in the |
| 454 | repository. Keep reading to see how you create a local snapshot of a |
| 455 | Yocto Project Release. |
| 456 | |
| 457 | Git uses "tags" to mark specific changes in a repository branch |
| 458 | structure. Typically, a tag is used to mark a special point such as the |
| 459 | final change (or commit) before a project is released. You can see the |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 460 | tags used with the ``poky`` Git repository by going to :yocto_git:`/poky/` |
| 461 | and clicking on the ``[...]`` link beneath the "Tag" heading. |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 462 | |
| 463 | Some key tags for the ``poky`` repository are ``jethro-14.0.3``, |
| 464 | ``morty-16.0.1``, ``pyro-17.0.0``, and |
Andrew Geissler | 9aee500 | 2022-03-30 16:27:02 +0000 | [diff] [blame] | 465 | ``&DISTRO_NAME_NO_CAP;-&DISTRO;``. These tags represent Yocto Project |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 466 | releases. |
| 467 | |
| 468 | When you create a local copy of the Git repository, you also have access |
| 469 | to all the tags in the upstream repository. Similar to branches, you can |
| 470 | create and checkout a local working Git branch based on a tag name. When |
| 471 | you do this, you get a snapshot of the Git repository that reflects the |
| 472 | state of the files when the change was made associated with that tag. |
| 473 | The most common use is to checkout a working branch that matches a |
Andrew Geissler | c926e17 | 2021-05-07 16:11:35 -0500 | [diff] [blame] | 474 | specific Yocto Project release. Here is an example:: |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 475 | |
| 476 | $ cd ~ |
| 477 | $ git clone git://git.yoctoproject.org/poky |
| 478 | $ cd poky |
| 479 | $ git fetch --tags |
| 480 | $ git checkout tags/rocko-18.0.0 -b my_rocko-18.0.0 |
| 481 | |
| 482 | In this example, the name |
| 483 | of the top-level directory of your local Yocto Project repository is |
| 484 | ``poky``. After moving to the ``poky`` directory, the ``git fetch`` |
| 485 | command makes all the upstream tags available locally in your |
| 486 | repository. Finally, the ``git checkout`` command creates and checks out |
| 487 | a branch named "my-rocko-18.0.0" that is based on the upstream branch |
| 488 | whose "HEAD" matches the commit in the repository associated with the |
| 489 | "rocko-18.0.0" tag. The files in your repository now exactly match that |
| 490 | particular Yocto Project release as it is tagged in the upstream Git |
| 491 | repository. It is important to understand that when you create and |
| 492 | checkout a local working branch based on a tag, your environment matches |
| 493 | a specific point in time and not the entire development branch (i.e. |
| 494 | from the "tip" of the branch backwards). |
| 495 | |
| 496 | Basic Commands |
| 497 | -------------- |
| 498 | |
| 499 | Git has an extensive set of commands that lets you manage changes and |
| 500 | perform collaboration over the life of a project. Conveniently though, |
| 501 | you can manage with a small set of basic operations and workflows once |
| 502 | you understand the basic philosophy behind Git. You do not have to be an |
| 503 | expert in Git to be functional. A good place to look for instruction on |
| 504 | a minimal set of Git commands is |
Andrew Geissler | d1e8949 | 2021-02-12 15:35:20 -0600 | [diff] [blame] | 505 | `here <https://git-scm.com/documentation>`__. |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 506 | |
| 507 | The following list of Git commands briefly describes some basic Git |
| 508 | operations as a way to get started. As with any set of commands, this |
| 509 | list (in most cases) simply shows the base command and omits the many |
| 510 | arguments it supports. See the Git documentation for complete |
| 511 | descriptions and strategies on how to use these commands: |
| 512 | |
| 513 | - *git init:* Initializes an empty Git repository. You cannot use |
| 514 | Git commands unless you have a ``.git`` repository. |
| 515 | |
| 516 | - *git clone:* Creates a local clone of a Git repository that is on |
| 517 | equal footing with a fellow developer's Git repository or an upstream |
| 518 | repository. |
| 519 | |
| 520 | - *git add:* Locally stages updated file contents to the index that |
| 521 | Git uses to track changes. You must stage all files that have changed |
| 522 | before you can commit them. |
| 523 | |
| 524 | - *git commit:* Creates a local "commit" that documents the changes |
| 525 | you made. Only changes that have been staged can be committed. |
| 526 | Commits are used for historical purposes, for determining if a |
| 527 | maintainer of a project will allow the change, and for ultimately |
| 528 | pushing the change from your local Git repository into the project's |
| 529 | upstream repository. |
| 530 | |
| 531 | - *git status:* Reports any modified files that possibly need to be |
| 532 | staged and gives you a status of where you stand regarding local |
| 533 | commits as compared to the upstream repository. |
| 534 | |
| 535 | - *git checkout branch-name:* Changes your local working branch and |
| 536 | in this form assumes the local branch already exists. This command is |
| 537 | analogous to "cd". |
| 538 | |
Andrew Geissler | 3b8a17c | 2021-04-15 15:55:55 -0500 | [diff] [blame] | 539 | - *git checkout -b working-branch upstream-branch:* Creates and |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 540 | checks out a working branch on your local machine. The local branch |
| 541 | tracks the upstream branch. You can use your local branch to isolate |
| 542 | your work. It is a good idea to use local branches when adding |
| 543 | specific features or changes. Using isolated branches facilitates |
| 544 | easy removal of changes if they do not work out. |
| 545 | |
| 546 | - *git branch:* Displays the existing local branches associated |
| 547 | with your local repository. The branch that you have currently |
| 548 | checked out is noted with an asterisk character. |
| 549 | |
| 550 | - *git branch -D branch-name:* Deletes an existing local branch. |
| 551 | You need to be in a local branch other than the one you are deleting |
| 552 | in order to delete branch-name. |
| 553 | |
Patrick Williams | 4585273 | 2022-04-02 08:58:32 -0500 | [diff] [blame] | 554 | - *git pull \-\-rebase*: Retrieves information from an upstream Git |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 555 | repository and places it in your local Git repository. You use this |
| 556 | command to make sure you are synchronized with the repository from |
Andrew Geissler | 595f630 | 2022-01-24 19:11:47 +0000 | [diff] [blame] | 557 | which you are basing changes (e.g. the "&DISTRO_NAME_NO_CAP;" |
Patrick Williams | 4585273 | 2022-04-02 08:58:32 -0500 | [diff] [blame] | 558 | branch). The ``--rebase`` option ensures that any local commits you |
Andrew Geissler | 595f630 | 2022-01-24 19:11:47 +0000 | [diff] [blame] | 559 | have in your branch are preserved at the top of your local branch. |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 560 | |
| 561 | - *git push repo-name local-branch:upstream-branch:* Sends |
| 562 | all your committed local changes to the upstream Git repository that |
| 563 | your local repository is tracking (e.g. a contribution repository). |
| 564 | The maintainer of the project draws from these repositories to merge |
| 565 | changes (commits) into the appropriate branch of project's upstream |
| 566 | repository. |
| 567 | |
| 568 | - *git merge:* Combines or adds changes from one local branch of |
| 569 | your repository with another branch. When you create a local Git |
Andrew Geissler | 595f630 | 2022-01-24 19:11:47 +0000 | [diff] [blame] | 570 | repository, the default branch may be named "main". A typical |
| 571 | workflow is to create a temporary branch that is based off "main" |
| 572 | that you would use for isolated work. You would make your changes in |
| 573 | that isolated branch, stage and commit them locally, switch to the |
| 574 | "main" branch, and then use the ``git merge`` command to apply the |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 575 | changes from your isolated branch into the currently checked out |
Andrew Geissler | 595f630 | 2022-01-24 19:11:47 +0000 | [diff] [blame] | 576 | branch (e.g. "main"). After the merge is complete and if you are |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 577 | done with working in that isolated branch, you can safely delete the |
| 578 | isolated branch. |
| 579 | |
| 580 | - *git cherry-pick commits:* Choose and apply specific commits from |
| 581 | one branch into another branch. There are times when you might not be |
| 582 | able to merge all the changes in one branch with another but need to |
| 583 | pick out certain ones. |
| 584 | |
| 585 | - *gitk:* Provides a GUI view of the branches and changes in your |
| 586 | local Git repository. This command is a good way to graphically see |
| 587 | where things have diverged in your local repository. |
| 588 | |
| 589 | .. note:: |
| 590 | |
| 591 | You need to install the |
| 592 | gitk |
| 593 | package on your development system to use this command. |
| 594 | |
| 595 | - *git log:* Reports a history of your commits to the repository. |
| 596 | This report lists all commits regardless of whether you have pushed |
| 597 | them upstream or not. |
| 598 | |
| 599 | - *git diff:* Displays line-by-line differences between a local |
| 600 | working file and the same file as understood by Git. This command is |
| 601 | useful to see what you have changed in any given file. |
| 602 | |
| 603 | Licensing |
| 604 | ========= |
| 605 | |
| 606 | Because open source projects are open to the public, they have different |
| 607 | licensing structures in place. License evolution for both Open Source |
| 608 | and Free Software has an interesting history. If you are interested in |
| 609 | this history, you can find basic information here: |
| 610 | |
| 611 | - `Open source license |
Andrew Geissler | d1e8949 | 2021-02-12 15:35:20 -0600 | [diff] [blame] | 612 | history <https://en.wikipedia.org/wiki/Open-source_license>`__ |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 613 | |
| 614 | - `Free software license |
Andrew Geissler | d1e8949 | 2021-02-12 15:35:20 -0600 | [diff] [blame] | 615 | history <https://en.wikipedia.org/wiki/Free_software_license>`__ |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 616 | |
| 617 | In general, the Yocto Project is broadly licensed under the |
| 618 | Massachusetts Institute of Technology (MIT) License. MIT licensing |
| 619 | permits the reuse of software within proprietary software as long as the |
Patrick Williams | 03907ee | 2022-05-01 06:28:52 -0500 | [diff] [blame] | 620 | license is distributed with that software. Patches to the Yocto Project |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 621 | follow the upstream licensing scheme. You can find information on the |
Patrick Williams | 03907ee | 2022-05-01 06:28:52 -0500 | [diff] [blame] | 622 | MIT license `here <https://en.wikipedia.org/wiki/MIT_License>`__. |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 623 | |
| 624 | When you build an image using the Yocto Project, the build process uses |
| 625 | a known list of licenses to ensure compliance. You can find this list in |
| 626 | the :term:`Source Directory` at |
| 627 | ``meta/files/common-licenses``. Once the build completes, the list of |
| 628 | all licenses found and used during that build are kept in the |
| 629 | :term:`Build Directory` at |
| 630 | ``tmp/deploy/licenses``. |
| 631 | |
| 632 | If a module requires a license that is not in the base list, the build |
| 633 | process generates a warning during the build. These tools make it easier |
| 634 | for a developer to be certain of the licenses with which their shipped |
| 635 | products must comply. However, even with these tools it is still up to |
| 636 | the developer to resolve potential licensing issues. |
| 637 | |
| 638 | The base list of licenses used by the build process is a combination of |
| 639 | the Software Package Data Exchange (SPDX) list and the Open Source |
Andrew Geissler | d1e8949 | 2021-02-12 15:35:20 -0600 | [diff] [blame] | 640 | Initiative (OSI) projects. `SPDX Group <https://spdx.org>`__ is a working |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 641 | group of the Linux Foundation that maintains a specification for a |
| 642 | standard format for communicating the components, licenses, and |
| 643 | copyrights associated with a software package. |
Andrew Geissler | d1e8949 | 2021-02-12 15:35:20 -0600 | [diff] [blame] | 644 | `OSI <https://opensource.org>`__ is a corporation dedicated to the Open |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 645 | Source Definition and the effort for reviewing and approving licenses |
| 646 | that conform to the Open Source Definition (OSD). |
| 647 | |
| 648 | You can find a list of the combined SPDX and OSI licenses that the Yocto |
| 649 | Project uses in the ``meta/files/common-licenses`` directory in your |
| 650 | :term:`Source Directory`. |
| 651 | |
| 652 | For information that can help you maintain compliance with various open |
| 653 | source licensing during the lifecycle of a product created using the |
| 654 | Yocto Project, see the |
Andrew Geissler | 09209ee | 2020-12-13 08:44:15 -0600 | [diff] [blame] | 655 | ":ref:`dev-manual/common-tasks:maintaining open source license compliance during your product's lifecycle`" |
Andrew Geissler | c9f7865 | 2020-09-18 14:11:35 -0500 | [diff] [blame] | 656 | section in the Yocto Project Development Tasks Manual. |