blob: 63d63fc27e8c6fae80037048259e30162eeca0d3 [file] [log] [blame]
Andrew Geisslerf0343792020-11-18 10:42:21 -06001.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Andrew Geisslerc9f78652020-09-18 14:11:35 -05002
3***********************************
4Setting Up to Use the Yocto Project
5***********************************
6
7This chapter provides guidance on how to prepare to use the Yocto
8Project. You can learn about creating a team environment to develop
9using the Yocto Project, how to set up a :ref:`build
Andrew Geissler09209ee2020-12-13 08:44:15 -060010host <dev-manual/start:preparing the build host>`, how to locate
Andrew Geisslerc9f78652020-09-18 14:11:35 -050011Yocto Project source repositories, and how to create local Git
12repositories.
13
Andrew Geisslerc9f78652020-09-18 14:11:35 -050014Creating a Team Development Environment
15=======================================
16
17It might not be immediately clear how you can use the Yocto Project in a
18team development environment, or how to scale it for a large team of
19developers. You can adapt the Yocto Project to many different use cases
20and scenarios; however, this flexibility could cause difficulties if you
21are trying to create a working setup that scales effectively.
22
23To help you understand how to set up this type of environment, this
24section presents a procedure that gives you information that can help
25you get the results you want. The procedure is high-level and presents
26some of the project's most successful experiences, practices, solutions,
27and available technologies that have proved to work well in the past;
28however, keep in mind, the procedure here is simply a starting point.
29You can build off these steps and customize the procedure to fit any
30particular working environment and set of practices.
31
321. *Determine Who is Going to be Developing:* You first need to
33 understand who is going to be doing anything related to the Yocto
34 Project and determine their roles. Making this determination is
35 essential to completing subsequent steps, which are to get your
36 equipment together and set up your development environment's
37 hardware topology.
38
William A. Kennington IIIac69b482021-06-02 12:28:27 -070039 Here are possible roles:
Andrew Geisslerc9f78652020-09-18 14:11:35 -050040
41 - *Application Developer:* This type of developer does application
42 level work on top of an existing software stack.
43
44 - *Core System Developer:* This type of developer works on the
45 contents of the operating system image itself.
46
47 - *Build Engineer:* This type of developer manages Autobuilders and
48 releases. Depending on the specifics of the environment, not all
49 situations might need a Build Engineer.
50
51 - *Test Engineer:* This type of developer creates and manages
52 automated tests that are used to ensure all application and core
53 system development meets desired quality standards.
54
552. *Gather the Hardware:* Based on the size and make-up of the team,
56 get the hardware together. Ideally, any development, build, or test
57 engineer uses a system that runs a supported Linux distribution.
58 These systems, in general, should be high performance (e.g. dual,
59 six-core Xeons with 24 Gbytes of RAM and plenty of disk space). You
60 can help ensure efficiency by having any machines used for testing
61 or that run Autobuilders be as high performance as possible.
62
63 .. note::
64
65 Given sufficient processing power, you might also consider
66 building Yocto Project development containers to be run under
67 Docker, which is described later.
68
693. *Understand the Hardware Topology of the Environment:* Once you
70 understand the hardware involved and the make-up of the team, you
71 can understand the hardware topology of the development environment.
72 You can get a visual idea of the machines and their roles across the
73 development environment.
74
754. *Use Git as Your Source Control Manager (SCM):* Keeping your
76 :term:`Metadata` (i.e. recipes,
77 configuration files, classes, and so forth) and any software you are
78 developing under the control of an SCM system that is compatible
79 with the OpenEmbedded build system is advisable. Of all of the SCMs
80 supported by BitBake, the Yocto Project team strongly recommends using
Andrew Geissler09209ee2020-12-13 08:44:15 -060081 :ref:`overview-manual/development-environment:git`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -050082 Git is a distributed system
83 that is easy to back up, allows you to work remotely, and then
84 connects back to the infrastructure.
85
86 .. note::
87
88 For information about BitBake, see the
Andrew Geissler4c19ea12020-10-27 13:52:24 -050089 :doc:`bitbake:index`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -050090
91 It is relatively easy to set up Git services and create
Andrew Geissler4c19ea12020-10-27 13:52:24 -050092 infrastructure like :yocto_git:`/`, which is based on
Andrew Geisslerc9f78652020-09-18 14:11:35 -050093 server software called ``gitolite`` with ``cgit`` being used to
94 generate the web interface that lets you view the repositories. The
95 ``gitolite`` software identifies users using SSH keys and allows
96 branch-based access controls to repositories that you can control as
97 little or as much as necessary.
98
99 .. note::
100
101 The setup of these services is beyond the scope of this manual.
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700102 However, here are sites describing how to perform setup:
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500103
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500104 - `Gitolite <https://gitolite.com>`__: Information for
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500105 ``gitolite``.
106
107 - `Interfaces, frontends, and
108 tools <https://git.wiki.kernel.org/index.php/Interfaces,_frontends,_and_tools>`__:
109 Documentation on how to create interfaces and frontends for
110 Git.
111
1125. *Set up the Application Development Machines:* As mentioned earlier,
113 application developers are creating applications on top of existing
114 software stacks. Following are some best practices for setting up
115 machines used for application development:
116
117 - Use a pre-built toolchain that contains the software stack
118 itself. Then, develop the application code on top of the stack.
119 This method works well for small numbers of relatively isolated
120 applications.
121
122 - Keep your cross-development toolchains updated. You can do this
123 through provisioning either as new toolchain downloads or as
124 updates through a package update mechanism using ``opkg`` to
125 provide updates to an existing toolchain. The exact mechanics of
126 how and when to do this depend on local policy.
127
128 - Use multiple toolchains installed locally into different
129 locations to allow development across versions.
130
1316. *Set up the Core Development Machines:* As mentioned earlier, core
132 developers work on the contents of the operating system itself.
133 Following are some best practices for setting up machines used for
134 developing images:
135
136 - Have the :term:`OpenEmbedded Build System` available on
137 the developer workstations so developers can run their own builds
138 and directly rebuild the software stack.
139
140 - Keep the core system unchanged as much as possible and do your
141 work in layers on top of the core system. Doing so gives you a
142 greater level of portability when upgrading to new versions of
143 the core system or Board Support Packages (BSPs).
144
145 - Share layers amongst the developers of a particular project and
146 contain the policy configuration that defines the project.
147
1487. *Set up an Autobuilder:* Autobuilders are often the core of the
149 development environment. It is here that changes from individual
150 developers are brought together and centrally tested. Based on this
151 automated build and test environment, subsequent decisions about
152 releases can be made. Autobuilders also allow for "continuous
153 integration" style testing of software components and regression
154 identification and tracking.
155
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500156 See ":yocto_ab:`Yocto Project Autobuilder <>`" for more
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500157 information and links to buildbot. The Yocto Project team has found
158 this implementation works well in this role. A public example of
159 this is the Yocto Project Autobuilders, which the Yocto Project team
160 uses to test the overall health of the project.
161
162 The features of this system are:
163
164 - Highlights when commits break the build.
165
166 - Populates an :ref:`sstate
Andrew Geissler09209ee2020-12-13 08:44:15 -0600167 cache <overview-manual/concepts:shared state cache>` from which
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500168 developers can pull rather than requiring local builds.
169
170 - Allows commit hook triggers, which trigger builds when commits
171 are made.
172
173 - Allows triggering of automated image booting and testing under
174 the QuickEMUlator (QEMU).
175
176 - Supports incremental build testing and from-scratch builds.
177
178 - Shares output that allows developer testing and historical
179 regression investigation.
180
181 - Creates output that can be used for releases.
182
183 - Allows scheduling of builds so that resources can be used
184 efficiently.
185
1868. *Set up Test Machines:* Use a small number of shared, high
187 performance systems for testing purposes. Developers can use these
188 systems for wider, more extensive testing while they continue to
189 develop locally using their primary development system.
190
1919. *Document Policies and Change Flow:* The Yocto Project uses a
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700192 hierarchical structure and a pull model. There are scripts to create and
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500193 send pull requests (i.e. ``create-pull-request`` and
194 ``send-pull-request``). This model is in line with other open source
195 projects where maintainers are responsible for specific areas of the
196 project and a single maintainer handles the final "top-of-tree"
197 merges.
198
199 .. note::
200
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500201 You can also use a more collective push model. The ``gitolite``
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500202 software supports both the push and pull models quite easily.
203
204 As with any development environment, it is important to document the
205 policy used as well as any main project guidelines so they are
206 understood by everyone. It is also a good idea to have
207 well-structured commit messages, which are usually a part of a
208 project's guidelines. Good commit messages are essential when
209 looking back in time and trying to understand why changes were made.
210
211 If you discover that changes are needed to the core layer of the
212 project, it is worth sharing those with the community as soon as
213 possible. Chances are if you have discovered the need for changes,
214 someone else in the community needs them also.
215
21610. *Development Environment Summary:* Aside from the previous steps,
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700217 here are best practices within the Yocto Project development
218 environment:
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500219
Andrew Geissler09209ee2020-12-13 08:44:15 -0600220 - Use :ref:`overview-manual/development-environment:git` as the source control
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500221 system.
222
223 - Maintain your Metadata in layers that make sense for your
Andrew Geissler09209ee2020-12-13 08:44:15 -0600224 situation. See the ":ref:`overview-manual/yp-intro:the yocto project layer model`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500225 section in the Yocto Project Overview and Concepts Manual and the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600226 ":ref:`dev-manual/common-tasks:understanding and creating layers`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500227 section for more information on layers.
228
229 - Separate the project's Metadata and code by using separate Git
Andrew Geissler09209ee2020-12-13 08:44:15 -0600230 repositories. See the ":ref:`overview-manual/development-environment:yocto project source repositories`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500231 section in the Yocto Project Overview and Concepts Manual for
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500232 information on these repositories. See the
233 ":ref:`dev-manual/start:locating yocto project source files`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500234 section for information on how to set up local Git repositories
235 for related upstream Yocto Project Git repositories.
236
237 - Set up the directory for the shared state cache
238 (:term:`SSTATE_DIR`) where
239 it makes sense. For example, set up the sstate cache on a system
240 used by developers in the same organization and share the same
241 source directories on their machines.
242
243 - Set up an Autobuilder and have it populate the sstate cache and
244 source directories.
245
246 - The Yocto Project community encourages you to send patches to the
247 project to fix bugs or add features. If you do submit patches,
248 follow the project commit guidelines for writing good commit
249 messages. See the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600250 ":ref:`dev-manual/common-tasks:submitting a change to the yocto project`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500251 section.
252
253 - Send changes to the core sooner than later as others are likely
254 to run into the same issues. For some guidance on mailing lists
255 to use, see the list in the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600256 ":ref:`dev-manual/common-tasks:submitting a change to the yocto project`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500257 section. For a description
258 of the available mailing lists, see the ":ref:`resources-mailinglist`" section in
259 the Yocto Project Reference Manual.
260
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500261Preparing the Build Host
262========================
263
264This section provides procedures to set up a system to be used as your
265:term:`Build Host` for
266development using the Yocto Project. Your build host can be a native
267Linux machine (recommended), it can be a machine (Linux, Mac, or
268Windows) that uses `CROPS <https://github.com/crops/poky-container>`__,
269which leverages `Docker Containers <https://www.docker.com/>`__ or it
Patrick Williams2390b1b2022-11-03 13:47:49 -0500270can be a Windows machine capable of running version 2 of Windows Subsystem
271For Linux (WSL 2).
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500272
273.. note::
274
Patrick Williams2390b1b2022-11-03 13:47:49 -0500275 The Yocto Project is not compatible with version 1 of
Patrick Williams7784c422022-11-17 07:29:11 -0600276 :wikipedia:`Windows Subsystem for Linux <Windows_Subsystem_for_Linux>`.
Patrick Williams2390b1b2022-11-03 13:47:49 -0500277 It is compatible but neither officially supported nor validated with
278 WSL 2. If you still decide to use WSL please upgrade to
279 `WSL 2 <https://learn.microsoft.com/en-us/windows/wsl/install>`__.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500280
281Once your build host is set up to use the Yocto Project, further steps
282are necessary depending on what you want to accomplish. See the
283following references for information on how to prepare for Board Support
284Package (BSP) development and kernel development:
285
286- *BSP Development:* See the ":ref:`bsp-guide/bsp:preparing your build host to work with bsp layers`"
287 section in the Yocto Project Board Support Package (BSP) Developer's
288 Guide.
289
Andrew Geissler09209ee2020-12-13 08:44:15 -0600290- *Kernel Development:* See the ":ref:`kernel-dev/common:preparing the build host to work on the kernel`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500291 section in the Yocto Project Linux Kernel Development Manual.
292
293Setting Up a Native Linux Host
294------------------------------
295
296Follow these steps to prepare a native Linux machine as your Yocto
297Project Build Host:
298
2991. *Use a Supported Linux Distribution:* You should have a reasonably
300 current Linux-based host system. You will have the best results with
301 a recent release of Fedora, openSUSE, Debian, Ubuntu, RHEL or CentOS
302 as these releases are frequently tested against the Yocto Project and
303 officially supported. For a list of the distributions under
304 validation and their status, see the ":ref:`Supported Linux
305 Distributions <detailed-supported-distros>`"
306 section in the Yocto Project Reference Manual and the wiki page at
Andrew Geissler09209ee2020-12-13 08:44:15 -0600307 :yocto_wiki:`Distribution Support </Distribution_Support>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500308
3092. *Have Enough Free Memory:* Your system should have at least 50 Gbytes
310 of free disk space for building images.
311
3123. *Meet Minimal Version Requirements:* The OpenEmbedded build system
313 should be able to run on any modern distribution that has the
Andrew Geissler615f2f12022-07-15 14:00:58 -0500314 following versions for Git, tar, Python, gcc and make.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500315
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500316 - Git &MIN_GIT_VERSION; or greater
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500317
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500318 - tar &MIN_TAR_VERSION; or greater
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500319
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500320 - Python &MIN_PYTHON_VERSION; or greater.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500321
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500322 - gcc &MIN_GCC_VERSION; or greater.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500323
Andrew Geissler615f2f12022-07-15 14:00:58 -0500324 - GNU make &MIN_MAKE_VERSION; or greater
325
Andrew Geissler5199d832021-09-24 16:47:35 -0500326 If your build host does not meet any of these listed version
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500327 requirements, you can take steps to prepare the system so that you
328 can still use the Yocto Project. See the
Andrew Geissler615f2f12022-07-15 14:00:58 -0500329 ":ref:`ref-manual/system-requirements:required git, tar, python, make and gcc versions`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500330 section in the Yocto Project Reference Manual for information.
331
3324. *Install Development Host Packages:* Required development host
333 packages vary depending on your build host and what you want to do
334 with the Yocto Project. Collectively, the number of required packages
335 is large if you want to be able to cover all cases.
336
337 For lists of required packages for all scenarios, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600338 ":ref:`ref-manual/system-requirements:required packages for the build host`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500339 section in the Yocto Project Reference Manual.
340
341Once you have completed the previous steps, you are ready to continue
342using a given development path on your native Linux machine. If you are
343going to use BitBake, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600344":ref:`dev-manual/start:cloning the \`\`poky\`\` repository`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500345section. If you are going
Andrew Geissler09209ee2020-12-13 08:44:15 -0600346to use the Extensible SDK, see the ":doc:`/sdk-manual/extensible`" Chapter in the Yocto
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500347Project Application Development and the Extensible Software Development
Andrew Geissler09209ee2020-12-13 08:44:15 -0600348Kit (eSDK) manual. If you want to work on the kernel, see the :doc:`/kernel-dev/index`. If you are going to use
349Toaster, see the ":doc:`/toaster-manual/setup-and-use`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500350section in the Toaster User Manual.
351
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500352Setting Up to Use CROss PlatformS (CROPS)
353-----------------------------------------
354
355With `CROPS <https://github.com/crops/poky-container>`__, which
356leverages `Docker Containers <https://www.docker.com/>`__, you can
357create a Yocto Project development environment that is operating system
358agnostic. You can set up a container in which you can develop using the
359Yocto Project on a Windows, Mac, or Linux machine.
360
361Follow these general steps to prepare a Windows, Mac, or Linux machine
362as your Yocto Project build host:
363
3641. *Determine What Your Build Host Needs:*
365 `Docker <https://www.docker.com/what-docker>`__ is a software
366 container platform that you need to install on the build host.
367 Depending on your build host, you might have to install different
368 software to support Docker containers. Go to the Docker installation
369 page and read about the platform requirements in "`Supported
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500370 Platforms <https://docs.docker.com/engine/install/#supported-platforms>`__"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500371 your build host needs to run containers.
372
3732. *Choose What To Install:* Depending on whether or not your build host
374 meets system requirements, you need to install "Docker CE Stable" or
375 the "Docker Toolbox". Most situations call for Docker CE. However, if
376 you have a build host that does not meet requirements (e.g.
377 Pre-Windows 10 or Windows 10 "Home" version), you must install Docker
378 Toolbox instead.
379
3803. *Go to the Install Site for Your Platform:* Click the link for the
381 Docker edition associated with your build host's native software. For
382 example, if your build host is running Microsoft Windows Version 10
383 and you want the Docker CE Stable edition, click that link under
384 "Supported Platforms".
385
3864. *Install the Software:* Once you have understood all the
387 pre-requisites, you can download and install the appropriate
388 software. Follow the instructions for your specific machine and the
389 type of the software you need to install:
390
Andrew Geisslerc926e172021-05-07 16:11:35 -0500391 - Install `Docker Desktop on
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500392 Windows <https://docs.docker.com/docker-for-windows/install/#install-docker-desktop-on-windows>`__
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500393 for Windows build hosts that meet requirements.
394
Andrew Geisslerc926e172021-05-07 16:11:35 -0500395 - Install `Docker Desktop on
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500396 MacOs <https://docs.docker.com/docker-for-mac/install/#install-and-run-docker-desktop-on-mac>`__
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500397 for Mac build hosts that meet requirements.
398
Andrew Geisslerc926e172021-05-07 16:11:35 -0500399 - Install `Docker Engine on
400 CentOS <https://docs.docker.com/engine/install/centos/>`__
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500401 for Linux build hosts running the CentOS distribution.
402
Andrew Geisslerc926e172021-05-07 16:11:35 -0500403 - Install `Docker Engine on
404 Debian <https://docs.docker.com/engine/install/debian/>`__
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500405 for Linux build hosts running the Debian distribution.
406
Andrew Geisslerc926e172021-05-07 16:11:35 -0500407 - Install `Docker Engine for
408 Fedora <https://docs.docker.com/engine/install/fedora/>`__
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500409 for Linux build hosts running the Fedora distribution.
410
Andrew Geisslerc926e172021-05-07 16:11:35 -0500411 - Install `Docker Engine for
412 Ubuntu <https://docs.docker.com/engine/install/ubuntu/>`__
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500413 for Linux build hosts running the Ubuntu distribution.
414
4155. *Optionally Orient Yourself With Docker:* If you are unfamiliar with
416 Docker and the container concept, you can learn more here -
417 https://docs.docker.com/get-started/.
418
4196. *Launch Docker or Docker Toolbox:* You should be able to launch
420 Docker or the Docker Toolbox and have a terminal shell on your
421 development host.
422
4237. *Set Up the Containers to Use the Yocto Project:* Go to
424 https://github.com/crops/docker-win-mac-docs/wiki and follow
425 the directions for your particular build host (i.e. Linux, Mac, or
426 Windows).
427
428 Once you complete the setup instructions for your machine, you have
429 the Poky, Extensible SDK, and Toaster containers available. You can
430 click those links from the page and learn more about using each of
431 those containers.
432
433Once you have a container set up, everything is in place to develop just
434as if you were running on a native Linux machine. If you are going to
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500435use the Poky container, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600436":ref:`dev-manual/start:cloning the \`\`poky\`\` repository`"
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500437section. If you are going to use the Extensible SDK container, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600438":doc:`/sdk-manual/extensible`" Chapter in the Yocto
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500439Project Application Development and the Extensible Software Development
440Kit (eSDK) manual. If you are going to use the Toaster container, see
Andrew Geissler09209ee2020-12-13 08:44:15 -0600441the ":doc:`/toaster-manual/setup-and-use`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500442section in the Toaster User Manual.
443
Patrick Williams2390b1b2022-11-03 13:47:49 -0500444Setting Up to Use Windows Subsystem For Linux (WSL 2)
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500445-----------------------------------------------------
446
Patrick Williams2390b1b2022-11-03 13:47:49 -0500447With `Windows Subsystem for Linux (WSL 2)
448<https://learn.microsoft.com/en-us/windows/wsl/>`__,
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500449you can create a Yocto Project development environment that allows you
450to build on Windows. You can set up a Linux distribution inside Windows
451in which you can develop using the Yocto Project.
452
Patrick Williams2390b1b2022-11-03 13:47:49 -0500453Follow these general steps to prepare a Windows machine using WSL 2 as
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500454your Yocto Project build host:
455
Patrick Williams2390b1b2022-11-03 13:47:49 -05004561. *Make sure your Windows machine is capable of running WSL 2:*
457
458 While all Windows 11 and Windows Server 2022 builds support WSL 2,
459 the first versions of Windows 10 and Windows Server 2019 didn't.
460 Check the minimum build numbers for `Windows 10
461 <https://learn.microsoft.com/en-us/windows/wsl/install-manual#step-2---check-requirements-for-running-wsl-2>`__
462 and for `Windows Server 2019
463 <https://learn.microsoft.com/en-us/windows/wsl/install-on-server>`__.
464
465 To check which build version you are running, you may open a command
466 prompt on Windows and execute the command "ver"::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500467
468 C:\Users\myuser> ver
469
470 Microsoft Windows [Version 10.0.19041.153]
471
Patrick Williams2390b1b2022-11-03 13:47:49 -05004722. *Install the Linux distribution of your choice inside WSL 2:*
473 Once you know your version of Windows supports WSL 2, you can
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500474 install the distribution of your choice from the Microsoft Store.
475 Open the Microsoft Store and search for Linux. While there are
476 several Linux distributions available, the assumption is that your
477 pick will be one of the distributions supported by the Yocto Project
478 as stated on the instructions for using a native Linux host. After
479 making your selection, simply click "Get" to download and install the
480 distribution.
481
Patrick Williams2390b1b2022-11-03 13:47:49 -05004823. *Check which Linux distribution WSL 2 is using:* Open a Windows
Andrew Geisslerc926e172021-05-07 16:11:35 -0500483 PowerShell and run::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500484
485 C:\WINDOWS\system32> wsl -l -v
486 NAME STATE VERSION
487 *Ubuntu Running 2
488
Patrick Williams2390b1b2022-11-03 13:47:49 -0500489 Note that WSL 2 supports running as many different Linux distributions
490 as you want to install.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500491
Patrick Williams2390b1b2022-11-03 13:47:49 -05004924. *Optionally Get Familiar with WSL:* You can learn more on
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500493 https://docs.microsoft.com/en-us/windows/wsl/wsl2-about.
494
4955. *Launch your WSL Distibution:* From the Windows start menu simply
496 launch your WSL distribution just like any other application.
497
Patrick Williams2390b1b2022-11-03 13:47:49 -05004986. *Optimize your WSL 2 storage often:* Due to the way storage is
499 handled on WSL 2, the storage space used by the underlying Linux
Patrick Williams975a06f2022-10-21 14:42:47 -0500500 distribution is not reflected immediately, and since BitBake heavily
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500501 uses storage, after several builds, you may be unaware you are
Patrick Williams2390b1b2022-11-03 13:47:49 -0500502 running out of space. As WSL 2 uses a VHDX file for storage, this issue
503 can be easily avoided by regularly optimizing this file in a manual way:
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500504
Andrew Geissler9aee5002022-03-30 16:27:02 +0000505 1. *Find the location of your VHDX file:*
506
507 First you need to find the distro app package directory, to achieve this
508 open a Windows Powershell as Administrator and run::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500509
510 C:\WINDOWS\system32> Get-AppxPackage -Name "*Ubuntu*" | Select PackageFamilyName
511 PackageFamilyName
512 -----------------
513 CanonicalGroupLimited.UbuntuonWindows_79abcdefgh
514
515
516 You should now
517 replace the PackageFamilyName and your user on the following path
Andrew Geisslerc926e172021-05-07 16:11:35 -0500518 to find your VHDX file::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500519
Andrew Geissler9aee5002022-03-30 16:27:02 +0000520 ls C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\
521 Mode LastWriteTime Length Name
522 -a---- 3/14/2020 9:52 PM 57418973184 ext4.vhdx
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500523
524 Your VHDX file path is:
525 ``C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\ext4.vhdx``
526
Andrew Geissler9aee5002022-03-30 16:27:02 +0000527 2a. *Optimize your VHDX file using Windows Powershell:*
528
529 To use the ``optimize-vhd`` cmdlet below, first install the Hyper-V
530 option on Windows. Then, open a Windows Powershell as Administrator to
531 optimize your VHDX file, shutting down WSL first::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500532
533 C:\WINDOWS\system32> wsl --shutdown
534 C:\WINDOWS\system32> optimize-vhd -Path C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\ext4.vhdx -Mode full
535
Andrew Geissler9aee5002022-03-30 16:27:02 +0000536 A progress bar should be shown while optimizing the
537 VHDX file, and storage should now be reflected correctly on the
538 Windows Explorer.
539
540 2b. *Optimize your VHDX file using DiskPart:*
541
542 The ``optimize-vhd`` cmdlet noted in step 2a above is provided by
543 Hyper-V. Not all SKUs of Windows can install Hyper-V. As an alternative,
544 use the DiskPart tool. To start, open a Windows command prompt as
545 Administrator to optimize your VHDX file, shutting down WSL first::
546
547 C:\WINDOWS\system32> wsl --shutdown
548 C:\WINDOWS\system32> diskpart
549
550 DISKPART> select vdisk file="<path_to_VHDX_file>"
551 DISKPART> attach vdisk readonly
552 DISKPART> compact vdisk
553 DISKPART> exit
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500554
555.. note::
556
Patrick Williams2390b1b2022-11-03 13:47:49 -0500557 The current implementation of WSL 2 does not have out-of-the-box
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500558 access to external devices such as those connected through a USB
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500559 port, but it automatically mounts your ``C:`` drive on ``/mnt/c/``
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500560 (and others), which you can use to share deploy artifacts to be later
Patrick Williams2390b1b2022-11-03 13:47:49 -0500561 flashed on hardware through Windows, but your :term:`Build Directory`
562 should not reside inside this mountpoint.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500563
Patrick Williams2390b1b2022-11-03 13:47:49 -0500564Once you have WSL 2 set up, everything is in place to develop just as if
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500565you were running on a native Linux machine. If you are going to use the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600566Extensible SDK container, see the ":doc:`/sdk-manual/extensible`" Chapter in the Yocto
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500567Project Application Development and the Extensible Software Development
568Kit (eSDK) manual. If you are going to use the Toaster container, see
Andrew Geissler09209ee2020-12-13 08:44:15 -0600569the ":doc:`/toaster-manual/setup-and-use`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500570section in the Toaster User Manual.
571
572Locating Yocto Project Source Files
573===================================
574
575This section shows you how to locate, fetch and configure the source
576files you'll need to work with the Yocto Project.
577
578.. note::
579
580 - For concepts and introductory information about Git as it is used
Andrew Geissler09209ee2020-12-13 08:44:15 -0600581 in the Yocto Project, see the ":ref:`overview-manual/development-environment:git`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500582 section in the Yocto Project Overview and Concepts Manual.
583
584 - For concepts on Yocto Project source repositories, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600585 ":ref:`overview-manual/development-environment:yocto project source repositories`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500586 section in the Yocto Project Overview and Concepts Manual."
587
588Accessing Source Repositories
589-----------------------------
590
Andrew Geissler09209ee2020-12-13 08:44:15 -0600591Working from a copy of the upstream :ref:`dev-manual/start:accessing source repositories` is the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500592preferred method for obtaining and using a Yocto Project release. You
593can view the Yocto Project Source Repositories at
594:yocto_git:`/`. In particular, you can find the ``poky``
Andrew Geissler09209ee2020-12-13 08:44:15 -0600595repository at :yocto_git:`/poky`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500596
597Use the following procedure to locate the latest upstream copy of the
598``poky`` Git repository:
599
6001. *Access Repositories:* Open a browser and go to
601 :yocto_git:`/` to access the GUI-based interface into the
602 Yocto Project source repositories.
603
6042. *Select the Repository:* Click on the repository in which you are
605 interested (e.g. ``poky``).
606
6073. *Find the URL Used to Clone the Repository:* At the bottom of the
608 page, note the URL used to clone that repository
Andrew Geissler09209ee2020-12-13 08:44:15 -0600609 (e.g. :yocto_git:`/poky`).
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500610
611 .. note::
612
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500613 For information on cloning a repository, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600614 ":ref:`dev-manual/start:cloning the \`\`poky\`\` repository`" section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500615
616Accessing Index of Releases
617---------------------------
618
619Yocto Project maintains an Index of Releases area that contains related
620files that contribute to the Yocto Project. Rather than Git
621repositories, these files are tarballs that represent snapshots in time
622of a given component.
623
624.. note::
625
626 The recommended method for accessing Yocto Project components is to
627 use Git to clone the upstream repository and work from within that
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700628 locally cloned repository. However, this section documents how to
629 use a tarball snapshot of any given component.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500630
631Follow these steps to locate and download a particular tarball:
632
6331. *Access the Index of Releases:* Open a browser and go to
Andrew Geisslerc3d88e42020-10-02 09:45:00 -0500634 :yocto_dl:`Index of Releases </releases>`. The
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500635 list represents released components (e.g. ``bitbake``, ``sato``, and
636 so on).
637
638 .. note::
639
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500640 The ``yocto`` directory contains the full array of released Poky
641 tarballs. The ``poky`` directory in the Index of Releases was
642 historically used for very early releases and exists now only for
643 retroactive completeness.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500644
6452. *Select a Component:* Click on any released component in which you
646 are interested (e.g. ``yocto``).
647
6483. *Find the Tarball:* Drill down to find the associated tarball. For
649 example, click on ``yocto-&DISTRO;`` to view files associated with the
Andrew Geissler9aee5002022-03-30 16:27:02 +0000650 Yocto Project &DISTRO; release.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500651
6524. *Download the Tarball:* Click the tarball to download and save a
653 snapshot of the given component.
654
655Using the Downloads Page
656------------------------
657
658The :yocto_home:`Yocto Project Website <>` uses a "DOWNLOADS" page
659from which you can locate and download tarballs of any Yocto Project
660release. Rather than Git repositories, these files represent snapshot
661tarballs similar to the tarballs located in the Index of Releases
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500662described in the ":ref:`dev-manual/start:accessing index of releases`" section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500663
Andrew Geisslerc9f78652020-09-18 14:11:35 -05006641. *Go to the Yocto Project Website:* Open The
665 :yocto_home:`Yocto Project Website <>` in your browser.
666
6672. *Get to the Downloads Area:* Select the "DOWNLOADS" item from the
668 pull-down "SOFTWARE" tab menu near the top of the page.
669
6703. *Select a Yocto Project Release:* Use the menu next to "RELEASE" to
671 display and choose a recent or past supported Yocto Project release
672 (e.g. &DISTRO_NAME_NO_CAP;, &DISTRO_NAME_NO_CAP_MINUS_ONE;, and so forth).
673
674 .. note::
675
676 For a "map" of Yocto Project releases to version numbers, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600677 :yocto_wiki:`Releases </Releases>` wiki page.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500678
679 You can use the "RELEASE ARCHIVE" link to reveal a menu of all Yocto
680 Project releases.
681
6824. *Download Tools or Board Support Packages (BSPs):* From the
683 "DOWNLOADS" page, you can download tools or BSPs as well. Just scroll
684 down the page and look for what you need.
685
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500686Cloning and Checking Out Branches
687=================================
688
689To use the Yocto Project for development, you need a release locally
690installed on your development system. This locally installed set of
691files is referred to as the :term:`Source Directory`
692in the Yocto Project documentation.
693
694The preferred method of creating your Source Directory is by using
Andrew Geissler09209ee2020-12-13 08:44:15 -0600695:ref:`overview-manual/development-environment:git` to clone a local copy of the upstream
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500696``poky`` repository. Working from a cloned copy of the upstream
697repository allows you to contribute back into the Yocto Project or to
698simply work with the latest software on a development branch. Because
699Git maintains and creates an upstream repository with a complete history
700of changes and you are working with a local clone of that repository,
701you have access to all the Yocto Project development branches and tag
702names used in the upstream repository.
703
704Cloning the ``poky`` Repository
705-------------------------------
706
707Follow these steps to create a local version of the upstream
708:term:`Poky` Git repository.
709
7101. *Set Your Directory:* Change your working directory to where you want
711 to create your local copy of ``poky``.
712
7132. *Clone the Repository:* The following example command clones the
714 ``poky`` repository and uses the default name "poky" for your local
Andrew Geisslerc926e172021-05-07 16:11:35 -0500715 repository::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500716
717 $ git clone git://git.yoctoproject.org/poky
718 Cloning into 'poky'...
719 remote: Counting objects: 432160, done.
720 remote: Compressing objects: 100% (102056/102056), done.
721 remote: Total 432160 (delta 323116), reused 432037 (delta 323000)
722 Receiving objects: 100% (432160/432160), 153.81 MiB | 8.54 MiB/s, done.
723 Resolving deltas: 100% (323116/323116), done.
724 Checking connectivity... done.
725
726 Unless you
727 specify a specific development branch or tag name, Git clones the
728 "master" branch, which results in a snapshot of the latest
729 development changes for "master". For information on how to check out
730 a specific development branch or on how to check out a local branch
Andrew Geissler3b8a17c2021-04-15 15:55:55 -0500731 based on a tag name, see the
732 ":ref:`dev-manual/start:checking out by branch in poky`" and
733 ":ref:`dev-manual/start:checking out by tag in poky`" sections, respectively.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500734
735 Once the local repository is created, you can change to that
William A. Kennington IIIac69b482021-06-02 12:28:27 -0700736 directory and check its status. The ``master`` branch is checked out
737 by default::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500738
Andrew Geissler95ac1b82021-03-31 14:34:31 -0500739 $ cd poky
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500740 $ git status
741 On branch master
742 Your branch is up-to-date with 'origin/master'.
743 nothing to commit, working directory clean
744 $ git branch
745 * master
746
747 Your local repository of poky is identical to the
748 upstream poky repository at the time from which it was cloned. As you
749 work with the local branch, you can periodically use the
750 ``git pull --rebase`` command to be sure you are up-to-date
751 with the upstream branch.
752
753Checking Out by Branch in Poky
754------------------------------
755
756When you clone the upstream poky repository, you have access to all its
757development branches. Each development branch in a repository is unique
758as it forks off the "master" branch. To see and use the files of a
759particular development branch locally, you need to know the branch name
760and then specifically check out that development branch.
761
762.. note::
763
764 Checking out an active development branch by branch name gives you a
765 snapshot of that particular branch at the time you check it out.
766 Further development on top of the branch that occurs after check it
767 out can occur.
768
7691. *Switch to the Poky Directory:* If you have a local poky Git
770 repository, switch to that directory. If you do not have the local
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500771 copy of poky, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600772 ":ref:`dev-manual/start:cloning the \`\`poky\`\` repository`"
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500773 section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500774
7752. *Determine Existing Branch Names:*
776 ::
777
778 $ git branch -a
779 * master
780 remotes/origin/1.1_M1
781 remotes/origin/1.1_M2
782 remotes/origin/1.1_M3
783 remotes/origin/1.1_M4
784 remotes/origin/1.2_M1
785 remotes/origin/1.2_M2
786 remotes/origin/1.2_M3
787 . . .
788 remotes/origin/thud
789 remotes/origin/thud-next
790 remotes/origin/warrior
791 remotes/origin/warrior-next
792 remotes/origin/zeus
793 remotes/origin/zeus-next
794 ... and so on ...
795
7963. *Check out the Branch:* Check out the development branch in which you
797 want to work. For example, to access the files for the Yocto Project
Andrew Geisslerc926e172021-05-07 16:11:35 -0500798 &DISTRO; Release (&DISTRO_NAME;), use the following command::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500799
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500800 $ git checkout -b &DISTRO_NAME_NO_CAP; origin/&DISTRO_NAME_NO_CAP;
801 Branch &DISTRO_NAME_NO_CAP; set up to track remote branch &DISTRO_NAME_NO_CAP; from origin.
802 Switched to a new branch '&DISTRO_NAME_NO_CAP;'
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500803
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500804 The previous command checks out the "&DISTRO_NAME_NO_CAP;" development
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500805 branch and reports that the branch is tracking the upstream
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500806 "origin/&DISTRO_NAME_NO_CAP;" branch.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500807
808 The following command displays the branches that are now part of your
809 local poky repository. The asterisk character indicates the branch
Andrew Geisslerc926e172021-05-07 16:11:35 -0500810 that is currently checked out for work::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500811
812 $ git branch
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500813 master
814 * &DISTRO_NAME_NO_CAP;
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500815
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500816Checking Out by Tag in Poky
817---------------------------
818
819Similar to branches, the upstream repository uses tags to mark specific
820commits associated with significant points in a development branch (i.e.
821a release point or stage of a release). You might want to set up a local
822branch based on one of those points in the repository. The process is
823similar to checking out by branch name except you use tag names.
824
825.. note::
826
827 Checking out a branch based on a tag gives you a stable set of files
828 not affected by development on the branch above the tag.
829
8301. *Switch to the Poky Directory:* If you have a local poky Git
831 repository, switch to that directory. If you do not have the local
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500832 copy of poky, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600833 ":ref:`dev-manual/start:cloning the \`\`poky\`\` repository`"
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500834 section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500835
8362. *Fetch the Tag Names:* To checkout the branch based on a tag name,
Andrew Geisslerc926e172021-05-07 16:11:35 -0500837 you need to fetch the upstream tags into your local repository::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500838
839 $ git fetch --tags
840 $
841
Andrew Geisslerc926e172021-05-07 16:11:35 -05008423. *List the Tag Names:* You can list the tag names now::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500843
844 $ git tag
845 1.1_M1.final
846 1.1_M1.rc1
847 1.1_M1.rc2
848 1.1_M2.final
849 1.1_M2.rc1
850 .
851 .
852 .
853 yocto-2.5
854 yocto-2.5.1
855 yocto-2.5.2
856 yocto-2.5.3
857 yocto-2.6
858 yocto-2.6.1
859 yocto-2.6.2
860 yocto-2.7
861 yocto_1.5_M5.rc8
862
863
8644. *Check out the Branch:*
865 ::
866
867 $ git checkout tags/yocto-&DISTRO; -b my_yocto_&DISTRO;
868 Switched to a new branch 'my_yocto_&DISTRO;'
869 $ git branch
870 master
871 * my_yocto_&DISTRO;
872
873 The previous command creates and
874 checks out a local branch named "my_yocto_&DISTRO;", which is based on
875 the commit in the upstream poky repository that has the same tag. In
876 this example, the files you have available locally as a result of the
877 ``checkout`` command are a snapshot of the "&DISTRO_NAME_NO_CAP;"
878 development branch at the point where Yocto Project &DISTRO; was
879 released.