blob: 4fac78bdfb155628c8b5dc8f79900a357fee7b67 [file] [log] [blame]
Andrew Geissler517393d2023-01-13 08:55:19 -06001.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
2
3Upgrading Recipes
4*****************
5
6Over time, upstream developers publish new versions for software built
7by layer recipes. It is recommended to keep recipes up-to-date with
8upstream version releases.
9
10While there are several methods to upgrade a recipe, you might
11consider checking on the upgrade status of a recipe first. You can do so
12using the ``devtool check-upgrade-status`` command. See the
13":ref:`devtool-checking-on-the-upgrade-status-of-a-recipe`"
14section in the Yocto Project Reference Manual for more information.
15
16The remainder of this section describes three ways you can upgrade a
17recipe. You can use the Automated Upgrade Helper (AUH) to set up
18automatic version upgrades. Alternatively, you can use
19``devtool upgrade`` to set up semi-automatic version upgrades. Finally,
20you can manually upgrade a recipe by editing the recipe itself.
21
22Using the Auto Upgrade Helper (AUH)
23===================================
24
25The AUH utility works in conjunction with the OpenEmbedded build system
26in order to automatically generate upgrades for recipes based on new
27versions being published upstream. Use AUH when you want to create a
28service that performs the upgrades automatically and optionally sends
29you an email with the results.
30
31AUH allows you to update several recipes with a single use. You can also
32optionally perform build and integration tests using images with the
33results saved to your hard drive and emails of results optionally sent
34to recipe maintainers. Finally, AUH creates Git commits with appropriate
35commit messages in the layer's tree for the changes made to recipes.
36
37.. note::
38
39 In some conditions, you should not use AUH to upgrade recipes
40 and should instead use either ``devtool upgrade`` or upgrade your
41 recipes manually:
42
43 - When AUH cannot complete the upgrade sequence. This situation
44 usually results because custom patches carried by the recipe
45 cannot be automatically rebased to the new version. In this case,
46 ``devtool upgrade`` allows you to manually resolve conflicts.
47
48 - When for any reason you want fuller control over the upgrade
49 process. For example, when you want special arrangements for
50 testing.
51
52The following steps describe how to set up the AUH utility:
53
54#. *Be Sure the Development Host is Set Up:* You need to be sure that
55 your development host is set up to use the Yocto Project. For
56 information on how to set up your host, see the
57 ":ref:`dev-manual/start:Preparing the Build Host`" section.
58
59#. *Make Sure Git is Configured:* The AUH utility requires Git to be
60 configured because AUH uses Git to save upgrades. Thus, you must have
61 Git user and email configured. The following command shows your
62 configurations::
63
64 $ git config --list
65
66 If you do not have the user and
67 email configured, you can use the following commands to do so::
68
69 $ git config --global user.name some_name
70 $ git config --global user.email username@domain.com
71
72#. *Clone the AUH Repository:* To use AUH, you must clone the repository
73 onto your development host. The following command uses Git to create
74 a local copy of the repository on your system::
75
Patrick Williams520786c2023-06-25 16:20:36 -050076 $ git clone git://git.yoctoproject.org/auto-upgrade-helper
Andrew Geissler517393d2023-01-13 08:55:19 -060077 Cloning into 'auto-upgrade-helper'... remote: Counting objects: 768, done.
78 remote: Compressing objects: 100% (300/300), done.
79 remote: Total 768 (delta 499), reused 703 (delta 434)
80 Receiving objects: 100% (768/768), 191.47 KiB | 98.00 KiB/s, done.
81 Resolving deltas: 100% (499/499), done.
82 Checking connectivity... done.
83
84 AUH is not part of the :term:`OpenEmbedded-Core (OE-Core)` or
85 :term:`Poky` repositories.
86
87#. *Create a Dedicated Build Directory:* Run the :ref:`structure-core-script`
88 script to create a fresh :term:`Build Directory` that you use exclusively
89 for running the AUH utility::
90
91 $ cd poky
92 $ source oe-init-build-env your_AUH_build_directory
93
94 Re-using an existing :term:`Build Directory` and its configurations is not
95 recommended as existing settings could cause AUH to fail or behave
96 undesirably.
97
98#. *Make Configurations in Your Local Configuration File:* Several
99 settings are needed in the ``local.conf`` file in the build
100 directory you just created for AUH. Make these following
101 configurations:
102
103 - If you want to enable :ref:`Build
104 History <dev-manual/build-quality:maintaining build output quality>`,
105 which is optional, you need the following lines in the
106 ``conf/local.conf`` file::
107
108 INHERIT =+ "buildhistory"
109 BUILDHISTORY_COMMIT = "1"
110
111 With this configuration and a successful
112 upgrade, a build history "diff" file appears in the
113 ``upgrade-helper/work/recipe/buildhistory-diff.txt`` file found in
114 your :term:`Build Directory`.
115
116 - If you want to enable testing through the :ref:`ref-classes-testimage`
117 class, which is optional, you need to have the following set in
118 your ``conf/local.conf`` file::
119
Andrew Geissler20137392023-10-12 04:59:14 -0600120 IMAGE_CLASSES += "testimage"
Andrew Geissler517393d2023-01-13 08:55:19 -0600121
122 .. note::
123
124 If your distro does not enable by default ptest, which Poky
125 does, you need the following in your ``local.conf`` file::
126
127 DISTRO_FEATURES:append = " ptest"
128
129
130#. *Optionally Start a vncserver:* If you are running in a server
131 without an X11 session, you need to start a vncserver::
132
133 $ vncserver :1
134 $ export DISPLAY=:1
135
136#. *Create and Edit an AUH Configuration File:* You need to have the
137 ``upgrade-helper/upgrade-helper.conf`` configuration file in your
138 :term:`Build Directory`. You can find a sample configuration file in the
139 :yocto_git:`AUH source repository </auto-upgrade-helper/tree/>`.
140
141 Read through the sample file and make configurations as needed. For
142 example, if you enabled build history in your ``local.conf`` as
143 described earlier, you must enable it in ``upgrade-helper.conf``.
144
145 Also, if you are using the default ``maintainers.inc`` file supplied
146 with Poky and located in ``meta-yocto`` and you do not set a
147 "maintainers_whitelist" or "global_maintainer_override" in the
148 ``upgrade-helper.conf`` configuration, and you specify "-e all" on
149 the AUH command-line, the utility automatically sends out emails to
150 all the default maintainers. Please avoid this.
151
152This next set of examples describes how to use the AUH:
153
154- *Upgrading a Specific Recipe:* To upgrade a specific recipe, use the
155 following form::
156
157 $ upgrade-helper.py recipe_name
158
159 For example, this command upgrades the ``xmodmap`` recipe::
160
161 $ upgrade-helper.py xmodmap
162
163- *Upgrading a Specific Recipe to a Particular Version:* To upgrade a
164 specific recipe to a particular version, use the following form::
165
166 $ upgrade-helper.py recipe_name -t version
167
168 For example, this command upgrades the ``xmodmap`` recipe to version 1.2.3::
169
170 $ upgrade-helper.py xmodmap -t 1.2.3
171
172- *Upgrading all Recipes to the Latest Versions and Suppressing Email
173 Notifications:* To upgrade all recipes to their most recent versions
174 and suppress the email notifications, use the following command::
175
176 $ upgrade-helper.py all
177
178- *Upgrading all Recipes to the Latest Versions and Send Email
179 Notifications:* To upgrade all recipes to their most recent versions
180 and send email messages to maintainers for each attempted recipe as
181 well as a status email, use the following command::
182
183 $ upgrade-helper.py -e all
184
185Once you have run the AUH utility, you can find the results in the AUH
186:term:`Build Directory`::
187
188 ${BUILDDIR}/upgrade-helper/timestamp
189
190The AUH utility
191also creates recipe update commits from successful upgrade attempts in
192the layer tree.
193
194You can easily set up to run the AUH utility on a regular basis by using
195a cron job. See the
196:yocto_git:`weeklyjob.sh </auto-upgrade-helper/tree/weeklyjob.sh>`
197file distributed with the utility for an example.
198
199Using ``devtool upgrade``
200=========================
201
202As mentioned earlier, an alternative method for upgrading recipes to
203newer versions is to use
204:doc:`devtool upgrade </ref-manual/devtool-reference>`.
205You can read about ``devtool upgrade`` in general in the
206":ref:`sdk-manual/extensible:use \`\`devtool upgrade\`\` to create a version of the recipe that supports a newer version of the software`"
207section in the Yocto Project Application Development and the Extensible
208Software Development Kit (eSDK) Manual.
209
210To see all the command-line options available with ``devtool upgrade``,
211use the following help command::
212
213 $ devtool upgrade -h
214
215If you want to find out what version a recipe is currently at upstream
216without any attempt to upgrade your local version of the recipe, you can
217use the following command::
218
219 $ devtool latest-version recipe_name
220
221As mentioned in the previous section describing AUH, ``devtool upgrade``
222works in a less-automated manner than AUH. Specifically,
223``devtool upgrade`` only works on a single recipe that you name on the
224command line, cannot perform build and integration testing using images,
225and does not automatically generate commits for changes in the source
226tree. Despite all these "limitations", ``devtool upgrade`` updates the
227recipe file to the new upstream version and attempts to rebase custom
228patches contained by the recipe as needed.
229
230.. note::
231
232 AUH uses much of ``devtool upgrade`` behind the scenes making AUH somewhat
233 of a "wrapper" application for ``devtool upgrade``.
234
235A typical scenario involves having used Git to clone an upstream
236repository that you use during build operations. Because you have built the
237recipe in the past, the layer is likely added to your
238configuration already. If for some reason, the layer is not added, you
239could add it easily using the
240":ref:`bitbake-layers <bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script>`"
241script. For example, suppose you use the ``nano.bb`` recipe from the
242``meta-oe`` layer in the ``meta-openembedded`` repository. For this
243example, assume that the layer has been cloned into following area::
244
245 /home/scottrif/meta-openembedded
246
247The following command from your :term:`Build Directory` adds the layer to
248your build configuration (i.e. ``${BUILDDIR}/conf/bblayers.conf``)::
249
250 $ bitbake-layers add-layer /home/scottrif/meta-openembedded/meta-oe
251 NOTE: Starting bitbake server...
252 Parsing recipes: 100% |##########################################| Time: 0:00:55
253 Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors.
254 Removing 12 recipes from the x86_64 sysroot: 100% |##############| Time: 0:00:00
255 Removing 1 recipes from the x86_64_i586 sysroot: 100% |##########| Time: 0:00:00
256 Removing 5 recipes from the i586 sysroot: 100% |#################| Time: 0:00:00
257 Removing 5 recipes from the qemux86 sysroot: 100% |##############| Time: 0:00:00
258
259For this example, assume that the ``nano.bb`` recipe that
260is upstream has a 2.9.3 version number. However, the version in the
261local repository is 2.7.4. The following command from your build
262directory automatically upgrades the recipe for you::
263
264 $ devtool upgrade nano -V 2.9.3
265 NOTE: Starting bitbake server...
266 NOTE: Creating workspace layer in /home/scottrif/poky/build/workspace
267 Parsing recipes: 100% |##########################################| Time: 0:00:46
268 Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors.
269 NOTE: Extracting current version source...
270 NOTE: Resolving any missing task queue dependencies
271 .
272 .
273 .
274 NOTE: Executing SetScene Tasks
275 NOTE: Executing RunQueue Tasks
276 NOTE: Tasks Summary: Attempted 74 tasks of which 72 didn't need to be rerun and all succeeded.
277 Adding changed files: 100% |#####################################| Time: 0:00:00
278 NOTE: Upgraded source extracted to /home/scottrif/poky/build/workspace/sources/nano
279 NOTE: New recipe is /home/scottrif/poky/build/workspace/recipes/nano/nano_2.9.3.bb
280
281.. note::
282
283 Using the ``-V`` option is not necessary. Omitting the version number causes
284 ``devtool upgrade`` to upgrade the recipe to the most recent version.
285
286Continuing with this example, you can use ``devtool build`` to build the
287newly upgraded recipe::
288
289 $ devtool build nano
290 NOTE: Starting bitbake server...
291 Loading cache: 100% |################################################################################################| Time: 0:00:01
292 Loaded 2040 entries from dependency cache.
293 Parsing recipes: 100% |##############################################################################################| Time: 0:00:00
294 Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors.
295 NOTE: Resolving any missing task queue dependencies
296 .
297 .
298 .
299 NOTE: Executing SetScene Tasks
300 NOTE: Executing RunQueue Tasks
301 NOTE: nano: compiling from external source tree /home/scottrif/poky/build/workspace/sources/nano
302 NOTE: Tasks Summary: Attempted 520 tasks of which 304 didn't need to be rerun and all succeeded.
303
304Within the ``devtool upgrade`` workflow, you can
305deploy and test your rebuilt software. For this example,
306however, running ``devtool finish`` cleans up the workspace once the
307source in your workspace is clean. This usually means using Git to stage
308and submit commits for the changes generated by the upgrade process.
309
310Once the tree is clean, you can clean things up in this example with the
311following command from the ``${BUILDDIR}/workspace/sources/nano``
312directory::
313
314 $ devtool finish nano meta-oe
315 NOTE: Starting bitbake server...
316 Loading cache: 100% |################################################################################################| Time: 0:00:00
317 Loaded 2040 entries from dependency cache.
318 Parsing recipes: 100% |##############################################################################################| Time: 0:00:01
319 Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors.
320 NOTE: Adding new patch 0001-nano.bb-Stuff-I-changed-when-upgrading-nano.bb.patch
321 NOTE: Updating recipe nano_2.9.3.bb
322 NOTE: Removing file /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano/nano_2.7.4.bb
323 NOTE: Moving recipe file to /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano
324 NOTE: Leaving source tree /home/scottrif/poky/build/workspace/sources/nano as-is; if you no longer need it then please delete it manually
325
326
327Using the ``devtool finish`` command cleans up the workspace and creates a patch
328file based on your commits. The tool puts all patch files back into the
329source directory in a sub-directory named ``nano`` in this case.
330
331Manually Upgrading a Recipe
332===========================
333
334If for some reason you choose not to upgrade recipes using
335:ref:`dev-manual/upgrading-recipes:Using the Auto Upgrade Helper (AUH)` or
336by :ref:`dev-manual/upgrading-recipes:Using \`\`devtool upgrade\`\``,
337you can manually edit the recipe files to upgrade the versions.
338
339.. note::
340
341 Manually updating multiple recipes scales poorly and involves many
342 steps. The recommendation to upgrade recipe versions is through AUH
343 or ``devtool upgrade``, both of which automate some steps and provide
344 guidance for others needed for the manual process.
345
346To manually upgrade recipe versions, follow these general steps:
347
348#. *Change the Version:* Rename the recipe such that the version (i.e.
349 the :term:`PV` part of the recipe name)
350 changes appropriately. If the version is not part of the recipe name,
351 change the value as it is set for :term:`PV` within the recipe itself.
352
353#. *Update* :term:`SRCREV` *if Needed*: If the source code your recipe builds
354 is fetched from Git or some other version control system, update
355 :term:`SRCREV` to point to the
356 commit hash that matches the new version.
357
358#. *Build the Software:* Try to build the recipe using BitBake. Typical
359 build failures include the following:
360
361 - License statements were updated for the new version. For this
362 case, you need to review any changes to the license and update the
363 values of :term:`LICENSE` and
364 :term:`LIC_FILES_CHKSUM`
365 as needed.
366
367 .. note::
368
369 License changes are often inconsequential. For example, the
370 license text's copyright year might have changed.
371
372 - Custom patches carried by the older version of the recipe might
373 fail to apply to the new version. For these cases, you need to
374 review the failures. Patches might not be necessary for the new
375 version of the software if the upgraded version has fixed those
376 issues. If a patch is necessary and failing, you need to rebase it
377 into the new version.
378
379#. *Optionally Attempt to Build for Several Architectures:* Once you
380 successfully build the new software for a given architecture, you
381 could test the build for other architectures by changing the
382 :term:`MACHINE` variable and
383 rebuilding the software. This optional step is especially important
384 if the recipe is to be released publicly.
385
386#. *Check the Upstream Change Log or Release Notes:* Checking both these
387 reveals if there are new features that could break
388 backwards-compatibility. If so, you need to take steps to mitigate or
389 eliminate that situation.
390
391#. *Optionally Create a Bootable Image and Test:* If you want, you can
392 test the new software by booting it onto actual hardware.
393
394#. *Create a Commit with the Change in the Layer Repository:* After all
395 builds work and any testing is successful, you can create commits for
396 any changes in the layer holding your upgraded recipe.
397