| .. SPDX-License-Identifier: CC-BY-SA-2.0-UK |
| |
| Upgrading Recipes |
| ***************** |
| |
| Over time, upstream developers publish new versions for software built |
| by layer recipes. It is recommended to keep recipes up-to-date with |
| upstream version releases. |
| |
| While there are several methods to upgrade a recipe, you might |
| consider checking on the upgrade status of a recipe first. You can do so |
| using the ``devtool check-upgrade-status`` command. See the |
| ":ref:`devtool-checking-on-the-upgrade-status-of-a-recipe`" |
| section in the Yocto Project Reference Manual for more information. |
| |
| The remainder of this section describes three ways you can upgrade a |
| recipe. You can use the Automated Upgrade Helper (AUH) to set up |
| automatic version upgrades. Alternatively, you can use |
| ``devtool upgrade`` to set up semi-automatic version upgrades. Finally, |
| you can manually upgrade a recipe by editing the recipe itself. |
| |
| Using the Auto Upgrade Helper (AUH) |
| =================================== |
| |
| The AUH utility works in conjunction with the OpenEmbedded build system |
| in order to automatically generate upgrades for recipes based on new |
| versions being published upstream. Use AUH when you want to create a |
| service that performs the upgrades automatically and optionally sends |
| you an email with the results. |
| |
| AUH allows you to update several recipes with a single use. You can also |
| optionally perform build and integration tests using images with the |
| results saved to your hard drive and emails of results optionally sent |
| to recipe maintainers. Finally, AUH creates Git commits with appropriate |
| commit messages in the layer's tree for the changes made to recipes. |
| |
| .. note:: |
| |
| In some conditions, you should not use AUH to upgrade recipes |
| and should instead use either ``devtool upgrade`` or upgrade your |
| recipes manually: |
| |
| - When AUH cannot complete the upgrade sequence. This situation |
| usually results because custom patches carried by the recipe |
| cannot be automatically rebased to the new version. In this case, |
| ``devtool upgrade`` allows you to manually resolve conflicts. |
| |
| - When for any reason you want fuller control over the upgrade |
| process. For example, when you want special arrangements for |
| testing. |
| |
| The following steps describe how to set up the AUH utility: |
| |
| #. *Be Sure the Development Host is Set Up:* You need to be sure that |
| your development host is set up to use the Yocto Project. For |
| information on how to set up your host, see the |
| ":ref:`dev-manual/start:Preparing the Build Host`" section. |
| |
| #. *Make Sure Git is Configured:* The AUH utility requires Git to be |
| configured because AUH uses Git to save upgrades. Thus, you must have |
| Git user and email configured. The following command shows your |
| configurations:: |
| |
| $ git config --list |
| |
| If you do not have the user and |
| email configured, you can use the following commands to do so:: |
| |
| $ git config --global user.name some_name |
| $ git config --global user.email username@domain.com |
| |
| #. *Clone the AUH Repository:* To use AUH, you must clone the repository |
| onto your development host. The following command uses Git to create |
| a local copy of the repository on your system:: |
| |
| $ git clone git://git.yoctoproject.org/auto-upgrade-helper |
| Cloning into 'auto-upgrade-helper'... remote: Counting objects: 768, done. |
| remote: Compressing objects: 100% (300/300), done. |
| remote: Total 768 (delta 499), reused 703 (delta 434) |
| Receiving objects: 100% (768/768), 191.47 KiB | 98.00 KiB/s, done. |
| Resolving deltas: 100% (499/499), done. |
| Checking connectivity... done. |
| |
| AUH is not part of the :term:`OpenEmbedded-Core (OE-Core)` or |
| :term:`Poky` repositories. |
| |
| #. *Create a Dedicated Build Directory:* Run the :ref:`structure-core-script` |
| script to create a fresh :term:`Build Directory` that you use exclusively |
| for running the AUH utility:: |
| |
| $ cd poky |
| $ source oe-init-build-env your_AUH_build_directory |
| |
| Re-using an existing :term:`Build Directory` and its configurations is not |
| recommended as existing settings could cause AUH to fail or behave |
| undesirably. |
| |
| #. *Make Configurations in Your Local Configuration File:* Several |
| settings are needed in the ``local.conf`` file in the build |
| directory you just created for AUH. Make these following |
| configurations: |
| |
| - If you want to enable :ref:`Build |
| History <dev-manual/build-quality:maintaining build output quality>`, |
| which is optional, you need the following lines in the |
| ``conf/local.conf`` file:: |
| |
| INHERIT =+ "buildhistory" |
| BUILDHISTORY_COMMIT = "1" |
| |
| With this configuration and a successful |
| upgrade, a build history "diff" file appears in the |
| ``upgrade-helper/work/recipe/buildhistory-diff.txt`` file found in |
| your :term:`Build Directory`. |
| |
| - If you want to enable testing through the :ref:`ref-classes-testimage` |
| class, which is optional, you need to have the following set in |
| your ``conf/local.conf`` file:: |
| |
| IMAGE_CLASSES += "testimage" |
| |
| .. note:: |
| |
| If your distro does not enable by default ptest, which Poky |
| does, you need the following in your ``local.conf`` file:: |
| |
| DISTRO_FEATURES:append = " ptest" |
| |
| |
| #. *Optionally Start a vncserver:* If you are running in a server |
| without an X11 session, you need to start a vncserver:: |
| |
| $ vncserver :1 |
| $ export DISPLAY=:1 |
| |
| #. *Create and Edit an AUH Configuration File:* You need to have the |
| ``upgrade-helper/upgrade-helper.conf`` configuration file in your |
| :term:`Build Directory`. You can find a sample configuration file in the |
| :yocto_git:`AUH source repository </auto-upgrade-helper/tree/>`. |
| |
| Read through the sample file and make configurations as needed. For |
| example, if you enabled build history in your ``local.conf`` as |
| described earlier, you must enable it in ``upgrade-helper.conf``. |
| |
| Also, if you are using the default ``maintainers.inc`` file supplied |
| with Poky and located in ``meta-yocto`` and you do not set a |
| "maintainers_whitelist" or "global_maintainer_override" in the |
| ``upgrade-helper.conf`` configuration, and you specify "-e all" on |
| the AUH command-line, the utility automatically sends out emails to |
| all the default maintainers. Please avoid this. |
| |
| This next set of examples describes how to use the AUH: |
| |
| - *Upgrading a Specific Recipe:* To upgrade a specific recipe, use the |
| following form:: |
| |
| $ upgrade-helper.py recipe_name |
| |
| For example, this command upgrades the ``xmodmap`` recipe:: |
| |
| $ upgrade-helper.py xmodmap |
| |
| - *Upgrading a Specific Recipe to a Particular Version:* To upgrade a |
| specific recipe to a particular version, use the following form:: |
| |
| $ upgrade-helper.py recipe_name -t version |
| |
| For example, this command upgrades the ``xmodmap`` recipe to version 1.2.3:: |
| |
| $ upgrade-helper.py xmodmap -t 1.2.3 |
| |
| - *Upgrading all Recipes to the Latest Versions and Suppressing Email |
| Notifications:* To upgrade all recipes to their most recent versions |
| and suppress the email notifications, use the following command:: |
| |
| $ upgrade-helper.py all |
| |
| - *Upgrading all Recipes to the Latest Versions and Send Email |
| Notifications:* To upgrade all recipes to their most recent versions |
| and send email messages to maintainers for each attempted recipe as |
| well as a status email, use the following command:: |
| |
| $ upgrade-helper.py -e all |
| |
| Once you have run the AUH utility, you can find the results in the AUH |
| :term:`Build Directory`:: |
| |
| ${BUILDDIR}/upgrade-helper/timestamp |
| |
| The AUH utility |
| also creates recipe update commits from successful upgrade attempts in |
| the layer tree. |
| |
| You can easily set up to run the AUH utility on a regular basis by using |
| a cron job. See the |
| :yocto_git:`weeklyjob.sh </auto-upgrade-helper/tree/weeklyjob.sh>` |
| file distributed with the utility for an example. |
| |
| Using ``devtool upgrade`` |
| ========================= |
| |
| As mentioned earlier, an alternative method for upgrading recipes to |
| newer versions is to use |
| :doc:`devtool upgrade </ref-manual/devtool-reference>`. |
| You can read about ``devtool upgrade`` in general in the |
| ":ref:`sdk-manual/extensible:use \`\`devtool upgrade\`\` to create a version of the recipe that supports a newer version of the software`" |
| section in the Yocto Project Application Development and the Extensible |
| Software Development Kit (eSDK) Manual. |
| |
| To see all the command-line options available with ``devtool upgrade``, |
| use the following help command:: |
| |
| $ devtool upgrade -h |
| |
| If you want to find out what version a recipe is currently at upstream |
| without any attempt to upgrade your local version of the recipe, you can |
| use the following command:: |
| |
| $ devtool latest-version recipe_name |
| |
| As mentioned in the previous section describing AUH, ``devtool upgrade`` |
| works in a less-automated manner than AUH. Specifically, |
| ``devtool upgrade`` only works on a single recipe that you name on the |
| command line, cannot perform build and integration testing using images, |
| and does not automatically generate commits for changes in the source |
| tree. Despite all these "limitations", ``devtool upgrade`` updates the |
| recipe file to the new upstream version and attempts to rebase custom |
| patches contained by the recipe as needed. |
| |
| .. note:: |
| |
| AUH uses much of ``devtool upgrade`` behind the scenes making AUH somewhat |
| of a "wrapper" application for ``devtool upgrade``. |
| |
| A typical scenario involves having used Git to clone an upstream |
| repository that you use during build operations. Because you have built the |
| recipe in the past, the layer is likely added to your |
| configuration already. If for some reason, the layer is not added, you |
| could add it easily using the |
| ":ref:`bitbake-layers <bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script>`" |
| script. For example, suppose you use the ``nano.bb`` recipe from the |
| ``meta-oe`` layer in the ``meta-openembedded`` repository. For this |
| example, assume that the layer has been cloned into following area:: |
| |
| /home/scottrif/meta-openembedded |
| |
| The following command from your :term:`Build Directory` adds the layer to |
| your build configuration (i.e. ``${BUILDDIR}/conf/bblayers.conf``):: |
| |
| $ bitbake-layers add-layer /home/scottrif/meta-openembedded/meta-oe |
| NOTE: Starting bitbake server... |
| Parsing recipes: 100% |##########################################| Time: 0:00:55 |
| Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors. |
| Removing 12 recipes from the x86_64 sysroot: 100% |##############| Time: 0:00:00 |
| Removing 1 recipes from the x86_64_i586 sysroot: 100% |##########| Time: 0:00:00 |
| Removing 5 recipes from the i586 sysroot: 100% |#################| Time: 0:00:00 |
| Removing 5 recipes from the qemux86 sysroot: 100% |##############| Time: 0:00:00 |
| |
| For this example, assume that the ``nano.bb`` recipe that |
| is upstream has a 2.9.3 version number. However, the version in the |
| local repository is 2.7.4. The following command from your build |
| directory automatically upgrades the recipe for you:: |
| |
| $ devtool upgrade nano -V 2.9.3 |
| NOTE: Starting bitbake server... |
| NOTE: Creating workspace layer in /home/scottrif/poky/build/workspace |
| Parsing recipes: 100% |##########################################| Time: 0:00:46 |
| Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors. |
| NOTE: Extracting current version source... |
| NOTE: Resolving any missing task queue dependencies |
| . |
| . |
| . |
| NOTE: Executing SetScene Tasks |
| NOTE: Executing RunQueue Tasks |
| NOTE: Tasks Summary: Attempted 74 tasks of which 72 didn't need to be rerun and all succeeded. |
| Adding changed files: 100% |#####################################| Time: 0:00:00 |
| NOTE: Upgraded source extracted to /home/scottrif/poky/build/workspace/sources/nano |
| NOTE: New recipe is /home/scottrif/poky/build/workspace/recipes/nano/nano_2.9.3.bb |
| |
| .. note:: |
| |
| Using the ``-V`` option is not necessary. Omitting the version number causes |
| ``devtool upgrade`` to upgrade the recipe to the most recent version. |
| |
| Continuing with this example, you can use ``devtool build`` to build the |
| newly upgraded recipe:: |
| |
| $ devtool build nano |
| NOTE: Starting bitbake server... |
| Loading cache: 100% |################################################################################################| Time: 0:00:01 |
| Loaded 2040 entries from dependency cache. |
| Parsing recipes: 100% |##############################################################################################| Time: 0:00:00 |
| Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. |
| NOTE: Resolving any missing task queue dependencies |
| . |
| . |
| . |
| NOTE: Executing SetScene Tasks |
| NOTE: Executing RunQueue Tasks |
| NOTE: nano: compiling from external source tree /home/scottrif/poky/build/workspace/sources/nano |
| NOTE: Tasks Summary: Attempted 520 tasks of which 304 didn't need to be rerun and all succeeded. |
| |
| Within the ``devtool upgrade`` workflow, you can |
| deploy and test your rebuilt software. For this example, |
| however, running ``devtool finish`` cleans up the workspace once the |
| source in your workspace is clean. This usually means using Git to stage |
| and submit commits for the changes generated by the upgrade process. |
| |
| Once the tree is clean, you can clean things up in this example with the |
| following command from the ``${BUILDDIR}/workspace/sources/nano`` |
| directory:: |
| |
| $ devtool finish nano meta-oe |
| NOTE: Starting bitbake server... |
| Loading cache: 100% |################################################################################################| Time: 0:00:00 |
| Loaded 2040 entries from dependency cache. |
| Parsing recipes: 100% |##############################################################################################| Time: 0:00:01 |
| Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. |
| NOTE: Adding new patch 0001-nano.bb-Stuff-I-changed-when-upgrading-nano.bb.patch |
| NOTE: Updating recipe nano_2.9.3.bb |
| NOTE: Removing file /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano/nano_2.7.4.bb |
| NOTE: Moving recipe file to /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano |
| NOTE: Leaving source tree /home/scottrif/poky/build/workspace/sources/nano as-is; if you no longer need it then please delete it manually |
| |
| |
| Using the ``devtool finish`` command cleans up the workspace and creates a patch |
| file based on your commits. The tool puts all patch files back into the |
| source directory in a sub-directory named ``nano`` in this case. |
| |
| Manually Upgrading a Recipe |
| =========================== |
| |
| If for some reason you choose not to upgrade recipes using |
| :ref:`dev-manual/upgrading-recipes:Using the Auto Upgrade Helper (AUH)` or |
| by :ref:`dev-manual/upgrading-recipes:Using \`\`devtool upgrade\`\``, |
| you can manually edit the recipe files to upgrade the versions. |
| |
| .. note:: |
| |
| Manually updating multiple recipes scales poorly and involves many |
| steps. The recommendation to upgrade recipe versions is through AUH |
| or ``devtool upgrade``, both of which automate some steps and provide |
| guidance for others needed for the manual process. |
| |
| To manually upgrade recipe versions, follow these general steps: |
| |
| #. *Change the Version:* Rename the recipe such that the version (i.e. |
| the :term:`PV` part of the recipe name) |
| changes appropriately. If the version is not part of the recipe name, |
| change the value as it is set for :term:`PV` within the recipe itself. |
| |
| #. *Update* :term:`SRCREV` *if Needed*: If the source code your recipe builds |
| is fetched from Git or some other version control system, update |
| :term:`SRCREV` to point to the |
| commit hash that matches the new version. |
| |
| #. *Build the Software:* Try to build the recipe using BitBake. Typical |
| build failures include the following: |
| |
| - License statements were updated for the new version. For this |
| case, you need to review any changes to the license and update the |
| values of :term:`LICENSE` and |
| :term:`LIC_FILES_CHKSUM` |
| as needed. |
| |
| .. note:: |
| |
| License changes are often inconsequential. For example, the |
| license text's copyright year might have changed. |
| |
| - Custom patches carried by the older version of the recipe might |
| fail to apply to the new version. For these cases, you need to |
| review the failures. Patches might not be necessary for the new |
| version of the software if the upgraded version has fixed those |
| issues. If a patch is necessary and failing, you need to rebase it |
| into the new version. |
| |
| #. *Optionally Attempt to Build for Several Architectures:* Once you |
| successfully build the new software for a given architecture, you |
| could test the build for other architectures by changing the |
| :term:`MACHINE` variable and |
| rebuilding the software. This optional step is especially important |
| if the recipe is to be released publicly. |
| |
| #. *Check the Upstream Change Log or Release Notes:* Checking both these |
| reveals if there are new features that could break |
| backwards-compatibility. If so, you need to take steps to mitigate or |
| eliminate that situation. |
| |
| #. *Optionally Create a Bootable Image and Test:* If you want, you can |
| test the new software by booting it onto actual hardware. |
| |
| #. *Create a Commit with the Change in the Layer Repository:* After all |
| builds work and any testing is successful, you can create commits for |
| any changes in the layer holding your upgraded recipe. |
| |