blob: a9b1057206c9188af86c45498fb7a3cb1792ec42 [file] [log] [blame]
Andrew Geissler517393d2023-01-13 08:55:19 -06001.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
2
Andrew Geissler7e0e3c02022-02-25 20:34:39 +00003Migration notes for 3.4 (honister)
4----------------------------------
5
Patrick Williams213cb262021-08-07 19:21:33 -05006This section provides migration information for moving to the Yocto
7Project 3.4 Release (codename "honister") from the prior release.
8
9Override syntax changes
Andrew Geissler7e0e3c02022-02-25 20:34:39 +000010~~~~~~~~~~~~~~~~~~~~~~~
Patrick Williams213cb262021-08-07 19:21:33 -050011
Andrew Geisslereff27472021-10-29 15:35:00 -050012In this release, the ``:`` character replaces the use of ``_`` to
13refer to an override, most commonly when making a conditional assignment
14of a variable. This means that an entry like::
Patrick Williams213cb262021-08-07 19:21:33 -050015
16 SRC_URI_qemux86 = "file://somefile"
17
Andrew Geisslereff27472021-10-29 15:35:00 -050018now becomes::
Patrick Williams213cb262021-08-07 19:21:33 -050019
20 SRC_URI:qemux86 = "file://somefile"
21
Andrew Geisslereff27472021-10-29 15:35:00 -050022since ``qemux86`` is an override. This applies to any use of override
23syntax, so the following::
Patrick Williams213cb262021-08-07 19:21:33 -050024
25 SRC_URI_append = " file://somefile"
26 SRC_URI_append_qemux86 = " file://somefile2"
Patrick Williams2194f502022-10-16 14:26:09 -050027 SRC_URI_remove_qemux86-64 = "file://somefile3"
Patrick Williams213cb262021-08-07 19:21:33 -050028 SRC_URI_prepend_qemuarm = "file://somefile4 "
29 FILES_${PN}-ptest = "${bindir}/xyz"
30 IMAGE_CMD_tar = "tar"
31 BASE_LIB_tune-cortexa76 = "lib"
32 SRCREV_pn-bash = "abc"
33 BB_TASK_NICE_LEVEL_task-testimage = '0'
34
Andrew Geisslereff27472021-10-29 15:35:00 -050035would now become::
Patrick Williams213cb262021-08-07 19:21:33 -050036
37 SRC_URI:append = " file://somefile"
38 SRC_URI:append:qemux86 = " file://somefile2"
Patrick Williams2194f502022-10-16 14:26:09 -050039 SRC_URI:remove:qemux86-64 = "file://somefile3"
Patrick Williams213cb262021-08-07 19:21:33 -050040 SRC_URI:prepend:qemuarm = "file://somefile4 "
41 FILES:${PN}-ptest = "${bindir}/xyz"
42 IMAGE_CMD:tar = "tar"
43 BASE_LIB:tune-cortexa76 = "lib"
44 SRCREV:pn-bash = "abc"
45 BB_TASK_NICE_LEVEL:task-testimage = '0'
46
47This also applies to
Andrew Geisslerfc113ea2023-03-31 09:59:46 -050048:ref:`variable queries to the datastore <bitbake-user-manual/bitbake-user-manual-metadata:functions for accessing datastore variables>`,
Patrick Williams213cb262021-08-07 19:21:33 -050049for example using ``getVar`` and similar so ``d.getVar("RDEPENDS_${PN}")``
50becomes ``d.getVar("RDEPENDS:${PN}")``.
51
52Whilst some of these are fairly obvious such as :term:`MACHINE` and :term:`DISTRO`
53overrides, some are less obvious, for example the packaging variables such as
54:term:`RDEPENDS`, :term:`FILES` and so on taking package names (e.g. ``${PN}``,
55``${PN}-ptest``) as overrides. These overrides are not always in
56:term:`OVERRIDES` but applied conditionally in specific contexts
57such as packaging. ``task-<taskname>`` is another context specific override, the
58context being specific tasks in that case. Tune overrides are another special
59case where some code does use them as overrides but some does not. We plan to try
60and make the tune code use overrides more consistently in the future.
61
Patrick Williams0ca19cc2021-08-16 14:03:13 -050062There are some variables which do not use override syntax which include the
63suffix to variables in ``layer.conf`` files such as :term:`BBFILE_PATTERN`,
64:term:`SRCREV`\ ``_xxx`` where ``xxx`` is a name from :term:`SRC_URI` and
65:term:`PREFERRED_VERSION`\ ``_xxx``. In particular, ``layer.conf`` suffixes
66may be the same as a :term:`DISTRO` override causing some confusion. We do
67plan to try and improve consistency as these issues are identified.
68
Andrew Geisslereff27472021-10-29 15:35:00 -050069To help with migration of layers, a script has been provided in OE-Core.
70Once configured with the overrides used by a layer, this can be run as::
Patrick Williams213cb262021-08-07 19:21:33 -050071
72 <oe-core>/scripts/contrib/convert-overrides.py <layerdir>
73
74.. note::
75
76 Please read the notes in the script as it isn't entirely automatic and it isn't
77 expected to handle every case. In particular, it needs to be told which overrides
78 the layer uses (usually machine and distro names/overrides) and the result should
79 be carefully checked since it can be a little enthusiastic and will convert
Andrew Geisslereff27472021-10-29 15:35:00 -050080 references to ``_append``, ``_remove`` and ``_prepend`` in function and variable
81 names.
Patrick Williams213cb262021-08-07 19:21:33 -050082
Andrew Geisslereff27472021-10-29 15:35:00 -050083For reference, this conversion is important as it allows BitBake to more reliably
84determine what is an override and what is not, as underscores are also used in
85variable names without intending to be overrides. This should allow us to proceed
86with other syntax improvements and simplifications for usability. It also means
87BitBake no longer has to guess and maintain large lookup lists just in case
88e.g. ``functionname`` in ``my_functionname`` is an override, and thus should improve
89efficiency.
90
Andrew Geisslereff27472021-10-29 15:35:00 -050091New host dependencies
Andrew Geissler7e0e3c02022-02-25 20:34:39 +000092~~~~~~~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -050093
94The ``lz4c``, ``pzstd`` and ``zstd`` commands are now required to be
95installed on the build host to support LZ4 and Zstandard compression
96functionality. These are typically provided by ``lz4`` and ``zstd``
97packages in most Linux distributions. Alternatively they are available
Andrew Geissler517393d2023-01-13 08:55:19 -060098as part of :term:`buildtools` tarball if your distribution does not provide
Andrew Geisslereff27472021-10-29 15:35:00 -050099them. For more information see
100:ref:`ref-manual/system-requirements:required packages for the build host`.
101
Andrew Geisslereff27472021-10-29 15:35:00 -0500102Removed recipes
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000103~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500104
105The following recipes have been removed in this release:
106
107- ``assimp``: problematic from a licensing perspective and no longer
108 needed by anything else
109- ``clutter-1.0``: legacy component moved to meta-gnome
110- ``clutter-gst-3.0``: legacy component moved to meta-gnome
111- ``clutter-gtk-1.0``: legacy component moved to meta-gnome
112- ``cogl-1.0``: legacy component moved to meta-gnome
113- ``core-image-clutter``: removed along with clutter
114- ``linux-yocto``: removed version 5.4 recipes (5.14 and 5.10 still
115 provided)
116- ``mklibs-native``: not actively tested and upstream mklibs still
117 requires Python 2
118- ``mx-1.0``: obsolete (last release 2012) and isn't used by anything in
119 any known layer
120- ``packagegroup-core-clutter``: removed along with clutter
121
Andrew Geisslereff27472021-10-29 15:35:00 -0500122Removed classes
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000123~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500124
125- ``clutter``: moved to meta-gnome along with clutter itself
126- ``image-mklibs``: not actively tested and upstream mklibs still
127 requires Python 2
128- ``meta``: no longer useful. Recipes that need to skip installing
Andrew Geissler517393d2023-01-13 08:55:19 -0600129 packages should inherit :ref:`ref-classes-nopackages` instead.
Andrew Geisslereff27472021-10-29 15:35:00 -0500130
Andrew Geisslereff27472021-10-29 15:35:00 -0500131Prelinking disabled by default
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000132~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500133
134Recent tests have shown that prelinking works only when PIE is not
135enabled (see `here <https://rlbl.me/prelink-1>`__ and `here <https://rlbl.me/prelink-2>`__),
136and as PIE is both a desirable security feature, and the only
137configuration provided and tested by the Yocto Project, there is
138simply no sense in continuing to enable prelink.
139
140There's also a concern that no one is maintaining the code, and there
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000141are open bugs (including :yocto_bugs:`this serious one </show_bug.cgi?id=14429>`).
Andrew Geisslereff27472021-10-29 15:35:00 -0500142Given that prelink does intricate address arithmetic and rewriting
143of binaries the best option is to disable the feature. It is recommended
144that you consider disabling this feature in your own configuration if
145it is currently enabled.
146
Andrew Geisslereff27472021-10-29 15:35:00 -0500147Virtual runtime provides
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000148~~~~~~~~~~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500149
150Recipes shouldn't use the ``virtual/`` string in :term:`RPROVIDES` and
Andrew Geissler615f2f12022-07-15 14:00:58 -0500151:term:`RDEPENDS` --- it is confusing because ``virtual/`` has no special
Andrew Geisslereff27472021-10-29 15:35:00 -0500152meaning in :term:`RPROVIDES` and :term:`RDEPENDS` (unlike in the
153corresponding build-time :term:`PROVIDES` and :term:`DEPENDS`).
154
Andrew Geisslereff27472021-10-29 15:35:00 -0500155Tune files moved to architecture-specific directories
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000156~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500157
158The tune files found in ``conf/machine/include`` have now been moved
159into their respective architecture name directories under that same
160location; e.g. x86 tune files have moved into an ``x86`` subdirectory,
161MIPS tune files have moved into a ``mips`` subdirectory, etc.
162The ARM tunes have an extra level (``armv8a``, ``armv8m``, etc.) and
163some have been renamed to make them uniform with the rest of the tunes.
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000164See :yocto_git:`this commit </poky/commit/?id=1d381f21f5f13aa0c4e1a45683ed656ebeedd37d>`
Andrew Geisslereff27472021-10-29 15:35:00 -0500165for reference.
166
167If you have any references to tune files (e.g. in custom machine
168configuration files) they will need to be updated.
169
Andrew Geisslereff27472021-10-29 15:35:00 -0500170Extensible SDK host extension
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000171~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500172
173For a normal SDK, some layers append to :term:`TOOLCHAIN_HOST_TASK`
174unconditionally which is fine, until the eSDK tries to override the
175variable to its own values. Instead of installing packages specified
Andrew Geissler615f2f12022-07-15 14:00:58 -0500176in this variable it uses native recipes instead --- a very different
Andrew Geisslereff27472021-10-29 15:35:00 -0500177approach. This has led to confusing errors when binaries are added
178to the SDK but not relocated.
179
180To avoid these issues, a new :term:`TOOLCHAIN_HOST_TASK_ESDK` variable has
181been created. If you wish to extend what is installed in the host
182portion of the eSDK then you will now need to set this variable.
183
Andrew Geisslereff27472021-10-29 15:35:00 -0500184Package/recipe splitting
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000185~~~~~~~~~~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500186
187- ``perl-cross`` has been split out from the main ``perl`` recipe to
188 its own ``perlcross`` recipe for maintenance reasons. If you have
189 bbappends for the perl recipe then these may need extending.
190
191- The ``wayland`` recipe now packages its binaries in a
192 ``wayland-tools`` package rather than putting them into
193 ``wayland-dev``.
194
195- Xwayland has been split out of the xserver-xorg tree and thus is now
196 in its own ``xwayland`` recipe. If you need Xwayland in your image
197 then you may now need to add it explicitly.
198
199- The ``rpm`` package no longer has ``rpm-build`` in its :term:`RRECOMMENDS`;
Patrick Williams520786c2023-06-25 16:20:36 -0500200 if by chance you still need rpm package building functionality in
Andrew Geisslereff27472021-10-29 15:35:00 -0500201 your image and you have not already done so then you should add
202 ``rpm-build`` to your image explicitly.
203
204- The Python ``statistics`` standard module is now packaged in its own
205 ``python3-statistics`` package instead of ``python3-misc`` as
206 previously.
207
Andrew Geisslereff27472021-10-29 15:35:00 -0500208Image / SDK generation changes
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000209~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500210
Patrick Williams2194f502022-10-16 14:26:09 -0500211- Recursive dependencies on the :ref:`ref-tasks-build` task are now disabled when
Andrew Geisslereff27472021-10-29 15:35:00 -0500212 building SDKs. These are generally not needed; in the unlikely event
213 that you do encounter problems then it will probably be as a result of
214 missing explicit dependencies that need to be added.
215
216- Errors during "complementary" package installation (e.g. for ``*-dbg``
217 and ``*-dev`` packages) during image construction are no longer
218 ignored. Historically some of these packages had installation problems,
219 that is no longer the case. In the unlikely event that you see errors
220 as a result, you will need to fix the installation/packaging issues.
221
222- When building an image, only packages that will be used in building
223 the image (i.e. the first entry in :term:`PACKAGE_CLASSES`) will be
224 produced if multiple package types are enabled (which is not a typical
225 configuration). If in your CI system you need to have the original
226 behaviour, use ``bitbake --runall build <target>``.
227
228- The ``-lic`` package is no longer automatically added to
229 :term:`RRECOMMENDS` for every other package when
230 :term:`LICENSE_CREATE_PACKAGE` is set to "1". If you wish all license
231 packages to be installed corresponding to packages in your image, then
232 you should instead add the new ``lic-pkgs`` feature to
233 :term:`IMAGE_FEATURES`.
234
Andrew Geisslereff27472021-10-29 15:35:00 -0500235Miscellaneous
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000236~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500237
Andrew Geisslerd5838332022-05-27 11:33:10 -0500238- Certificates are now properly checked when BitBake fetches sources
Andrew Geisslereff27472021-10-29 15:35:00 -0500239 over HTTPS. If you receive errors as a result for your custom recipes,
240 you will need to use a mirror or address the issue with the operators
241 of the server in question.
242
243- ``avahi`` has had its GTK+ support disabled by default. If you wish to
244 re-enable it, set ``AVAHI_GTK = "gtk3"`` in a bbappend for the
245 ``avahi`` recipe or in your custom distro configuration file.
246
247- Setting the ``BUILD_REPRODUCIBLE_BINARIES`` variable to "0" no longer
248 uses a strangely old fallback date of April 2011, it instead disables
249 building reproducible binaries as you would logically expect.
250
251- Setting noexec/nostamp/fakeroot varflags to any value besides "1" will
252 now trigger a warning. These should be either set to "1" to enable, or
253 not set at all to disable.
254
255- The previously deprecated ``COMPRESS_CMD`` and
256 ``CVE_CHECK_CVE_WHITELIST`` variables have been removed. Use
Andrew Geissler615f2f12022-07-15 14:00:58 -0500257 :term:`CONVERSION_CMD` and ``CVE_CHECK_WHITELIST`` (replaced by
Patrick Williamsda295312023-12-05 16:48:56 -0600258 :term:`CVE_CHECK_IGNORE` in version 4.0) respectively
Andrew Geisslereff27472021-10-29 15:35:00 -0500259 instead.
260
261- The obsolete ``oe_machinstall`` function previously provided in the
Andrew Geissler517393d2023-01-13 08:55:19 -0600262 :ref:`ref-classes-utils` class has been removed. For
Andrew Geisslereff27472021-10-29 15:35:00 -0500263 machine-specific installation it is recommended that you use the
264 built-in override support in the fetcher or overrides in general
265 instead.
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000266
Patrick Williams03907ee2022-05-01 06:28:52 -0500267- The ``-P`` (``--clear-password``) option can no longer be used with
268 ``useradd`` and ``usermod`` entries in :term:`EXTRA_USERS_PARAMS`.
269 It was being implemented using a custom patch to the ``shadow`` recipe
270 which clashed with a ``-P`` option that was added upstream in
271 ``shadow`` version 4.9, and in any case is fundamentally insecure.
272 Hardcoded passwords are still supported but they need to be hashed, see
273 examples in :term:`EXTRA_USERS_PARAMS`.
274
275