blob: bc2c75d42e9fbd13394c3d5e2c46aefd25591be7 [file] [log] [blame]
Andrew Geissler7e0e3c02022-02-25 20:34:39 +00001Migration notes for 3.4 (honister)
2----------------------------------
3
Patrick Williams213cb262021-08-07 19:21:33 -05004This section provides migration information for moving to the Yocto
5Project 3.4 Release (codename "honister") from the prior release.
6
7Override syntax changes
Andrew Geissler7e0e3c02022-02-25 20:34:39 +00008~~~~~~~~~~~~~~~~~~~~~~~
Patrick Williams213cb262021-08-07 19:21:33 -05009
Andrew Geisslereff27472021-10-29 15:35:00 -050010In this release, the ``:`` character replaces the use of ``_`` to
11refer to an override, most commonly when making a conditional assignment
12of a variable. This means that an entry like::
Patrick Williams213cb262021-08-07 19:21:33 -050013
14 SRC_URI_qemux86 = "file://somefile"
15
Andrew Geisslereff27472021-10-29 15:35:00 -050016now becomes::
Patrick Williams213cb262021-08-07 19:21:33 -050017
18 SRC_URI:qemux86 = "file://somefile"
19
Andrew Geisslereff27472021-10-29 15:35:00 -050020since ``qemux86`` is an override. This applies to any use of override
21syntax, so the following::
Patrick Williams213cb262021-08-07 19:21:33 -050022
23 SRC_URI_append = " file://somefile"
24 SRC_URI_append_qemux86 = " file://somefile2"
25 SRC_URI_remove_qemux86-64 = " file://somefile3"
26 SRC_URI_prepend_qemuarm = "file://somefile4 "
27 FILES_${PN}-ptest = "${bindir}/xyz"
28 IMAGE_CMD_tar = "tar"
29 BASE_LIB_tune-cortexa76 = "lib"
30 SRCREV_pn-bash = "abc"
31 BB_TASK_NICE_LEVEL_task-testimage = '0'
32
Andrew Geisslereff27472021-10-29 15:35:00 -050033would now become::
Patrick Williams213cb262021-08-07 19:21:33 -050034
35 SRC_URI:append = " file://somefile"
36 SRC_URI:append:qemux86 = " file://somefile2"
37 SRC_URI:remove:qemux86-64 = " file://somefile3"
38 SRC_URI:prepend:qemuarm = "file://somefile4 "
39 FILES:${PN}-ptest = "${bindir}/xyz"
40 IMAGE_CMD:tar = "tar"
41 BASE_LIB:tune-cortexa76 = "lib"
42 SRCREV:pn-bash = "abc"
43 BB_TASK_NICE_LEVEL:task-testimage = '0'
44
45This also applies to
46:ref:`variable queries to the datastore <bitbake:bitbake-user-manual/bitbake-user-manual-metadata:functions for accessing datastore variables>`,
47for example using ``getVar`` and similar so ``d.getVar("RDEPENDS_${PN}")``
48becomes ``d.getVar("RDEPENDS:${PN}")``.
49
50Whilst some of these are fairly obvious such as :term:`MACHINE` and :term:`DISTRO`
51overrides, some are less obvious, for example the packaging variables such as
52:term:`RDEPENDS`, :term:`FILES` and so on taking package names (e.g. ``${PN}``,
53``${PN}-ptest``) as overrides. These overrides are not always in
54:term:`OVERRIDES` but applied conditionally in specific contexts
55such as packaging. ``task-<taskname>`` is another context specific override, the
56context being specific tasks in that case. Tune overrides are another special
57case where some code does use them as overrides but some does not. We plan to try
58and make the tune code use overrides more consistently in the future.
59
Patrick Williams0ca19cc2021-08-16 14:03:13 -050060There are some variables which do not use override syntax which include the
61suffix to variables in ``layer.conf`` files such as :term:`BBFILE_PATTERN`,
62:term:`SRCREV`\ ``_xxx`` where ``xxx`` is a name from :term:`SRC_URI` and
63:term:`PREFERRED_VERSION`\ ``_xxx``. In particular, ``layer.conf`` suffixes
64may be the same as a :term:`DISTRO` override causing some confusion. We do
65plan to try and improve consistency as these issues are identified.
66
Andrew Geisslereff27472021-10-29 15:35:00 -050067To help with migration of layers, a script has been provided in OE-Core.
68Once configured with the overrides used by a layer, this can be run as::
Patrick Williams213cb262021-08-07 19:21:33 -050069
70 <oe-core>/scripts/contrib/convert-overrides.py <layerdir>
71
72.. note::
73
74 Please read the notes in the script as it isn't entirely automatic and it isn't
75 expected to handle every case. In particular, it needs to be told which overrides
76 the layer uses (usually machine and distro names/overrides) and the result should
77 be carefully checked since it can be a little enthusiastic and will convert
Andrew Geisslereff27472021-10-29 15:35:00 -050078 references to ``_append``, ``_remove`` and ``_prepend`` in function and variable
79 names.
Patrick Williams213cb262021-08-07 19:21:33 -050080
Andrew Geisslereff27472021-10-29 15:35:00 -050081For reference, this conversion is important as it allows BitBake to more reliably
82determine what is an override and what is not, as underscores are also used in
83variable names without intending to be overrides. This should allow us to proceed
84with other syntax improvements and simplifications for usability. It also means
85BitBake no longer has to guess and maintain large lookup lists just in case
86e.g. ``functionname`` in ``my_functionname`` is an override, and thus should improve
87efficiency.
88
Andrew Geisslereff27472021-10-29 15:35:00 -050089New host dependencies
Andrew Geissler7e0e3c02022-02-25 20:34:39 +000090~~~~~~~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -050091
92The ``lz4c``, ``pzstd`` and ``zstd`` commands are now required to be
93installed on the build host to support LZ4 and Zstandard compression
94functionality. These are typically provided by ``lz4`` and ``zstd``
95packages in most Linux distributions. Alternatively they are available
96as part of ``buildtools-tarball`` if your distribution does not provide
97them. For more information see
98:ref:`ref-manual/system-requirements:required packages for the build host`.
99
Andrew Geisslereff27472021-10-29 15:35:00 -0500100Removed recipes
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000101~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500102
103The following recipes have been removed in this release:
104
105- ``assimp``: problematic from a licensing perspective and no longer
106 needed by anything else
107- ``clutter-1.0``: legacy component moved to meta-gnome
108- ``clutter-gst-3.0``: legacy component moved to meta-gnome
109- ``clutter-gtk-1.0``: legacy component moved to meta-gnome
110- ``cogl-1.0``: legacy component moved to meta-gnome
111- ``core-image-clutter``: removed along with clutter
112- ``linux-yocto``: removed version 5.4 recipes (5.14 and 5.10 still
113 provided)
114- ``mklibs-native``: not actively tested and upstream mklibs still
115 requires Python 2
116- ``mx-1.0``: obsolete (last release 2012) and isn't used by anything in
117 any known layer
118- ``packagegroup-core-clutter``: removed along with clutter
119
Andrew Geisslereff27472021-10-29 15:35:00 -0500120Removed classes
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000121~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500122
123- ``clutter``: moved to meta-gnome along with clutter itself
124- ``image-mklibs``: not actively tested and upstream mklibs still
125 requires Python 2
126- ``meta``: no longer useful. Recipes that need to skip installing
127 packages should inherit ``nopackages`` instead.
128
Andrew Geisslereff27472021-10-29 15:35:00 -0500129Prelinking disabled by default
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000130~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500131
132Recent tests have shown that prelinking works only when PIE is not
133enabled (see `here <https://rlbl.me/prelink-1>`__ and `here <https://rlbl.me/prelink-2>`__),
134and as PIE is both a desirable security feature, and the only
135configuration provided and tested by the Yocto Project, there is
136simply no sense in continuing to enable prelink.
137
138There's also a concern that no one is maintaining the code, and there
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000139are open bugs (including :yocto_bugs:`this serious one </show_bug.cgi?id=14429>`).
Andrew Geisslereff27472021-10-29 15:35:00 -0500140Given that prelink does intricate address arithmetic and rewriting
141of binaries the best option is to disable the feature. It is recommended
142that you consider disabling this feature in your own configuration if
143it is currently enabled.
144
Andrew Geisslereff27472021-10-29 15:35:00 -0500145Virtual runtime provides
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000146~~~~~~~~~~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500147
148Recipes shouldn't use the ``virtual/`` string in :term:`RPROVIDES` and
Andrew Geissler615f2f12022-07-15 14:00:58 -0500149:term:`RDEPENDS` --- it is confusing because ``virtual/`` has no special
Andrew Geisslereff27472021-10-29 15:35:00 -0500150meaning in :term:`RPROVIDES` and :term:`RDEPENDS` (unlike in the
151corresponding build-time :term:`PROVIDES` and :term:`DEPENDS`).
152
Andrew Geisslereff27472021-10-29 15:35:00 -0500153Tune files moved to architecture-specific directories
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000154~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500155
156The tune files found in ``conf/machine/include`` have now been moved
157into their respective architecture name directories under that same
158location; e.g. x86 tune files have moved into an ``x86`` subdirectory,
159MIPS tune files have moved into a ``mips`` subdirectory, etc.
160The ARM tunes have an extra level (``armv8a``, ``armv8m``, etc.) and
161some have been renamed to make them uniform with the rest of the tunes.
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000162See :yocto_git:`this commit </poky/commit/?id=1d381f21f5f13aa0c4e1a45683ed656ebeedd37d>`
Andrew Geisslereff27472021-10-29 15:35:00 -0500163for reference.
164
165If you have any references to tune files (e.g. in custom machine
166configuration files) they will need to be updated.
167
Andrew Geisslereff27472021-10-29 15:35:00 -0500168Extensible SDK host extension
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000169~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500170
171For a normal SDK, some layers append to :term:`TOOLCHAIN_HOST_TASK`
172unconditionally which is fine, until the eSDK tries to override the
173variable to its own values. Instead of installing packages specified
Andrew Geissler615f2f12022-07-15 14:00:58 -0500174in this variable it uses native recipes instead --- a very different
Andrew Geisslereff27472021-10-29 15:35:00 -0500175approach. This has led to confusing errors when binaries are added
176to the SDK but not relocated.
177
178To avoid these issues, a new :term:`TOOLCHAIN_HOST_TASK_ESDK` variable has
179been created. If you wish to extend what is installed in the host
180portion of the eSDK then you will now need to set this variable.
181
Andrew Geisslereff27472021-10-29 15:35:00 -0500182Package/recipe splitting
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000183~~~~~~~~~~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500184
185- ``perl-cross`` has been split out from the main ``perl`` recipe to
186 its own ``perlcross`` recipe for maintenance reasons. If you have
187 bbappends for the perl recipe then these may need extending.
188
189- The ``wayland`` recipe now packages its binaries in a
190 ``wayland-tools`` package rather than putting them into
191 ``wayland-dev``.
192
193- Xwayland has been split out of the xserver-xorg tree and thus is now
194 in its own ``xwayland`` recipe. If you need Xwayland in your image
195 then you may now need to add it explicitly.
196
197- The ``rpm`` package no longer has ``rpm-build`` in its :term:`RRECOMMENDS`;
198 if by chance you still need rpm package building functionality in
199 your image and you have not already done so then you should add
200 ``rpm-build`` to your image explicitly.
201
202- The Python ``statistics`` standard module is now packaged in its own
203 ``python3-statistics`` package instead of ``python3-misc`` as
204 previously.
205
Andrew Geisslereff27472021-10-29 15:35:00 -0500206Image / SDK generation changes
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000207~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500208
209- Recursive dependencies on the ``do_build`` task are now disabled when
210 building SDKs. These are generally not needed; in the unlikely event
211 that you do encounter problems then it will probably be as a result of
212 missing explicit dependencies that need to be added.
213
214- Errors during "complementary" package installation (e.g. for ``*-dbg``
215 and ``*-dev`` packages) during image construction are no longer
216 ignored. Historically some of these packages had installation problems,
217 that is no longer the case. In the unlikely event that you see errors
218 as a result, you will need to fix the installation/packaging issues.
219
220- When building an image, only packages that will be used in building
221 the image (i.e. the first entry in :term:`PACKAGE_CLASSES`) will be
222 produced if multiple package types are enabled (which is not a typical
223 configuration). If in your CI system you need to have the original
224 behaviour, use ``bitbake --runall build <target>``.
225
226- The ``-lic`` package is no longer automatically added to
227 :term:`RRECOMMENDS` for every other package when
228 :term:`LICENSE_CREATE_PACKAGE` is set to "1". If you wish all license
229 packages to be installed corresponding to packages in your image, then
230 you should instead add the new ``lic-pkgs`` feature to
231 :term:`IMAGE_FEATURES`.
232
Andrew Geisslereff27472021-10-29 15:35:00 -0500233Miscellaneous
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000234~~~~~~~~~~~~~
Andrew Geisslereff27472021-10-29 15:35:00 -0500235
Andrew Geisslerd5838332022-05-27 11:33:10 -0500236- Certificates are now properly checked when BitBake fetches sources
Andrew Geisslereff27472021-10-29 15:35:00 -0500237 over HTTPS. If you receive errors as a result for your custom recipes,
238 you will need to use a mirror or address the issue with the operators
239 of the server in question.
240
241- ``avahi`` has had its GTK+ support disabled by default. If you wish to
242 re-enable it, set ``AVAHI_GTK = "gtk3"`` in a bbappend for the
243 ``avahi`` recipe or in your custom distro configuration file.
244
245- Setting the ``BUILD_REPRODUCIBLE_BINARIES`` variable to "0" no longer
246 uses a strangely old fallback date of April 2011, it instead disables
247 building reproducible binaries as you would logically expect.
248
249- Setting noexec/nostamp/fakeroot varflags to any value besides "1" will
250 now trigger a warning. These should be either set to "1" to enable, or
251 not set at all to disable.
252
253- The previously deprecated ``COMPRESS_CMD`` and
254 ``CVE_CHECK_CVE_WHITELIST`` variables have been removed. Use
Andrew Geissler615f2f12022-07-15 14:00:58 -0500255 :term:`CONVERSION_CMD` and ``CVE_CHECK_WHITELIST`` (replaced by
Andrew Geissler9aee5002022-03-30 16:27:02 +0000256 :term:`CVE_CHECK_IGNORE` in version 3.5) respectively
Andrew Geisslereff27472021-10-29 15:35:00 -0500257 instead.
258
259- The obsolete ``oe_machinstall`` function previously provided in the
260 :ref:`utils <ref-classes-utils>` class has been removed. For
261 machine-specific installation it is recommended that you use the
262 built-in override support in the fetcher or overrides in general
263 instead.
Andrew Geissler7e0e3c02022-02-25 20:34:39 +0000264
Patrick Williams03907ee2022-05-01 06:28:52 -0500265- The ``-P`` (``--clear-password``) option can no longer be used with
266 ``useradd`` and ``usermod`` entries in :term:`EXTRA_USERS_PARAMS`.
267 It was being implemented using a custom patch to the ``shadow`` recipe
268 which clashed with a ``-P`` option that was added upstream in
269 ``shadow`` version 4.9, and in any case is fundamentally insecure.
270 Hardcoded passwords are still supported but they need to be hashed, see
271 examples in :term:`EXTRA_USERS_PARAMS`.
272
273