blob: b79bc541c547a24374260f84ea37abf4abccfc1c [file] [log] [blame]
Andrew Geissler09036742021-06-25 14:25:14 -05001Release 2.3 (pyro)
2==================
Andrew Geisslerc9f78652020-09-18 14:11:35 -05003
4This section provides migration information for moving to the Yocto
Andrew Geissler3b8a17c2021-04-15 15:55:55 -05005Project 2.3 Release (codename "pyro") from the prior release.
Andrew Geisslerc9f78652020-09-18 14:11:35 -05006
7.. _migration-2.3-recipe-specific-sysroots:
8
9Recipe-specific Sysroots
10------------------------
11
12The OpenEmbedded build system now uses one sysroot per recipe to resolve
13long-standing issues with configuration script auto-detection of
14undeclared dependencies. Consequently, you might find that some of your
15previously written custom recipes are missing declared dependencies,
16particularly those dependencies that are incidentally built earlier in a
17typical build process and thus are already likely to be present in the
18shared sysroot in previous releases.
19
20Consider the following:
21
22- *Declare Build-Time Dependencies:* Because of this new feature, you
23 must explicitly declare all build-time dependencies for your recipe.
24 If you do not declare these dependencies, they are not populated into
25 the sysroot for the recipe.
26
27- *Specify Pre-Installation and Post-Installation Native Tool
28 Dependencies:* You must specifically specify any special native tool
29 dependencies of ``pkg_preinst`` and ``pkg_postinst`` scripts by using
30 the :term:`PACKAGE_WRITE_DEPS` variable.
31 Specifying these dependencies ensures that these tools are available
32 if these scripts need to be run on the build host during the
33 :ref:`ref-tasks-rootfs` task.
34
35 As an example, see the ``dbus`` recipe. You will see that this recipe
36 has a ``pkg_postinst`` that calls ``systemctl`` if "systemd" is in
37 :term:`DISTRO_FEATURES`. In the example,
Andrew Geissler09036742021-06-25 14:25:14 -050038 ``systemd-systemctl-native`` is added to :term:`PACKAGE_WRITE_DEPS`,
Andrew Geissler5f350902021-07-23 13:09:54 -040039 which is also conditional on "systemd" being in :term:`DISTRO_FEATURES`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -050040
41- Examine Recipes that Use ``SSTATEPOSTINSTFUNCS``: You need to
42 examine any recipe that uses ``SSTATEPOSTINSTFUNCS`` and determine
43 steps to take.
44
45 Functions added to ``SSTATEPOSTINSTFUNCS`` are still called as they
46 were in previous Yocto Project releases. However, since a separate
47 sysroot is now being populated for every recipe and if existing
48 functions being called through ``SSTATEPOSTINSTFUNCS`` are doing
49 relocation, then you will need to change these to use a
50 post-installation script that is installed by a function added to
51 :term:`SYSROOT_PREPROCESS_FUNCS`.
52
53 For an example, see the ``pixbufcache`` class in ``meta/classes/`` in
Andrew Geissler09209ee2020-12-13 08:44:15 -060054 the :ref:`overview-manual/development-environment:yocto project source repositories`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -050055
56 .. note::
57
58 The
59 SSTATEPOSTINSTFUNCS
60 variable itself is now deprecated in favor of the
61 do_populate_sysroot[postfuncs]
62 task. Consequently, if you do still have any function or functions
63 that need to be called after the sysroot component is created for
64 a recipe, then you would be well advised to take steps to use a
65 post installation script as described previously. Taking these
66 steps prepares your code for when
67 SSTATEPOSTINSTFUNCS
68 is removed in a future Yocto Project release.
69
70- *Specify the Sysroot when Using Certain External Scripts:* Because
71 the shared sysroot is now gone, the scripts
72 ``oe-find-native-sysroot`` and ``oe-run-native`` have been changed
73 such that you need to specify which recipe's
74 :term:`STAGING_DIR_NATIVE` is used.
75
76.. note::
77
78 You can find more information on how recipe-specific sysroots work in
Andrew Geissler4c19ea12020-10-27 13:52:24 -050079 the ":ref:`ref-classes-staging`" section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -050080
81.. _migration-2.3-path-variable:
82
83``PATH`` Variable
84-----------------
85
86Within the environment used to run build tasks, the environment variable
87``PATH`` is now sanitized such that the normal native binary paths
88(``/bin``, ``/sbin``, ``/usr/bin`` and so forth) are removed and a
89directory containing symbolic links linking only to the binaries from
90the host mentioned in the :term:`HOSTTOOLS` and
91:term:`HOSTTOOLS_NONFATAL` variables is added
92to ``PATH``.
93
94Consequently, any native binaries provided by the host that you need to
95call needs to be in one of these two variables at the configuration
96level.
97
98Alternatively, you can add a native recipe (i.e. ``-native``) that
99provides the binary to the recipe's :term:`DEPENDS`
100value.
101
102.. note::
103
104 PATH
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500105 is not sanitized in the same way within ``devshell``.
106 If it were, you would have difficulty running host tools for
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500107 development and debugging within the shell.
108
109.. _migration-2.3-scripts:
110
111Changes to Scripts
112------------------
113
114The following changes to scripts took place:
115
116- ``oe-find-native-sysroot``: The usage for the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500117 ``oe-find-native-sysroot`` script has changed to the following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500118
119 $ . oe-find-native-sysroot recipe
120
121 You must now supply a recipe for recipe
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500122 as part of the command. Prior to the Yocto Project 2.3 release, it
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500123 was not necessary to provide the script with the command.
124
125- ``oe-run-native``: The usage for the ``oe-run-native`` script has
Andrew Geisslerc926e172021-05-07 16:11:35 -0500126 changed to the following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500127
128 $ oe-run-native native_recipe tool
129
130 You must
131 supply the name of the native recipe and the tool you want to run as
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500132 part of the command. Prior to the Yocto Project 2.3 release, it
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500133 was not necessary to provide the native recipe with the command.
134
135- ``cleanup-workdir``: The ``cleanup-workdir`` script has been
136 removed because the script was found to be deleting files it should
137 not have, which lead to broken build trees. Rather than trying to
138 delete portions of :term:`TMPDIR` and getting it wrong,
Andrew Geissler09036742021-06-25 14:25:14 -0500139 it is recommended that you delete :term:`TMPDIR` and have it restored
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500140 from shared state (sstate) on subsequent builds.
141
142- ``wipe-sysroot``: The ``wipe-sysroot`` script has been removed as
143 it is no longer needed with recipe-specific sysroots.
144
145.. _migration-2.3-functions:
146
147Changes to Functions
148--------------------
149
150The previously deprecated ``bb.data.getVar()``, ``bb.data.setVar()``,
151and related functions have been removed in favor of ``d.getVar()``,
152``d.setVar()``, and so forth.
153
154You need to fix any references to these old functions.
155
156.. _migration-2.3-bitbake-changes:
157
158BitBake Changes
159---------------
160
161The following changes took place for BitBake:
162
163- *BitBake's Graphical Dependency Explorer UI Replaced:* BitBake's
164 graphical dependency explorer UI ``depexp`` was replaced by
165 ``taskexp`` ("Task Explorer"), which provides a graphical way of
166 exploring the ``task-depends.dot`` file. The data presented by Task
167 Explorer is much more accurate than the data that was presented by
168 ``depexp``. Being able to visualize the data is an often requested
169 feature as standard ``*.dot`` file viewers cannot usual cope with the
170 size of the ``task-depends.dot`` file.
171
172- *BitBake "-g" Output Changes:* The ``package-depends.dot`` and
173 ``pn-depends.dot`` files as previously generated using the
174 ``bitbake -g`` command have been removed. A ``recipe-depends.dot``
175 file is now generated as a collapsed version of ``task-depends.dot``
176 instead.
177
178 The reason for this change is because ``package-depends.dot`` and
179 ``pn-depends.dot`` largely date back to a time before task-based
180 execution and do not take into account task-level dependencies
181 between recipes, which could be misleading.
182
183- *Mirror Variable Splitting Changes:* Mirror variables including
184 :term:`MIRRORS`, :term:`PREMIRRORS`,
185 and :term:`SSTATE_MIRRORS` can now separate
186 values entirely with spaces. Consequently, you no longer need "\\n".
187 BitBake looks for pairs of values, which simplifies usage. There
188 should be no change required to existing mirror variable values
189 themselves.
190
191- *The Subversion (SVN) Fetcher Uses an "ssh" Parameter and Not an
192 "rsh" Parameter:* The SVN fetcher now takes an "ssh" parameter
193 instead of an "rsh" parameter. This new optional parameter is used
194 when the "protocol" parameter is set to "svn+ssh". You can only use
195 the new parameter to specify the ``ssh`` program used by SVN. The SVN
196 fetcher passes the new parameter through the ``SVN_SSH`` environment
197 variable during the :ref:`ref-tasks-fetch` task.
198
Andrew Geissler09209ee2020-12-13 08:44:15 -0600199 See the ":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-fetching:subversion (svn) fetcher (\`\`svn://\`\`)`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500200 section in the BitBake
201 User Manual for additional information.
202
203- ``BB_SETSCENE_VERIFY_FUNCTION`` and ``BB_SETSCENE_VERIFY_FUNCTION2``
204 Removed: Because the mechanism they were part of is no longer
205 necessary with recipe-specific sysroots, the
206 ``BB_SETSCENE_VERIFY_FUNCTION`` and ``BB_SETSCENE_VERIFY_FUNCTION2``
207 variables have been removed.
208
209.. _migration-2.3-absolute-symlinks:
210
211Absolute Symbolic Links
212-----------------------
213
214Absolute symbolic links (symlinks) within staged files are no longer
215permitted and now trigger an error. Any explicit creation of symlinks
216can use the ``lnr`` script, which is a replacement for ``ln -r``.
217
218If the build scripts in the software that the recipe is building are
219creating a number of absolute symlinks that need to be corrected, you
220can inherit ``relative_symlinks`` within the recipe to turn those
221absolute symlinks into relative symlinks.
222
223.. _migration-2.3-gplv2-and-gplv3-moves:
224
225GPLv2 Versions of GPLv3 Recipes Moved
226-------------------------------------
227
228Older GPLv2 versions of GPLv3 recipes have moved to a separate
229``meta-gplv2`` layer.
230
231If you use :term:`INCOMPATIBLE_LICENSE` to
232exclude GPLv3 or set :term:`PREFERRED_VERSION`
233to substitute a GPLv2 version of a GPLv3 recipe, then you must add the
234``meta-gplv2`` layer to your configuration.
235
236.. note::
237
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500238 You can ``find meta-gplv2`` layer in the OpenEmbedded layer index at
Andrew Geisslerd1e89492021-02-12 15:35:20 -0600239 :oe_layer:`/meta-gplv2`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500240
241These relocated GPLv2 recipes do not receive the same level of
242maintenance as other core recipes. The recipes do not get security fixes
243and upstream no longer maintains them. In fact, the upstream community
244is actively hostile towards people that use the old versions of the
245recipes. Moving these recipes into a separate layer both makes the
246different needs of the recipes clearer and clearly identifies the number
247of these recipes.
248
249.. note::
250
251 The long-term solution might be to move to BSD-licensed replacements
252 of the GPLv3 components for those that need to exclude GPLv3-licensed
253 components from the target system. This solution will be investigated
254 for future Yocto Project releases.
255
256.. _migration-2.3-package-management-changes:
257
258Package Management Changes
259--------------------------
260
261The following package management changes took place:
262
263- Smart package manager is replaced by DNF package manager. Smart has
264 become unmaintained upstream, is not ported to Python 3.x.
265 Consequently, Smart needed to be replaced. DNF is the only feasible
266 candidate.
267
268 The change in functionality is that the on-target runtime package
269 management from remote package feeds is now done with a different
270 tool that has a different set of command-line options. If you have
271 scripts that call the tool directly, or use its API, they need to be
272 fixed.
273
274 For more information, see the `DNF
Andrew Geisslerd1e89492021-02-12 15:35:20 -0600275 Documentation <https://dnf.readthedocs.io/en/latest/>`__.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500276
277- Rpm 5.x is replaced with Rpm 4.x. This is done for two major reasons:
278
279 - DNF is API-incompatible with Rpm 5.x and porting it and
280 maintaining the port is non-trivial.
281
282 - Rpm 5.x itself has limited maintenance upstream, and the Yocto
283 Project is one of the very few remaining users.
284
285- Berkeley DB 6.x is removed and Berkeley DB 5.x becomes the default:
286
287 - Version 6.x of Berkeley DB has largely been rejected by the open
288 source community due to its AGPLv3 license. As a result, most
289 mainstream open source projects that require DB are still
290 developed and tested with DB 5.x.
291
292 - In OE-core, the only thing that was requiring DB 6.x was Rpm 5.x.
293 Thus, no reason exists to continue carrying DB 6.x in OE-core.
294
295- ``createrepo`` is replaced with ``createrepo_c``.
296
297 ``createrepo_c`` is the current incarnation of the tool that
298 generates remote repository metadata. It is written in C as compared
299 to ``createrepo``, which is written in Python. ``createrepo_c`` is
300 faster and is maintained.
301
302- Architecture-independent RPM packages are "noarch" instead of "all".
303
304 This change was made because too many places in DNF/RPM4 stack
305 already make that assumption. Only the filenames and the architecture
306 tag has changed. Nothing else has changed in OE-core system,
307 particularly in the :ref:`allarch.bbclass <ref-classes-allarch>`
308 class.
309
310- Signing of remote package feeds using ``PACKAGE_FEED_SIGN`` is not
311 currently supported. This issue will be fully addressed in a future
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500312 Yocto Project release. See :yocto_bugs:`defect 11209 </show_bug.cgi?id=11209>`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500313 for more information on a solution to package feed signing with RPM
314 in the Yocto Project 2.3 release.
315
316- OPKG now uses the libsolv backend for resolving package dependencies
317 by default. This is vastly superior to OPKG's internal ad-hoc solver
318 that was previously used. This change does have a small impact on
319 disk (around 500 KB) and memory footprint.
320
321 .. note::
322
323 For further details on this change, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600324 :yocto_git:`commit message </poky/commit/?id=f4d4f99cfbc2396e49c1613a7d237b9e57f06f81>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500325
326.. _migration-2.3-removed-recipes:
327
328Removed Recipes
329---------------
330
331The following recipes have been removed:
332
333- ``linux-yocto 4.8``: Version 4.8 has been removed. Versions 4.1
334 (LTSI), 4.4 (LTS), 4.9 (LTS/LTSI) and 4.10 are now present.
335
336- ``python-smartpm``: Functionally replaced by ``dnf``.
337
338- ``createrepo``: Replaced by the ``createrepo-c`` recipe.
339
340- ``rpmresolve``: No longer needed with the move to RPM 4 as RPM
341 itself is used instead.
342
343- ``gstreamer``: Removed the GStreamer Git version recipes as they
344 have been stale. ``1.10.``\ x recipes are still present.
345
346- ``alsa-conf-base``: Merged into ``alsa-conf`` since ``libasound``
347 depended on both. Essentially, no way existed to install only one of
348 these.
349
350- ``tremor``: Moved to ``meta-multimedia``. Fixed-integer Vorbis
351 decoding is not needed by current hardware. Thus, GStreamer's ivorbis
352 plugin has been disabled by default eliminating the need for the
353 ``tremor`` recipe in :term:`OpenEmbedded-Core (OE-Core)`.
354
355- ``gummiboot``: Replaced by ``systemd-boot``.
356
357.. _migration-2.3-wic-changes:
358
359Wic Changes
360-----------
361
362The following changes have been made to Wic:
363
364.. note::
365
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500366 For more information on Wic, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600367 ":ref:`dev-manual/common-tasks:creating partitioned images using wic`"
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500368 section in the Yocto Project Development Tasks Manual.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500369
370- *Default Output Directory Changed:* Wic's default output directory is
371 now the current directory by default instead of the unusual
372 ``/var/tmp/wic``.
373
374 The "-o" and "--outdir" options remain unchanged and are used to
375 specify your preferred output directory if you do not want to use the
376 default directory.
377
378- *fsimage Plug-in Removed:* The Wic fsimage plugin has been removed as
379 it duplicates functionality of the rawcopy plugin.
380
381.. _migration-2.3-qa-changes:
382
383QA Changes
384----------
385
386The following QA checks have changed:
387
388- ``unsafe-references-in-binaries``: The
389 ``unsafe-references-in-binaries`` QA check, which was disabled by
390 default, has now been removed. This check was intended to detect
391 binaries in ``/bin`` that link to libraries in ``/usr/lib`` and have
392 the case where the user has ``/usr`` on a separate filesystem to
393 ``/``.
394
395 The removed QA check was buggy. Additionally, ``/usr`` residing on a
396 separate partition from ``/`` is now a rare configuration.
397 Consequently, ``unsafe-references-in-binaries`` was removed.
398
399- ``file-rdeps``: The ``file-rdeps`` QA check is now an error by
400 default instead of a warning. Because it is an error instead of a
401 warning, you need to address missing runtime dependencies.
402
403 For additional information, see the
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500404 :ref:`insane <ref-classes-insane>` class and the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600405 ":ref:`ref-manual/qa-checks:errors and warnings`" section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500406
407.. _migration-2.3-miscellaneous-changes:
408
409Miscellaneous Changes
410---------------------
411
412The following miscellaneous changes have occurred:
413
414- In this release, a number of recipes have been changed to ignore the
415 ``largefile`` :term:`DISTRO_FEATURES` item,
416 enabling large file support unconditionally. This feature has always
417 been enabled by default. Disabling the feature has not been widely
418 tested.
419
420 .. note::
421
422 Future releases of the Yocto Project will remove entirely the
423 ability to disable the
424 largefile
425 feature, which would make it unconditionally enabled everywhere.
426
427- If the :term:`DISTRO_VERSION` value contains
428 the value of the :term:`DATE` variable, which is the
Andrew Geissler09036742021-06-25 14:25:14 -0500429 default between Poky releases, the :term:`DATE` value is explicitly
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500430 excluded from ``/etc/issue`` and ``/etc/issue.net``, which is
431 displayed at the login prompt, in order to avoid conflicts with
Andrew Geissler09036742021-06-25 14:25:14 -0500432 Multilib enabled. Regardless, the :term:`DATE` value is inaccurate if the
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500433 ``base-files`` recipe is restored from shared state (sstate) rather
434 than rebuilt.
435
436 If you need the build date recorded in ``/etc/issue*`` or anywhere
437 else in your image, a better method is to define a post-processing
438 function to do it and have the function called from
439 :term:`ROOTFS_POSTPROCESS_COMMAND`.
440 Doing so ensures the value is always up-to-date with the created
441 image.
442
443- Dropbear's ``init`` script now disables DSA host keys by default.
444 This change is in line with the systemd service file, which supports
445 RSA keys only, and with recent versions of OpenSSH, which deprecates
446 DSA host keys.
447
448- The :ref:`buildhistory <ref-classes-buildhistory>` class now
449 correctly uses tabs as separators between all columns in
450 ``installed-package-sizes.txt`` in order to aid import into other
451 tools.
452
453- The ``USE_LDCONFIG`` variable has been replaced with the "ldconfig"
Andrew Geissler09036742021-06-25 14:25:14 -0500454 :term:`DISTRO_FEATURES` feature. Distributions that previously set::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500455
456 USE_LDCONFIG = "0"
457
Andrew Geisslerc926e172021-05-07 16:11:35 -0500458 should now instead use the following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500459
460 DISTRO_FEATURES_BACKFILL_CONSIDERED_append = " ldconfig"
461
462- The default value of
463 :term:`COPYLEFT_LICENSE_INCLUDE` now
464 includes all versions of AGPL licenses in addition to GPL and LGPL.
465
466 .. note::
467
468 The default list is not intended to be guaranteed as a complete
469 safe list. You should seek legal advice based on what you are
470 distributing if you are unsure.
471
472- Kernel module packages are now suffixed with the kernel version in
473 order to allow module packages from multiple kernel versions to
474 co-exist on a target system. If you wish to return to the previous
475 naming scheme that does not include the version suffix, use the
Andrew Geisslerc926e172021-05-07 16:11:35 -0500476 following::
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500477
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500478 KERNEL_MODULE_PACKAGE_SUFFIX = ""
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500479
480- Removal of ``libtool`` ``*.la`` files is now enabled by default. The
481 ``*.la`` files are not actually needed on Linux and relocating them
482 is an unnecessary burden.
483
484 If you need to preserve these ``.la`` files (e.g. in a custom
485 distribution), you must change
486 :term:`INHERIT_DISTRO` such that
487 "remove-libtool" is not included in the value.
488
489- Extensible SDKs built for GCC 5+ now refuse to install on a
490 distribution where the host GCC version is 4.8 or 4.9. This change
491 resulted from the fact that the installation is known to fail due to
492 the way the ``uninative`` shared state (sstate) package is built. See
493 the :ref:`uninative <ref-classes-uninative>` class for additional
494 information.
495
496- All native and nativesdk recipes now use a separate
Andrew Geissler09036742021-06-25 14:25:14 -0500497 :term:`DISTRO_FEATURES` value instead of sharing the value used by
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500498 recipes for the target, in order to avoid unnecessary rebuilds.
499
Andrew Geissler09036742021-06-25 14:25:14 -0500500 The :term:`DISTRO_FEATURES` for ``native`` recipes is
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500501 :term:`DISTRO_FEATURES_NATIVE` added to
Andrew Geissler09036742021-06-25 14:25:14 -0500502 an intersection of :term:`DISTRO_FEATURES` and
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500503 :term:`DISTRO_FEATURES_FILTER_NATIVE`.
504
505 For nativesdk recipes, the corresponding variables are
506 :term:`DISTRO_FEATURES_NATIVESDK`
507 and
508 :term:`DISTRO_FEATURES_FILTER_NATIVESDK`.
509
510- The ``FILESDIR`` variable, which was previously deprecated and rarely
511 used, has now been removed. You should change any recipes that set
512 ``FILESDIR`` to set :term:`FILESPATH` instead.
513
514- The ``MULTIMACH_HOST_SYS`` variable has been removed as it is no
515 longer needed with recipe-specific sysroots.
516
517