blob: 04b11daa7dee1f86e13d897473cc63a509fec5a8 [file] [log] [blame]
Andrew Geisslerc9f78652020-09-18 14:11:35 -05001Moving to the Yocto Project 2.3 Release
2=======================================
3
4This section provides migration information for moving to the Yocto
5Project 2.3 Release from the prior release.
6
7.. _migration-2.3-recipe-specific-sysroots:
8
9Recipe-specific Sysroots
10------------------------
11
12The OpenEmbedded build system now uses one sysroot per recipe to resolve
13long-standing issues with configuration script auto-detection of
14undeclared dependencies. Consequently, you might find that some of your
15previously written custom recipes are missing declared dependencies,
16particularly those dependencies that are incidentally built earlier in a
17typical build process and thus are already likely to be present in the
18shared sysroot in previous releases.
19
20Consider the following:
21
22- *Declare Build-Time Dependencies:* Because of this new feature, you
23 must explicitly declare all build-time dependencies for your recipe.
24 If you do not declare these dependencies, they are not populated into
25 the sysroot for the recipe.
26
27- *Specify Pre-Installation and Post-Installation Native Tool
28 Dependencies:* You must specifically specify any special native tool
29 dependencies of ``pkg_preinst`` and ``pkg_postinst`` scripts by using
30 the :term:`PACKAGE_WRITE_DEPS` variable.
31 Specifying these dependencies ensures that these tools are available
32 if these scripts need to be run on the build host during the
33 :ref:`ref-tasks-rootfs` task.
34
35 As an example, see the ``dbus`` recipe. You will see that this recipe
36 has a ``pkg_postinst`` that calls ``systemctl`` if "systemd" is in
37 :term:`DISTRO_FEATURES`. In the example,
38 ``systemd-systemctl-native`` is added to ``PACKAGE_WRITE_DEPS``,
39 which is also conditional on "systemd" being in ``DISTRO_FEATURES``.
40
41- Examine Recipes that Use ``SSTATEPOSTINSTFUNCS``: You need to
42 examine any recipe that uses ``SSTATEPOSTINSTFUNCS`` and determine
43 steps to take.
44
45 Functions added to ``SSTATEPOSTINSTFUNCS`` are still called as they
46 were in previous Yocto Project releases. However, since a separate
47 sysroot is now being populated for every recipe and if existing
48 functions being called through ``SSTATEPOSTINSTFUNCS`` are doing
49 relocation, then you will need to change these to use a
50 post-installation script that is installed by a function added to
51 :term:`SYSROOT_PREPROCESS_FUNCS`.
52
53 For an example, see the ``pixbufcache`` class in ``meta/classes/`` in
Andrew Geissler09209ee2020-12-13 08:44:15 -060054 the :ref:`overview-manual/development-environment:yocto project source repositories`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -050055
56 .. note::
57
58 The
59 SSTATEPOSTINSTFUNCS
60 variable itself is now deprecated in favor of the
61 do_populate_sysroot[postfuncs]
62 task. Consequently, if you do still have any function or functions
63 that need to be called after the sysroot component is created for
64 a recipe, then you would be well advised to take steps to use a
65 post installation script as described previously. Taking these
66 steps prepares your code for when
67 SSTATEPOSTINSTFUNCS
68 is removed in a future Yocto Project release.
69
70- *Specify the Sysroot when Using Certain External Scripts:* Because
71 the shared sysroot is now gone, the scripts
72 ``oe-find-native-sysroot`` and ``oe-run-native`` have been changed
73 such that you need to specify which recipe's
74 :term:`STAGING_DIR_NATIVE` is used.
75
76.. note::
77
78 You can find more information on how recipe-specific sysroots work in
Andrew Geissler4c19ea12020-10-27 13:52:24 -050079 the ":ref:`ref-classes-staging`" section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -050080
81.. _migration-2.3-path-variable:
82
83``PATH`` Variable
84-----------------
85
86Within the environment used to run build tasks, the environment variable
87``PATH`` is now sanitized such that the normal native binary paths
88(``/bin``, ``/sbin``, ``/usr/bin`` and so forth) are removed and a
89directory containing symbolic links linking only to the binaries from
90the host mentioned in the :term:`HOSTTOOLS` and
91:term:`HOSTTOOLS_NONFATAL` variables is added
92to ``PATH``.
93
94Consequently, any native binaries provided by the host that you need to
95call needs to be in one of these two variables at the configuration
96level.
97
98Alternatively, you can add a native recipe (i.e. ``-native``) that
99provides the binary to the recipe's :term:`DEPENDS`
100value.
101
102.. note::
103
104 PATH
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500105 is not sanitized in the same way within ``devshell``.
106 If it were, you would have difficulty running host tools for
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500107 development and debugging within the shell.
108
109.. _migration-2.3-scripts:
110
111Changes to Scripts
112------------------
113
114The following changes to scripts took place:
115
116- ``oe-find-native-sysroot``: The usage for the
117 ``oe-find-native-sysroot`` script has changed to the following:
118 ::
119
120 $ . oe-find-native-sysroot recipe
121
122 You must now supply a recipe for recipe
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500123 as part of the command. Prior to the Yocto Project 2.3 release, it
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500124 was not necessary to provide the script with the command.
125
126- ``oe-run-native``: The usage for the ``oe-run-native`` script has
127 changed to the following:
128 ::
129
130 $ oe-run-native native_recipe tool
131
132 You must
133 supply the name of the native recipe and the tool you want to run as
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500134 part of the command. Prior to the Yocto Project 2.3 release, it
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500135 was not necessary to provide the native recipe with the command.
136
137- ``cleanup-workdir``: The ``cleanup-workdir`` script has been
138 removed because the script was found to be deleting files it should
139 not have, which lead to broken build trees. Rather than trying to
140 delete portions of :term:`TMPDIR` and getting it wrong,
141 it is recommended that you delete ``TMPDIR`` and have it restored
142 from shared state (sstate) on subsequent builds.
143
144- ``wipe-sysroot``: The ``wipe-sysroot`` script has been removed as
145 it is no longer needed with recipe-specific sysroots.
146
147.. _migration-2.3-functions:
148
149Changes to Functions
150--------------------
151
152The previously deprecated ``bb.data.getVar()``, ``bb.data.setVar()``,
153and related functions have been removed in favor of ``d.getVar()``,
154``d.setVar()``, and so forth.
155
156You need to fix any references to these old functions.
157
158.. _migration-2.3-bitbake-changes:
159
160BitBake Changes
161---------------
162
163The following changes took place for BitBake:
164
165- *BitBake's Graphical Dependency Explorer UI Replaced:* BitBake's
166 graphical dependency explorer UI ``depexp`` was replaced by
167 ``taskexp`` ("Task Explorer"), which provides a graphical way of
168 exploring the ``task-depends.dot`` file. The data presented by Task
169 Explorer is much more accurate than the data that was presented by
170 ``depexp``. Being able to visualize the data is an often requested
171 feature as standard ``*.dot`` file viewers cannot usual cope with the
172 size of the ``task-depends.dot`` file.
173
174- *BitBake "-g" Output Changes:* The ``package-depends.dot`` and
175 ``pn-depends.dot`` files as previously generated using the
176 ``bitbake -g`` command have been removed. A ``recipe-depends.dot``
177 file is now generated as a collapsed version of ``task-depends.dot``
178 instead.
179
180 The reason for this change is because ``package-depends.dot`` and
181 ``pn-depends.dot`` largely date back to a time before task-based
182 execution and do not take into account task-level dependencies
183 between recipes, which could be misleading.
184
185- *Mirror Variable Splitting Changes:* Mirror variables including
186 :term:`MIRRORS`, :term:`PREMIRRORS`,
187 and :term:`SSTATE_MIRRORS` can now separate
188 values entirely with spaces. Consequently, you no longer need "\\n".
189 BitBake looks for pairs of values, which simplifies usage. There
190 should be no change required to existing mirror variable values
191 themselves.
192
193- *The Subversion (SVN) Fetcher Uses an "ssh" Parameter and Not an
194 "rsh" Parameter:* The SVN fetcher now takes an "ssh" parameter
195 instead of an "rsh" parameter. This new optional parameter is used
196 when the "protocol" parameter is set to "svn+ssh". You can only use
197 the new parameter to specify the ``ssh`` program used by SVN. The SVN
198 fetcher passes the new parameter through the ``SVN_SSH`` environment
199 variable during the :ref:`ref-tasks-fetch` task.
200
Andrew Geissler09209ee2020-12-13 08:44:15 -0600201 See the ":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-fetching:subversion (svn) fetcher (\`\`svn://\`\`)`"
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500202 section in the BitBake
203 User Manual for additional information.
204
205- ``BB_SETSCENE_VERIFY_FUNCTION`` and ``BB_SETSCENE_VERIFY_FUNCTION2``
206 Removed: Because the mechanism they were part of is no longer
207 necessary with recipe-specific sysroots, the
208 ``BB_SETSCENE_VERIFY_FUNCTION`` and ``BB_SETSCENE_VERIFY_FUNCTION2``
209 variables have been removed.
210
211.. _migration-2.3-absolute-symlinks:
212
213Absolute Symbolic Links
214-----------------------
215
216Absolute symbolic links (symlinks) within staged files are no longer
217permitted and now trigger an error. Any explicit creation of symlinks
218can use the ``lnr`` script, which is a replacement for ``ln -r``.
219
220If the build scripts in the software that the recipe is building are
221creating a number of absolute symlinks that need to be corrected, you
222can inherit ``relative_symlinks`` within the recipe to turn those
223absolute symlinks into relative symlinks.
224
225.. _migration-2.3-gplv2-and-gplv3-moves:
226
227GPLv2 Versions of GPLv3 Recipes Moved
228-------------------------------------
229
230Older GPLv2 versions of GPLv3 recipes have moved to a separate
231``meta-gplv2`` layer.
232
233If you use :term:`INCOMPATIBLE_LICENSE` to
234exclude GPLv3 or set :term:`PREFERRED_VERSION`
235to substitute a GPLv2 version of a GPLv3 recipe, then you must add the
236``meta-gplv2`` layer to your configuration.
237
238.. note::
239
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500240 You can ``find meta-gplv2`` layer in the OpenEmbedded layer index at
Andrew Geisslerd1e89492021-02-12 15:35:20 -0600241 :oe_layer:`/meta-gplv2`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500242
243These relocated GPLv2 recipes do not receive the same level of
244maintenance as other core recipes. The recipes do not get security fixes
245and upstream no longer maintains them. In fact, the upstream community
246is actively hostile towards people that use the old versions of the
247recipes. Moving these recipes into a separate layer both makes the
248different needs of the recipes clearer and clearly identifies the number
249of these recipes.
250
251.. note::
252
253 The long-term solution might be to move to BSD-licensed replacements
254 of the GPLv3 components for those that need to exclude GPLv3-licensed
255 components from the target system. This solution will be investigated
256 for future Yocto Project releases.
257
258.. _migration-2.3-package-management-changes:
259
260Package Management Changes
261--------------------------
262
263The following package management changes took place:
264
265- Smart package manager is replaced by DNF package manager. Smart has
266 become unmaintained upstream, is not ported to Python 3.x.
267 Consequently, Smart needed to be replaced. DNF is the only feasible
268 candidate.
269
270 The change in functionality is that the on-target runtime package
271 management from remote package feeds is now done with a different
272 tool that has a different set of command-line options. If you have
273 scripts that call the tool directly, or use its API, they need to be
274 fixed.
275
276 For more information, see the `DNF
Andrew Geisslerd1e89492021-02-12 15:35:20 -0600277 Documentation <https://dnf.readthedocs.io/en/latest/>`__.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500278
279- Rpm 5.x is replaced with Rpm 4.x. This is done for two major reasons:
280
281 - DNF is API-incompatible with Rpm 5.x and porting it and
282 maintaining the port is non-trivial.
283
284 - Rpm 5.x itself has limited maintenance upstream, and the Yocto
285 Project is one of the very few remaining users.
286
287- Berkeley DB 6.x is removed and Berkeley DB 5.x becomes the default:
288
289 - Version 6.x of Berkeley DB has largely been rejected by the open
290 source community due to its AGPLv3 license. As a result, most
291 mainstream open source projects that require DB are still
292 developed and tested with DB 5.x.
293
294 - In OE-core, the only thing that was requiring DB 6.x was Rpm 5.x.
295 Thus, no reason exists to continue carrying DB 6.x in OE-core.
296
297- ``createrepo`` is replaced with ``createrepo_c``.
298
299 ``createrepo_c`` is the current incarnation of the tool that
300 generates remote repository metadata. It is written in C as compared
301 to ``createrepo``, which is written in Python. ``createrepo_c`` is
302 faster and is maintained.
303
304- Architecture-independent RPM packages are "noarch" instead of "all".
305
306 This change was made because too many places in DNF/RPM4 stack
307 already make that assumption. Only the filenames and the architecture
308 tag has changed. Nothing else has changed in OE-core system,
309 particularly in the :ref:`allarch.bbclass <ref-classes-allarch>`
310 class.
311
312- Signing of remote package feeds using ``PACKAGE_FEED_SIGN`` is not
313 currently supported. This issue will be fully addressed in a future
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500314 Yocto Project release. See :yocto_bugs:`defect 11209 </show_bug.cgi?id=11209>`
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500315 for more information on a solution to package feed signing with RPM
316 in the Yocto Project 2.3 release.
317
318- OPKG now uses the libsolv backend for resolving package dependencies
319 by default. This is vastly superior to OPKG's internal ad-hoc solver
320 that was previously used. This change does have a small impact on
321 disk (around 500 KB) and memory footprint.
322
323 .. note::
324
325 For further details on this change, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600326 :yocto_git:`commit message </poky/commit/?id=f4d4f99cfbc2396e49c1613a7d237b9e57f06f81>`.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500327
328.. _migration-2.3-removed-recipes:
329
330Removed Recipes
331---------------
332
333The following recipes have been removed:
334
335- ``linux-yocto 4.8``: Version 4.8 has been removed. Versions 4.1
336 (LTSI), 4.4 (LTS), 4.9 (LTS/LTSI) and 4.10 are now present.
337
338- ``python-smartpm``: Functionally replaced by ``dnf``.
339
340- ``createrepo``: Replaced by the ``createrepo-c`` recipe.
341
342- ``rpmresolve``: No longer needed with the move to RPM 4 as RPM
343 itself is used instead.
344
345- ``gstreamer``: Removed the GStreamer Git version recipes as they
346 have been stale. ``1.10.``\ x recipes are still present.
347
348- ``alsa-conf-base``: Merged into ``alsa-conf`` since ``libasound``
349 depended on both. Essentially, no way existed to install only one of
350 these.
351
352- ``tremor``: Moved to ``meta-multimedia``. Fixed-integer Vorbis
353 decoding is not needed by current hardware. Thus, GStreamer's ivorbis
354 plugin has been disabled by default eliminating the need for the
355 ``tremor`` recipe in :term:`OpenEmbedded-Core (OE-Core)`.
356
357- ``gummiboot``: Replaced by ``systemd-boot``.
358
359.. _migration-2.3-wic-changes:
360
361Wic Changes
362-----------
363
364The following changes have been made to Wic:
365
366.. note::
367
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500368 For more information on Wic, see the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600369 ":ref:`dev-manual/common-tasks:creating partitioned images using wic`"
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500370 section in the Yocto Project Development Tasks Manual.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500371
372- *Default Output Directory Changed:* Wic's default output directory is
373 now the current directory by default instead of the unusual
374 ``/var/tmp/wic``.
375
376 The "-o" and "--outdir" options remain unchanged and are used to
377 specify your preferred output directory if you do not want to use the
378 default directory.
379
380- *fsimage Plug-in Removed:* The Wic fsimage plugin has been removed as
381 it duplicates functionality of the rawcopy plugin.
382
383.. _migration-2.3-qa-changes:
384
385QA Changes
386----------
387
388The following QA checks have changed:
389
390- ``unsafe-references-in-binaries``: The
391 ``unsafe-references-in-binaries`` QA check, which was disabled by
392 default, has now been removed. This check was intended to detect
393 binaries in ``/bin`` that link to libraries in ``/usr/lib`` and have
394 the case where the user has ``/usr`` on a separate filesystem to
395 ``/``.
396
397 The removed QA check was buggy. Additionally, ``/usr`` residing on a
398 separate partition from ``/`` is now a rare configuration.
399 Consequently, ``unsafe-references-in-binaries`` was removed.
400
401- ``file-rdeps``: The ``file-rdeps`` QA check is now an error by
402 default instead of a warning. Because it is an error instead of a
403 warning, you need to address missing runtime dependencies.
404
405 For additional information, see the
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500406 :ref:`insane <ref-classes-insane>` class and the
Andrew Geissler09209ee2020-12-13 08:44:15 -0600407 ":ref:`ref-manual/qa-checks:errors and warnings`" section.
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500408
409.. _migration-2.3-miscellaneous-changes:
410
411Miscellaneous Changes
412---------------------
413
414The following miscellaneous changes have occurred:
415
416- In this release, a number of recipes have been changed to ignore the
417 ``largefile`` :term:`DISTRO_FEATURES` item,
418 enabling large file support unconditionally. This feature has always
419 been enabled by default. Disabling the feature has not been widely
420 tested.
421
422 .. note::
423
424 Future releases of the Yocto Project will remove entirely the
425 ability to disable the
426 largefile
427 feature, which would make it unconditionally enabled everywhere.
428
429- If the :term:`DISTRO_VERSION` value contains
430 the value of the :term:`DATE` variable, which is the
431 default between Poky releases, the ``DATE`` value is explicitly
432 excluded from ``/etc/issue`` and ``/etc/issue.net``, which is
433 displayed at the login prompt, in order to avoid conflicts with
434 Multilib enabled. Regardless, the ``DATE`` value is inaccurate if the
435 ``base-files`` recipe is restored from shared state (sstate) rather
436 than rebuilt.
437
438 If you need the build date recorded in ``/etc/issue*`` or anywhere
439 else in your image, a better method is to define a post-processing
440 function to do it and have the function called from
441 :term:`ROOTFS_POSTPROCESS_COMMAND`.
442 Doing so ensures the value is always up-to-date with the created
443 image.
444
445- Dropbear's ``init`` script now disables DSA host keys by default.
446 This change is in line with the systemd service file, which supports
447 RSA keys only, and with recent versions of OpenSSH, which deprecates
448 DSA host keys.
449
450- The :ref:`buildhistory <ref-classes-buildhistory>` class now
451 correctly uses tabs as separators between all columns in
452 ``installed-package-sizes.txt`` in order to aid import into other
453 tools.
454
455- The ``USE_LDCONFIG`` variable has been replaced with the "ldconfig"
456 ``DISTRO_FEATURES`` feature. Distributions that previously set:
457 ::
458
459 USE_LDCONFIG = "0"
460
461 should now instead use the following:
462
463 ::
464
465 DISTRO_FEATURES_BACKFILL_CONSIDERED_append = " ldconfig"
466
467- The default value of
468 :term:`COPYLEFT_LICENSE_INCLUDE` now
469 includes all versions of AGPL licenses in addition to GPL and LGPL.
470
471 .. note::
472
473 The default list is not intended to be guaranteed as a complete
474 safe list. You should seek legal advice based on what you are
475 distributing if you are unsure.
476
477- Kernel module packages are now suffixed with the kernel version in
478 order to allow module packages from multiple kernel versions to
479 co-exist on a target system. If you wish to return to the previous
480 naming scheme that does not include the version suffix, use the
481 following:
482 ::
483
Andrew Geissler4c19ea12020-10-27 13:52:24 -0500484 KERNEL_MODULE_PACKAGE_SUFFIX = ""
Andrew Geisslerc9f78652020-09-18 14:11:35 -0500485
486- Removal of ``libtool`` ``*.la`` files is now enabled by default. The
487 ``*.la`` files are not actually needed on Linux and relocating them
488 is an unnecessary burden.
489
490 If you need to preserve these ``.la`` files (e.g. in a custom
491 distribution), you must change
492 :term:`INHERIT_DISTRO` such that
493 "remove-libtool" is not included in the value.
494
495- Extensible SDKs built for GCC 5+ now refuse to install on a
496 distribution where the host GCC version is 4.8 or 4.9. This change
497 resulted from the fact that the installation is known to fail due to
498 the way the ``uninative`` shared state (sstate) package is built. See
499 the :ref:`uninative <ref-classes-uninative>` class for additional
500 information.
501
502- All native and nativesdk recipes now use a separate
503 ``DISTRO_FEATURES`` value instead of sharing the value used by
504 recipes for the target, in order to avoid unnecessary rebuilds.
505
506 The ``DISTRO_FEATURES`` for ``native`` recipes is
507 :term:`DISTRO_FEATURES_NATIVE` added to
508 an intersection of ``DISTRO_FEATURES`` and
509 :term:`DISTRO_FEATURES_FILTER_NATIVE`.
510
511 For nativesdk recipes, the corresponding variables are
512 :term:`DISTRO_FEATURES_NATIVESDK`
513 and
514 :term:`DISTRO_FEATURES_FILTER_NATIVESDK`.
515
516- The ``FILESDIR`` variable, which was previously deprecated and rarely
517 used, has now been removed. You should change any recipes that set
518 ``FILESDIR`` to set :term:`FILESPATH` instead.
519
520- The ``MULTIMACH_HOST_SYS`` variable has been removed as it is no
521 longer needed with recipe-specific sysroots.
522
523