poky/documentation/ref-manual/migration-3.1.rst

Moving to the Yocto Project 3.1 Release (dunfell)
=================================================

This section provides migration information for moving to the Yocto
Project 3.1 Release (codename "dunfell") from the prior release.

.. _migration-3.1-minimum-system-requirements:

Minimum system requirements
---------------------------

The following versions / requirements of build host components have been
updated:

-  gcc 5.0

-  python 3.5

-  tar 1.28

-  ``rpcgen`` is now required on the host (part of the ``libc-dev-bin``
   package on Ubuntu, Debian and related distributions, and the
   ``glibc`` package on RPM-based distributions).

Additionally, the ``makeinfo`` and ``pod2man`` tools are *no longer*
required on the host.

.. _migration-3.1-mpc8315e-rdb-removed:

mpc8315e-rdb machine removed
----------------------------

The MPC8315E-RDB machine is old/obsolete and unobtainable, thus given
the maintenance burden the ``mpc8315e-rdb`` machine configuration that
supported it has been removed in this release. The removal does leave a
gap in official PowerPC reference hardware support; this may change in
future if a suitable machine with accompanying support resources is
found.

.. _migration-3.1-python-2-removed:

Python 2 removed
----------------

Due to the expiration of upstream support in January 2020, support for
Python 2 has now been removed; it is recommended that you use Python 3
instead. If absolutely needed there is a meta-python2 community layer
containing Python 2, related classes and various Python 2-based modules,
however it should not be considered as supported.

.. _migration-3.1-reproducible-builds:

Reproducible builds now enabled by default
------------------------------------------

In order to avoid unnecessary differences in output files (aiding binary
reproducibility), the Poky distribution configuration
(``DISTRO = "poky"``) now inherits the ``reproducible_build`` class by
default.

.. _migration-3.1-ptest-feature-impact:

Impact of ptest feature is now more significant
-----------------------------------------------

The Poky distribution configuration (``DISTRO = "poky"``) enables ptests
by default to enable runtime testing of various components. In this
release, a dependency needed to be added that has resulted in a
significant increase in the number of components that will be built just
when building a simple image such as core-image-minimal. If you do not
need runtime tests enabled for core components, then it is recommended
that you remove "ptest" from
:term:`DISTRO_FEATURES` to save a significant
amount of build time e.g. by adding the following in your configuration::

   DISTRO_FEATURES_remove = "ptest"

.. _migration-3.1-removed-recipes:

Removed recipes
---------------

The following recipes have been removed:

-  ``chkconfig``: obsolete

-  ``console-tools``: obsolete

-  ``enchant``: replaced by ``enchant2``

-  ``foomatic-filters``: obsolete

-  ``libidn``: no longer needed, moved to meta-oe

-  ``libmodulemd``: replaced by ``libmodulemd-v1``

-  ``linux-yocto``: drop 4.19, 5.2 version recipes (5.4 now provided)

-  ``nspr``: no longer needed, moved to meta-oe

-  ``nss``: no longer needed, moved to meta-oe

-  ``python``: Python 2 removed (Python 3 preferred)

-  ``python-setuptools``: Python 2 version removed (python3-setuptools
   preferred)

-  ``sysprof``: no longer needed, moved to meta-oe

-  ``texi2html``: obsolete

-  ``u-boot-fw-utils``: functionally replaced by ``libubootenv``

.. _migration-3.1-features-check:

features_check class replaces distro_features_check
---------------------------------------------------

The ``distro_features_check`` class has had its functionality expanded,
now supporting ``ANY_OF_MACHINE_FEATURES``,
``REQUIRED_MACHINE_FEATURES``, ``CONFLICT_MACHINE_FEATURES``,
``ANY_OF_COMBINED_FEATURES``, ``REQUIRED_COMBINED_FEATURES``,
``CONFLICT_COMBINED_FEATURES``. As a result the class has now been
renamed to ``features_check``; the ``distro_features_check`` class still
exists but generates a warning and redirects to the new class. In
preparation for a future removal of the old class it is recommended that
you update recipes currently inheriting ``distro_features_check`` to
inherit ``features_check`` instead.

.. _migration-3.1-removed-classes:

Removed classes
---------------

The following classes have been removed:

-  ``distutils-base``: moved to meta-python2

-  ``distutils``: moved to meta-python2

-  ``libc-common``: merged into the glibc recipe as nothing else used
   it.

-  ``python-dir``: moved to meta-python2

-  ``pythonnative``: moved to meta-python2

-  ``setuptools``: moved to meta-python2

-  ``tinderclient``: dropped as it was obsolete.

.. _migration-3.1-src-uri-checksums:

SRC_URI checksum behaviour
--------------------------

Previously, recipes by tradition included both SHA256 and MD5 checksums
for remotely fetched files in :term:`SRC_URI`, even
though only one is actually mandated. However, the MD5 checksum does not
add much given its inherent weakness; thus when a checksum fails only
the SHA256 sum will now be printed. The md5sum will still be verified if
it is specified.

.. _migration-3.1-npm:

npm fetcher changes
-------------------

The npm fetcher has been completely reworked in this release. The npm
fetcher now only fetches the package source itself and no longer the
dependencies; there is now also an npmsw fetcher which explicitly
fetches the shrinkwrap file and the dependencies. This removes the
slightly awkward ``NPM_LOCKDOWN`` and ``NPM_SHRINKWRAP`` variables which
pointed to local files; the lockdown file is no longer needed at all.
Additionally, the package name in ``npm://`` entries in
:term:`SRC_URI` is now specified using a ``package``
parameter instead of the earlier ``name`` which overlapped with the
generic ``name`` parameter. All recipes using the npm fetcher will need
to be changed as a result.

An example of the new scheme::

   SRC_URI = "npm://registry.npmjs.org;package=array-flatten;version=1.1.1 \
              npmsw://${THISDIR}/npm-shrinkwrap.json"

Another example where the sources are fetched from git rather than an npm repository::

   SRC_URI = "git://github.com/foo/bar.git;protocol=https \
              npmsw://${THISDIR}/npm-shrinkwrap.json"

devtool and recipetool have also been updated to match with the npm
fetcher changes. Other than producing working and more complete recipes
for npm sources, there is also a minor change to the command line for
devtool: the ``--fetch-dev`` option has been renamed to ``--npm-dev`` as
it is npm-specific.

.. _migration-3.1-packaging-changes:

Packaging changes
-----------------

-  ``intltool`` has been removed from ``packagegroup-core-sdk`` as it is
   rarely needed to build modern software - gettext can do most of the
   things it used to be needed for. ``intltool`` has also been removed
   from ``packagegroup-core-self-hosted`` as it is not needed to for
   standard builds.

-  git: ``git-am``, ``git-difftool``, ``git-submodule``, and
   ``git-request-pull`` are no longer perl-based, so are now installed
   with the main ``git`` package instead of within ``git-perltools``.

-  The ``ldconfig`` binary built as part of glibc has now been moved to
   its own ``ldconfig`` package (note no ``glibc-`` prefix). This
   package is in the :term:`RRECOMMENDS` of the main
   ``glibc`` package if ``ldconfig`` is present in
   :term:`DISTRO_FEATURES`.

-  ``libevent`` now splits each shared library into its own package (as
   Debian does). Since these are shared libraries and will be pulled in
   through the normal shared library dependency handling, there should
   be no impact to existing configurations other than less unnecessary
   libraries being installed in some cases.

-  linux-firmware now has a new package for ``bcm4366c`` and includes
   available NVRAM config files into the ``bcm43340``, ``bcm43362``,
   ``bcm43430`` and ``bcm4356-pcie`` packages.

-  ``harfbuzz`` now splits the new ``libharfbuzz-subset.so`` library
   into its own package to reduce the main package size in cases where
   ``libharfbuzz-subset.so`` is not needed.

.. _migration-3.1-package-qa-warnings:

Additional warnings
-------------------

Warnings will now be shown at ``do_package_qa`` time in the following
circumstances:

-  A recipe installs ``.desktop`` files containing ``MimeType`` keys but
   does not inherit the new ``mime-xdg`` class

-  A recipe installs ``.xml`` files into ``${datadir}/mime/packages``
   but does not inherit the ``mime`` class

.. _migration-3.1-x86-live-wic:

``wic`` image type now used instead of ``live`` by default for x86
------------------------------------------------------------------

``conf/machine/include/x86-base.inc`` (inherited by most x86 machine
configurations) now specifies ``wic`` instead of ``live`` by default in
:term:`IMAGE_FSTYPES`. The ``live`` image type will
likely be removed in a future release so it is recommended that you use
``wic`` instead.

.. _migration-3.1-misc:

Miscellaneous changes
---------------------

-  The undocumented ``SRC_DISTRIBUTE_LICENSES`` variable has now been
   removed in favour of a new ``AVAILABLE_LICENSES`` variable which is
   dynamically set based upon license files found in
   ``${COMMON_LICENSE_DIR}`` and ``${LICENSE_PATH}``.

-  The tune definition for big-endian microblaze machines is now
   ``microblaze`` instead of ``microblazeeb``.

-  ``newlib`` no longer has built-in syscalls. ``libgloss`` should then
   provide the syscalls, ``crt0.o`` and other functions that are no
   longer part of ``newlib`` itself. If you are using
   ``TCLIBC = "newlib"`` this now means that you must link applications
   with both ``newlib`` and ``libgloss``, whereas before ``newlib``
   would run in many configurations by itself.