poky: subtree update:c8075ed8f1..7d0988966c
Adrian (1):
weston-init: Set $HOME variable for SysVinit manager
Alejandro Hernandez Samaniego (6):
bitbake: bitbake: Add Azure Storage fetcher implementation
bitbake: docs: Add Az fetcher documentation
bitbake: docs: Add AZ_SAS definition to glossary
ref-manual: Add Az fetcher documentation
sanity.bbclass: Add az fetcher to accepted URI protocols for mirrors sanity
devtool: Fix do_kernel_configme task
Alexander Kanavin (23):
core-image.bbclass: add a weston IMAGE_FEATURE, similar to x11-base
rootfs-postcommands: correctly set systemd target for weston images
oeqa/weston: correctly run wayland-info when systemd is in use
dnf: upgrade 4.5.2 -> 4.6.0
libdnf: update 0.55.2 -> 0.58.0
libdnf: replace a musl fix with a better one
librepo: update 1.12.1 -> 1.13.0
libical: update 3.0.8 -> 3.0.9
libical: drop the ad hoc native generator build
bmap-tools: upgrade 3.5 -> 3.6
procps: update 3.3.16 -> 3.3.17
libxcrypt: fix sporadic failures in nativesdk-libxcrypt-compat
bind: upgrade 9.16.11 -> 9.16.12
spirv-headers: update to latest revision
spirv-tools: upgrade 2020.6 -> 2020.7
mesa: update 20.3.4 -> 21.0.0
attr: update 2.4.48 -> 2.5.1
asciidoc: update 9.0.4 -> 9.1.0
libhandy: update 1.0.3 -> 1.2.0
strace: update 5.10 -> 5.11
automake: update 1.16.2 - > 1.16.3
xxhash: import from meta-oe
apt: update 1.8.2.2 -> 2.2.2
Alistair Francis (4):
openssl: Enable building for RISC-V 32-bit
linux-yocto: Add qemuriscv32 as a compatible machine
machine: Initial commit of qemuriscv32
u-boot: Add support for building for qemuriscv32
Andrei Gherzan (3):
gcr: Add support for building without x11
epiphany: Add package configuration for building in developer mode
glibc: Backport patch to fix _SC_LEVEL1_ICACHE_LINESIZE
Anton D. Kachalov (1):
run-postinsts: do not remove postinsts directory.
Awais Belal (2):
libsdl2: disable shared memory for native builds
rootfs.py: uninstall the run-postinsts package if not needed
Bartosz Golaszewski (1):
python3: make pydoc rdepend on python3-io
Bruce Ashfield (27):
linux-yocto/5.10: fix x86 32bit boot warnings
linux-yocto/5.10: fix qemumips testimage failures
linux-yocto/5.10: update to v5.10.17 and -rt31
linux-yocto/5.4: update to v5.4.99
linux-yocto/5.10: update to v5.10.19
linux-yocto/5.4: update to v5.4.101
perf: fix reproducibility issues
linux-yocto/5.4: update to v5.4.103
kern-tools: symbol-why fix and README update
linux-yocto/5.10: update to v5.10.21
linux-yocto/qemuarmv5: fix configuration warning
yocto-bsp: update reference platforms to latest 5.10
perf: reproducibility fixes for pmu-events.c
reproducibile: remove perf from exclusions
linux-yocto/5.10: fix SERIAL_OF_PLATFORM warning
linux-yocto/5.4: update to v5.4.105
linux-yocto/5.10: update to v5.10.23
linux-yocto/5.10: cfg: features/numa: Remove NODES_SPAN_OTHER_NODES option
lttng-modules: update devupstream to 2.12.5+
lttng-modules: backport patches to fix build against 5.12+ kernel
kernel-devsrc: fix on-target scripts/prepare for v5.12+
linux-yocto/5.10: cfg: fix ppc64 configuration warnings
linux-yocto-dev: bump to v5.12-rc
linux-yocto/5.10: update qemuriscv32 v5.10.23
linux-yocto/5.10: update to v5.10.25
linux-yocto/5.4: update to v5.4.107
linux-yocto-rt/5.10: update to -rt34
Changqing Li (1):
go-helloworld: disable module-aware mode
Charlie Davies (9):
bitbake-bblayers/create: Fix incorrect priority help message
bitbake-bblayers/create: Add optional layerid argument
bitbake: bitbake: providers: remove unneeded logging call
bitbake: bitbake: providers: consistent single line formatting between functions
bitbake: bitbake: providers: introduce logic for REQUIRED_VERSION variable
bitbake: bitbake: providers: check for REQUIRED_VERSION in _filterProviders
bitbake: bitbake: lib: add Required Version to Bitbake --show-versions command
bitbake: bitbake: cooker: Add REQUIRED_VERSION checks
multilib_global: handle REQUIRED_VERSION
Chen Qi (3):
python3-jinja2: set CVE_PRODUCT
bitbake: event.py: fix regression about INVALIDCONF
populate_sdk_ext: record METADATA_REVISION
Christopher Larson (2):
buildhistory: add missing vardepsexcludes
image,populate_sdk_base: move 'func' flag setting for sdk command vars
Diego Santa Cruz (2):
rpm: split build and extra functionality into separate packages
sstate-diff-machines.sh: support rpm, deb and tar package types in analysis
Dorinda (8):
meta/recipes-rt: Add HOMEPAGE / DESCRIPTION
meta-skeleton: Add HOMEPAGE / DESCRIPTION
meta/recipes-extended: Add HOMEPAGE / DESCRIPTION
meta/recipes-support: Add HOMEPAGE / DESCRIPTION
meta/recipes-kernel: Add HOMEPAGE / DESCRIPTION
meta/recipes-multimedia: Add HOMEPAGE / DESCRIPTION
meta/recipes-graphics: Add HOMEPAGE / DESCRIPTION
dev-manual/common-task.rst: Added documentation for debuginfod support
Dorinda Bassey (4):
meta/recipes-devtools: Add HOMEPAGE / DESCRIPTION
distrodata.py: Test to track recipes without HOMEPAGE or DESCRIPTION
meta-selftest: Add HOMEPAGE / DESCRIPTION
devshell.bbclass: Exceptions displayed within devpyshell
Douglas Royds (2):
cmake: Fully-qualified path to ar
externalsrc: Detect code changes in submodules
Jon Mason (5):
runqemu: use "raw" instead of "bin" for ovmf
runqemu: add QB_GRAPHICS
runqemu: correct forcing of ttyS0
runqemu: modify novga to conform to documentation
tune-cortexa32: Add hard FPU
Jose Quaresma (4):
spirv-tools: python3 is need not only for the tests
spirv-tools: cleanup an old patch that is not used anymore
gstreamer1.0: upgrade 1.18.3 -> 1.18.4
gstreamer1.0-plugins-good: fix rtpjitterbuffer regression
Kai Kang (2):
toolchain-scripts.bbclass: customize prompt string for SDKs
local.conf.sample.extended: sample value for SDK_PS1
Kevin Hao (1):
meta-yocto-bsp: beaglebone: Set a fixed size for boot partition in WIC image
Khairul Rohaizzat Jamaluddin (1):
glibc: Fix CVE-2021-27645
Khem Raj (33):
python3-cython: Check for files before editing
webkitgtk: Use linker options to reduce memory overhead during linking
webkitgtk: Enhance check for atomics to include 1 byte CAS
qemuppc64: Add a QEMU machine definition for ppc64
linux-yocto: Enable powerpc64le QEMU
formfactor: Add machine config for qemuppc64
inittab: Add getty launch on hvc0 for qemuppc64
parselogs: Allow expected kernel messages for qemuppc64
spirv-tools: Replace strncpy with memcpy
python3-setuptools: Upgrade 54.1.0 -> 54.1.1
perf: Remove libunwind for rv32
Enable qemu usermode on ppc64
cmake.bbclass: Create cmake arch mapping for ppc64le
libucontext: Recognize ppc64le architecture
libunwind: Do not assume libdir for ppc64
gcc: Package new module mapper tool
glib-2.0: Drop volatile qualifier
go: Update to 1.16.2
go-dep: Remove
libunwind: Link with libucontext on musl
libedit: upgrade 20191231-3.1 -> 20210216-3.1
webkitgtk: Reduce stack and heap sizes for jsc on musl
webkitgtk: Add packageconfig to chose between size and speed
webkitgtk: enable strlen and stpcpy on musl/linux too
gcc-sanitizers: Package up hwasan files
apt: Support old resolver as fallback
apt: Do not disable NLS
valgrind: Fix ptests on ppc64 LE
documentation-audit.sh: Fix typo in specifying LICENSE_FLAGS_WHITELIST
image-uefi: Set efi_file for rv32/rv64
grub2: Disable for RISCV32
grub-efi: Re-introduce lost cast to long
grub2: Enable on riscv32
Lee Chee Yang (2):
cve-update-db-native: consider version suffix when update CVE db
cve-check: CVE_VERSION_SUFFIX to work with patched release
Li Wang (1):
linux-dummy: add empty dependent packages
Luca Boccassi (2):
util-linux: split uuid in separate recipe to allow bootstrapping
util-linux: backport patch to skip build of unused objects
Manuel Leonhardt (1):
bitbake: tinfoil: Honor quiet when parsing recipes
Mark Hatle (2):
populate_sdk_ext: Avoid copying and producing .pyc files
populate_sdk_ext: Add support for PR service
Martin Jansa (7):
glib-2.0: call os.path.normpath on THISDIR
glib-2.0: show an error about --cross-file only when such file exists
busybox: refresh the defconfig from 1.33.0
busybox-inittab: rename for PV to match with busybox's PV
iso-codes: fix protocol in SRC_URI
packagegroup-core-weston.bb: add pam to REQUIRED_DISTRO_FEATURES
python3: fix PACKAGECONFIG handling
Michael Opdenacker (4):
bitbake: doc: Update links to documentation
Do not assume working from $HOME
Update documentation URL
documentation/README minor improvements
Michael Trensch (1):
linux-firmware: Fix packaging
Mikko Rapeli (1):
openssl: update to 1.1.1k to fix CVE-2021-3450 and CVE-2021-3449
Ming Liu (2):
kernel-fitimage.bbclass: support both KERNEL_DEVICETREE/EXTERNAL_KERNEL_DEVICETREE
initramfs-framework:rootfs: fix some conditional check
Mingli Yu (1):
libtool: make sure autoheader run before autoconf
Minjae Kim (3):
qemu: fix CVE-2021-20203
git: fix CVE-2021-21300
git: upgrade 2.30.1 -> 2.31.1
Naveen Saini (1):
grub: upgrade 2.04 -> 2.06~rc1
Oleksandr Kravchuk (1):
python3-setuptools: update to 54.1.0
Paul Gortmaker (1):
bitbake: gitignore: ignore runqueue-tests/bitbake-cookerdaemon.log
Peter Kjellerstedt (4):
meson: Correctly set uid/gid of installed files
uninative-tarball: Add a dependency on nativesdk-glibc-dbg
metadata_scm.bbclass: Use immediate expansion for the METADATA_* variables
license.bbclass: Improve parsing time when INCOMPATIBLE_LICENSES is big
Petr Vorel (1):
scripts/verify-bashisms: Update checkbashisms.pl URL
Purushottam Choudhary (1):
shadow: whitelist CVE-2013-4235
Randy MacLeod (2):
ffmpeg: upgrade 4.3.1 -> 4.3.2
zstd: upgrade 1.4.8 -> 1.4.9
Richard Purdie (33):
ltp: Fix another determinism issue
build-appliance-image: Drop kernel module handling
ltp: Further extend reproducibility patch
bitbake: runqueue: Fix task execution corruption issue
bitbake: runqueue: Add setscene task overlap sanity check
bitbake: __init__.py: Fix bitbake debug log handling
ltp: Fix determinism issue
ltp: Fix libswapon issue properly
bitbake: event: Fix multiconfig event handler change performance regressions
bitbake: event: Fix another 'if d' test to test for None explictly
bitbake: tests/color: Fix event register to pass the datastore
bitbake: bblayers/query: Update to handle REQUIRED_VERSION
scripts/contrib/list-packageconfig-flags: Upate for tinfoil API changes
layer.conf: Update for hardknott release series
layer.conf: Update for hardknott release series
build-appliance-image: Update to master head revision
layer.conf: Add gatesgarth back for now to allow for transition
build-appliance-image: Update to master head revision
meson: Work around determinism issue
lttng-tools: upgrade 2.12.2 -> 2.12.3
lttng-modules: upgrade 2.12.4 -> 2.12.5
sqlite3: upgrade 3.34.1 -> 3.35.0
sudo: upgrade 1.9.5p2 -> 1.9.6
selftest/reproducible: Drop ovmf exclusion
selftest/wic: Fix dependency issue in rawcopy test
layer.conf: Drop gatesgarth from the layer series names
bitbake: build: Add find_stale_stamps function
bitbake: runqueue/event: Add an event for notifying of stale setscene tasks
sstate: Remove stale objects before the main build
sstate: Add documentation for eventhandlers and tweak naming
site/elfutils/libunistring: Drop patching for iconv and set in site file
build-appliance-image: Update to master head revision
util-linux-libuuid: Simplify recipe and rename from util-linux-uuid
Robert P. J. Day (8):
bitbake-whatchanged: change ending quote to proper period
bitbake: doc: move BBFILES_DYNAMIC for alphabetical order
ref-manual: remove erroneous '\*' from variables list
bitbake: doc: mention that addtask handles multiple dependencies
bitbake: doc: fix glossary link for BB_INVALIDCONF variable
bitbake: doc: fix syntax error in layer.conf example
bitbake.conf: correct description of HOSTTOOLS_DIR
packagegroups: delete useless "PROVIDES" lines
Robert Yang (1):
image.bbclass: deltask do_packagedata
Ross Burton (13):
libinput: less parallism to increase chances the test suite works
ptest-packagelists: remove libinput-ptest
insane: don't check for a warning string that is never output
gcc: add an option for --enable-poison-system-directories to be fatal
gcc-cross: make use of the system include directories fatal
oeqa/selftest: add test to verify that poisoned sysroots are detected
ovmf: make output binaries reproducible
base: respect downloadfilename when sniffing the SRC_URI for dependencies
lib/oe/utils: add directory size function
classes/image: use oe.utils.directory_size() instead of du
bitbake.conf: ensure BUILD_* tools match target tools
meson: fix native/host confusion in gobject-introspection
meson: use native-file instead of environment variables
Sakib Sajal (2):
buildstats.bbclass: improve timeout handling
scripts: add oe-time-dd-test.sh
Scott Murray (1):
u-boot: Fix CVE-2021-27097, CVE-2021-27138
Stefan Ghinea (1):
wpa-supplicant: fix CVE-2021-27803
Stefan Schmidt (1):
systemd-conf: do not ask for DHCP if configured on kernel command line
Tomasz Dziendzielski (2):
bitbake: runqueue: Print pseudo.log if fakeroot task failed
bitbake: codeparser: Fix TypeError in bitbake debug mode
Ulrich ?lmann (1):
gstreamer1.0: add support for coretracers
Ulrich Ölmann (3):
common-tasks.rst: fix 'wic cp' command
common-tasks.rst: fix typo
local.conf.sample: fix typo
Vivien Didelot (6):
systemd-container: sort RRECOMMENDS alphabetically
systemd-container: recommend tar
systemd-container: recommends StartExecPre modules
poky: fix typos
poky: don't use space with +=
beaglebone-yocto: allow other virtual/bootloader
Wang Mingyu (20):
shaderc: upgrade 2020.4 -> 2020.5
stress-ng: upgrade 0.12.03 -> 0.12.04
taglib: upgrade 1.11.1 -> 1.12
vala: upgrade 0.50.3 -> 0.50.4
python3-pygments: upgrade 2.8.0 -> 2.8.1
python3-zipp: upgrade 3.4.0 -> 3.4.1
sysklogd: upgrade 2.2.1 -> 2.2.2
help2man: upgrade 1.48.1 -> 1.48.2
iso-codes: upgrade 4.5.0 -> 4.6.0
jquery: upgrade 3.5.1 -> 3.6.0
ovmf: upgrade 202011 -> 202102
msmtp: upgrade 1.8.14 -> 1.8.15
gptfdisk: upgrade 1.0.6 -> 1.0.7
vulkan-headers: upgrade 1.2.162.0 -> 1.2.170.0
vulkan-loader: upgrade 1.2.162.0 -> 1.2.170.0
vulkan-tools: upgrade 1.2.162.1 -> 1.2.170.0
gcc: Update patch to only patch files once
openssh: upgrade 8.4p1 -> 8.5p1
openssh: Improve LICENSE to show BSD license variants.
gnutls: upgrade 3.7.0 -> 3.7.1
Yanfei Xu (1):
glibc: fix pthread_cond_destroy hangs with process-shared mutex
Yann Dirson (1):
ffmpeg: disable GPL features by default
Yi Fan Yu (1):
valgrind: Fix nlcontrolc.vgtest
jan (1):
cve-update-db-native: Allow to overrule the URL in a bbappend.
wangmy@fujitsu.com (1):
systemd: upgrade 247.3 -> 247.4
zhengruoqin (3):
sudo: upgrade 1.9.6 -> 1.9.6p1
liburi-perl: upgrade 1.74 -> 5.08
stress-ng: upgrade 0.12.04 -> 0.12.05
Signed-off-by: Andrew Geissler <geissonator@yahoo.com>
Change-Id: I04195d4c610ca1674dd736eceeb6442d974cb711
diff --git a/poky/bitbake/README b/poky/bitbake/README
index 479c376..96e6007 100644
--- a/poky/bitbake/README
+++ b/poky/bitbake/README
@@ -11,7 +11,7 @@
Bitbake plain documentation can be found under the doc directory or its integrated
html version at the Yocto Project website:
- http://yoctoproject.org/documentation
+ https://docs.yoctoproject.org
Contributing
------------
diff --git a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst
index d74e768..56abf77 100644
--- a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst
+++ b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst
@@ -244,7 +244,8 @@
BBFILE_COLLECTIONS = "upstream local"
BBFILE_PATTERN_upstream = "^/stuff/openembedded/"
BBFILE_PATTERN_local = "^/stuff/openembedded.modified/"
- BBFILE_PRIORITY_upstream = "5" BBFILE_PRIORITY_local = "10"
+ BBFILE_PRIORITY_upstream = "5"
+ BBFILE_PRIORITY_local = "10"
.. note::
diff --git a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
index 6760b10..e9a5f33 100644
--- a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
+++ b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
@@ -624,6 +624,34 @@
SRC_URI = "repo://REPOROOT;protocol=git;branch=some_branch;manifest=my_manifest.xml"
SRC_URI = "repo://REPOROOT;protocol=file;branch=some_branch;manifest=my_manifest.xml"
+.. _az-fetcher:
+
+Az Fetcher (``az://``)
+--------------------------
+
+This submodule fetches data from an
+`Azure Storage account <https://docs.microsoft.com/en-us/azure/storage/>`__ ,
+it inherits its functionality from the HTTP wget fetcher, but modifies its
+behavior to accomodate the usage of a
+`Shared Access Signature (SAS) <https://docs.microsoft.com/en-us/azure/storage/common/storage-sas-overview>`__
+for non-public data.
+
+Such functionality is set by the variable:
+
+- :term:`AZ_SAS`: The Azure Storage Shared Access Signature provides secure
+ delegate access to resources, if this variable is set, the Az Fetcher will
+ use it when fetching artifacts from the cloud.
+
+You can specify the AZ_SAS variable as shown below: ::
+
+ AZ_SAS = "se=2021-01-01&sp=r&sv=2018-11-09&sr=c&skoid=<skoid>&sig=<signature>"
+
+Here is an example URL: ::
+
+ SRC_URI = "az://<azure-storage-account>.blob.core.windows.net/<foo_container>/<bar_file>"
+
+It can also be used when setting mirrors definitions using the :term:`PREMIRRORS` variable.
+
Other Fetchers
--------------
diff --git a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
index 7ea68ad..d4190c2 100644
--- a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
+++ b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
@@ -1296,6 +1296,17 @@
See the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:variable flags`" section for information
on variable flags you can use with tasks.
+.. note::
+
+ While it's infrequent, it's possible to define multiple tasks as
+ dependencies when calling ``addtask``. For example, here's a snippet
+ from the OpenEmbedded class file ``package_tar.bbclass``::
+
+ addtask package_write_tar before do_build after do_packagedata do_package
+
+ Note how the ``package_write_tar`` task has to wait until both of
+ ``do_packagedata`` and ``do_package`` complete.
+
Deleting a Task
---------------
@@ -1569,7 +1580,7 @@
events when each of the workers parse the base configuration or if
the server changes configuration and reparses. Any given datastore
only has one such event executed against it, however. If
- ```BB_INVALIDCONF`` <#>`__ is set in the datastore by the event
+ :term:`BB_INVALIDCONF` is set in the datastore by the event
handler, the configuration is reparsed and a new event triggered,
allowing the metadata to update configuration.
diff --git a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst
index 6469f9d..1528b04 100644
--- a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst
+++ b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst
@@ -39,6 +39,19 @@
when specified allows for the Git binary from the host to be used
rather than building ``git-native``.
+ :term:`AZ_SAS`
+ Azure Storage Shared Access Signature, when using the
+ :ref:`Azure Storage fetcher <bitbake-user-manual/bitbake-user-manual-fetching:fetchers>`
+ This variable can be defined to be used by the fetcher to authenticate
+ and gain access to non-public artifacts.
+ ::
+
+ AZ_SAS = ""se=2021-01-01&sp=r&sv=2018-11-09&sr=c&skoid=<skoid>&sig=<signature>""
+
+ For more information see Microsoft's Azure Storage documentation at
+ https://docs.microsoft.com/en-us/azure/storage/common/storage-sas-overview
+
+
:term:`B`
The directory in which BitBake executes functions during a recipe's
build process.
@@ -257,45 +270,6 @@
``my-recipe.bb`` is executing, the ``BB_FILENAME`` variable contains
"/foo/path/my-recipe.bb".
- :term:`BBFILES_DYNAMIC`
- Activates content depending on presence of identified layers. You
- identify the layers by the collections that the layers define.
-
- Use the ``BBFILES_DYNAMIC`` variable to avoid ``.bbappend`` files whose
- corresponding ``.bb`` file is in a layer that attempts to modify other
- layers through ``.bbappend`` but does not want to introduce a hard
- dependency on those other layers.
-
- Additionally you can prefix the rule with "!" to add ``.bbappend`` and
- ``.bb`` files in case a layer is not present. Use this avoid hard
- dependency on those other layers.
-
- Use the following form for ``BBFILES_DYNAMIC``: ::
-
- collection_name:filename_pattern
-
- The following example identifies two collection names and two filename
- patterns: ::
-
- BBFILES_DYNAMIC += "\
- clang-layer:${LAYERDIR}/bbappends/meta-clang/*/*/*.bbappend \
- core:${LAYERDIR}/bbappends/openembedded-core/meta/*/*/*.bbappend \
- "
-
- When the collection name is prefixed with "!" it will add the file pattern in case
- the layer is absent: ::
-
- BBFILES_DYNAMIC += "\
- !clang-layer:${LAYERDIR}/backfill/meta-clang/*/*/*.bb \
- "
-
- This next example shows an error message that occurs because invalid
- entries are found, which cause parsing to abort: ::
-
- ERROR: BBFILES_DYNAMIC entries must be of the form {!}<collection name>:<filename pattern>, not:
- /work/my-layer/bbappends/meta-security-isafw/*/*/*.bbappend
- /work/my-layer/bbappends/openembedded-core/meta/*/*/*.bbappend
-
:term:`BB_GENERATE_MIRROR_TARBALLS`
Causes tarballs of the Git repositories, including the Git metadata,
to be placed in the :term:`DL_DIR` directory. Anyone
@@ -671,6 +645,45 @@
For details on the syntax, see the documentation by following the
previous link.
+ :term:`BBFILES_DYNAMIC`
+ Activates content depending on presence of identified layers. You
+ identify the layers by the collections that the layers define.
+
+ Use the ``BBFILES_DYNAMIC`` variable to avoid ``.bbappend`` files whose
+ corresponding ``.bb`` file is in a layer that attempts to modify other
+ layers through ``.bbappend`` but does not want to introduce a hard
+ dependency on those other layers.
+
+ Additionally you can prefix the rule with "!" to add ``.bbappend`` and
+ ``.bb`` files in case a layer is not present. Use this avoid hard
+ dependency on those other layers.
+
+ Use the following form for ``BBFILES_DYNAMIC``: ::
+
+ collection_name:filename_pattern
+
+ The following example identifies two collection names and two filename
+ patterns: ::
+
+ BBFILES_DYNAMIC += "\
+ clang-layer:${LAYERDIR}/bbappends/meta-clang/*/*/*.bbappend \
+ core:${LAYERDIR}/bbappends/openembedded-core/meta/*/*/*.bbappend \
+ "
+
+ When the collection name is prefixed with "!" it will add the file pattern in case
+ the layer is absent: ::
+
+ BBFILES_DYNAMIC += "\
+ !clang-layer:${LAYERDIR}/backfill/meta-clang/*/*/*.bb \
+ "
+
+ This next example shows an error message that occurs because invalid
+ entries are found, which cause parsing to abort: ::
+
+ ERROR: BBFILES_DYNAMIC entries must be of the form {!}<collection name>:<filename pattern>, not:
+ /work/my-layer/bbappends/meta-security-isafw/*/*/*.bbappend
+ /work/my-layer/bbappends/openembedded-core/meta/*/*/*.bbappend
+
:term:`BBINCLUDED`
Contains a space-separated list of all of all files that BitBake's
parser included during parsing of the current file.
@@ -1303,6 +1316,8 @@
- ``svn://`` : Fetches files from a Subversion (``svn``) revision
control repository.
+ - ``az://`` : Fetches files from an Azure Storage account using HTTPS.
+
Here are some additional options worth mentioning:
- ``unpack`` : Controls whether or not to unpack the file if it is
diff --git a/poky/bitbake/lib/bb/__init__.py b/poky/bitbake/lib/bb/__init__.py
index 84a9051..afce5cc 100644
--- a/poky/bitbake/lib/bb/__init__.py
+++ b/poky/bitbake/lib/bb/__init__.py
@@ -58,7 +58,7 @@
if not bb.event.worker_pid:
if self.name in bb.msg.loggerDefaultDomains and loglevel > (bb.msg.loggerDefaultDomains[self.name]):
return
- if loglevel > bb.msg.loggerDefaultLogLevel:
+ if loglevel < bb.msg.loggerDefaultLogLevel:
return
return self.log(loglevel, msg, *args, **kwargs)
diff --git a/poky/bitbake/lib/bb/build.py b/poky/bitbake/lib/bb/build.py
index f4f897e..b2715fc 100644
--- a/poky/bitbake/lib/bb/build.py
+++ b/poky/bitbake/lib/bb/build.py
@@ -854,6 +854,23 @@
file_name = d.getVar('BB_FILENAME')
bb.parse.siggen.dump_sigtask(file_name, task, stampbase, True)
+def find_stale_stamps(task, d, file_name=None):
+ current = stamp_internal(task, d, file_name)
+ current2 = stamp_internal(task + "_setscene", d, file_name)
+ cleanmask = stamp_cleanmask_internal(task, d, file_name)
+ found = []
+ for mask in cleanmask:
+ for name in glob.glob(mask):
+ if "sigdata" in name or "sigbasedata" in name:
+ continue
+ if name.endswith('.taint'):
+ continue
+ if name == current or name == current2:
+ continue
+ logger.debug2("Stampfile %s does not match %s or %s" % (name, current, current2))
+ found.append(name)
+ return found
+
def del_stamp(task, d, file_name = None):
"""
Removes a stamp for a given task
diff --git a/poky/bitbake/lib/bb/cache.py b/poky/bitbake/lib/bb/cache.py
index aea2b8b..27eb271 100644
--- a/poky/bitbake/lib/bb/cache.py
+++ b/poky/bitbake/lib/bb/cache.py
@@ -126,6 +126,7 @@
self.inherits = self.getvar('__inherit_cache', metadata, expand=False)
self.fakerootenv = self.getvar('FAKEROOTENV', metadata)
self.fakerootdirs = self.getvar('FAKEROOTDIRS', metadata)
+ self.fakerootlogs = self.getvar('FAKEROOTLOGS', metadata)
self.fakerootnoenv = self.getvar('FAKEROOTNOENV', metadata)
self.extradepsfunc = self.getvar('calculate_extra_depends', metadata)
@@ -163,6 +164,7 @@
cachedata.fakerootenv = {}
cachedata.fakerootnoenv = {}
cachedata.fakerootdirs = {}
+ cachedata.fakerootlogs = {}
cachedata.extradepsfunc = {}
def add_cacheData(self, cachedata, fn):
@@ -231,6 +233,7 @@
cachedata.fakerootenv[fn] = self.fakerootenv
cachedata.fakerootnoenv[fn] = self.fakerootnoenv
cachedata.fakerootdirs[fn] = self.fakerootdirs
+ cachedata.fakerootlogs[fn] = self.fakerootlogs
cachedata.extradepsfunc[fn] = self.extradepsfunc
def virtualfn2realfn(virtualfn):
diff --git a/poky/bitbake/lib/bb/codeparser.py b/poky/bitbake/lib/bb/codeparser.py
index 25a7ac6..0cec452 100644
--- a/poky/bitbake/lib/bb/codeparser.py
+++ b/poky/bitbake/lib/bb/codeparser.py
@@ -212,9 +212,9 @@
funcstr = codegen.to_source(func)
argstr = codegen.to_source(arg)
except TypeError:
- self.log.debug(2, 'Failed to convert function and argument to source form')
+ self.log.debug2('Failed to convert function and argument to source form')
else:
- self.log.debug(1, self.unhandled_message % (funcstr, argstr))
+ self.log.debug(self.unhandled_message % (funcstr, argstr))
def visit_Call(self, node):
name = self.called_node_name(node.func)
@@ -450,7 +450,7 @@
cmd = word[1]
if cmd.startswith("$"):
- self.log.debug(1, self.unhandled_template % cmd)
+ self.log.debug(self.unhandled_template % cmd)
elif cmd == "eval":
command = " ".join(word for _, word in words[1:])
self._parse_shell(command)
diff --git a/poky/bitbake/lib/bb/cooker.py b/poky/bitbake/lib/bb/cooker.py
index f4ab797..39e10e6 100644
--- a/poky/bitbake/lib/bb/cooker.py
+++ b/poky/bitbake/lib/bb/cooker.py
@@ -502,22 +502,30 @@
def showVersions(self):
- (latest_versions, preferred_versions) = self.findProviders()
+ (latest_versions, preferred_versions, required) = self.findProviders()
- logger.plain("%-35s %25s %25s", "Recipe Name", "Latest Version", "Preferred Version")
- logger.plain("%-35s %25s %25s\n", "===========", "==============", "=================")
+ logger.plain("%-35s %25s %25s %25s", "Recipe Name", "Latest Version", "Preferred Version", "Required Version")
+ logger.plain("%-35s %25s %25s %25s\n", "===========", "==============", "=================", "================")
for p in sorted(self.recipecaches[''].pkg_pn):
- pref = preferred_versions[p]
+ preferred = preferred_versions[p]
latest = latest_versions[p]
+ requiredstr = ""
+ preferredstr = ""
+ if required[p]:
+ if preferred[0] is not None:
+ requiredstr = preferred[0][0] + ":" + preferred[0][1] + '-' + preferred[0][2]
+ else:
+ bb.fatal("REQUIRED_VERSION of package %s not available" % p)
+ else:
+ preferredstr = preferred[0][0] + ":" + preferred[0][1] + '-' + preferred[0][2]
- prefstr = pref[0][0] + ":" + pref[0][1] + '-' + pref[0][2]
lateststr = latest[0][0] + ":" + latest[0][1] + "-" + latest[0][2]
- if pref == latest:
- prefstr = ""
+ if preferred == latest:
+ preferredstr = ""
- logger.plain("%-35s %25s %25s", p, lateststr, prefstr)
+ logger.plain("%-35s %25s %25s %25s", p, lateststr, preferredstr, requiredstr)
def showEnvironment(self, buildfile=None, pkgs_to_build=None):
"""
@@ -1063,10 +1071,16 @@
if pn in self.recipecaches[mc].providers:
filenames = self.recipecaches[mc].providers[pn]
eligible, foundUnique = bb.providers.filterProviders(filenames, pn, self.databuilder.mcdata[mc], self.recipecaches[mc])
- filename = eligible[0]
+ if eligible is not None:
+ filename = eligible[0]
+ else:
+ filename = None
return None, None, None, filename
elif pn in self.recipecaches[mc].pkg_pn:
- return bb.providers.findBestProvider(pn, self.databuilder.mcdata[mc], self.recipecaches[mc], self.recipecaches[mc].pkg_pn)
+ (latest, latest_f, preferred_ver, preferred_file, required) = bb.providers.findBestProvider(pn, self.databuilder.mcdata[mc], self.recipecaches[mc], self.recipecaches[mc].pkg_pn)
+ if required and preferred_file is None:
+ return None, None, None, None
+ return (latest, latest_f, preferred_ver, preferred_file)
else:
return None, None, None, None
diff --git a/poky/bitbake/lib/bb/event.py b/poky/bitbake/lib/bb/event.py
index 23e1f31..0454c75 100644
--- a/poky/bitbake/lib/bb/event.py
+++ b/poky/bitbake/lib/bb/event.py
@@ -118,7 +118,7 @@
if _eventfilter:
if not _eventfilter(name, handler, event, d):
continue
- if d and not name in (d.getVar("__BBHANDLERS_MC") or []):
+ if d is not None and not name in (d.getVar("__BBHANDLERS_MC") or set()):
continue
execute_handler(name, handler, event, d)
@@ -232,12 +232,16 @@
def register(name, handler, mask=None, filename=None, lineno=None, data=None):
"""Register an Event handler"""
- if data and data.getVar("BB_CURRENT_MC"):
+ if data is not None and data.getVar("BB_CURRENT_MC"):
mc = data.getVar("BB_CURRENT_MC")
name = '%s%s' % (mc.replace('-', '_'), name)
# already registered
if name in _handlers:
+ if data is not None:
+ bbhands_mc = (data.getVar("__BBHANDLERS_MC") or set())
+ bbhands_mc.add(name)
+ data.setVar("__BBHANDLERS_MC", bbhands_mc)
return AlreadyRegistered
if handler is not None:
@@ -274,16 +278,16 @@
_event_handler_map[m] = {}
_event_handler_map[m][name] = True
- if data:
- bbhands_mc = (data.getVar("__BBHANDLERS_MC") or [])
- bbhands_mc.append(name)
+ if data is not None:
+ bbhands_mc = (data.getVar("__BBHANDLERS_MC") or set())
+ bbhands_mc.add(name)
data.setVar("__BBHANDLERS_MC", bbhands_mc)
return Registered
def remove(name, handler, data=None):
"""Remove an Event handler"""
- if data:
+ if data is not None:
if data.getVar("BB_CURRENT_MC"):
mc = data.getVar("BB_CURRENT_MC")
name = '%s%s' % (mc.replace('-', '_'), name)
@@ -295,8 +299,8 @@
if name in _event_handler_map[event]:
_event_handler_map[event].pop(name)
- if data:
- bbhands_mc = (data.getVar("__BBHANDLERS_MC") or [])
+ if data is not None:
+ bbhands_mc = (data.getVar("__BBHANDLERS_MC") or set())
if name in bbhands_mc:
bbhands_mc.remove(name)
data.setVar("__BBHANDLERS_MC", bbhands_mc)
@@ -666,6 +670,17 @@
Event.__init__(self)
self.stamps = stamps
+class StaleSetSceneTasks(Event):
+ """
+ An event listing setscene tasks which are 'stale' and will
+ be rerun. The metadata may use to clean up stale data.
+ tasks is a mapping of tasks and matching stale stamps.
+ """
+
+ def __init__(self, tasks):
+ Event.__init__(self)
+ self.tasks = tasks
+
class FilesMatchingFound(Event):
"""
Event when a list of files matching the supplied pattern has
diff --git a/poky/bitbake/lib/bb/fetch2/__init__.py b/poky/bitbake/lib/bb/fetch2/__init__.py
index 19169d7..cf0201c 100644
--- a/poky/bitbake/lib/bb/fetch2/__init__.py
+++ b/poky/bitbake/lib/bb/fetch2/__init__.py
@@ -1243,7 +1243,7 @@
if checksum_name in self.parm:
checksum_expected = self.parm[checksum_name]
- elif self.type not in ["http", "https", "ftp", "ftps", "sftp", "s3"]:
+ elif self.type not in ["http", "https", "ftp", "ftps", "sftp", "s3", "az"]:
checksum_expected = None
else:
checksum_expected = d.getVarFlag("SRC_URI", checksum_name)
@@ -1908,6 +1908,7 @@
from . import clearcase
from . import npm
from . import npmsw
+from . import az
methods.append(local.Local())
methods.append(wget.Wget())
@@ -1927,3 +1928,4 @@
methods.append(clearcase.ClearCase())
methods.append(npm.Npm())
methods.append(npmsw.NpmShrinkWrap())
+methods.append(az.Az())
diff --git a/poky/bitbake/lib/bb/fetch2/az.py b/poky/bitbake/lib/bb/fetch2/az.py
new file mode 100644
index 0000000..3ccc594
--- /dev/null
+++ b/poky/bitbake/lib/bb/fetch2/az.py
@@ -0,0 +1,93 @@
+"""
+BitBake 'Fetch' Azure Storage implementation
+
+"""
+
+# Copyright (C) 2021 Alejandro Hernandez Samaniego
+#
+# Based on bb.fetch2.wget:
+# Copyright (C) 2003, 2004 Chris Larson
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Based on functions from the base bb module, Copyright 2003 Holger Schurig
+
+import shlex
+import os
+import bb
+from bb.fetch2 import FetchError
+from bb.fetch2 import logger
+from bb.fetch2.wget import Wget
+
+
+class Az(Wget):
+
+ def supports(self, ud, d):
+ """
+ Check to see if a given url can be fetched from Azure Storage
+ """
+ return ud.type in ['az']
+
+
+ def checkstatus(self, fetch, ud, d, try_again=True):
+
+ # checkstatus discards parameters either way, we need to do this before adding the SAS
+ ud.url = ud.url.replace('az://','https://').split(';')[0]
+
+ az_sas = d.getVar('AZ_SAS')
+ if az_sas and az_sas not in ud.url:
+ ud.url += az_sas
+
+ return Wget.checkstatus(self, fetch, ud, d, try_again)
+
+ # Override download method, include retries
+ def download(self, ud, d, retries=3):
+ """Fetch urls"""
+
+ # If were reaching the account transaction limit we might be refused a connection,
+ # retrying allows us to avoid false negatives since the limit changes over time
+ fetchcmd = self.basecmd + ' --retry-connrefused --waitretry=5'
+
+ # We need to provide a localpath to avoid wget using the SAS
+ # ud.localfile either has the downloadfilename or ud.path
+ localpath = os.path.join(d.getVar("DL_DIR"), ud.localfile)
+ bb.utils.mkdirhier(os.path.dirname(localpath))
+ fetchcmd += " -O %s" % shlex.quote(localpath)
+
+
+ if ud.user and ud.pswd:
+ fetchcmd += " --user=%s --password=%s --auth-no-challenge" % (ud.user, ud.pswd)
+
+ # Check if a Shared Access Signature was given and use it
+ az_sas = d.getVar('AZ_SAS')
+
+ if az_sas:
+ azuri = '%s%s%s%s' % ('https://', ud.host, ud.path, az_sas)
+ else:
+ azuri = '%s%s%s' % ('https://', ud.host, ud.path)
+
+ if os.path.exists(ud.localpath):
+ # file exists, but we didnt complete it.. trying again.
+ fetchcmd += d.expand(" -c -P ${DL_DIR} '%s'" % azuri)
+ else:
+ fetchcmd += d.expand(" -P ${DL_DIR} '%s'" % azuri)
+
+ try:
+ self._runwget(ud, d, fetchcmd, False)
+ except FetchError as e:
+ # Azure fails on handshake sometimes when using wget after some stress, producing a
+ # FetchError from the fetcher, if the artifact exists retyring should succeed
+ if 'Unable to establish SSL connection' in str(e):
+ logger.debug2('Unable to establish SSL connection: Retries remaining: %s, Retrying...' % retries)
+ self.download(ud, d, retries -1)
+
+ # Sanity check since wget can pretend it succeed when it didn't
+ # Also, this used to happen if sourceforge sent us to the mirror page
+ if not os.path.exists(ud.localpath):
+ raise FetchError("The fetch command returned success for url %s but %s doesn't exist?!" % (azuri, ud.localpath), azuri)
+
+ if os.path.getsize(ud.localpath) == 0:
+ os.remove(ud.localpath)
+ raise FetchError("The fetch of %s resulted in a zero size file?! Deleting and failing since this isn't right." % (azuri), azuri)
+
+ return True
diff --git a/poky/bitbake/lib/bb/providers.py b/poky/bitbake/lib/bb/providers.py
index b5a6cd0..3ec11a4 100644
--- a/poky/bitbake/lib/bb/providers.py
+++ b/poky/bitbake/lib/bb/providers.py
@@ -38,16 +38,17 @@
localdata = data.createCopy(cfgData)
bb.data.expandKeys(localdata)
+ required = {}
preferred_versions = {}
latest_versions = {}
for pn in pkg_pn:
- (last_ver, last_file, pref_ver, pref_file) = findBestProvider(pn, localdata, dataCache, pkg_pn)
+ (last_ver, last_file, pref_ver, pref_file, req) = findBestProvider(pn, localdata, dataCache, pkg_pn)
preferred_versions[pn] = (pref_ver, pref_file)
latest_versions[pn] = (last_ver, last_file)
+ required[pn] = req
- return (latest_versions, preferred_versions)
-
+ return (latest_versions, preferred_versions, required)
def allProviders(dataCache):
"""
@@ -59,7 +60,6 @@
all_providers[pn].append((ver, fn))
return all_providers
-
def sortPriorities(pn, dataCache, pkg_pn = None):
"""
Reorder pkg_pn by file priority and default preference
@@ -87,6 +87,21 @@
return tmp_pn
+def versionVariableMatch(cfgData, keyword, pn):
+ """
+ Return the value of the <keyword>_VERSION variable if set.
+ """
+
+ # pn can contain '_', e.g. gcc-cross-x86_64 and an override cannot
+ # hence we do this manually rather than use OVERRIDES
+ ver = cfgData.getVar("%s_VERSION_pn-%s" % (keyword, pn))
+ if not ver:
+ ver = cfgData.getVar("%s_VERSION_%s" % (keyword, pn))
+ if not ver:
+ ver = cfgData.getVar("%s_VERSION" % keyword)
+
+ return ver
+
def preferredVersionMatch(pe, pv, pr, preferred_e, preferred_v, preferred_r):
"""
Check if the version pe,pv,pr is the preferred one.
@@ -102,19 +117,28 @@
def findPreferredProvider(pn, cfgData, dataCache, pkg_pn = None, item = None):
"""
- Find the first provider in pkg_pn with a PREFERRED_VERSION set.
+ Find the first provider in pkg_pn with REQUIRED_VERSION or PREFERRED_VERSION set.
"""
preferred_file = None
preferred_ver = None
+ required = False
- # pn can contain '_', e.g. gcc-cross-x86_64 and an override cannot
- # hence we do this manually rather than use OVERRIDES
- preferred_v = cfgData.getVar("PREFERRED_VERSION_pn-%s" % pn)
- if not preferred_v:
- preferred_v = cfgData.getVar("PREFERRED_VERSION_%s" % pn)
- if not preferred_v:
- preferred_v = cfgData.getVar("PREFERRED_VERSION")
+ required_v = versionVariableMatch(cfgData, "REQUIRED", pn)
+ preferred_v = versionVariableMatch(cfgData, "PREFERRED", pn)
+
+ itemstr = ""
+ if item:
+ itemstr = " (for item %s)" % item
+
+ if required_v is not None:
+ if preferred_v is not None:
+ logger.warn("REQUIRED_VERSION and PREFERRED_VERSION for package %s%s are both set using REQUIRED_VERSION %s", pn, itemstr, required_v)
+ else:
+ logger.debug("REQUIRED_VERSION is set for package %s%s", pn, itemstr)
+ # REQUIRED_VERSION always takes precedence over PREFERRED_VERSION
+ preferred_v = required_v
+ required = True
if preferred_v:
m = re.match(r'(\d+:)*(.*)(_.*)*', preferred_v)
@@ -147,11 +171,9 @@
pv_str = preferred_v
if not (preferred_e is None):
pv_str = '%s:%s' % (preferred_e, pv_str)
- itemstr = ""
- if item:
- itemstr = " (for item %s)" % item
if preferred_file is None:
- logger.warn("preferred version %s of %s not available%s", pv_str, pn, itemstr)
+ if not required:
+ logger.warn("preferred version %s of %s not available%s", pv_str, pn, itemstr)
available_vers = []
for file_set in pkg_pn:
for f in file_set:
@@ -164,11 +186,15 @@
if available_vers:
available_vers.sort()
logger.warn("versions of %s available: %s", pn, ' '.join(available_vers))
+ if required:
+ logger.error("required version %s of %s not available%s", pv_str, pn, itemstr)
else:
- logger.debug("selecting %s as PREFERRED_VERSION %s of package %s%s", preferred_file, pv_str, pn, itemstr)
+ if required:
+ logger.debug("selecting %s as REQUIRED_VERSION %s of package %s%s", preferred_file, pv_str, pn, itemstr)
+ else:
+ logger.debug("selecting %s as PREFERRED_VERSION %s of package %s%s", preferred_file, pv_str, pn, itemstr)
- return (preferred_ver, preferred_file)
-
+ return (preferred_ver, preferred_file, required)
def findLatestProvider(pn, cfgData, dataCache, file_set):
"""
@@ -189,7 +215,6 @@
return (latest, latest_f)
-
def findBestProvider(pn, cfgData, dataCache, pkg_pn = None, item = None):
"""
If there is a PREFERRED_VERSION, find the highest-priority bbfile
@@ -198,17 +223,16 @@
"""
sortpkg_pn = sortPriorities(pn, dataCache, pkg_pn)
- # Find the highest priority provider with a PREFERRED_VERSION set
- (preferred_ver, preferred_file) = findPreferredProvider(pn, cfgData, dataCache, sortpkg_pn, item)
+ # Find the highest priority provider with a REQUIRED_VERSION or PREFERRED_VERSION set
+ (preferred_ver, preferred_file, required) = findPreferredProvider(pn, cfgData, dataCache, sortpkg_pn, item)
# Find the latest version of the highest priority provider
(latest, latest_f) = findLatestProvider(pn, cfgData, dataCache, sortpkg_pn[0])
- if preferred_file is None:
+ if not required and preferred_file is None:
preferred_file = latest_f
preferred_ver = latest
- return (latest, latest_f, preferred_ver, preferred_file)
-
+ return (latest, latest_f, preferred_ver, preferred_file, required)
def _filterProviders(providers, item, cfgData, dataCache):
"""
@@ -234,10 +258,13 @@
logger.debug("providers for %s are: %s", item, list(sorted(pkg_pn.keys())))
- # First add PREFERRED_VERSIONS
+ # First add REQUIRED_VERSIONS or PREFERRED_VERSIONS
for pn in sorted(pkg_pn):
sortpkg_pn[pn] = sortPriorities(pn, dataCache, pkg_pn)
- preferred_versions[pn] = findPreferredProvider(pn, cfgData, dataCache, sortpkg_pn[pn], item)
+ preferred_ver, preferred_file, required = findPreferredProvider(pn, cfgData, dataCache, sortpkg_pn[pn], item)
+ if required and preferred_file is None:
+ return eligible
+ preferred_versions[pn] = (preferred_ver, preferred_file)
if preferred_versions[pn][1]:
eligible.append(preferred_versions[pn][1])
@@ -249,7 +276,6 @@
eligible.append(preferred_versions[pn][1])
if not eligible:
- logger.error("no eligible providers for %s", item)
return eligible
# If pn == item, give it a slight default preference
@@ -266,7 +292,6 @@
return eligible
-
def filterProviders(providers, item, cfgData, dataCache):
"""
Take a list of providers and filter/reorder according to the
@@ -388,7 +413,6 @@
return rproviders
-
def buildWorldTargetList(dataCache, task=None):
"""
Build package list for "bitbake world"
diff --git a/poky/bitbake/lib/bb/runqueue.py b/poky/bitbake/lib/bb/runqueue.py
index 54ef245..80d7f6c 100644
--- a/poky/bitbake/lib/bb/runqueue.py
+++ b/poky/bitbake/lib/bb/runqueue.py
@@ -1242,6 +1242,7 @@
magic = "decafbad"
if self.cooker.configuration.profile:
magic = "decafbadbad"
+ fakerootlogs = None
if fakeroot:
magic = magic + "beef"
mcdata = self.cooker.databuilder.mcdata[mc]
@@ -1251,10 +1252,11 @@
for key, value in (var.split('=') for var in fakerootenv):
env[key] = value
worker = subprocess.Popen(fakerootcmd + ["bitbake-worker", magic], stdout=subprocess.PIPE, stdin=subprocess.PIPE, env=env)
+ fakerootlogs = self.rqdata.dataCaches[mc].fakerootlogs
else:
worker = subprocess.Popen(["bitbake-worker", magic], stdout=subprocess.PIPE, stdin=subprocess.PIPE)
bb.utils.nonblockingfd(worker.stdout)
- workerpipe = runQueuePipe(worker.stdout, None, self.cfgData, self, rqexec)
+ workerpipe = runQueuePipe(worker.stdout, None, self.cfgData, self, rqexec, fakerootlogs=fakerootlogs)
workerdata = {
"taskdeps" : self.rqdata.dataCaches[mc].task_deps,
@@ -1772,7 +1774,7 @@
self.sqdata = SQData()
build_scenequeue_data(self.sqdata, self.rqdata, self.rq, self.cooker, self.stampcache, self)
- def runqueue_process_waitpid(self, task, status):
+ def runqueue_process_waitpid(self, task, status, fakerootlog=None):
# self.build_stamps[pid] may not exist when use shared work directory.
if task in self.build_stamps:
@@ -1787,7 +1789,7 @@
self.sq_live.remove(task)
else:
if status != 0:
- self.task_fail(task, status)
+ self.task_fail(task, status, fakerootlog=fakerootlog)
else:
self.task_complete(task)
return True
@@ -1908,14 +1910,31 @@
self.task_completeoutright(task)
self.runq_tasksrun.add(task)
- def task_fail(self, task, exitcode):
+ def task_fail(self, task, exitcode, fakerootlog=None):
"""
Called when a task has failed
Updates the state engine with the failure
"""
self.stats.taskFailed()
self.failed_tids.append(task)
- bb.event.fire(runQueueTaskFailed(task, self.stats, exitcode, self.rq), self.cfgData)
+
+ fakeroot_log = ""
+ if fakerootlog and os.path.exists(fakerootlog):
+ with open(fakerootlog) as fakeroot_log_file:
+ fakeroot_failed = False
+ for line in reversed(fakeroot_log_file.readlines()):
+ for fakeroot_error in ['mismatch', 'error', 'fatal']:
+ if fakeroot_error in line.lower():
+ fakeroot_failed = True
+ if 'doing new pid setup and server start' in line:
+ break
+ fakeroot_log = line + fakeroot_log
+
+ if not fakeroot_failed:
+ fakeroot_log = None
+
+ bb.event.fire(runQueueTaskFailed(task, self.stats, exitcode, self.rq, fakeroot_log=fakeroot_log), self.cfgData)
+
if self.rqdata.taskData[''].abort:
self.rq.state = runQueueCleanUp
@@ -1943,6 +1962,10 @@
logger.error("Scenequeue had holdoff tasks: %s" % pprint.pformat(self.holdoff_tasks))
err = True
+ for tid in self.scenequeue_covered.intersection(self.scenequeue_notcovered):
+ # No task should end up in both covered and uncovered, that is a bug.
+ logger.error("Setscene task %s in both covered and notcovered." % tid)
+
for tid in self.rqdata.runq_setscene_tids:
if tid not in self.scenequeue_covered and tid not in self.scenequeue_notcovered:
err = True
@@ -2431,6 +2454,9 @@
for dep in sorted(self.sqdata.sq_deps[task]):
if fail and task in self.sqdata.sq_harddeps and dep in self.sqdata.sq_harddeps[task]:
+ if dep in self.scenequeue_covered or dep in self.scenequeue_notcovered:
+ # dependency could be already processed, e.g. noexec setscene task
+ continue
logger.debug2("%s was unavailable and is a hard dependency of %s so skipping" % (task, dep))
self.sq_task_failoutright(dep)
continue
@@ -2755,6 +2781,20 @@
update_scenequeue_data(sqdata.sq_revdeps, sqdata, rqdata, rq, cooker, stampcache, sqrq, summary=True)
+ # Compute a list of 'stale' sstate tasks where the current hash does not match the one
+ # in any stamp files. Pass the list out to metadata as an event.
+ found = {}
+ for tid in rqdata.runq_setscene_tids:
+ (mc, fn, taskname, taskfn) = split_tid_mcfn(tid)
+ stamps = bb.build.find_stale_stamps(taskname, rqdata.dataCaches[mc], taskfn)
+ if stamps:
+ if mc not in found:
+ found[mc] = {}
+ found[mc][tid] = stamps
+ for mc in found:
+ event = bb.event.StaleSetSceneTasks(found[mc])
+ bb.event.fire(event, cooker.databuilder.mcdata[mc])
+
def update_scenequeue_data(tids, sqdata, rqdata, rq, cooker, stampcache, sqrq, summary=True):
tocheck = set()
@@ -2876,12 +2916,16 @@
"""
Event notifying a task failed
"""
- def __init__(self, task, stats, exitcode, rq):
+ def __init__(self, task, stats, exitcode, rq, fakeroot_log=None):
runQueueEvent.__init__(self, task, stats, rq)
self.exitcode = exitcode
+ self.fakeroot_log = fakeroot_log
def __str__(self):
- return "Task (%s) failed with exit code '%s'" % (self.taskstring, self.exitcode)
+ if self.fakeroot_log:
+ return "Task (%s) failed with exit code '%s' \nPseudo log:\n%s" % (self.taskstring, self.exitcode, self.fakeroot_log)
+ else:
+ return "Task (%s) failed with exit code '%s'" % (self.taskstring, self.exitcode)
class sceneQueueTaskFailed(sceneQueueEvent):
"""
@@ -2933,7 +2977,7 @@
"""
Abstraction for a pipe between a worker thread and the server
"""
- def __init__(self, pipein, pipeout, d, rq, rqexec):
+ def __init__(self, pipein, pipeout, d, rq, rqexec, fakerootlogs=None):
self.input = pipein
if pipeout:
pipeout.close()
@@ -2942,6 +2986,7 @@
self.d = d
self.rq = rq
self.rqexec = rqexec
+ self.fakerootlogs = fakerootlogs
def setrunqueueexec(self, rqexec):
self.rqexec = rqexec
@@ -2987,7 +3032,11 @@
task, status = pickle.loads(self.queue[10:index])
except (ValueError, pickle.UnpicklingError, AttributeError, IndexError) as e:
bb.msg.fatal("RunQueue", "failed load pickle '%s': '%s'" % (e, self.queue[10:index]))
- self.rqexec.runqueue_process_waitpid(task, status)
+ (_, _, _, taskfn) = split_tid_mcfn(task)
+ fakerootlog = None
+ if self.fakerootlogs and taskfn and taskfn in self.fakerootlogs:
+ fakerootlog = self.fakerootlogs[taskfn]
+ self.rqexec.runqueue_process_waitpid(task, status, fakerootlog=fakerootlog)
found = True
self.queue = self.queue[index+11:]
index = self.queue.find(b"</exitcode>")
diff --git a/poky/bitbake/lib/bb/tests/color.py b/poky/bitbake/lib/bb/tests/color.py
index bf03750..88dd278 100644
--- a/poky/bitbake/lib/bb/tests/color.py
+++ b/poky/bitbake/lib/bb/tests/color.py
@@ -31,7 +31,7 @@
def setUp(self):
self.d = bb.data.init()
self._progress_watcher = ProgressWatcher()
- bb.event.register("bb.build.TaskProgress", self._progress_watcher.handle_event)
+ bb.event.register("bb.build.TaskProgress", self._progress_watcher.handle_event, data=self.d)
def tearDown(self):
bb.event.remove("bb.build.TaskProgress", None)
diff --git a/poky/bitbake/lib/bb/tests/fetch.py b/poky/bitbake/lib/bb/tests/fetch.py
index 7b2dac7..ddf6e97 100644
--- a/poky/bitbake/lib/bb/tests/fetch.py
+++ b/poky/bitbake/lib/bb/tests/fetch.py
@@ -1345,7 +1345,7 @@
"http://downloads.yoctoproject.org/releases/sato/sato-engine-0.2.tar.gz",
"http://downloads.yoctoproject.org/releases/sato/sato-engine-0.3.tar.gz",
"https://yoctoproject.org/",
- "https://yoctoproject.org/documentation",
+ "https://docs.yoctoproject.org",
"http://downloads.yoctoproject.org/releases/opkg/opkg-0.1.7.tar.gz",
"http://downloads.yoctoproject.org/releases/opkg/opkg-0.3.0.tar.gz",
"ftp://sourceware.org/pub/libffi/libffi-1.20.tar.gz",
diff --git a/poky/bitbake/lib/bb/tinfoil.py b/poky/bitbake/lib/bb/tinfoil.py
index 763c329..796a98f 100644
--- a/poky/bitbake/lib/bb/tinfoil.py
+++ b/poky/bitbake/lib/bb/tinfoil.py
@@ -440,7 +440,7 @@
to initialise Tinfoil and use it with config_only=True first and
then conditionally call this function to parse recipes later.
"""
- config_params = TinfoilConfigParameters(config_only=False)
+ config_params = TinfoilConfigParameters(config_only=False, quiet=self.quiet)
self.run_actions(config_params)
self.recipes_parsed = True
diff --git a/poky/bitbake/lib/bblayers/query.py b/poky/bitbake/lib/bblayers/query.py
index f5e3c84..947422a 100644
--- a/poky/bitbake/lib/bblayers/query.py
+++ b/poky/bitbake/lib/bblayers/query.py
@@ -128,7 +128,7 @@
sys.exit(1)
pkg_pn = self.tinfoil.cooker.recipecaches[mc].pkg_pn
- (latest_versions, preferred_versions) = self.tinfoil.find_providers(mc)
+ (latest_versions, preferred_versions, required_versions) = self.tinfoil.find_providers(mc)
allproviders = self.tinfoil.get_all_providers(mc)
# Ensure we list skipped recipes