subtree updates
poky: 492205ea83..94dfcaff64:
Alejandro Hernandez Samaniego (1):
baremetal-helloworld: Enable RISC-V 32 port
Alexandre Belloni (1):
oeqa/runtime/cases: make date.DateTest.test_date more reliable
Anton Blanchard (3):
libjpeg-turbo: Handle powerpc64le without Altivec
kmod: use nonarch_base_libdir for depmod.d and modprobe.d
pixman: Handle PowerPC without Altivec
Changqing Li (1):
libconvert-asn1-perl: 0.27 -> 0.31
Chen Qi (4):
convert-overrides.py: also convert comments without a leading whitespace
meta: use new override syntax in comments
multilib.bbclass: fix new override syntax for virtclass-multilib
util-linux: add back manpages related settings
Daniel Gomez (1):
docs: fix typo in releases
Dmitry Baryshkov (1):
linux-firmware: add more Qualcomm firmware packages
Dragos-Marian Panait (1):
util-linux: fix CVE-2021-37600
Joe Slater (1):
terminal.bbclass: force bash for devshell
Jon Mason (1):
tune-cortexm*: add support for all Arm Cortex-M processors
Jose Quaresma (1):
sstate.bbclass: fix error handling when sstate mirrors is ro
Joshua Watt (2):
classes/cve-check: Move get_patches_cves to library
lib/packagedata: Fix for new overrides
Khem Raj (4):
glibc: Upgrade to 2.34 release
glibc: Remove obsolete --enable-stackguard-randomization
glibc: Drop DUMMY_LOCALE_T define patch
glibc: Add missing symlinks for libpthread and librt dev files
Michael Halstead (1):
releases: update to include 3.1.10
Michael Opdenacker (12):
manuals: mention license information in footer
manuals: further documentation for cve-check
cve-check: remove deprecated CVE_CHECK_CVE_WHITELIST
bsp-guide: overrides syntax updates
dev-manual: overrides syntax updates
kernel-dev manual: overrides syntax updates
ref-manual: overrides syntax updates
sdk-manual: overrides syntax updates
test-manual: overrides syntax updates
sdk-manual: reference obsolete reference to ADT
Manuals: replace "file name" by "filename"
dev-manual: fix grammar in post-install script explanations
Nisha Parrakat (1):
dbus_%.bbappend: stop using selinux_set_mapping
Olaf Mandel (1):
kickstart: document which options accept units
Patrick Williams (3):
pixman: re-disable iwmmxt
systemd: add zstd PACKAGECONFIG
systemd: set zstd as default PACKAGECONFIG
Paul Barker (2):
u-boot: Package extlinux.conf separately
pypi: Allow override of PyPI archive name
Quentin Schulz (3):
insane.bbclass: fix new override syntax migration
docs: fix new override syntax migration
docs: overview-manual: concepts: remove long-gone BBHASHDEPS variable
Richard Purdie (6):
test-manual: Add extra detail to YP Compatible section
migration-3.4: Add extra notes to override syntax changes
ruby: Fix DEBUG_PREFIX_MAP in LDFLAGS issue
gettext: Fix reproducibility issue with LDFLAGS
curl: Fix reproducibility issue with LDFLAGS
libtool: Fix lto option passing for reproducible builds
Ross Burton (11):
e2fsprogs: ensure small images have 256-byte inodes
wic: don't forcibly pass -T default
parted: drop unneeded ld-is-gold patch
parted: update patch status
buildtools-tarball: add testsdk task
oeqa/sdk: add some buildtools tests
bitbake: utils: add environment updating context manager
bitbake: fetch2: expose environment variable names that need to be exported
bitbake: fetch2/wget: ensure all variables are set when calling urllib
bitbake: fetch2/wget: fetch securely by default
tar: ignore node-tar CVEs
Thomas Perrot (2):
kernel-fitimage: images should not be signed with the same keys as the configurations
oeqa/selftest/fitimage: update tests to use two keys
Tim Orling (3):
python3-scons{-native}: upgrade 4.1.0 -> 4.2.0
perl: do_create_rdepends_inc override syntax
package.bbclass: FILER* override syntax
Tom Rini (2):
common-tasks: Add a summary to the end of the bbappend example
manuals: Rename the "Using .bbappend Files in Your Layer" section
Tony Battersby (2):
bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS
ruby: Fix reproducibility issue with LDFLAGS
Tony Tascioglu (1):
valgrind: skip broken ptests for glibc 2.34
Vyacheslav Yurkov (7):
lib/oe: add generic functions for overlayfs
overlayfs.bbclass: generate overlayfs mount units
rootfs-postcommands: add QA check for overlayfs
systemd-machine-units: add bbappend for meta-selftest
overlayfs: meta-selftest recipe
oeqa/selftest: overlayfs unit tests
MAINTAINERS: add overlayfs maintainer
Yi Zhao (3):
dbus: add PACKAGECONFIG for audit and selinux
glib-2.0: add PACKAGECONFIG for selinux
shadow: add PACKAGECONFIG for audit and selinux
hongxu (1):
sdk: fix relocate symlink failed
wangmy (1):
ell: upgrade 0.41 -> 0.42
meta-raspberrypi: c7f4c739a3..32921fc9bd:
Omer Akram (1):
linux-firmware-rpidistro: fix wifi driver loading on cm4
Otavio Salvador (1):
rpi-config: Allow setting hdmi_cvt
meta-openembedded: 3cf2475ea0..a13db91f19:
Changqing Li (1):
ndpi: fix CVE-2021-36082
Chen Qi (1):
Convert to new override syntax using latest convert-overrides.py script
Dmitry Baryshkov (1):
image_types_sparse: fix sparse image generation
Geoff Parker (1):
cifs-utils: typo fix fakse --> false
Kai Kang (2):
libdbi-perl: fix CVE-2014-10402
python3-m2crypto: fix for new overrides syntax
Khem Raj (1):
packagegroup-meta-oe: Add ttf-ipa
Leon Anavi (15):
python3-astroid: Upgrade 2.6.5 -> 2.6.6
python3-gast: Upgrade 0.5.1 -> 0.5.2
python3-greenlet: Upgrade 1.1.0 -> 1.1.1
python3-bitarray: Upgrade 2.2.3 -> 2.2.5
python3-send2trash: Upgrade 1.7.1 -> 1.8.0
python3-zeroconf: Upgrade 0.33.2 -> 0.34.3
python3-aiohue: Upgrade 2.5.1 -> 2.6.1
python3-configargparse: Upgrade 1.5.1 -> 1.5.2
python3-pycurl: Upgrade 7.43.0.6 -> 7.44.0
python3-distro: Upgrade 1.5.0 -> 1.6.0
python3-google-api-core: Upgrade 1.30.0 -> 1.31.1
python3-google-auth: Upgrade 1.32.0 -> 1.34.0
python3-google-api-python-client: Upgrade 2.12.0 -> 2.15.0
python3-huey: Upgrade 2.3.2 -> 2.4.0
python3-apply-defaults: Upgrade 0.1.4 -> 0.1.6
Martin Jansa (1):
python3-grpcio: make sure that GRPC_CFLAGS is expanded to empty
Michael Opdenacker (3):
vorbis-tools: update to 1.4.2 (latest in 1.4.x series)
bigbuckbunny-1080p: fix sample video URL
opus-tools: update to 0.2, move to meta-multimedia and fix license
Mingli Yu (3):
jemalloc: fix the race during do_install
jemalloc: add ptest support
jemalloc: improve the ptest output
Naveen Saini (1):
python3-defusedxml: extend recipe to add native support
Philippe Coval (1):
mycroft: Install more tools needed by scripts
Tony Battersby (3):
curlpp: fix QA Issue after LDFLAGS change
ldns: fix QA Issue after LDFLAGS change
tcsh: fix compile error after LDFLAGS change
Yi Zhao (5):
audit: upgrade 3.0.3 -> 3.0.4
augeas: rename PACKAGECONFIG[libselinux] to PACKAGECONFIG[selinux]
network-manager-applet: add selinux to PACKAGECONFIG if enable selinux distro feature
networkmanager: add PACKAGECONFIG for audit and selinux
augeas: add selinux to PACKAGECONFIG if enable selinux distro feature
leimaohui (1):
ttf-ipa: Added a new font.
wangmy (1):
iwd: upgrade 1.15 -> 1.16
zangrc (1):
python3-humanize: upgrade 3.10.0 -> 3.11.0
zhengruoqin (3):
python3-engineio: upgrade 4.2.0 -> 4.2.1
python3-ipython: upgrade 7.25.0 -> 7.26.0
python3-isort: upgrade 5.9.2 -> 5.9.3
Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I7a8bd19709f465db51254ed3fcaf2486fe64dcaf
diff --git a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
index 593de61..40b245b 100644
--- a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
+++ b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
@@ -144,6 +144,10 @@
make any attempted network access a fatal error, which is useful for
checking that mirrors are complete as well as other things.
+If :term:`BB_CHECK_SSL_CERTS` is set to ``0`` then SSL certificate checking will
+be disabled. This variable defaults to ``1`` so SSL certificates are normally
+checked.
+
.. _bb-the-unpack:
The Unpack
diff --git a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst
index 6283c26..2392ec4 100644
--- a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst
+++ b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst
@@ -93,6 +93,10 @@
fetcher does not attempt to use the host listed in :term:`SRC_URI` after
a successful fetch from the :term:`PREMIRRORS` occurs.
+ :term:`BB_CHECK_SSL_CERTS`
+ Specifies if SSL certificates should be checked when fetching. The default
+ value is ``1`` and certificates are not checked if the value is set to ``0``.
+
:term:`BB_CONSOLELOG`
Specifies the path to a log file into which BitBake's user interface
writes output during the build.
diff --git a/poky/bitbake/lib/bb/fetch2/__init__.py b/poky/bitbake/lib/bb/fetch2/__init__.py
index ad89868..914fa5c 100644
--- a/poky/bitbake/lib/bb/fetch2/__init__.py
+++ b/poky/bitbake/lib/bb/fetch2/__init__.py
@@ -808,6 +808,29 @@
fetcher = bb.fetch2.Fetch([url], d)
return fetcher.localpath(url)
+# Need to export PATH as binary could be in metadata paths
+# rather than host provided
+# Also include some other variables.
+FETCH_EXPORT_VARS = ['HOME', 'PATH',
+ 'HTTP_PROXY', 'http_proxy',
+ 'HTTPS_PROXY', 'https_proxy',
+ 'FTP_PROXY', 'ftp_proxy',
+ 'FTPS_PROXY', 'ftps_proxy',
+ 'NO_PROXY', 'no_proxy',
+ 'ALL_PROXY', 'all_proxy',
+ 'GIT_PROXY_COMMAND',
+ 'GIT_SSH',
+ 'GIT_SSL_CAINFO',
+ 'GIT_SMART_HTTP',
+ 'SSH_AUTH_SOCK', 'SSH_AGENT_PID',
+ 'SOCKS5_USER', 'SOCKS5_PASSWD',
+ 'DBUS_SESSION_BUS_ADDRESS',
+ 'P4CONFIG',
+ 'SSL_CERT_FILE',
+ 'AWS_ACCESS_KEY_ID',
+ 'AWS_SECRET_ACCESS_KEY',
+ 'AWS_DEFAULT_REGION']
+
def runfetchcmd(cmd, d, quiet=False, cleanup=None, log=None, workdir=None):
"""
Run cmd returning the command output
@@ -816,28 +839,7 @@
Optionally remove the files/directories listed in cleanup upon failure
"""
- # Need to export PATH as binary could be in metadata paths
- # rather than host provided
- # Also include some other variables.
- # FIXME: Should really include all export varaiables?
- exportvars = ['HOME', 'PATH',
- 'HTTP_PROXY', 'http_proxy',
- 'HTTPS_PROXY', 'https_proxy',
- 'FTP_PROXY', 'ftp_proxy',
- 'FTPS_PROXY', 'ftps_proxy',
- 'NO_PROXY', 'no_proxy',
- 'ALL_PROXY', 'all_proxy',
- 'GIT_PROXY_COMMAND',
- 'GIT_SSH',
- 'GIT_SSL_CAINFO',
- 'GIT_SMART_HTTP',
- 'SSH_AUTH_SOCK', 'SSH_AGENT_PID',
- 'SOCKS5_USER', 'SOCKS5_PASSWD',
- 'DBUS_SESSION_BUS_ADDRESS',
- 'P4CONFIG',
- 'AWS_ACCESS_KEY_ID',
- 'AWS_SECRET_ACCESS_KEY',
- 'AWS_DEFAULT_REGION']
+ exportvars = FETCH_EXPORT_VARS
if not cleanup:
cleanup = []
diff --git a/poky/bitbake/lib/bb/fetch2/wget.py b/poky/bitbake/lib/bb/fetch2/wget.py
index 784df70..29fcfbb 100644
--- a/poky/bitbake/lib/bb/fetch2/wget.py
+++ b/poky/bitbake/lib/bb/fetch2/wget.py
@@ -52,13 +52,19 @@
class Wget(FetchMethod):
+ """Class to fetch urls via 'wget'"""
# CDNs like CloudFlare may do a 'browser integrity test' which can fail
# with the standard wget/urllib User-Agent, so pretend to be a modern
# browser.
user_agent = "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:84.0) Gecko/20100101 Firefox/84.0"
- """Class to fetch urls via 'wget'"""
+ def check_certs(self, d):
+ """
+ Should certificates be checked?
+ """
+ return (d.getVar("BB_CHECK_SSL_CERTS") or "1") != "0"
+
def supports(self, ud, d):
"""
Check to see if a given url can be fetched with wget.
@@ -82,7 +88,10 @@
if not ud.localfile:
ud.localfile = d.expand(urllib.parse.unquote(ud.host + ud.path).replace("/", "."))
- self.basecmd = d.getVar("FETCHCMD_wget") or "/usr/bin/env wget -t 2 -T 30 --passive-ftp --no-check-certificate"
+ self.basecmd = d.getVar("FETCHCMD_wget") or "/usr/bin/env wget -t 2 -T 30 --passive-ftp"
+
+ if not self.check_certs(d):
+ self.basecmd += " --no-check-certificate"
def _runwget(self, ud, d, command, quiet, workdir=None):
@@ -282,19 +291,44 @@
newreq = urllib.request.HTTPRedirectHandler.redirect_request(self, req, fp, code, msg, headers, newurl)
newreq.get_method = req.get_method
return newreq
- exported_proxies = export_proxies(d)
- handlers = [FixedHTTPRedirectHandler, HTTPMethodFallback]
- if exported_proxies:
- handlers.append(urllib.request.ProxyHandler())
- handlers.append(CacheHTTPHandler())
- # Since Python 2.7.9 ssl cert validation is enabled by default
- # see PEP-0476, this causes verification errors on some https servers
- # so disable by default.
- import ssl
- if hasattr(ssl, '_create_unverified_context'):
- handlers.append(urllib.request.HTTPSHandler(context=ssl._create_unverified_context()))
- opener = urllib.request.build_opener(*handlers)
+ # We need to update the environment here as both the proxy and HTTPS
+ # handlers need variables set. The proxy needs http_proxy and friends to
+ # be set, and HTTPSHandler ends up calling into openssl to load the
+ # certificates. In buildtools configurations this will be looking at the
+ # wrong place for certificates by default: we set SSL_CERT_FILE to the
+ # right location in the buildtools environment script but as BitBake
+ # prunes prunes the environment this is lost. When binaries are executed
+ # runfetchcmd ensures these values are in the environment, but this is
+ # pure Python so we need to update the environment.
+ #
+ # Avoid tramping the environment too much by using bb.utils.environment
+ # to scope the changes to the build_opener request, which is when the
+ # environment lookups happen.
+ newenv = {}
+ for name in bb.fetch2.FETCH_EXPORT_VARS:
+ value = d.getVar(name)
+ if not value:
+ origenv = d.getVar("BB_ORIGENV")
+ if origenv:
+ value = origenv.getVar(name)
+ if value:
+ newenv[name] = value
+
+ with bb.utils.environment(**newenv):
+ import ssl
+
+ if self.check_certs(d):
+ context = ssl.create_default_context()
+ else:
+ context = ssl._create_unverified_context()
+
+ handlers = [FixedHTTPRedirectHandler,
+ HTTPMethodFallback,
+ urllib.request.ProxyHandler(),
+ CacheHTTPHandler(),
+ urllib.request.HTTPSHandler(context=context)]
+ opener = urllib.request.build_opener(*handlers)
try:
uri = ud.url.split(";")[0]
diff --git a/poky/bitbake/lib/bb/tests/utils.py b/poky/bitbake/lib/bb/tests/utils.py
index a7ff33d..4d5e21b 100644
--- a/poky/bitbake/lib/bb/tests/utils.py
+++ b/poky/bitbake/lib/bb/tests/utils.py
@@ -666,3 +666,21 @@
layers = [{"SRC_URI"}, {"QT_GIT", "QT_MODULE", "QT_MODULE_BRANCH_PARAM", "QT_GIT_PROTOCOL"}, {"QT_GIT_PROJECT", "QT_MODULE_BRANCH", "BPN"}, {"PN", "SPECIAL_PKGSUFFIX"}]
self.check_referenced("${SRC_URI}", layers)
+
+
+class EnvironmentTests(unittest.TestCase):
+ def test_environment(self):
+ os.environ["A"] = "this is A"
+ self.assertIn("A", os.environ)
+ self.assertEqual(os.environ["A"], "this is A")
+ self.assertNotIn("B", os.environ)
+
+ with bb.utils.environment(B="this is B"):
+ self.assertIn("A", os.environ)
+ self.assertEqual(os.environ["A"], "this is A")
+ self.assertIn("B", os.environ)
+ self.assertEqual(os.environ["B"], "this is B")
+
+ self.assertIn("A", os.environ)
+ self.assertEqual(os.environ["A"], "this is A")
+ self.assertNotIn("B", os.environ)
diff --git a/poky/bitbake/lib/bb/utils.py b/poky/bitbake/lib/bb/utils.py
index e6e82d1..7063491 100644
--- a/poky/bitbake/lib/bb/utils.py
+++ b/poky/bitbake/lib/bb/utils.py
@@ -1681,3 +1681,19 @@
shutil.move(src, dst)
else:
raise err
+
+@contextmanager
+def environment(**envvars):
+ """
+ Context manager to selectively update the environment with the specified mapping.
+ """
+ backup = dict(os.environ)
+ try:
+ os.environ.update(envvars)
+ yield
+ finally:
+ for var in envvars:
+ if var in backup:
+ os.environ[var] = backup[var]
+ else:
+ del os.environ[var]