subtree updates
meta-security: de6712a806..a85fbe980e:
Anton Antonov (1):
Upgrade parsec-service 0.8.1 and parsec-tool 0.4.0
Armin Kuster (1):
chkrootkit: update to 0.55
Bhupesh Sharma (1):
recipes-security/fscrypt: Add fscrypt .bb file
Christer Fletcher (1):
dmverity: Make use of DATA_BLOCK_SIZE variable in initrdscript.
Kristian Klausen (1):
libtpm: update to 0.8.7
Zoltán Böszörményi (1):
clamav: Set clamav:clamav ownership on /var/lib/clamav in do_install
poky: 06dcace68b..80f2b56ad8:
Anibal Limon (1):
recipes-support/ptest-runner: Bump to v2.4.2
Bruce Ashfield (5):
linux-yocto-dev: update to v5.15-rcX
lttng-modules/dev-upstream: update to 2.13-latest
lttng-modules: fix build against 5.15+
linux-yocto/5.13: drop recipes
yocto-bsp/5.13: drop recipes
Chandana kalluri (1):
scriptutils.py: Add check before deleting path
Daniel Wagenknecht (2):
common-tasks: add note about license implications of bundled initramfs
ref-manual: add note about license implications of bundled initramfs
Joshua Watt (2):
lib/oe/spdx.py: Add comments
python3: Fix sysroot reproducibility
Kenfe-Mickael Laventure (1):
package_ipk: Use localdata store when signing packages
Kiran Surendran (1):
ffmpeg: fix CVE-2021-38171
Kristian Klausen (2):
ovmf: add TPM PACKAGECONFIG and enable if tpm is in MACHINE_FEATURES
wic/bootimg-efi: Add Unified Kernel Image option
Markus Volk (1):
wic:direct.py: ignore invalid mountpoints during fstab update
Matt Madison (1):
autotools.bbclass: use ordinary append for file-checksums update
Michael Halstead (1):
releases: update to include 3.1.11
Minjae Kim (1):
vim: fix CVE-2021-3778
Quentin Schulz (1):
ref-manual: fix missed override syntax change
Rasmus Villemoes (1):
kernel.bbclass: remove unnecessary dead code
Richard Purdie (29):
oeqa/qemurunner: Use oe._exit(), not sys.exit()
pseudo: Add in ability to flush database with shutdown request
packagegroup-core-tools-profile: Exclude systemtap from riscv32 as well
bitbake: bitbake-worker: Allow shutdown/database flush of pseudo server at task exit
bitbake: siggen: Fix sorting in diff output
bitbake: cooker/command: Add a dummy event for tinfoil testing
oeqa/selftest/gotoolchain: Fix temp file cleanup
oeqa/buildproject: Ensure temp directories are cleaned up
libc_package/buildstats: Fix python regex quoting warnings
oeqa/selftest/tinfoil: Update to use test command
glew: Stop polluting /tmp during builds
rpm: Ensure compression parallelism isn't coded into rpms
package: Ensure pclist files are deterministic and don't use full paths
gnupg: Be deterministic about sendmail
mesa: Ensure megadrivers runtime mappings are deterministic
util-linux: Fix reproducibility
libtool: Allow libtool-cross to reproduce
gobject-introspection: Don't write $HOME into scripts
oeqa/selftest/bbtests: Add uuid to force build test
image: Exclude IMAGE_VERSION_SUFFIX from expansion in image tasks
sstatesig: Revert "Test cross/native hashserv method extension"
bitbake: data: Ensure functions are defined in a deterministic order
bitbake.conf: Set vardepvalue for PARALLEL_MAKEINST
externalsrc: Fix a source date epoch race in reproducible builds
sstatesig: Add processing for full build paths in sysroot files
python3: Drop broken pyc files
image-artifact-names: Use SOURCE_DATE_EPOCH when making reproducible builds for deploy
abi_version/sstate: Bump HASH_VERSION and SSTATE_VERSION
reproducible_build: Work around caching issues
Robert P. J. Day (3):
ref-manual: extend explanation of PACKAGE_DEBUG_SPLIT_STYLE
ref-manual: mention INHIBIT_PACKAGE_DEBUG_SPLIT variable
overview-manual: delete bad backslashes in SSTATE_MIRRORS example
Saul Wold (3):
spdx-licenses.json: Use 3.14 tagged version
spdx.py: Add SPDXAnnotation Object
create-spdx: Use SPDXAnnotation to track native recipes
Thomas Perrot (2):
libevent: mark util/monotonic_prc_fallback as retriable
ruby: fix the reproducibility issue
Tom Pollard (2):
bzip2: Update soname for libbz2 1.0.8
libsamplerate0: Set correct soname for 0.1.9
Trevor Woerner (1):
hello-mod/hello.c: convert printk to pr_xxx
William A. Kennington III (1):
rm_work.bbclass: Fix for files starting with -
Yi Zhao (1):
inetutils: fix CVE-2021-40491
wangmy (1):
strace: upgrade 5.13 -> 5.14
meta-openembedded: cff8331f96..23dc4f060f:
Armin Kuster (1):
README: update to main repo
Chandana kalluri (1):
python3-humanfriendly: Add nativesdk to BBCLASSEXTEND
Changqing Li (1):
layer.conf: add openembedded-layer as LAYERDEPENDS
Khem Raj (3):
smcroute: Add missing pkgconfig inherit
packagegroup-meta-oe: Add new packages smarty and libjs-jquery-icheck
gattlib: Upgrade to latest
LiweiSong (1):
chipsec: platform security assessment framework
Martin Jansa (5):
opencv: fix build with protobuf-3.18 when dnn PACKAGECONFIG is enabled
libeigen: backport fix for -Werror=class-memaccess issues when NEON is enabled
README: mention linux-libc-dev:i386 for luajit on ubuntu-21.10
gpsd: inherit pkgconfig
pahole: use MACHINE_ARCH
Matteo Croce (1):
pahole: don't download vendored libbpf
Mingli Yu (1):
libqb: Upgrade to 2.0.3
Nandor Han (1):
libiio: depend on avahi only when network backed is used
Peter Kjellerstedt (1):
netdata: Move the version to the file name and correct the SRC_URI
Richard Purdie (1):
gattlib: Place pkgconfig file in correct package
Yi Zhao (1):
phpmyadmin: upgrade 5.1.0 -> 5.1.1
wangmy (7):
unionfs-fuse: upgrade 2.1 -> 2.2
smcroute: upgrade 2.4.4 -> 2.5.3
snort: upgrade 2.9.18 -> 2.9.18.1
libsass: upgrade 3.6.4 -> 3.6.5
sanlock: upgrade 3.8.3 -> 3.8.4
sassc: upgrade 3.6.1 -> 3.6.2
valijson: upgrade 0.5 -> 0.6
zangrc (8):
python3-pychromecast: upgrade 9.2.0 -> 9.2.1
python3-pyro4: upgrade 4.80 -> 4.81
python3-pyzmq: upgrade 22.2.1 -> 22.3.0
python3-robotframework: upgrade 4.1 -> 4.1.1
python3-sqlparse: upgrade 0.4.1 -> 0.4.2
python3-tqdm: upgrade 4.62.2 -> 4.62.3
libjs-jquery-icheck: Add recipe
smarty: Add recipe
zhengruoqin (6):
python3-cmd2: upgrade 2.1.2 -> 2.2.0
python3-huey: upgrade 2.4.0 -> 2.4.1
python3-humanfriendly: upgrade 9.2 -> 10.0
cifs-utils: upgrade 6.13 -> 6.14
cmark: upgrade 0.30.1 -> 0.30.2
gpsd: upgrade 3.23 -> 3.23.1
Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Ie782ff5d7f3004fb1f1ac9a4c8644a178bae46ad
diff --git a/poky/meta/lib/buildstats.py b/poky/meta/lib/buildstats.py
index 8627ed3..c52b6c3 100644
--- a/poky/meta/lib/buildstats.py
+++ b/poky/meta/lib/buildstats.py
@@ -43,8 +43,8 @@
# depends on the heartbeat event, which fires less often.
self.min_seconds = 1
- self.meminfo_regex = re.compile(b'^(MemTotal|MemFree|Buffers|Cached|SwapTotal|SwapFree):\s*(\d+)')
- self.diskstats_regex = re.compile(b'^([hsv]d.|mtdblock\d|mmcblk\d|cciss/c\d+d\d+.*)$')
+ self.meminfo_regex = re.compile(rb'^(MemTotal|MemFree|Buffers|Cached|SwapTotal|SwapFree):\s*(\d+)')
+ self.diskstats_regex = re.compile(rb'^([hsv]d.|mtdblock\d|mmcblk\d|cciss/c\d+d\d+.*)$')
self.diskstats_ltime = None
self.diskstats_data = None
self.stat_ltimes = None
diff --git a/poky/meta/lib/oe/spdx.py b/poky/meta/lib/oe/spdx.py
index 9814fbf..4416194 100644
--- a/poky/meta/lib/oe/spdx.py
+++ b/poky/meta/lib/oe/spdx.py
@@ -2,6 +2,18 @@
# SPDX-License-Identifier: GPL-2.0-only
#
+#
+# This library is intended to capture the JSON SPDX specification in a type
+# safe manner. It is not intended to encode any particular OE specific
+# behaviors, see the sbom.py for that.
+#
+# The documented SPDX spec document doesn't cover the JSON syntax for
+# particular configuration, which can make it hard to determine what the JSON
+# syntax should be. I've found it is actually much simpler to read the official
+# SPDX JSON schema which can be found here: https://github.com/spdx/spdx-spec
+# in schemas/spdx-schema.json
+#
+
import hashlib
import itertools
import json
@@ -9,7 +21,16 @@
SPDX_VERSION = "2.2"
+#
+# The following are the support classes that are used to implement SPDX object
+#
+
class _Property(object):
+ """
+ A generic SPDX object property. The different types will derive from this
+ class
+ """
+
def __init__(self, *, default=None):
self.default = default
@@ -19,6 +40,10 @@
class _String(_Property):
+ """
+ A scalar string property for an SPDX object
+ """
+
def __init__(self, **kwargs):
super().__init__(**kwargs)
@@ -39,6 +64,10 @@
class _Object(_Property):
+ """
+ A scalar SPDX object property of a SPDX object
+ """
+
def __init__(self, cls, **kwargs):
super().__init__(**kwargs)
self.cls = cls
@@ -62,6 +91,10 @@
class _ListProperty(_Property):
+ """
+ A list of SPDX properties
+ """
+
def __init__(self, prop, **kwargs):
super().__init__(**kwargs)
self.prop = prop
@@ -82,16 +115,28 @@
class _StringList(_ListProperty):
+ """
+ A list of strings as a property for an SPDX object
+ """
+
def __init__(self, **kwargs):
super().__init__(_String(), **kwargs)
class _ObjectList(_ListProperty):
+ """
+ A list of SPDX objects as a property for an SPDX object
+ """
+
def __init__(self, cls, **kwargs):
super().__init__(_Object(cls), **kwargs)
class MetaSPDXObject(type):
+ """
+ A metaclass that allows properties (anything derived from a _Property
+ class) to be defined for a SPDX object
+ """
def __new__(mcls, name, bases, attrs):
attrs["_properties"] = {}
@@ -105,6 +150,9 @@
class SPDXObject(metaclass=MetaSPDXObject):
+ """
+ The base SPDX object; all SPDX spec classes must derive from this class
+ """
def __init__(self, **d):
self._spdx = {}
@@ -122,6 +170,21 @@
return
raise KeyError("%r is not a valid SPDX property" % name)
+#
+# These are the SPDX objects implemented from the spec. The *only* properties
+# that can be added to these objects are ones directly specified in the SPDX
+# spec, however you may add helper functions to make operations easier.
+#
+# Defaults should *only* be specified if the SPDX spec says there is a certain
+# required value for a field (e.g. dataLicense), or if the field is mandatory
+# and has some sane "this field is unknown" (e.g. "NOASSERTION")
+#
+
+class SPDXAnnotation(SPDXObject):
+ annotationDate = _String()
+ annotationType = _String()
+ annotator = _String()
+ comment = _String()
class SPDXChecksum(SPDXObject):
algorithm = _String()
@@ -164,6 +227,7 @@
packageVerificationCode = _Object(SPDXPackageVerificationCode)
hasFiles = _StringList()
packageFileName = _String()
+ annotations = _ObjectList(SPDXAnnotation)
class SPDXFile(SPDXObject):
diff --git a/poky/meta/lib/oe/sstatesig.py b/poky/meta/lib/oe/sstatesig.py
index dd6b9de..0c3b458 100644
--- a/poky/meta/lib/oe/sstatesig.py
+++ b/poky/meta/lib/oe/sstatesig.py
@@ -108,7 +108,6 @@
self.unlockedrecipes = (data.getVar("SIGGEN_UNLOCKED_RECIPES") or
"").split()
self.unlockedrecipes = { k: "" for k in self.unlockedrecipes }
- self.buildarch = data.getVar('BUILD_ARCH')
self._internal = False
pass
@@ -147,13 +146,6 @@
self.dump_lockedsigs(sigfile)
return super(bb.siggen.SignatureGeneratorBasicHash, self).dump_sigs(dataCache, options)
- def prep_taskhash(self, tid, deps, dataCaches):
- super().prep_taskhash(tid, deps, dataCaches)
- if hasattr(self, "extramethod"):
- (mc, _, _, fn) = bb.runqueue.split_tid_mcfn(tid)
- inherits = " ".join(dataCaches[mc].inherits[fn])
- if inherits.find("/native.bbclass") != -1 or inherits.find("/cross.bbclass") != -1:
- self.extramethod[tid] = ":" + self.buildarch
def get_taskhash(self, tid, deps, dataCaches):
if tid in self.lockedhashes:
@@ -478,6 +470,8 @@
import stat
import pwd
import grp
+ import re
+ import fnmatch
def update_hash(s):
s = s.encode('utf-8')
@@ -487,6 +481,8 @@
h = hashlib.sha256()
prev_dir = os.getcwd()
+ corebase = d.getVar("COREBASE")
+ tmpdir = d.getVar("TMPDIR")
include_owners = os.environ.get('PSEUDO_DISABLED') == '0'
if "package_write_" in task or task == "package_qa":
include_owners = False
@@ -497,8 +493,17 @@
include_root = False
extra_content = d.getVar('HASHEQUIV_HASH_VERSION')
+ filemaps = {}
+ for m in (d.getVar('SSTATE_HASHEQUIV_FILEMAP') or '').split():
+ entry = m.split(":")
+ if len(entry) != 3 or entry[0] != task:
+ continue
+ filemaps.setdefault(entry[1], [])
+ filemaps[entry[1]].append(entry[2])
+
try:
os.chdir(path)
+ basepath = os.path.normpath(path)
update_hash("OEOuthashBasic\n")
if extra_content:
@@ -580,8 +585,13 @@
else:
update_hash(" " * 9)
+ filterfile = False
+ for entry in filemaps:
+ if fnmatch.fnmatch(path, entry):
+ filterfile = True
+
update_hash(" ")
- if stat.S_ISREG(s.st_mode):
+ if stat.S_ISREG(s.st_mode) and not filterfile:
update_hash("%10d" % s.st_size)
else:
update_hash(" " * 10)
@@ -590,9 +600,24 @@
fh = hashlib.sha256()
if stat.S_ISREG(s.st_mode):
# Hash file contents
- with open(path, 'rb') as d:
- for chunk in iter(lambda: d.read(4096), b""):
+ if filterfile:
+ # Need to ignore paths in crossscripts and postinst-useradd files.
+ with open(path, 'rb') as d:
+ chunk = d.read()
+ chunk = chunk.replace(bytes(basepath, encoding='utf8'), b'')
+ for entry in filemaps:
+ if not fnmatch.fnmatch(path, entry):
+ continue
+ for r in filemaps[entry]:
+ if r.startswith("regex-"):
+ chunk = re.sub(bytes(r[6:], encoding='utf8'), b'', chunk)
+ else:
+ chunk = chunk.replace(bytes(r, encoding='utf8'), b'')
fh.update(chunk)
+ else:
+ with open(path, 'rb') as d:
+ for chunk in iter(lambda: d.read(4096), b""):
+ fh.update(chunk)
update_hash(fh.hexdigest())
else:
update_hash(" " * len(fh.hexdigest()))
diff --git a/poky/meta/lib/oeqa/selftest/cases/bbtests.py b/poky/meta/lib/oeqa/selftest/cases/bbtests.py
index 8831de6..6562364 100644
--- a/poky/meta/lib/oeqa/selftest/cases/bbtests.py
+++ b/poky/meta/lib/oeqa/selftest/cases/bbtests.py
@@ -83,8 +83,10 @@
def test_force_task_1(self):
# test 1 from bug 5875
+ import uuid
test_recipe = 'zlib'
- test_data = "Microsoft Made No Profit From Anyone's Zunes Yo"
+ # Need to use uuid otherwise hash equivlance would change the workflow
+ test_data = "Microsoft Made No Profit From Anyone's Zunes Yo %s" % uuid.uuid1()
bb_vars = get_bb_vars(['D', 'PKGDEST', 'mandir'], test_recipe)
image_dir = bb_vars['D']
pkgsplit_dir = bb_vars['PKGDEST']
diff --git a/poky/meta/lib/oeqa/selftest/cases/gotoolchain.py b/poky/meta/lib/oeqa/selftest/cases/gotoolchain.py
index 4fc3605..c809d7c 100644
--- a/poky/meta/lib/oeqa/selftest/cases/gotoolchain.py
+++ b/poky/meta/lib/oeqa/selftest/cases/gotoolchain.py
@@ -43,6 +43,12 @@
@classmethod
def tearDownClass(cls):
+ # Go creates file which are readonly
+ for dirpath, dirnames, filenames in os.walk(cls.tmpdir_SDKQA):
+ for filename in filenames + dirnames:
+ f = os.path.join(dirpath, filename)
+ if not os.path.islink(f):
+ os.chmod(f, 0o775)
shutil.rmtree(cls.tmpdir_SDKQA, ignore_errors=True)
super(oeGoToolchainSelfTest, cls).tearDownClass()
diff --git a/poky/meta/lib/oeqa/selftest/cases/tinfoil.py b/poky/meta/lib/oeqa/selftest/cases/tinfoil.py
index 5109280..8fd48bb 100644
--- a/poky/meta/lib/oeqa/selftest/cases/tinfoil.py
+++ b/poky/meta/lib/oeqa/selftest/cases/tinfoil.py
@@ -94,14 +94,13 @@
pass
pattern = 'conf'
- res = tinfoil.run_command('findFilesMatchingInDir', pattern, 'conf/machine')
+ res = tinfoil.run_command('testCookerCommandEvent', pattern)
self.assertTrue(res)
eventreceived = False
commandcomplete = False
start = time.time()
# Wait for maximum 60s in total so we'd detect spurious heartbeat events for example
- # The test is IO load sensitive too
while (not (eventreceived == True and commandcomplete == True)
and (time.time() - start < 60)):
# if we received both events (on let's say a good day), we are done
@@ -111,7 +110,8 @@
commandcomplete = True
elif isinstance(event, bb.event.FilesMatchingFound):
self.assertEqual(pattern, event._pattern)
- self.assertIn('qemuarm.conf', event._matches)
+ self.assertIn('A', event._matches)
+ self.assertIn('B', event._matches)
eventreceived = True
elif isinstance(event, logging.LogRecord):
continue
diff --git a/poky/meta/lib/oeqa/selftest/cases/wic.py b/poky/meta/lib/oeqa/selftest/cases/wic.py
index dc7b9e6..5fc8e65 100644
--- a/poky/meta/lib/oeqa/selftest/cases/wic.py
+++ b/poky/meta/lib/oeqa/selftest/cases/wic.py
@@ -1158,6 +1158,35 @@
out = glob(self.resultdir + "%s-*.direct" % wksname)
self.assertEqual(1, len(out))
+ @only_for_arch(['i586', 'i686', 'x86_64'])
+ def test_efi_plugin_unified_kernel_image_qemu(self):
+ """Test efi plugin's Unified Kernel Image feature in qemu"""
+ config = 'IMAGE_FSTYPES = "wic"\n'\
+ 'INITRAMFS_IMAGE = "core-image-minimal-initramfs"\n'\
+ 'WKS_FILE = "test_efi_plugin.wks"\n'\
+ 'MACHINE_FEATURES:append = " efi"\n'
+ self.append_config(config)
+ self.assertEqual(0, bitbake('core-image-minimal core-image-minimal-initramfs ovmf').status)
+ self.remove_config(config)
+
+ with runqemu('core-image-minimal', ssh=False,
+ runqemuparams='ovmf', image_fstype='wic') as qemu:
+ # Check that /boot has EFI bootx64.efi (required for EFI)
+ cmd = "ls /boot/EFI/BOOT/bootx64.efi | wc -l"
+ status, output = qemu.run_serial(cmd)
+ self.assertEqual(1, status, 'Failed to run command "%s": %s' % (cmd, output))
+ self.assertEqual(output, '1')
+ # Check that /boot has EFI/Linux/linux.efi (required for Unified Kernel Images auto detection)
+ cmd = "ls /boot/EFI/Linux/linux.efi | wc -l"
+ status, output = qemu.run_serial(cmd)
+ self.assertEqual(1, status, 'Failed to run command "%s": %s' % (cmd, output))
+ self.assertEqual(output, '1')
+ # Check that /boot doesn't have loader/entries/boot.conf (Unified Kernel Images are auto detected by the bootloader)
+ cmd = "ls /boot/loader/entries/boot.conf 2&>/dev/null | wc -l"
+ status, output = qemu.run_serial(cmd)
+ self.assertEqual(1, status, 'Failed to run command "%s": %s' % (cmd, output))
+ self.assertEqual(output, '0')
+
def test_fs_types(self):
"""Test filesystem types for empty and not empty partitions"""
img = 'core-image-minimal'
diff --git a/poky/meta/lib/oeqa/utils/buildproject.py b/poky/meta/lib/oeqa/utils/buildproject.py
index e6d80cc..dfb9661 100644
--- a/poky/meta/lib/oeqa/utils/buildproject.py
+++ b/poky/meta/lib/oeqa/utils/buildproject.py
@@ -18,6 +18,7 @@
def __init__(self, uri, foldername=None, tmpdir=None, dl_dir=None):
self.uri = uri
self.archive = os.path.basename(uri)
+ self.tempdirobj = None
if not tmpdir:
self.tempdirobj = tempfile.TemporaryDirectory(prefix='buildproject-')
tmpdir = self.tempdirobj.name
@@ -57,6 +58,8 @@
return self._run('cd %s; make install %s' % (self.targetdir, install_args))
def clean(self):
+ if self.tempdirobj:
+ self.tempdirobj.cleanup()
if not self.needclean:
return
self._run('rm -rf %s' % self.targetdir)
diff --git a/poky/meta/lib/oeqa/utils/qemurunner.py b/poky/meta/lib/oeqa/utils/qemurunner.py
index d55248c..d961a9a 100644
--- a/poky/meta/lib/oeqa/utils/qemurunner.py
+++ b/poky/meta/lib/oeqa/utils/qemurunner.py
@@ -265,7 +265,7 @@
r = os.fdopen(r)
x = r.read()
os.killpg(os.getpgid(self.runqemu.pid), signal.SIGTERM)
- sys.exit(0)
+ os._exit(0)
self.logger.debug("runqemu started, pid is %s" % self.runqemu.pid)
self.logger.debug("waiting at most %s seconds for qemu pid (%s)" %
diff --git a/poky/meta/lib/oeqa/utils/targetbuild.py b/poky/meta/lib/oeqa/utils/targetbuild.py
index 1055810..09738ad 100644
--- a/poky/meta/lib/oeqa/utils/targetbuild.py
+++ b/poky/meta/lib/oeqa/utils/targetbuild.py
@@ -19,6 +19,7 @@
self.d = d
self.uri = uri
self.archive = os.path.basename(uri)
+ self.tempdirobj = None
if not tmpdir:
tmpdir = self.d.getVar('WORKDIR')
if not tmpdir:
@@ -71,9 +72,10 @@
return self._run('cd %s; make install %s' % (self.targetdir, install_args))
def clean(self):
+ if self.tempdirobj:
+ self.tempdirobj.cleanup()
self._run('rm -rf %s' % self.targetdir)
subprocess.check_call('rm -f %s' % self.localarchive, shell=True)
- pass
class TargetBuildProject(BuildProject):