subtree updates
meta-openembedded: a9e6d16e66..11df15765c:
Adrian Fiergolski (1):
python3-matplotlib: add missing dependency
Adrian Freihofer (6):
conntrack-tools: fix postinst script
networkmanager: improve dependency handling
networkmanager: simplify selective installation
networkmanager: use nftables by default
networkmanager: udpate to 1.38.0
modemmanager: update to 1.18.8
Armin Kuster (2):
mariadb: update to 10.7.4
mariadb: Fix i386 Clang builds
Bartosz Golaszewski (2):
python3-uinput: new package
python3-speedtest-cli: fix RDEPENDS
Changqing Li (1):
redis: upgrade 7.0-rc3 -> 7.0.0
Denys Dmytriyenko (1):
devmem2: the source and patches moved to github repo
Enrico Scholz (1):
nodejs-oe-cache-native: initial checkin
Jiaqing Zhao (1):
openldap: Remove unnecessary use-urandom.patch
Kai Kang (2):
libportal: add distro features check
graphviz: rrecommends on liberation-fonts
Khem Raj (5):
ubi-utils-klibc: Disable lzo compression by default
unattended-upgrades: Disable auto-detecting modules
sdbus-c++: Link with libatomic for rv32
sdbus-c++-libsystemd: Fix patch fuzz
python3-uinput: Fix build on 32bit arches using 64bit times_t
Luca Boccassi (1):
dbus-broker: update 29 -> 31
Marcel Ziswiler (1):
libavtp: add recipe for audio video transport protocol (avtp)
Markus Volk (6):
jack: allow to build native/nativesdk
pipewire: reduce native/nativesdk dependencies; add backport patch
p8platform: unbreak do_populate_sdk
pavucontrol: update; fix build for wayland only
gnome-disk-utility: fix build for wayland only
unblock some recipes for wayland
Martin Jansa (1):
mm-common: package the files from ${PN} in ${PN}-dev and use allarch
Ming Liu (1):
bluealsa: uprev to 4.0.0
Nikhil R (1):
duktape: Add ptest
Peter Marko (1):
libgpiod: move test dependencies to ptest package
Ross Burton (8):
python3-cppy: fix inherits and DEPENDS
python3-setuptools-scm-git-archive: add new recipe
python3-traitlets: upgrade to 5.2.1
python3-pathspec: add new recipe
python3-hatchling: add new recipe and build class
python3-editables: add new recipe
python3-setuptools-declarative-requirements: add new recipe
lzop: add (from oe-core)
Samuli Piippo (2):
flite: add recipe
libtomcrypt: add recipe
Thomas Perrot (1):
nbd: update 3.20 -> 3.24
Vyacheslav Yurkov (4):
packagegroup-meta-filesystems: fix build issue
overlayfs-progs: add new recipe
overlayfs-tools: add new recipe
xfstests: add new recipe
Wang Mingyu (38):
babeld: upgrade 1.12 -> 1.12.1
ctags: upgrade 5.9.20220508.0 -> 5.9.20220515.0
libbpf: upgrade 0.7.0 -> 0.8.0
evtest: upgrade 1.34 -> 1.35
nbdkit: upgrade 1.31.5 -> 1.31.7
smarty: upgrade 4.1.0 -> 4.1.1
thingsboard-gateway: upgrade 2.9 -> 3.1
opencl-headers: upgrade 2022.01.04 -> 2022.05.18
python3-robotframework: upgrade 5.0 -> 5.0.1
python3-watchdog: upgrade 2.1.7 -> 2.1.8
python3-web3: upgrade 5.29.0 -> 5.29.1
python3-xmlschema: upgrade 1.10.0 -> 1.11.0
python3-sqlalchemy: upgrade 1.4.35 -> 1.4.36
python3-yappi: upgrade 1.3.3 -> 1.3.5
apitrace: upgrade 11.0 -> 11.1
ctags: upgrade 5.9.20220515.0 -> 5.9.20220529.0
gedit: upgrade 42.0 -> 42.1
hidapi: upgrade 0.11.2 -> 0.12.0
libbytesize: upgrade 2.6 -> 2.7
libdvdread: upgrade 6.1.2 -> 6.1.3
links: upgrade 2.26 -> 2.27
libxmlb: upgrade 0.3.8 -> 0.3.9
ser2net: upgrade 4.3.5 -> 4.3.6
python3-awesomeversion: upgrade 22.5.1 -> 22.5.2
htop: upgrade 3.2.0 -> 3.2.1
hwdata: upgrade 0.359 -> 0.360
libnet-dns-perl: upgrade 1.33 -> 1.34
tinyproxy: upgrade 1.11.0 -> 1.11.1
function2: upgrade 4.2.0 -> 4.2.1
openvpn: upgrade 2.5.6 -> 2.5.7
poppler: upgrade 22.05.0 -> 22.06.0
sshfs-fuse: upgrade 3.7.2 -> 3.7.3
tgt: upgrade 1.0.82 -> 1.0.83
tracker: upgrade 3.3.0 -> 3.3.1
unbound: upgrade 1.15.0 -> 1.16.0
zabbix: upgrade 6.0.4 -> 6.0.5
botan: upgrade 2.19.1 -> 2.19.2
evolution-data-server: upgrade 3.44.1 -> 3.44.2
Wolfgang Meyer (1):
fbida: remove bash from RDEPENDS
Xu Huan (17):
python3-pint: upgrade 0.19.1 -> 0.19.2
python3-pylint: upgrade 2.13.7 -> 2.13.9
python3-redis: upgrade 4.2.2 -> 4.3.1
python3-werkzeug: upgrade 2.1.1 -> 2.1.2
python3-zeroconf: upgrade 0.38.4 -> 0.38.6
python3-sentry-sdk: upgrade 1.5.10 -> 1.5.12
python3-astroid: upgrade 2.11.3 -> 2.11.5
python3-cachetools: upgrade 5.0.0 -> 5.1.0
python3-imageio: upgrade 2.19.1 -> 2.19.2
python3-asyncinotify: upgrade 2.0.2 -> 2.0.3
python3-croniter: upgrade 1.3.4 -> 1.3.5
python3-google-api-core: upgrade 2.7.3 -> 2.8.0
python3-flask-socketio: upgrade 5.1.2 -> 5.2.0
python3-h5py: upgrade 3.6.0 -> 3.7.0
python3-lz4: upgrade 4.0.0 -> 4.0.1
python3-mypy: upgrade 0.950 -> 0.960
python3-pyscaffold: upgrade 4.2.1 -> 4.2.2
zhengrq.fnst (10):
python3-google-api-python-client: upgrade 2.45.0 -> 2.48.0
python3-grpcio-tools: upgrade 1.46.0 -> 1.46.3
python3-openpyxl: upgrade 3.0.9 -> 3.0.10
python3-paramiko: upgrade 2.10.4 -> 2.11.0
python3-humanize: upgrade 4.0.0 -> 4.1.0
python3-pychromecast: upgrade 12.1.1 -> 12.1.2
python3-cachetools: upgrade 5.1.0 -> 5.2.0
python3-google-api-python-client: upgrade 2.48.0 -> 2.49.0
python3-googleapis-common-protos: upgrade 1.56.1 -> 1.56.2
python3-imageio: upgrade 2.19.2 -> 2.19.3
zhengruoqin (6):
python3-bitarray: upgrade 2.5.0 -> 2.5.1
python3-eventlet: upgrade 0.33.0 -> 0.33.1
python3-googleapis-common-protos: upgrade 1.56.0 -> 1.56.1
python3-imageio: upgrade 2.18.0 -> 2.19.1
python3-pyjwt: upgrade 2.3.0 -> 2.4.0
python3-wrapt: upgrade 1.14.0 -> 1.14.1
poky: 13d70e57f8..ee0d001b81:
Alex Stewart (1):
opkg: upgrade to version 0.6.0
Alexander Kanavin (23):
bash: submit patch upstream
valgrind: submit arm patches upstream
apt: fix upstream version check
zip/unzip: mark all submittable patches as Inactive-Upstream
less: mark upstream version as unknown
wayland: exclude pre-releases from version check
mesa-demos: update 8.4.0 -> 8.5.0
seatd: update 0.6.4 -> 0.7.0
systemd: update 250.5 -> 251.2
btrfs-tools: update 5.16.2 -> 5.18
llvm: update 14.0.3 -> 14.0.4
python3-psutil: update 5.9.0 -> 5.9.1
tiff: update 4.3.0 -> 4.4.0
pulseaudio: update 15.0 -> 16.0
alsa-utils-scripts: merge into alsa-utils
alsa-utils: update 1.2.6 -> 1.2.7
ovmf: update 202202 -> 202205
cmake: update 3.23.1 -> 3.23.2
ltp: upgrade 20220121 -> 20220527
perl: update 5.34.1 -> 5.36.0
perl: drop perltoc regeneration
perl: clean prior to build
perl: enable _GNU_SOURCE define via d_gnulibc
Bruce Ashfield (7):
linux-yocto/5.15: bpf: explicitly disable unpriv eBPF by default
linux-yocto/5.15: update to v5.15.43
linux-yocto/5.10: update to v5.10.118
linux-yocto/5.15: Enable MDIO bus config
linux-yocto/5.15: cfg/xen: Move x86 configs to separate file
linux-yocto/5.15: update to v5.15.44
linux-yocto/5.10: update to v5.10.119
Chen Qi (1):
libsdl2: add back xvm and xinerama options
Daiane Angolini (1):
python3-pip: Fix RDEPENDS after the update
Davide Gardenal (2):
efivar: add musl libc compatibility
baremetal-image: fix broken symlink in do_rootfs
Dmitry Baryshkov (2):
go.bbclass: fix path to linker in native Go builds
linux-firmware: add support for building snapshots
Ernst Sjöstrand (2):
cve-check: Add helper for symlink handling
cve-check: Only include installed packages for rootfs manifest
He Zhe (1):
lttng-modules: Fix build failure for 5.10.119+ and 5.15.44+ kernel
Jack Mitchell (1):
meson.bbclass: add cython binary to cross/native toolchain config
Jeremy Puhlman (1):
gcc: depend on zstd-native
Jiaqing Zhao (1):
systemd: Correct 0001-pass-correct-parameters-to-getdents64.patch
Joerg Vehlow (1):
libseccomp: Add missing files for ptests
Jose Quaresma (1):
archiver: use bb.note instead of echo
Kai Kang (1):
xxhash: fix build with gcc 12
Marcel Ziswiler (2):
alsa-plugins: fix libavtp vs. avtp packageconfig
gstreamer1.0-plugins-bad: add libavtp packageconfig
Markus Volk (1):
gcr: build with gtk+3 for wayland
Marta Rybczynska (4):
cve-check: move update_symlinks to a library
cve-check: write empty fragment files in the text mode
cve-check: fix return type in check_cves
cve-update-db-native: make it possible to disable database updates
Martin Jansa (9):
makedevs: Don't use COPYING.patch just to add license file into ${S}
insane.bbclass: make sure to close .patch files
staging.bbclass: process direct dependencies in deterministic order
patch.py: make sure that patches/series file exists before quilt pop
lttng-modules: fix shell syntax
buildhistory.bbclass: fix shell syntax when using dash
rootfs.py: close kernel_abi_ver_file
ltp: use bfd even when gold is used with ld-is-gold
systemd: Fix build without utmp
Michael Opdenacker (1):
migration guides: release notes for 4.0.1
Mikko Rapeli (1):
bitbake: event.py: ignore exceptions from stdout and sterr operations in atexit
Ming Liu (1):
udev-extraconf: let automount base directory configurable
Mingli Yu (4):
perl: Fix build with gcc-12
ccache: Fix build with gcc-12
oescripts: change compare logic in OEListPackageconfigTests
python3-cryptography: remove test_x509.py
Naveen Saini (1):
pciutils: avoid lspci conflict with busybox
Pavel Zhukov (6):
bitbake.conf: Make TCLIBC and TCMODE lazy assigned
bitbake: fetch2: Honour BB_FETCH_PREMIRRORONLY option
bitbake: Add tests to cover BB_FETCH_PREMIRRORONLY functionality
dbus: Specify runstatedir configure option
bitbake: tests/fetch: Drop unnecessary duplicated function
bitbake: tests/fetch: Add tests for premirror using real project
Peter Kjellerstedt (2):
libseccomp: Correct LIC_FILES_CHKSUM
license.bbclass: Bound beginline and endline in copy_license_files()
Quentin Schulz (2):
docs: set_versions.py: remove honister from active releases list
docs: set_versions.py: check for first latest release tag
Rasmus Villemoes (2):
vim: put xxd in its own package
e2fsprogs: add alternatives handling of lsattr as well
Ricardo Salveti (1):
gnu-efi: enable for riscv64
Richard Purdie (51):
cve-extra-exclusions: Add kernel CVEs
lzo: Add further info to a patch and mark as Inactive-Upstream
python3: Remove problematic paths from sysroot files
python3: Ensure stale empty python module directories don't break the build
Revert "qemu.inc: Remove empty egg-info directories before running meson"
Revert "meson.bblcass: Remove empty egg-info directories before running meson"
vim: Upgrade 8.2.4912 -> 8.2.5034 to fix 9 CVEs
tiff: Add jbig PACKAGECONFIG and clarify CVE-2022-1210
libxslt: Mark CVE-2022-29824 as not applying
oeqa/imagefeatures: Replace lzo with zst
oeqa/imagefeatures: Disable squashfs-lzo
cve-check: Allow warnings to be disabled
openssl: Backport fix for ptest cert expiry
bitbake: runqueue: Fix unihash cache mismatch issues
bitbake: cache/siggen: Add unihash cache copy function
bitbake: bitbake: Bump to version 2.0.1
populate_sdk_ext: Fix race condition on bb_unihashes.dat
gcc-cross-canadian: Add nativesdk-zstd dependency
glib-2.0: upgrade 2.72.1 -> 2.72.2
dnf: upgrade 4.12.0 -> 4.13.0
python3-dtschema: upgrade 2022.4 -> 2022.5
python3-sphinx: upgrade 4.5.0 -> 5.0.0
python3-pip: upgrade 22.1.1 -> 22.1.2
alsa-lib: upgrade 1.2.6.1 -> 1.2.7
sysklogd: upgrade 2.3.0 -> 2.4.0
libxkbcommon: upgrade 1.4.0 -> 1.4.1
piglit: upgrade to latest revision
sysstat: upgrade 12.4.5 -> 12.6.0
harfbuzz: upgrade 4.2.1 -> 4.3.0
gtk+3: upgrade 3.24.33 -> 3.24.34
xwayland: upgrade 22.1.1 -> 22.1.2
alsa-ucm-conf: upgrade 1.2.6.3 -> 1.2.7
gnutls: upgrade 3.7.5 -> 3.7.6
webkitgtk: upgrade 2.36.1 -> 2.36.3
diffoscope: upgrade 212 -> 215
populate_sdk_ext: Fix second bb_unihashes reference
sanity: Switch to make 4.0 as a minimum version
perl: Add dependency on make-native to avoid race issues
glibc: Drop make-native dependency
bitbake: fetch/wget: Move files into place atomically
bitbake: server/process: Avoid risk of exception deadlocks
bitbake: server/process: Remove daemonic thread usage
bitbake: server/process: Avoid tracebacks at exit
uboot-sign: Fix potential index error issues
selftest/multiconfig: Test that multiconfigs in separate layers works
bitbake: cooker: Drop sre_constants usage
classes/buildcfg: Move git/layer revision code into new OE module buildcfg
lib/buildcfg: Share common clean/dirty layer function
buildcfg: Drop unused svn revision function
base/buildhistory/image-buildinfo: Use common buildcfg function
image-buildinfo: Improve and extend to SDK coverage too
Robert Yang (1):
systemd: Set RebootWatchdogSec to 60s as watchdog
Ross Burton (8):
python3-pluggy: add BBCLASSEXTEND for native/nativesdk
btrfs-tools: add a PACKAGECONFIG for lzo
tiff: mark CVE-2022-1622 and CVE-2022-1623 as invalid
packagegroup-self-hosted: remove lzo
libarchive: disable LZO by default
squashfs-tools: disable LZO by default
lzop: remove recipe from oe-core
setuptools3: clean up class
Rusty Howell (1):
oe-depends-dot: Handle new format for task-depends.dot
Sean Anderson (1):
rootfs.py: find .ko.zst kernel modules
Stefan Wiehler (1):
kernel-yocto.bbclass: Reset to exiting on non-zero return code at end of task
Tobias Schmidl (2):
oeqa/selftest/wic.py: Repaired test_qemu()
wic/plugins/images/direct: Allow changes in fstab on rootfs
Vyacheslav Yurkov (2):
files: rootfs-postcommands: move helper commands to script
files: respect overlayfs owner from lower layer
Xiaobing Luo (1):
devtool: Fix _copy_file() TypeError
Zach Welch (2):
test-manual/intro: reorder bitbake-selftest steps
test-manual/intro: bitbake-selftest needs bitbake
leimaohui (1):
gnutls: Added fips option.
wangmy (30):
bind: upgrade 9.18.2 -> 9.18.3
ccache: upgrade 4.6 -> 4.6.1
init-system-helpers: upgrade 1.62 -> 1.63
ninja: upgrade 1.10.2 -> 1.11.0
python3-certifi: upgrade 2021.10.8 -> 2022.5.18.1
python3-cython: upgrade 0.29.28 -> 0.29.30
python3-hypothesis: upgrade 6.46.4 -> 6.46.7
python3-importlib-metadata: upgrade 4.11.3 -> 4.11.4
python3-magic: upgrade 0.4.25 -> 0.4.26
python3-pip: upgrade 22.1 -> 22.1.1
python3-setuptools: upgrade 62.3.1 -> 62.3.2
python3-hypothesis: upgrade 6.46.7 -> 6.46.9
python3-semantic-version: upgrade 2.9.0 -> 2.10.0
python3-webcolors: upgrade 1.11.1 -> 1.12
python3-pytest-subtests: upgrade 0.7.0 -> 0.8.0
asciidoc: upgrade 10.1.4 -> 10.2.0
cups: upgrade 2.4.1 -> 2.4.2
iproute2: upgrade 5.17.0 -> 5.18.0
iw: upgrade 5.16 -> 5.19
logrotate: upgrade 3.19.0 -> 3.20.1
dpkg: upgrade 1.21.7 -> 1.21.8
repo: upgrade 2.25 -> 2.26
iso-codes: upgrade 4.9.0 -> 4.10.0
lttng-ust: upgrade 2.13.2 -> 2.13.3
meson: upgrade 0.62.1 -> 0.62.2
mtools: upgrade 4.0.39 -> 4.0.40
nettle: upgrade 3.7.3 -> 3.8
kbd: upgrade 2.4.0 -> 2.5.0
python3-hypothesis: upgrade 6.46.9 -> 6.46.11
xkeyboard-config: upgrade 2.35.1 -> 2.36
meta-security: 7628a3e90b..8c6fe006a1:
Armin Kuster (18):
swtpm: enable seccomp if DISTRO is enabled
security-tpm2-image: add swtpm
swtpm: enable gnutls
oeqa/swtpm: add swtpm runtime
oeqa/tpm2: fix and cleanup tests
tpm2-pkcs11: we really need the symlinks
smack-test: switch to python3
oeqa/smack: consolidate classes
checksec: update 2.6.0
chkrootkit: update SRC_URI
packagegroup-core-security: add arpwatch and chkrootkit to pkg grp
layer.conf: Post release codename changes
README: Update for dynamic layers
arpwatch: riscv not supported
packagegroup-core-security: drop arpwatch for riscv from pkg grp
chkrootkit: Fix missing includes for musl
arpwatch: update to 3.3
packagegroup-core-security: don't include aprwatch for musl
Signed-off-by: Andrew Geissler <geissonator@yahoo.com>
Change-Id: Ic83db16445cf0a1286685f11d378e1e3e9b794c3
diff --git a/poky/bitbake/lib/bb/__init__.py b/poky/bitbake/lib/bb/__init__.py
index b8333bd..6b470aa 100644
--- a/poky/bitbake/lib/bb/__init__.py
+++ b/poky/bitbake/lib/bb/__init__.py
@@ -9,7 +9,7 @@
# SPDX-License-Identifier: GPL-2.0-only
#
-__version__ = "2.0.0"
+__version__ = "2.0.1"
import sys
if sys.version_info < (3, 6, 0):
diff --git a/poky/bitbake/lib/bb/cache.py b/poky/bitbake/lib/bb/cache.py
index 92e9a3c..988c596 100644
--- a/poky/bitbake/lib/bb/cache.py
+++ b/poky/bitbake/lib/bb/cache.py
@@ -24,6 +24,7 @@
import bb.utils
from bb import PrefixLoggerAdapter
import re
+import shutil
logger = logging.getLogger("BitBake.Cache")
@@ -998,3 +999,11 @@
p.dump([data, self.cacheversion])
bb.utils.unlockfile(glf)
+
+ def copyfile(self, target):
+ if not self.cachefile:
+ return
+
+ glf = bb.utils.lockfile(self.cachefile + ".lock")
+ shutil.copy(self.cachefile, target)
+ bb.utils.unlockfile(glf)
diff --git a/poky/bitbake/lib/bb/cooker.py b/poky/bitbake/lib/bb/cooker.py
index 6da9291..2adf4d2 100644
--- a/poky/bitbake/lib/bb/cooker.py
+++ b/poky/bitbake/lib/bb/cooker.py
@@ -13,7 +13,6 @@
import itertools
import logging
import multiprocessing
-import sre_constants
import threading
from io import StringIO, UnsupportedOperation
from contextlib import closing
@@ -1907,7 +1906,7 @@
try:
re.compile(mask)
bbmasks.append(mask)
- except sre_constants.error:
+ except re.error:
collectlog.critical("BBMASK contains an invalid regular expression, ignoring: %s" % mask)
# Then validate the combined regular expressions. This should never
@@ -1915,7 +1914,7 @@
bbmask = "|".join(bbmasks)
try:
bbmask_compiled = re.compile(bbmask)
- except sre_constants.error:
+ except re.error:
collectlog.critical("BBMASK is not a valid regular expression, ignoring: %s" % bbmask)
bbmask = None
diff --git a/poky/bitbake/lib/bb/event.py b/poky/bitbake/lib/bb/event.py
index df02055..9766860 100644
--- a/poky/bitbake/lib/bb/event.py
+++ b/poky/bitbake/lib/bb/event.py
@@ -132,8 +132,14 @@
if not _uiready:
from bb.msg import BBLogFormatter
# Flush any existing buffered content
- sys.stdout.flush()
- sys.stderr.flush()
+ try:
+ sys.stdout.flush()
+ except:
+ pass
+ try:
+ sys.stderr.flush()
+ except:
+ pass
stdout = logging.StreamHandler(sys.stdout)
stderr = logging.StreamHandler(sys.stderr)
formatter = BBLogFormatter("%(levelname)s: %(message)s")
diff --git a/poky/bitbake/lib/bb/fetch2/git.py b/poky/bitbake/lib/bb/fetch2/git.py
index 23f8c0d..07b3d9a 100644
--- a/poky/bitbake/lib/bb/fetch2/git.py
+++ b/poky/bitbake/lib/bb/fetch2/git.py
@@ -353,10 +353,15 @@
if ud.shallow and os.path.exists(ud.fullshallow) and self.need_update(ud, d):
ud.localpath = ud.fullshallow
return
- elif os.path.exists(ud.fullmirror) and not os.path.exists(ud.clonedir):
- bb.utils.mkdirhier(ud.clonedir)
- runfetchcmd("tar -xzf %s" % ud.fullmirror, d, workdir=ud.clonedir)
-
+ elif os.path.exists(ud.fullmirror) and self.need_update(ud, d):
+ if not os.path.exists(ud.clonedir):
+ bb.utils.mkdirhier(ud.clonedir)
+ runfetchcmd("tar -xzf %s" % ud.fullmirror, d, workdir=ud.clonedir)
+ else:
+ tmpdir = tempfile.mkdtemp(dir=d.getVar('DL_DIR'))
+ runfetchcmd("tar -xzf %s" % ud.fullmirror, d, workdir=tmpdir)
+ fetch_cmd = "LANG=C %s fetch -f --progress %s " % (ud.basecmd, shlex.quote(tmpdir))
+ runfetchcmd(fetch_cmd, d, workdir=ud.clonedir)
repourl = self._get_repo_url(ud)
# If the repo still doesn't exist, fallback to cloning it
diff --git a/poky/bitbake/lib/bb/fetch2/wget.py b/poky/bitbake/lib/bb/fetch2/wget.py
index b3a3de5..b2b542e 100644
--- a/poky/bitbake/lib/bb/fetch2/wget.py
+++ b/poky/bitbake/lib/bb/fetch2/wget.py
@@ -106,10 +106,9 @@
fetchcmd = self.basecmd
- if 'downloadfilename' in ud.parm:
- localpath = os.path.join(d.getVar("DL_DIR"), ud.localfile)
- bb.utils.mkdirhier(os.path.dirname(localpath))
- fetchcmd += " -O %s" % shlex.quote(localpath)
+ localpath = os.path.join(d.getVar("DL_DIR"), ud.localfile) + ".tmp"
+ bb.utils.mkdirhier(os.path.dirname(localpath))
+ fetchcmd += " -O %s" % shlex.quote(localpath)
if ud.user and ud.pswd:
fetchcmd += " --auth-no-challenge"
@@ -133,6 +132,10 @@
self._runwget(ud, d, fetchcmd, False)
+ # Remove the ".tmp" and move the file into position atomically
+ # Our lock prevents multiple writers but mirroring code may grab incomplete files
+ os.rename(localpath, localpath[:-4])
+
# Sanity check since wget can pretend it succeed when it didn't
# Also, this used to happen if sourceforge sent us to the mirror page
if not os.path.exists(ud.localpath):
diff --git a/poky/bitbake/lib/bb/runqueue.py b/poky/bitbake/lib/bb/runqueue.py
index f34f156..1e47fe7 100644
--- a/poky/bitbake/lib/bb/runqueue.py
+++ b/poky/bitbake/lib/bb/runqueue.py
@@ -2299,6 +2299,9 @@
self.rqdata.runtaskentries[hashtid].unihash = unihash
bb.parse.siggen.set_unihash(hashtid, unihash)
toprocess.add(hashtid)
+ if torehash:
+ # Need to save after set_unihash above
+ bb.parse.siggen.save_unitaskhashes()
# Work out all tasks which depend upon these
total = set()
diff --git a/poky/bitbake/lib/bb/server/process.py b/poky/bitbake/lib/bb/server/process.py
index 613956f..43790b6 100644
--- a/poky/bitbake/lib/bb/server/process.py
+++ b/poky/bitbake/lib/bb/server/process.py
@@ -437,6 +437,7 @@
self.socket_connection = sock
def terminate(self):
+ self.events.close()
self.socket_connection.close()
self.connection.connection.close()
self.connection.recv.close()
@@ -662,23 +663,18 @@
self.reader = ConnectionReader(readfd)
self.t = threading.Thread()
- self.t.daemon = True
self.t.run = self.startCallbackHandler
self.t.start()
def getEvent(self):
- self.eventQueueLock.acquire()
+ with self.eventQueueLock:
+ if len(self.eventQueue) == 0:
+ return None
- if len(self.eventQueue) == 0:
- self.eventQueueLock.release()
- return None
+ item = self.eventQueue.pop(0)
+ if len(self.eventQueue) == 0:
+ self.eventQueueNotify.clear()
- item = self.eventQueue.pop(0)
-
- if len(self.eventQueue) == 0:
- self.eventQueueNotify.clear()
-
- self.eventQueueLock.release()
return item
def waitEvent(self, delay):
@@ -686,10 +682,9 @@
return self.getEvent()
def queue_event(self, event):
- self.eventQueueLock.acquire()
- self.eventQueue.append(event)
- self.eventQueueNotify.set()
- self.eventQueueLock.release()
+ with self.eventQueueLock:
+ self.eventQueue.append(event)
+ self.eventQueueNotify.set()
def send_event(self, event):
self.queue_event(pickle.loads(event))
@@ -698,13 +693,17 @@
bb.utils.set_process_name("UIEventQueue")
while True:
try:
- self.reader.wait()
- event = self.reader.get()
- self.queue_event(event)
- except EOFError:
+ ready = self.reader.wait(0.25)
+ if ready:
+ event = self.reader.get()
+ self.queue_event(event)
+ except (EOFError, OSError, TypeError):
# Easiest way to exit is to close the file descriptor to cause an exit
break
+
+ def close(self):
self.reader.close()
+ self.t.join()
class ConnectionReader(object):
diff --git a/poky/bitbake/lib/bb/siggen.py b/poky/bitbake/lib/bb/siggen.py
index 08eca78..3f3d6df 100644
--- a/poky/bitbake/lib/bb/siggen.py
+++ b/poky/bitbake/lib/bb/siggen.py
@@ -120,6 +120,9 @@
def save_unitaskhashes(self):
return
+ def copy_unitaskhashes(self, targetdir):
+ return
+
def set_setscene_tasks(self, setscene_tasks):
return
@@ -358,6 +361,9 @@
def save_unitaskhashes(self):
self.unihash_cache.save(self.unitaskhashes)
+ def copy_unitaskhashes(self, targetdir):
+ self.unihash_cache.copyfile(targetdir)
+
def dump_sigtask(self, fn, task, stampbase, runtime):
tid = fn + ":" + task
diff --git a/poky/bitbake/lib/bb/tests/fetch.py b/poky/bitbake/lib/bb/tests/fetch.py
index 1152e89..622c46a 100644
--- a/poky/bitbake/lib/bb/tests/fetch.py
+++ b/poky/bitbake/lib/bb/tests/fetch.py
@@ -2802,3 +2802,96 @@
fetcher.unpack(self.unpackdir)
alt = os.path.join(self.unpackdir, 'git/.git/objects/info/alternates')
self.assertFalse(os.path.exists(alt))
+
+
+class FetchPremirroronlyLocalTest(FetcherTest):
+
+ def setUp(self):
+ super(FetchPremirroronlyLocalTest, self).setUp()
+ self.mirrordir = os.path.join(self.tempdir, "mirrors")
+ os.mkdir(self.mirrordir)
+ self.reponame = "bitbake"
+ self.gitdir = os.path.join(self.tempdir, "git", self.reponame)
+ self.recipe_url = "git://git.fake.repo/bitbake"
+ self.d.setVar("BB_FETCH_PREMIRRORONLY", "1")
+ self.d.setVar("BB_NO_NETWORK", "1")
+ self.d.setVar("PREMIRRORS", self.recipe_url + " " + "file://{}".format(self.mirrordir) + " \n")
+
+ def make_git_repo(self):
+ self.mirrorname = "git2_git.fake.repo.bitbake.tar.gz"
+ recipeurl = "git:/git.fake.repo/bitbake"
+ os.makedirs(self.gitdir)
+ self.git("init", self.gitdir)
+ for i in range(0):
+ self.git_new_commit()
+ bb.process.run('tar -czvf {} .'.format(os.path.join(self.mirrordir, self.mirrorname)), cwd = self.gitdir)
+
+ def git_new_commit(self):
+ import random
+ testfilename = "bibake-fetch.test"
+ os.unlink(os.path.join(self.mirrordir, self.mirrorname))
+ with open(os.path.join(self.gitdir, testfilename), "w") as testfile:
+ testfile.write("Useless random data {}".format(random.random()))
+ self.git("add {}".format(testfilename), self.gitdir)
+ self.git("commit -a -m \"This random commit {}. I'm useless.\"".format(random.random()), self.gitdir)
+ bb.process.run('tar -czvf {} .'.format(os.path.join(self.mirrordir, self.mirrorname)), cwd = self.gitdir)
+ return self.git("rev-parse HEAD", self.gitdir).strip()
+
+ def test_mirror_commit_nonexistent(self):
+ self.make_git_repo()
+ self.d.setVar("SRCREV", "0"*40)
+ fetcher = bb.fetch.Fetch([self.recipe_url], self.d)
+ with self.assertRaises(bb.fetch2.NetworkAccess):
+ fetcher.download()
+
+ def test_mirror_commit_exists(self):
+ self.make_git_repo()
+ self.d.setVar("SRCREV", self.git_new_commit())
+ fetcher = bb.fetch.Fetch([self.recipe_url], self.d)
+ fetcher.download()
+ fetcher.unpack(self.unpackdir)
+
+ def test_mirror_tarball_nonexistent(self):
+ self.d.setVar("SRCREV", "0"*40)
+ fetcher = bb.fetch.Fetch([self.recipe_url], self.d)
+ with self.assertRaises(bb.fetch2.NetworkAccess):
+ fetcher.download()
+
+class FetchPremirroronlyNetworkTest(FetcherTest):
+
+ def setUp(self):
+ super(FetchPremirroronlyNetworkTest, self).setUp()
+ self.mirrordir = os.path.join(self.tempdir, "mirrors")
+ os.mkdir(self.mirrordir)
+ self.reponame = "fstests"
+ self.clonedir = os.path.join(self.tempdir, "git")
+ self.gitdir = os.path.join(self.tempdir, "git", "{}.git".format(self.reponame))
+ self.recipe_url = "git://git.yoctoproject.org/fstests"
+ self.d.setVar("BB_FETCH_PREMIRRORONLY", "1")
+ self.d.setVar("BB_NO_NETWORK", "0")
+ self.d.setVar("PREMIRRORS", self.recipe_url + " " + "file://{}".format(self.mirrordir) + " \n")
+
+ def make_git_repo(self):
+ import shutil
+ self.mirrorname = "git2_git.yoctoproject.org.fstests.tar.gz"
+ os.makedirs(self.clonedir)
+ self.git("clone --bare --shallow-since=\"01.01.2013\" {}".format(self.recipe_url), self.clonedir)
+ bb.process.run('tar -czvf {} .'.format(os.path.join(self.mirrordir, self.mirrorname)), cwd = self.gitdir)
+ shutil.rmtree(self.clonedir)
+
+ @skipIfNoNetwork()
+ def test_mirror_tarball_updated(self):
+ self.make_git_repo()
+ ## Upstream commit is in the mirror
+ self.d.setVar("SRCREV", "49d65d53c2bf558ae6e9185af0f3af7b79d255ec")
+ fetcher = bb.fetch.Fetch([self.recipe_url], self.d)
+ fetcher.download()
+
+ @skipIfNoNetwork()
+ def test_mirror_tarball_outdated(self):
+ self.make_git_repo()
+ ## Upstream commit not in the mirror
+ self.d.setVar("SRCREV", "15413486df1f5a5b5af699b6f3ba5f0984e52a9f")
+ fetcher = bb.fetch.Fetch([self.recipe_url], self.d)
+ with self.assertRaises(bb.fetch2.NetworkAccess):
+ fetcher.download()