subtree updates
meta-raspberrypi: b6a1645a97..c57b464b88:
Lluis Campos (1):
rpi-cmdline: do_compile: Use pure Python syntax to get `CMDLINE`
meta-openembedded: 2eb39477a7..a755af4fb5:
Adrian Zaharia (1):
lapack: add packageconfig for lapacke
Akash Hadke (1):
polkit: Add --shell /bin/nologin to polkitd user
Alex Kiernan (3):
ntpsec: Add UPSTREAM_CHECK_URI
libgpiod: Detect ptest using PTEST_ENABLED
ostree: Cleanup PACKAGECONFIGs
Anuj Mittal (1):
yasm: fix buildpaths warning
Atanas Bunchev (1):
python3-twitter: Upgrade 4.8.0 -> 4.10.1
Bartosz Golaszewski (4):
imagemagick: add PACKAGECONFIG for C++ bindings
python3-matplotlib: don't use PYTHON_PN
python3-matplotlib: add packaging to RDEPENDS
python3-matplotlib: bump to 3.5.2
Bruce Ashfield (3):
vboxguestdrivers: fix build against 5.19 kernel / libc-headers
zfs: update to v2.1.5
vboxguestdrivers: make kernel shared directory dependency explicit
Carsten Bäcker (1):
spdlog: Fix CMake flag
Changqing Li (3):
fuse3: support ptest
redis: fix do_patch fuzz warning
dlt-daemon: fix dlt-system.service failed since buffer overflow
Clément Péron (1):
python: add Pydantic data validation package
Devendra Tewari (1):
android-tools: sleep more in android-gadget-start
Ed Tanous (1):
Add python-requests-unixsocket recipe
Enguerrand de Ribaucourt (1):
mdio-tools: add recipes
Etienne Cordonnier (1):
uutils-coreutils: add recipe
Jagadeesh Krishnanjanappa (4):
python3-asgiref: add recipe
python3-django: make 3.2.x as default version
python3-django: Add python3-asgiref runtime dependency
python3-django: remove 2.2.x recipe
Jan Luebbe (2):
chrony: add support for config and source snippet includes
gensio: upgrade 2.3.1 -> 2.5.2
Jan Vermaete (1):
makeself: added makeself as new recipe
Jim Broadus (1):
networkmanager: fix iptables and nft paths
Jose Quaresma (2):
wireguard-module: 1.0.20210219 -> 1.0.20220627
wireguard-tools: Add a new package for wg-quick
Julian Haller (2):
pcsc-lite: upgrade 1.9.0 -> 1.9.8
ccid: upgrade 1.4.33 -> 1.5.0
Justin Bronder (1):
lmdb: only set SONAME on the shared library
Khem Raj (61):
mariadb: Inherit pkgconfig
mariadb: Add packageconfig for lz4 and enable it
ibus: Swith to use main branch instead of master
kronosnet: Upgrade to 1.24
ostree: Upgrade to 2022.5 release
sdbus-c++-libsystemd: Fix build with glibc 2.36
xfstests: Upgrade to v2022.07.10
autofs: Fix build with glibc 2.36
audit: Upgrade to 3.0.8 and fix build with linux 5.17+
pcp: Add to USERADD_PACKAGES instead of override
mozjs: Use RUST_HOST_SYS and RUST_TARGET_SYS
fluentbit: Fix build with clang
audit: Fix build with musl
fluentbit: Fix build with musl
klibc: Upgrade to 2.0.10
gnome-keyring,cunit,xfce4-panel: Do not inherit remove-libtool class here
mpd: Update to 0.23.8
openipmi: Enable largefile cflags
proftpd: Always enable largefile support
netperf: Always enable largefile support
openipmi: Always enable largefile support
unbound: Always enable largefile support
sysbench: Always enable largefile support
libmtp: Always enable largefile support
toybox: Fix build with glibc 2.36+
xfstests: Upgrade to 2022.07.31 release
libmpd: Fix function returns and casts
audit: Revert the tweak done in configure step in do_install
mpd: Upgrade to 0.23.9
fluentbit: Use CMAKE_C_STANDARD_LIBRARIES cmake var to pass libatomic
fluentbit: Upgrade to 1.9.7 and fix build on x86
klibc: Fix build with kernel 5.19 headers
ntpsec: Add -D_GNU_SOURCE and fix building with devtool
gd: Fix build with clang-15
cpulimit: Define -D_GNU_SOURCE
safec: Remove unused variable 'len'
ncftp: Enable autoreconf
ncftp: Fix TMPDIR path embedding into ncftpget
libb64: Switch to github fork and upgrade to 2.0.0.1+git
dhrystone: Disable warnings as errors with clang
dibbler: Fix build with musl
fio: Fix additional warnings seen with musl
ssmtp: Fix null pointer assignments
gst-editing-services: Add recipe
rygel: Upgrade to 0.40.4
libesmtp: Define _GNU_SOURCE
python3-grpcio: Enable largefile support explicitly
libteam: Include missing headers for strrchr and memcmp
neon: Upgrade to 0.32.2
satyr: Fix build on musl/clang
libmusicbrainz: Avoid -Wnonnull warning
aom: Upgrade to 3.4.0
vorbis-tools: Fix build on musl
dvb-apps: Use tarball for SRC_URI and fix build on musl
python3-netifaces: Fix build with python3 and musl
python3-pyephem: Fix build with python3 and musl
samba: Fix warnings in configure tests for rpath checks
lirc: Fix build on musl
mongodb: Fix boost build with clang-15
crda: Fix build with clang-15
monkey: Fix build with musl
Lei Maohui (2):
dnf-plugin-tui: Fix somw issue in postinstall process.
xrdp: Fix buildpaths warning.
Leon Anavi (16):
python3-nocasedict: Upgrade 1.0.3 -> 1.0.4
python3-frozenlist: Upgrade 1.3.0 -> 1.3.1
python3-networkx: Upgrade 2.8.4 -> 2.8.5
python3-pyhamcrest: Upgrade 2.0.3 -> 2.0.4
python3-aiohue: Upgrade 4.4.2 -> 4.5.0
python3-pyperf: Upgrade 2.3.0 -> 2.4.1
python3-eth-abi: Upgrade 3.0.0 -> 3.0.1
python3-cytoolz: Upgrade 0.11.2 -> 0.12.0
python3-yarl: Upgrade 1.7.2 -> 1.8.1
python3-term: Upgrade 2.3 -> 2.4
python3-coverage: Upgrade 6.4.1 -> 6.4.4
python3-regex: Upgrade 2022.7.25 -> 2022.8.17
python3-awesomeversion: Upgrade 22.6.0 -> 22.8.0
python3-typed-ast: Upgrade 1.5.2 -> 1.5.4
python3-prompt-toolkit: Upgrade 3.0.24 -> 3.0.30
python3-prettytable: Upgrade 3.1.1 -> 3.3.0
Markus Volk (6):
libass: update to v1.16.0
spdlog: update to v1.10.0
waylandpp: add recipe
wireplumber: update to v0.4.11
pipewire: update to v0.3.56
pipewire: improve runtime dependency settings
Marta Rybczynska (1):
polkit: update patches for musl compilation
Matthias Klein (1):
libftdi: update to 1.5
Mike Crowe (1):
yasm: Only depend on xmlto when docs are enabled
Mike Petersen (1):
sshpass: add recipe
Mingli Yu (10):
net-snmp: set ac_cv_path_PSPROG
postgresql: Fix the buildpaths issue
freeradius: Fix buildpaths issue
openipmi: Fix buildpaths issue
apache2: Fix the buildpaths issue
frr: fix buildpaths issue
nspr: fix buildpaths issue
liblockfile: fix buildpaths issue
freediameter: fix buildpaths issue
postgresql: make sure pam conf installed when pam enabled
Ovidiu Panait (1):
net-snmp: upgrade 5.9.1 -> 5.9.3
Paulo Neves (1):
fluentbit Upgrade to 1.3.5 -> 1.9.6
Philip Balister (2):
python3-pybind11: Update to Version 2.10.0.
Remove dead link and old information from the README.
Potin Lai (7):
libplist: add libplist_git.bb
libimobiledevice-glue: SRCREV bump bc6c44b..d2ff796
libimobiledevice: add libimobiledevice_git.bb
libirecovery: SRCREV bump e190945..ab5b4d8
libusbmuxd: add libusbmuxd_git.bb
usbmuxd: add usbmuxd_git.bb
idevicerestore: SRCREV bump 280575b..7d622d9
Richard Purdie (1):
lmdb: Don't inherit base
Sam Van Den Berge (1):
python3-jsonrpcserver: add patch to use importlib.resources instead of pkg_resources
Saul Wold (10):
libipc-signal-perl: Fix LICENSE string
libdigest-hmac-perl: Fix LICENSE string
libio-socket-ssl-perl: Fix LICENSE string
libdigest-sha1-perl: Fix LICENSE string
libmime-types-perl: Fix LICENSE string
libauthen-sasl-perl: Fix LICENSE string
libnet-ldap-perl: Fix LICENSE string
libxml-libxml-perl: Fix LICENSE string
libnet-telnet-perl: Fix LICENSE string
libproc-waitstat-perl: Fix LICENSE string
Sean Anderson (2):
image_types_sparse: Pad source image to block size
image_types_sparse: Generate "don't care" chunks
Vyacheslav Yurkov (4):
protobuf: correct ptest dependency
protobuf: 3.19.4 -> 3.21.5 upgrade
protobuf: change build system to cmake
protobuf: disable protoc binary for target
Wang Mingyu (60):
cifs-utils: upgrade 6.15 -> 7.0
geocode-glib: upgrade 3.26.3 -> 3.26.4
gjs: upgrade 1.72.1 -> 1.72.2
htpdate: upgrade 1.3.5 -> 1.3.6
icewm: upgrade 2.9.8 -> 2.9.9
ipc-run: upgrade 20200505.0 -> 20220807.0
iwd: upgrade 1.28 -> 1.29
ldns: upgrade 1.8.1 -> 1.8.2
libadwaita: upgrade 1.1.3 -> 1.1.4
libencode-perl: upgrade 3.18 -> 3.19
libmime-charset-perl: upgrade 1.012.2 -> 1.013.1
libtest-warn-perl: upgrade 0.36 -> 0.37
nano: upgrade 6.3 -> 6.4
nbdkit: upgrade 1.31.15 -> 1.32.1
netdata: upgrade 1.35.1 -> 1.36.0
fio: upgrade 3.30 -> 3.31
nlohmann-json: upgrade 3.10.5 -> 3.11.2
poco: upgrade 1.12.1 -> 1.12.2
postgresql: upgrade 14.4 -> 14.5
poppler: upgrade 22.07.0 -> 22.08.0
smarty: upgrade 4.1.1 -> 4.2.0
tracker: upgrade 3.3.2 -> 3.3.3
uftp: upgrade 5.0 -> 5.0.1
xdg-user-dirs: upgrade 0.17 -> 0.18
python3-pycodestyle: upgrade 2.9.0 -> 2.9.1
python3-pyzmq: upgrade 23.2.0 -> 23.2.1
python3-setuptools-declarative-requirements: upgrade 1.2.0 -> 1.3.0
python3-sqlalchemy: upgrade 1.4.39 -> 1.4.40
python3-werkzeug: upgrade 2.2.1 -> 2.2.2
python3-xmlschema: upgrade 2.0.1 -> 2.0.2
python3-yappi: upgrade 1.3.5 -> 1.3.6
ade: upgrade 0.1.1f -> 0.1.2
babl: upgrade 0.1.92 -> 0.1.94
ctags: upgrade 5.9.20220703.0 -> 5.9.20220821.0
grilo-plugins: upgrade 0.3.14 -> 0.3.15
ldns: upgrade 1.8.2 -> 1.8.3
libcurses-perl: upgrade 1.38 -> 1.41
mosquitto: upgrade 2.0.14 -> 2.0.15
nbdkit: upgrade 1.32.1 -> 1.33.1
netdata: upgrade 1.36.0 -> 1.36.1
libsdl2-ttf: upgrade 2.20.0 -> 2.20.1
xfstests: upgrade 2022.07.31 -> 2022.08.07
php: upgrade 8.1.8 -> 8.1.9
rdma-core: upgrade 41.0 -> 42.0
spitools: upgrade 1.0.1 -> 1.0.2
unbound: upgrade 1.16.1 -> 1.16.2
zlog: upgrade 1.2.15 -> 1.2.16
python3-hexbytes: upgrade 0.2.3 -> 0.3.0
python3-pythonping: upgrade 1.1.2 -> 1.1.3
python3-jsonrpcserver: Add dependence python3-typing-extensions
feh: upgrade 3.9 -> 3.9.1
gnome-bluetooth: upgrade 42.2 -> 42.3
hunspell: upgrade 1.7.0 -> 1.7.1
gtk4: upgrade 4.6.6 -> 4.6.7
logwatch: upgrade 7.6 -> 7.7
bdwgc: upgrade 8.2.0 -> 8.2.2
tcpreplay: upgrade 4.4.1 -> 4.4.2
tree: upgrade 2.0.2 -> 2.0.3
xfsdump: upgrade 3.1.10 -> 3.1.11
babl: upgrade 0.1.94 -> 0.1.96
Wolfgang Meyer (1):
libsdl2-ttf: upgrade 2.0.18 -> 2.20.0
Xu Huan (18):
python3-protobuf: upgrade 4.21.3 -> 4.21.4
python3-pycodestyle: upgrade 2.8.0 -> 2.9.0
python3-pyflakes: upgrade 2.4.0 -> 2.5.0
python3-pythonping: upgrade 1.1.1 -> 1.1.2
python3-regex: upgrade 2022.7.24 -> 2022.7.25
python3-werkzeug: upgrade 2.2.0 -> 2.2.1
python3-google-auth: upgrade 2.9.1 -> 2.10.0
python3-humanize: upgrade 4.2.3 -> 4.3.0
python3-hexbytes: upgrade 0.2.2 -> 0.2.3
python3-imageio: upgrade 2.21.0 -> 2.21.1
python3-nocaselist: upgrade 1.0.5 -> 1.0.6
python3-protobuf: upgrade 4.21.4 -> 4.21.5
python3-pycares: upgrade 4.2.1 -> 4.2.2
python3-fastjsonschema: upgrade 2.16.1 -> 2.16.2
python3-google-api-python-client: upgrade 2.56.0 -> 2.57.0
python3-google-auth: upgrade 2.10.0 -> 2.11.0
python3-grpcio-tools: upgrade 1.47.0 -> 1.48.0
python3-grpcio: upgrade 1.47.0 -> 1.48.0
Yi Zhao (5):
strongswan: upgrade 5.9.6 -> 5.9.7
libldb: upgrade 2.3.3 -> 2.3.4
samba: upgrade 4.14.13 -> 4.14.14
python3-jsonrpcserver: upgrade 5.0.7 -> 5.0.8
samba: fix buildpaths issue
wangmy (16):
gedit: upgrade 42.1 -> 42.2
libwacom: upgrade 2.3.0 -> 2.4.0
htpdate: upgrade 1.3.4 -> 1.3.5
nbdkit: upgrade 1.31.14 -> 1.31.15
pure-ftpd: upgrade 1.0.50 -> 1.0.51
avro-c: upgrade 1.11.0 -> 1.11.1
debootstrap: upgrade 1.0.126 -> 1.0.127
freerdp: upgrade 2.7.0 -> 2.8.0
icewm: upgrade 2.9.7 -> 2.9.8
libmxml: upgrade 3.3 -> 3.3.1
poco: upgrade 1.12.0 -> 1.12.1
xfontsel: upgrade 1.0.6 -> 1.1.0
xmessage: upgrade 1.0.5 -> 1.0.6
xrefresh: upgrade 1.0.6 -> 1.0.7
zabbix: upgrade 6.0.5 -> 6.2.1
xrdp: upgrade 0.9.18 -> 0.9.19
zhengrq.fnst (4):
python3-asttokens: upgrade 2.0.7 -> 2.0.8
python3-charset-normalizer: upgrade 2.1.0 -> 2.1.1
python3-eth-account: 0.6.1 -> 0.7.0
python3-cantools: upgrade 37.1.0 -> 37.1.2
zhengruoqin (12):
python3-dominate: upgrade 2.6.0 -> 2.7.0
python3-flask-login: upgrade 0.6.1 -> 0.6.2
python3-google-api-python-client: upgrade 2.54.0 -> 2.55.0
python3-haversine: upgrade 2.5.1 -> 2.6.0
python3-imageio: upgrade 2.19.5 -> 2.21.0
python3-autobahn: upgrade 22.6.1 -> 22.7.1
python3-engineio: upgrade 4.3.3 -> 4.3.4
python3-flask: upgrade 2.1.3 -> 2.2.2
python3-gcovr: upgrade 5.1 -> 5.2
python3-google-api-python-client: upgrade 2.55.0 -> 2.56.0
python3-asttokens: upgrade 2.0.5 -> 2.0.7
python3-zeroconf: upgrade 0.38.7 -> 0.39.0
meta-security: 2a2d650ee0..10fdc2b13a:
Anton Antonov (2):
Use CARGO_TARGET_SUBDIR in do_install
parsec-service: Update oeqa tests
Armin Kuster (8):
python3-privacyidea: update to 3.7.3
lkrg-module: update to 0.9.5
apparmor: update to 3.0.6
packagegroup-core-security: add space for appends
cryptmount: Add new pkg
packagegroup-core-security: add pkg to grp
cyptmount: Fix mount.h conflicts seen with glibc 2.36+
kas: update testimage inherit
John Edward Broadbent (1):
meta-security: Add recipe for Glome
Mingli Yu (1):
samhain-standalone: fix buildpaths issue
poky: fc59c28724..9b1db65e7d:
Alejandro Hernandez Samaniego (1):
baremetal-image.bbclass: Emulate image.bbclass to handle new classes scope
Alex Stewart (1):
maintainers: update opkg maintainer
Alexander Kanavin (113):
kmscube: address linux 5.19 fails
rpm: update 4.17.0 -> 4.17.1
go: update 1.18.4 -> 1.19
bluez5: update 5.64 -> 5.65
python3-pip: update 22.2.1 -> 22.2.2
ffmpeg: update 5.0.1 -> 5.1
iproute2: upgrade 5.18.0 -> 5.19.0
harfbuzz: upgrade 4.4.1 -> 5.1.0
libwpe: upgrade 1.12.0 -> 1.12.2
bind: upgrade 9.18.4 -> 9.18.5
diffoscope: upgrade 218 -> 220
ell: upgrade 0.51 -> 0.52
gnutls: upgrade 3.7.6 -> 3.7.7
iso-codes: upgrade 4.10.0 -> 4.11.0
kea: upgrade 2.0.2 -> 2.2.0
kexec-tools: upgrade 2.0.24 -> 2.0.25
libcap: upgrade 2.64 -> 2.65
libevdev: upgrade 1.12.1 -> 1.13.0
libnotify: upgrade 0.8.0 -> 0.8.1
libwebp: upgrade 1.2.2 -> 1.2.3
libxcvt: upgrade 0.1.1 -> 0.1.2
mesa: upgrade 22.1.3 -> 22.1.5
mobile-broadband-provider-info: upgrade 20220511 -> 20220725
nettle: upgrade 3.8 -> 3.8.1
piglit: upgrade to latest revision
puzzles: upgrade to latest revision
python3: upgrade 3.10.5 -> 3.10.6
python3-dtschema: upgrade 2022.7 -> 2022.8
python3-hypothesis: upgrade 6.50.1 -> 6.54.1
python3-jsonschema: upgrade 4.9.0 -> 4.9.1
python3-markdown: upgrade 3.3.7 -> 3.4.1
python3-setuptools: upgrade 63.3.0 -> 63.4.1
python3-sphinx: upgrade 5.0.2 -> 5.1.1
python3-urllib3: upgrade 1.26.10 -> 1.26.11
sqlite3: upgrade 3.39.1 -> 3.39.2
sysklogd: upgrade 2.4.0 -> 2.4.2
webkitgtk: upgrade 2.36.4 -> 2.36.5
kernel-dev: working with kernel using devtool does not require building and installing eSDK
sdk-manual: describe how to use extensible SDK functionality directly in a Yocto build
dropbear: merge .inc into .bb
rust: update 1.62.0 -> 1.62.1
cmake: update 3.23.2 -> 3.24.0
weston: upgrade 10.0.1 -> 10.0.2
patchelf: update 0.14.5 -> 0.15.0
patchelf: replace a rejected patch with an equivalent uninative.bbclass tweak
weston: exclude pre-releases from version check
tzdata: upgrade 2022a -> 2022b
libcgroup: update 2.0.2 -> 3.0.0
python3-setuptools-rust: update 1.4.1 -> 1.5.1
shadow: update 4.11.1 -> 4.12.1
slang: update 2.3.2 -> 2.3.3
xz: update 5.2.5 -> 5.2.6
gdk-pixbuf: update 2.42.8 -> 2.42.9
xorgproto: update 2022.1 -> 2022.2
boost-build-native: update 4.4.1 -> 1.80.0
boost: update 1.79.0 -> 1.80.0
vulkan-samples: update to latest revision
epiphany: upgrade 42.3 -> 42.4
git: upgrade 2.37.1 -> 2.37.2
glib-networking: upgrade 2.72.1 -> 2.72.2
gnu-efi: upgrade 3.0.14 -> 3.0.15
gpgme: upgrade 1.17.1 -> 1.18.0
libjpeg-turbo: upgrade 2.1.3 -> 2.1.4
libwebp: upgrade 1.2.3 -> 1.2.4
lighttpd: upgrade 1.4.65 -> 1.4.66
mesa: upgrade 22.1.5 -> 22.1.6
meson: upgrade 0.63.0 -> 0.63.1
mpg123: upgrade 1.30.1 -> 1.30.2
pango: upgrade 1.50.8 -> 1.50.9
piglit: upgrade to latest revision
pkgconf: upgrade 1.8.0 -> 1.9.2
python3-dtschema: upgrade 2022.8 -> 2022.8.1
python3-more-itertools: upgrade 8.13.0 -> 8.14.0
python3-numpy: upgrade 1.23.1 -> 1.23.2
python3-pbr: upgrade 5.9.0 -> 5.10.0
python3-pyelftools: upgrade 0.28 -> 0.29
python3-pytz: upgrade 2022.1 -> 2022.2.1
strace: upgrade 5.18 -> 5.19
sysklogd: upgrade 2.4.2 -> 2.4.4
wireless-regdb: upgrade 2022.06.06 -> 2022.08.12
wpebackend-fdo: upgrade 1.12.0 -> 1.12.1
python3-hatchling: update 1.6.0 -> 1.8.0
python3-setuptools: update 63.4.1 -> 65.0.2
devtool: do not leave behind source trees in workspace/sources
systemtap: add a patch to address a python 3.11 failure
bitbake: bitbake-layers: initialize tinfoil before registering command line arguments
scripts/oe-setup-builddir: add a check that TEMPLATECONF is valid
bitbake-layers: add a command to save the active build configuration as a template into a layer
bitbake-layers: add ability to save current layer repository configuration into a file
scripts/oe-setup-layers: add a script that restores the layer configuration from a json file
selftest/bblayers: add a test for creating a layer setup and using it to restore the layers
selftest/bblayers: adjust the revision for the layer setup test
perl: run builds from a pristine source tree
meta-poky/conf: move default templates to conf/templates/default/
syslinux: mark all pending patches as Inactive-Upstream
shadow: correct the pam patch status
mtd-utils: remove patch that adds -I option
gstreamer1.0-plugins-bad: remove an unneeded patch
ghostscript: remove unneeded patch
ovmf: drop the force no-stack-protector patch
python: submit CC to cc_basename patch upstream
mc: submit perl warnings patch upstream
sysvinit: send install.patch upstream
valgrind: (re)send ppc instructions patch upstream
gdk-pixbuf: submit fatal-loader.patch upstream
libsdl2: follow upstream version is even rule
python3-pip: submit reproducible.patch upstream
python3-pip: remove unneeded reproducible.patch
llvm: remove 0006-llvm-TargetLibraryInfo-Undefine-libc-functions-if-th.patch
scripts/oe-setup-builddir: migrate build/conf/templateconf.cfg to new template locations
meta/files/layers.schema.json: drop the layers property
scripts/oe-setup-builddir: write to conf/templateconf.cfg after the build is set up
scripts/oe-setup-builddir: make environment variable the highest priority source for TEMPLATECONF
Alexandre Belloni (1):
ruby: drop capstone support
Andrei Gherzan (7):
shadow: Enable subid support
rootfspostcommands.py: Restructure sort_passwd and related functions
rootfspostcommands.py: Cleanup subid backup files generated by shadow-utils
selftest: Add module for testing rootfs postcommands
rootfs-postcommands.bbclass: Follow function rename in rootfspostcommands.py
shadow: Avoid nss warning/error with musl
linux-yocto: Fix COMPATIBLE_MACHINE regex match
Andrey Konovalov (2):
mesa: add pipe-loader's libraries to libopencl-mesa package
mesa: build clover with native LLVM codegen support for freedreno
Anuj Mittal (1):
poky.conf: add ubuntu-22.04 to tested distros
Armin Kuster (1):
system-requirements.rst: remove EOL and Centos7 hosts
Aryaman Gupta (1):
bitbake: runqueue: add memory pressure regulation
Awais Belal (1):
kernel-fitimage.bbclass: only package unique DTBs
Beniamin Sandu (1):
libpam: use /run instead of /var/run in systemd tmpfiles
Bertrand Marquis (1):
sysvinit-inittab/start_getty: Fix respawn too fast
Bruce Ashfield (22):
linux-yocto/5.15: update to v5.15.58
linux-yocto/5.10: update to v5.10.134
linux-yocto-rt/5.15: update to -rt48 (and fix -stable merge)
linux-libc-headers: update to v5.19
kernel-devsrc: support arm v5.19+ on target build
kernel-devsrc: support powerpc on v5.19+
lttng-modules: fix build against mips and v5.19 kernel
linux-yocto: introduce v5.19 reference kernel recipes
meta/conf: update preferred linux-yocto version to v5.19
linux-yocto: drop v5.10 reference kernel recipes
linux-yocto/5.15: update to v5.15.59
linux-yocto/5.15: fix reproducibility issues
linux-yocto/5.19: cfg: update x32 configuration fragment
linux-yocto/5.19: fix reproducibility issues
poky: update preferred version to v5.19
poky: change preferred kernel version to 5.15 in poky-alt
yocto-bsp: drop v5.10 bbappend and create 5.19 placeholder
lttng-modules: replace mips compaction fix with upstream change
linux-yocto/5.15: update to v5.15.60
linux-yocto/5.19: update to v5.19.1
linux-yocto/5.19: update to v5.19.3
linux-yocto/5.15: update to v5.15.62
Changqing Li (1):
apt: fix nativesdk-apt build failure during the second time build
Chen Qi (2):
python3-hypothesis: revert back to 6.46.11
python3-requests: add python3-compression dependency
Drew Moseley (1):
rng-tools: Replace obsolete "wants systemd-udev-settle"
Enrico Scholz (2):
npm.bbclass: fix typo in 'fund' config option
npm.bbclass: fix architecture mapping
Ernst Sjöstrand (1):
cve-check: Don't use f-strings
Jacob Kroon (1):
python3-cython: Remove debug lines
Jan Luebbe (2):
openssh: sync local ssh_config + sshd_config files with upstream 8.7p1
openssh: add support for config snippet includes to ssh and sshd
JeongBong Seo (1):
wic: add 'none' fstype for custom image
Johannes Schneider (1):
classes: rootfs-postcommands: autologin root on serial-getty
Jon Mason (2):
oeqa/parselogs: add qemuarmv5 arm-charlcd masking
ref-manual: add numa to machine features
Jose Quaresma (4):
bitbake: build: prefix the tasks with a timestamp in the log task_order
archiver.bbclass: some recipes that uses the kernelsrc bbclass uses the shared source
linux-yocto: prepend the the value with a space when append to KERNEL_EXTRA_ARGS
shaderc: upgrade 2022.1 -> 2022.2
Joshua Watt (4):
bitbake: siggen: Fix insufficent entropy in sigtask file names
bitbake: utils: Pass lock argument in fileslocked
classes: cve-check: Get shared database lock
meta/files: add layer setup JSON schema and example
Kai Kang (1):
packagegroup-self-hosted: update for strace
Kevin Hao (1):
uboot-config.bbclass: Don't bail out early in multi configs
Khem Raj (83):
qemu: Fix build with glibc 2.36
mtd-utils: Fix build with glibc 2.36
stress-ng: Upgrade to 0.14.03
bootchart2: Fix build with glibc 2.36+
ltp: Fix sys/mount.h conflicts needed for glibc 2.36+ compile
efivar: Fix build with glibc 2.36
cracklib: Drop using register keyword
util-linux: Define pidfd_* function signatures
util-linux: Upgrade to 2.38.1
tcp-wrappers: Fix implicit-function-declaration warnings
perl-cross: Correct function signatures in configure_func.sh
perl: Pass additional flags to enable lfs and gnu source
sysvinit: Fix mount.h conflicts seen with glibc 2.36+
glibc: Bump to 2.36
glibc: Update patch status
zip: Enable largefile support based on distro feature
zip: Make configure checks to be more robust
unzip: Fix configure tests to use modern C
unzip: Enable largefile support when enabled in distro
iproute2: Fix netns check during configure
glibc: Bump to latest 2.36 branch
gstreamer1.0-plugins-base: Include required system headers for isspace() and sscanf()
musl: Upgrade to latest tip of trunk
zip: Always enable LARGE_FILE_SUPPORT
libmicrohttpd: Enable largefile support unconditionally
unzip: Always enable largefile support
default-distrovars: Remove largefile from defualt DISTRO_FEATURES
zlib: Resolve CVE-2022-37434
json-c: Fix function prototypes
rsync: Backport fix to address CVE-2022-29154
rsync: Upgrade to 3.2.5
libtirpc: Backport fix for CVE-2021-46828
libxml2: Ignore CVE-2016-3709
tiff: Backport a patch for CVE-2022-34526
libtirpc: Upgrade to 1.3.3
perf: Add packageconfig for libbfd support and use disabled as default
connman: Backports for security fixes
systemd: Upgrade to 251.4 and fix build with binutils 2.39
time: Add missing include for memset
screen: Add missing include files in configure checks
setserial: Fix build with clang
expect: Fix implicit-function-declaration warnings
spirv-tools: Remove default copy constructor in header
boost: Compile out stdlib unary/binary_functions for c++11 and newer
vulkan-samples: Qualify move as std::move
apt: Do not use std::binary_function
ltp: Fix sys/mount.h and linux/mount.h conflict
rpm: Remove -Wimplicit-function-declaration warnings
binutils: Upgrade to 2.39 release
binutils-cross: Disable gprofng for when building cross binutils
binutils: Package up gprofng
binutils: Disable gprofng when using clang
binutils-cross-canadian: Package up new gprofng.rc file
autoconf: Fix strict prototype errors in generated tests
rsync: Add missing prototypes to function declarations
nfs-utils: Upgrade to 2.6.2
webkitgtk: Upgrade to 2.36.6 minor update
musl: Update to tip
binutils: Disable gprofng on musl systems
binutils: Upgrade to latest on 2.39 release branch
cargo_common.bbclass: Add missing space in shell conditional code
rng-tools: Remove depndencies on hwrng
ccache: Update the patch status
ccache: Fix build with gcc12 on musl
alsa-plugins: Include missing string.h
xinetd: Pass missing -D_GNU_SOURCE
watchdog: Include needed system header for function decls
libcgroup: Use GNU strerror_r only when its available
pinentry: enable _XOPEN_SOURCE on musl for wchar usage in curses
apr: Use correct strerror_r implementation based on libc type
gcr: Define _GNU_SOURCE
ltp: Adjust types to match create_fifo_thread return
gcc: Upgrade to 12.2.0
glibc: Update to latest on 2.36
ltp: Remove -mfpmath=sse on x86-64 too
apr: Cache configure tests which use AC_TRY_RUN
rust: Fix build failure on riscv32
ncurses: Fix configure tests for exit and mbstate_t
rust-llvm: Update to matching LLVM_VERSION from rust-source
librepo: Fix build on musl
rsync: Turn on -pedantic-errors at the end of 'configure'
ccache: Upgrade to 4.6.2
xmlto: Update to use upstream tip of trunk
Konrad Weihmann (1):
python3: disable user site-pkg for native target
Lee Chee Yang (1):
migration guides: add release notes for 4.0.3
Luca Ceresoli (1):
libmnl: remove unneeded SRC_URI 'name' option
Markus Volk (2):
connman: add PACKAGECONFIG to support iwd
packagegroup-base.bb: add a configure option to set the wireless-daemon
Martin Jansa (5):
glibc: revert one upstream change to work around broken DEBUG_BUILD build
syslinux: Fix build with glibc-2.36
syslinux: refresh patches with devtool
glibc: fix new upstream build issue with DEBUG_BUILD build
glibc: apply proposed patch from upstream instead of revert
Mateusz Marciniec (2):
util-linux: Remove --enable-raw from EXTRA_OECONF
util-linux: Improve check for magic in configure.ac
Michael Halstead (1):
uninative: Upgrade to 3.7 to work with glibc 2.36
Michael Opdenacker (1):
dev-manual: use proper note directive
Mingli Yu (1):
bitbake: fetch: use BPN instead
Neil Horman (1):
bitbake: Fix npm to use https rather than http
Paul Eggleton (1):
relocate_sdk.py: ensure interpreter size error causes relocation to fail
Pavel Zhukov (6):
package_rpm: Do not replace square brackets in %files
selftest: Add regression test for rpm filesnames
parselogs: Ignore xf86OpenConsole error
bitbake: gitsm: Error out if submodule refers to parent repo
bitbake: tests: Add Timeout class
bitbake: tests: Add test for possible gitsm deadlock
Peter Bergin (3):
rust-cross-canadian: rename shell variables for easier appends
packagegroup-rust-cross-canadian: add native compiler environment
oeqa/sdk: extend rust test to also use a build script
Peter Marko (1):
create-spdx: handle links to inaccessible locations
Quentin Schulz (3):
docs: conf.py: update yocto_git base URL
docs: README: add TeX font package required for building PDF
docs: ref-manual: system-requirements: add missing packages
Randy MacLeod (1):
rust: update from 1.62.1 to 1.63.0
Rasmus Villemoes (1):
bitbake.conf: set BB_DEFAULT_UMASK using ??=
Richard Purdie (85):
oeqa/selftest/sstate: Ensure tests are deterministic
nativesdk: Clear TUNE_FEATURES
populate_sdk_base: Disable rust SDK for MIPS n32
selftest/reproducible: Exclude rust/rust-dbg for now until we can fix
conf/distro/no-static-libs: Allow static musl for rust
rust-target-config: Add mips n32 target information
rust-common: Add CXXFLAGS
rust-common: Drop export directive from wrappers
rust-common: Rework wrappers to handle musl
rust: Work around reproducibility issues
rust: Switch to use RUST_XXX_SYS consistently
rust.inc: Rename variables to make code clearer
rust.inc: Fix cross build llvm-config handling
rust/mesa: Drop obsolete YOCTO_ALTERNATE_MULTILIB_NAME
rust-target-config: Show clear error when target isn't defined
rust: Generate per recipe target configuration files
rust-common/rust: Improve bootstrap BUILD_SYS handling
cargo_common: Handle build SYS as well as HOST/TARGET
rust-llvm: Enable nativesdk variant
rust.inc: Fix for cross compilation configuration
rust-common: Update to match cross targets
rust-target-config: Make target workaround generic
rust-common: Simplify libc handling
cargo: Drop cross-canadian variant and fix/use nativesdk
rust-common: Set rustlibdir to match target expectation
rust-cross-canadian: Simplify and fix
rust: Drop cross/crosssdk
rust: Enable nativesdk and target builds + replace rust-tools-cross-canadian
rust: Fix musl builds
rust: Ensure buildpaths are handled in debug symbols correctly
rust: Update README
selftest/wic: Tweak test case to not depend on kernel size
bitbake: runqueue: Ensure deferred tasks are sorted by multiconfig
bitbake: runqueue: Improve deadlock warning messages
bitbake: runqueue: Drop deadlock breaking force fail
rust-common: Remove conflict with utils create_wrapper
kern-devsrc: Drop auto.conf creation
cargo: Work around host system library conflicts
rust-cross-canadian: Use shell from SDK, not the host
buildhistory: Only use image-artifact-names as an image class
rust: Remove unneeded RUST_TARGETGENS settings
meta-skeleton/hello-mod: Switch to SPDX-License-Identifier
perf: Fix reproducibility issues with 5.19 onwards
selftest/runtime_test/incompatible_lic: Use IMAGE_CLASSES for testimage
testexport: Fix to work as an image class
testexport: Use IMAGE_CLASSES for testimage
selftest/runtime_test: Use testexport in IMAGE_CLASSES, not globally
bitbake: BBHandler: Allow earlier exit for classes not found
bitbake: BBHandler: Make inherit calls more directly
bitbake: bitbake: Add copyright headers where missing
bitbake: BBHandler/cooker: Implement recipe and global classes
classes: Add copyright statements to files without one
scripts: Add copyright statements to files without one
classes: Add SPDX license identifiers
lib: Add copyright statements to files without one
insane: Update to allow for class layout changes
classes: Update classes to match new bitbake class scope functionality
recipetool: Update for class changes
package: Switch debug source handling to use prefix map
libgcc/gcc-runtime: Improve source reference handling
bitbake.conf: Handle S and B separately for debug mapping
python3-cython: Update code to match debug path changes
gcc-cross: Fix relative links
gcc: Resolve relative prefix-map filenames
gcc: Add a patch to avoid hardcoded paths in libgcc on powerpc
gcc: Update patch status to submitted for two patches
valgrind: Disable drd/tests/std_thread2 ptest
valgrind: Update to match debug file layout changes
skeleton/service: Ensure debug path handling works as intended
distrooverrides: Move back to classes whilst it's usage is clarified
vim: Upgrade 9.0.0115 -> 9.0.0242
icu: Drop binconfig support (icu-config)
libtirpc: Mark CVE-2021-46828 as resolved
bitbake: runqueue: Change pressure file warning to a note
rust-target-config: Drop has-elf-tls option
llvm: Add llvm-config wrapper to improve flags handling
mesa: Rework llvm handling
rust-target-config: Fix qemuppc target cpu option
rust: Fix crossbeam-utils for arches without atomics
pseudo: Update to include recent upstream minor fixes
bitbake: Revert "fetch: use BPN instead"
vim: Upgrade 9.0.0242 -> 9.0.0341
gcc-multilib-config: Fix i686 toolchain relocation issues
kernel: Always set CC and LD for the kernel build
kernel: Use consistent make flags for menuconfig
Robert Joslyn (1):
curl: Update to 7.85.0
Ross Burton (9):
oeqa/qemurunner: add run_serial() comment
oeqa/commands: add support for running cross tools to runCmd
oeqa/selftest: rewrite gdbserver test
libxml2: wrap xmllint to use the correct XML catalogues
oeqa/selftest: add test for debuginfod
libgcrypt: remove obsolete pkgconfig install
libgcrypt: remove obsolete patch
libgcrypt: rewrite ptest
cve-check: close cursors as soon as possible
Sakib Sajal (2):
qemu: fix CVE-2021-3507
qemu: fix CVE-2022-0216
Shubham Kulkarni (1):
sanity: add a comment to ensure CONNECTIVITY_CHECK_URIS is correct
Simone Weiss (1):
json-c: Add ptest for json-c
Sundeep KOKKONDA (1):
glibc : stable 2.35 branch updates
Thomas Roos (1):
oeqa devtool: Add tests to cover devtool handling of various git URL styles
Tom Hochstein (1):
piglit: Add PACKAGECONFIG for glx and opencl
Tom Rini (1):
qemux86-64: Allow higher tunes
Ulrich Ölmann (1):
scripts/runqemu.README: fix typos and trailing whitespaces
William A. Kennington III (1):
image_types: Set SOURCE_DATE_EPOCH for squashfs
Yang Xu (1):
insane.bbclass: Skip patches not in oe-core by full path
Yogesh Tyagi (1):
gdbserver : add selftest
Yongxin Liu (1):
grub2: fix several CVEs
wangmy (19):
msmtp: upgrade 1.8.20 -> 1.8.22
bind: upgrade 9.18.5 -> 9.18.6
btrfs-tools: upgrade 5.18.1 -> 5.19
libdnf: upgrade 0.67.0 -> 0.68.0
librepo: upgrade 1.14.3 -> 1.14.4
pkgconf: upgrade 1.9.2 -> 1.9.3
python3-pygments: upgrade 2.12.0 -> 2.13.0
ethtool: upgrade 5.18 -> 5.19
librsvg: upgrade 2.54.4 -> 2.54.5
libtasn1: upgrade 4.18.0 -> 4.19.0
liburcu: upgrade 0.13.1 -> 0.13.2
libwpe: upgrade 1.12.2 -> 1.12.3
lttng-tools: upgrade 2.13.7 -> 2.13.8
lttng-ust: upgrade 2.13.3 -> 2.13.4
libatomic-ops: upgrade 7.6.12 -> 7.6.14
lz4: upgrade 1.9.3 -> 1.9.4
python3-hatchling: upgrade 1.8.0 -> 1.8.1
python3-urllib3: upgrade 1.26.11 -> 1.26.12
repo: upgrade 2.28 -> 2.29.1
meta-arm: 20a629180c..52f07a4b0b:
Anton Antonov (11):
arm/optee-os: backport RWX permission error patch
work around for too few arguments to function init_disassemble_info() error
arm/optee-os: backport linker warning patches
arm/tf-a-tests: work around RWX permission error on segment
Recipes for Trusted Services dependencies.
Recipes for Trusted Services Secure Partitions
ARM-FFA kernel drivers and kernel configs for Trusted Services
Trusted Services test/demo NWd tools
psa-api-tests for Trusted Services
Include Trusted Services SPs into optee-os image
Define qemuarm64-secureboot-ts CI pipeline and include it into meta-arm
Gowtham Suresh Kumar (2):
arm-bsp/secure-partitions: fix SMM gateway bug for EFI GetVariable()
arm-bsp/u-boot: drop EFI GetVariable() workarounds patches
Jon Mason (11):
arm-bsp/fvp-base-arm32: Update kernel patch for v5.19
arm/qemuarm64-secureboot: remove tfa memory patch
arm/linux-yocto: remove optee num pages kernel config variable
arm-bsp/juno: drop scmi patch
arm/qemuarm-secureboot: remove vmalloc from QB_KERNEL_CMDLINE_APPEND
arm/fvp: use image-artifact-names as an image class
atp/atp: drop package inherits
arm/optee: Update to 3.18
arm-bsp/fvp-base: set preferred kernel to 5.15
arm/arm-bsp: Add yocto-kernel-cache bluetooth support
arm-bsp/corstone1000: use compressed kernel image
Khem Raj (2):
gator-daemon: Define _GNU_SOURCE feature test macro
optee-os: Add section attribute parameters when clang is used
Peter Hoyes (3):
docs: Update FVP_CONSOLES in runfvp documentation
docs: Introduce meta-arm OEQA documentation
arm/oeqa: Make linuxboot test case timeout configurable
Richard Purdie (1):
gem5/gem5-m5ops: Drop uneeded package inherit
Ross Burton (2):
arm/trusted-firmware-a: remove redundant patches
arm/trusted-firmware-a: work around RWX permission error on segment
Rui Miguel Silva (2):
arm-bsp:corstone500: rebase u-boot patches on v2022.07
arm-bsp/corstone1000: rebase u-boot patches on top v2022.07
Vishnu Banavath (3):
arm-bsp/trusted-firmware-a: Bump TF-A version for N1SDP
arm-bsp/optee: add optee-os support for N1SDP target
arm/optee: update optee-client to v3.18
Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I90aa0a94410dd208163af126566d22c77787abc2
diff --git a/poky/meta/classes-global/base.bbclass b/poky/meta/classes-global/base.bbclass
new file mode 100644
index 0000000..8203f54
--- /dev/null
+++ b/poky/meta/classes-global/base.bbclass
@@ -0,0 +1,789 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+BB_DEFAULT_TASK ?= "build"
+CLASSOVERRIDE ?= "class-target"
+
+inherit patch
+inherit staging
+
+inherit mirrors
+inherit utils
+inherit utility-tasks
+inherit logging
+
+OE_EXTRA_IMPORTS ?= ""
+
+OE_IMPORTS += "os sys time oe.path oe.utils oe.types oe.package oe.packagegroup oe.sstatesig oe.lsb oe.cachedpath oe.license oe.qa oe.reproducible oe.rust oe.buildcfg ${OE_EXTRA_IMPORTS}"
+OE_IMPORTS[type] = "list"
+
+PACKAGECONFIG_CONFARGS ??= ""
+
+def oe_import(d):
+ import sys
+
+ bbpath = [os.path.join(dir, "lib") for dir in d.getVar("BBPATH").split(":")]
+ sys.path[0:0] = [dir for dir in bbpath if dir not in sys.path]
+
+ import oe.data
+ for toimport in oe.data.typed_value("OE_IMPORTS", d):
+ try:
+ # Make a python object accessible from the metadata
+ bb.utils._context[toimport.split(".", 1)[0]] = __import__(toimport)
+ except AttributeError as e:
+ bb.error("Error importing OE modules: %s" % str(e))
+ return ""
+
+# We need the oe module name space early (before INHERITs get added)
+OE_IMPORTED := "${@oe_import(d)}"
+
+inherit metadata_scm
+
+def lsb_distro_identifier(d):
+ adjust = d.getVar('LSB_DISTRO_ADJUST')
+ adjust_func = None
+ if adjust:
+ try:
+ adjust_func = globals()[adjust]
+ except KeyError:
+ pass
+ return oe.lsb.distro_identifier(adjust_func)
+
+die() {
+ bbfatal_log "$*"
+}
+
+oe_runmake_call() {
+ bbnote ${MAKE} ${EXTRA_OEMAKE} "$@"
+ ${MAKE} ${EXTRA_OEMAKE} "$@"
+}
+
+oe_runmake() {
+ oe_runmake_call "$@" || die "oe_runmake failed"
+}
+
+
+def get_base_dep(d):
+ if d.getVar('INHIBIT_DEFAULT_DEPS', False):
+ return ""
+ return "${BASE_DEFAULT_DEPS}"
+
+BASE_DEFAULT_DEPS = "virtual/${HOST_PREFIX}gcc virtual/${HOST_PREFIX}compilerlibs virtual/libc"
+
+BASEDEPENDS = ""
+BASEDEPENDS:class-target = "${@get_base_dep(d)}"
+BASEDEPENDS:class-nativesdk = "${@get_base_dep(d)}"
+
+DEPENDS:prepend="${BASEDEPENDS} "
+
+FILESPATH = "${@base_set_filespath(["${FILE_DIRNAME}/${BP}", "${FILE_DIRNAME}/${BPN}", "${FILE_DIRNAME}/files"], d)}"
+# THISDIR only works properly with imediate expansion as it has to run
+# in the context of the location its used (:=)
+THISDIR = "${@os.path.dirname(d.getVar('FILE'))}"
+
+def extra_path_elements(d):
+ path = ""
+ elements = (d.getVar('EXTRANATIVEPATH') or "").split()
+ for e in elements:
+ path = path + "${STAGING_BINDIR_NATIVE}/" + e + ":"
+ return path
+
+PATH:prepend = "${@extra_path_elements(d)}"
+
+def get_lic_checksum_file_list(d):
+ filelist = []
+ lic_files = d.getVar("LIC_FILES_CHKSUM") or ''
+ tmpdir = d.getVar("TMPDIR")
+ s = d.getVar("S")
+ b = d.getVar("B")
+ workdir = d.getVar("WORKDIR")
+
+ urls = lic_files.split()
+ for url in urls:
+ # We only care about items that are absolute paths since
+ # any others should be covered by SRC_URI.
+ try:
+ (method, host, path, user, pswd, parm) = bb.fetch.decodeurl(url)
+ if method != "file" or not path:
+ raise bb.fetch.MalformedUrl(url)
+
+ if path[0] == '/':
+ if path.startswith((tmpdir, s, b, workdir)):
+ continue
+ filelist.append(path + ":" + str(os.path.exists(path)))
+ except bb.fetch.MalformedUrl:
+ bb.fatal(d.getVar('PN') + ": LIC_FILES_CHKSUM contains an invalid URL: " + url)
+ return " ".join(filelist)
+
+def setup_hosttools_dir(dest, toolsvar, d, fatal=True):
+ tools = d.getVar(toolsvar).split()
+ origbbenv = d.getVar("BB_ORIGENV", False)
+ path = origbbenv.getVar("PATH")
+ # Need to ignore our own scripts directories to avoid circular links
+ for p in path.split(":"):
+ if p.endswith("/scripts"):
+ path = path.replace(p, "/ignoreme")
+ bb.utils.mkdirhier(dest)
+ notfound = []
+ for tool in tools:
+ desttool = os.path.join(dest, tool)
+ if not os.path.exists(desttool):
+ # clean up dead symlink
+ if os.path.islink(desttool):
+ os.unlink(desttool)
+ srctool = bb.utils.which(path, tool, executable=True)
+ # gcc/g++ may link to ccache on some hosts, e.g.,
+ # /usr/local/bin/ccache/gcc -> /usr/bin/ccache, then which(gcc)
+ # would return /usr/local/bin/ccache/gcc, but what we need is
+ # /usr/bin/gcc, this code can check and fix that.
+ if "ccache" in srctool:
+ srctool = bb.utils.which(path, tool, executable=True, direction=1)
+ if srctool:
+ os.symlink(srctool, desttool)
+ else:
+ notfound.append(tool)
+
+ if notfound and fatal:
+ bb.fatal("The following required tools (as specified by HOSTTOOLS) appear to be unavailable in PATH, please install them in order to proceed:\n %s" % " ".join(notfound))
+
+addtask fetch
+do_fetch[dirs] = "${DL_DIR}"
+do_fetch[file-checksums] = "${@bb.fetch.get_checksum_file_list(d)}"
+do_fetch[file-checksums] += " ${@get_lic_checksum_file_list(d)}"
+do_fetch[vardeps] += "SRCREV"
+do_fetch[network] = "1"
+python base_do_fetch() {
+
+ src_uri = (d.getVar('SRC_URI') or "").split()
+ if not src_uri:
+ return
+
+ try:
+ fetcher = bb.fetch2.Fetch(src_uri, d)
+ fetcher.download()
+ except bb.fetch2.BBFetchException as e:
+ bb.fatal("Bitbake Fetcher Error: " + repr(e))
+}
+
+addtask unpack after do_fetch
+do_unpack[dirs] = "${WORKDIR}"
+
+do_unpack[cleandirs] = "${@d.getVar('S') if os.path.normpath(d.getVar('S')) != os.path.normpath(d.getVar('WORKDIR')) else os.path.join('${S}', 'patches')}"
+
+python base_do_unpack() {
+ src_uri = (d.getVar('SRC_URI') or "").split()
+ if not src_uri:
+ return
+
+ try:
+ fetcher = bb.fetch2.Fetch(src_uri, d)
+ fetcher.unpack(d.getVar('WORKDIR'))
+ except bb.fetch2.BBFetchException as e:
+ bb.fatal("Bitbake Fetcher Error: " + repr(e))
+}
+
+SSTATETASKS += "do_deploy_source_date_epoch"
+
+do_deploy_source_date_epoch () {
+ mkdir -p ${SDE_DEPLOYDIR}
+ if [ -e ${SDE_FILE} ]; then
+ echo "Deploying SDE from ${SDE_FILE} -> ${SDE_DEPLOYDIR}."
+ cp -p ${SDE_FILE} ${SDE_DEPLOYDIR}/__source_date_epoch.txt
+ else
+ echo "${SDE_FILE} not found!"
+ fi
+}
+
+python do_deploy_source_date_epoch_setscene () {
+ sstate_setscene(d)
+ bb.utils.mkdirhier(d.getVar('SDE_DIR'))
+ sde_file = os.path.join(d.getVar('SDE_DEPLOYDIR'), '__source_date_epoch.txt')
+ if os.path.exists(sde_file):
+ target = d.getVar('SDE_FILE')
+ bb.debug(1, "Moving setscene SDE file %s -> %s" % (sde_file, target))
+ bb.utils.rename(sde_file, target)
+ else:
+ bb.debug(1, "%s not found!" % sde_file)
+}
+
+do_deploy_source_date_epoch[dirs] = "${SDE_DEPLOYDIR}"
+do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DEPLOYDIR}"
+addtask do_deploy_source_date_epoch_setscene
+addtask do_deploy_source_date_epoch before do_configure after do_patch
+
+python create_source_date_epoch_stamp() {
+ # Version: 1
+ source_date_epoch = oe.reproducible.get_source_date_epoch(d, d.getVar('S'))
+ oe.reproducible.epochfile_write(source_date_epoch, d.getVar('SDE_FILE'), d)
+}
+do_unpack[postfuncs] += "create_source_date_epoch_stamp"
+
+def get_source_date_epoch_value(d):
+ return oe.reproducible.epochfile_read(d.getVar('SDE_FILE'), d)
+
+def get_layers_branch_rev(d):
+ revisions = oe.buildcfg.get_layer_revisions(d)
+ layers_branch_rev = ["%-20s = \"%s:%s\"" % (r[1], r[2], r[3]) for r in revisions]
+ i = len(layers_branch_rev)-1
+ p1 = layers_branch_rev[i].find("=")
+ s1 = layers_branch_rev[i][p1:]
+ while i > 0:
+ p2 = layers_branch_rev[i-1].find("=")
+ s2= layers_branch_rev[i-1][p2:]
+ if s1 == s2:
+ layers_branch_rev[i-1] = layers_branch_rev[i-1][0:p2]
+ i -= 1
+ else:
+ i -= 1
+ p1 = layers_branch_rev[i].find("=")
+ s1= layers_branch_rev[i][p1:]
+ return layers_branch_rev
+
+
+BUILDCFG_FUNCS ??= "buildcfg_vars get_layers_branch_rev buildcfg_neededvars"
+BUILDCFG_FUNCS[type] = "list"
+
+def buildcfg_vars(d):
+ statusvars = oe.data.typed_value('BUILDCFG_VARS', d)
+ for var in statusvars:
+ value = d.getVar(var)
+ if value is not None:
+ yield '%-20s = "%s"' % (var, value)
+
+def buildcfg_neededvars(d):
+ needed_vars = oe.data.typed_value("BUILDCFG_NEEDEDVARS", d)
+ pesteruser = []
+ for v in needed_vars:
+ val = d.getVar(v)
+ if not val or val == 'INVALID':
+ pesteruser.append(v)
+
+ if pesteruser:
+ bb.fatal('The following variable(s) were not set: %s\nPlease set them directly, or choose a MACHINE or DISTRO that sets them.' % ', '.join(pesteruser))
+
+addhandler base_eventhandler
+base_eventhandler[eventmask] = "bb.event.ConfigParsed bb.event.MultiConfigParsed bb.event.BuildStarted bb.event.RecipePreFinalise bb.event.RecipeParsed"
+python base_eventhandler() {
+ import bb.runqueue
+
+ if isinstance(e, bb.event.ConfigParsed):
+ if not d.getVar("NATIVELSBSTRING", False):
+ d.setVar("NATIVELSBSTRING", lsb_distro_identifier(d))
+ d.setVar("ORIGNATIVELSBSTRING", d.getVar("NATIVELSBSTRING", False))
+ d.setVar('BB_VERSION', bb.__version__)
+
+ # There might be no bb.event.ConfigParsed event if bitbake server is
+ # running, so check bb.event.BuildStarted too to make sure ${HOSTTOOLS_DIR}
+ # exists.
+ if isinstance(e, bb.event.ConfigParsed) or \
+ (isinstance(e, bb.event.BuildStarted) and not os.path.exists(d.getVar('HOSTTOOLS_DIR'))):
+ # Works with the line in layer.conf which changes PATH to point here
+ setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS', d)
+ setup_hosttools_dir(d.getVar('HOSTTOOLS_DIR'), 'HOSTTOOLS_NONFATAL', d, fatal=False)
+
+ if isinstance(e, bb.event.MultiConfigParsed):
+ # We need to expand SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS in each of the multiconfig data stores
+ # own contexts so the variables get expanded correctly for that arch, then inject back into
+ # the main data store.
+ deps = []
+ for config in e.mcdata:
+ deps.append(e.mcdata[config].getVar("SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS"))
+ deps = " ".join(deps)
+ e.mcdata[''].setVar("SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS", deps)
+
+ if isinstance(e, bb.event.BuildStarted):
+ localdata = bb.data.createCopy(d)
+ statuslines = []
+ for func in oe.data.typed_value('BUILDCFG_FUNCS', localdata):
+ g = globals()
+ if func not in g:
+ bb.warn("Build configuration function '%s' does not exist" % func)
+ else:
+ flines = g[func](localdata)
+ if flines:
+ statuslines.extend(flines)
+
+ statusheader = d.getVar('BUILDCFG_HEADER')
+ if statusheader:
+ bb.plain('\n%s\n%s\n' % (statusheader, '\n'.join(statuslines)))
+
+ # This code is to silence warnings where the SDK variables overwrite the
+ # target ones and we'd see dulpicate key names overwriting each other
+ # for various PREFERRED_PROVIDERS
+ if isinstance(e, bb.event.RecipePreFinalise):
+ if d.getVar("TARGET_PREFIX") == d.getVar("SDK_PREFIX"):
+ d.delVar("PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}binutils")
+ d.delVar("PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}gcc")
+ d.delVar("PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}g++")
+ d.delVar("PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}compilerlibs")
+
+ if isinstance(e, bb.event.RecipeParsed):
+ #
+ # If we have multiple providers of virtual/X and a PREFERRED_PROVIDER_virtual/X is set
+ # skip parsing for all the other providers which will mean they get uninstalled from the
+ # sysroot since they're now "unreachable". This makes switching virtual/kernel work in
+ # particular.
+ #
+ pn = d.getVar('PN')
+ source_mirror_fetch = d.getVar('SOURCE_MIRROR_FETCH', False)
+ if not source_mirror_fetch:
+ provs = (d.getVar("PROVIDES") or "").split()
+ multiprovidersallowed = (d.getVar("BB_MULTI_PROVIDER_ALLOWED") or "").split()
+ for p in provs:
+ if p.startswith("virtual/") and p not in multiprovidersallowed:
+ profprov = d.getVar("PREFERRED_PROVIDER_" + p)
+ if profprov and pn != profprov:
+ raise bb.parse.SkipRecipe("PREFERRED_PROVIDER_%s set to %s, not %s" % (p, profprov, pn))
+}
+
+CONFIGURESTAMPFILE = "${WORKDIR}/configure.sstate"
+CLEANBROKEN = "0"
+
+addtask configure after do_patch
+do_configure[dirs] = "${B}"
+base_do_configure() {
+ if [ -n "${CONFIGURESTAMPFILE}" -a -e "${CONFIGURESTAMPFILE}" ]; then
+ if [ "`cat ${CONFIGURESTAMPFILE}`" != "${BB_TASKHASH}" ]; then
+ cd ${B}
+ if [ "${CLEANBROKEN}" != "1" -a \( -e Makefile -o -e makefile -o -e GNUmakefile \) ]; then
+ oe_runmake clean
+ fi
+ # -ignore_readdir_race does not work correctly with -delete;
+ # use xargs to avoid spurious build failures
+ find ${B} -ignore_readdir_race -name \*.la -type f -print0 | xargs -0 rm -f
+ fi
+ fi
+ if [ -n "${CONFIGURESTAMPFILE}" ]; then
+ mkdir -p `dirname ${CONFIGURESTAMPFILE}`
+ echo ${BB_TASKHASH} > ${CONFIGURESTAMPFILE}
+ fi
+}
+
+addtask compile after do_configure
+do_compile[dirs] = "${B}"
+base_do_compile() {
+ if [ -e Makefile -o -e makefile -o -e GNUmakefile ]; then
+ oe_runmake || die "make failed"
+ else
+ bbnote "nothing to compile"
+ fi
+}
+
+addtask install after do_compile
+do_install[dirs] = "${B}"
+# Remove and re-create ${D} so that is it guaranteed to be empty
+do_install[cleandirs] = "${D}"
+
+base_do_install() {
+ :
+}
+
+base_do_package() {
+ :
+}
+
+addtask build after do_populate_sysroot
+do_build[noexec] = "1"
+do_build[recrdeptask] += "do_deploy"
+do_build () {
+ :
+}
+
+def set_packagetriplet(d):
+ archs = []
+ tos = []
+ tvs = []
+
+ archs.append(d.getVar("PACKAGE_ARCHS").split())
+ tos.append(d.getVar("TARGET_OS"))
+ tvs.append(d.getVar("TARGET_VENDOR"))
+
+ def settriplet(d, varname, archs, tos, tvs):
+ triplets = []
+ for i in range(len(archs)):
+ for arch in archs[i]:
+ triplets.append(arch + tvs[i] + "-" + tos[i])
+ triplets.reverse()
+ d.setVar(varname, " ".join(triplets))
+
+ settriplet(d, "PKGTRIPLETS", archs, tos, tvs)
+
+ variants = d.getVar("MULTILIB_VARIANTS") or ""
+ for item in variants.split():
+ localdata = bb.data.createCopy(d)
+ overrides = localdata.getVar("OVERRIDES", False) + ":virtclass-multilib-" + item
+ localdata.setVar("OVERRIDES", overrides)
+
+ archs.append(localdata.getVar("PACKAGE_ARCHS").split())
+ tos.append(localdata.getVar("TARGET_OS"))
+ tvs.append(localdata.getVar("TARGET_VENDOR"))
+
+ settriplet(d, "PKGMLTRIPLETS", archs, tos, tvs)
+
+python () {
+ import string, re
+
+ # Handle backfilling
+ oe.utils.features_backfill("DISTRO_FEATURES", d)
+ oe.utils.features_backfill("MACHINE_FEATURES", d)
+
+ if d.getVar("S")[-1] == '/':
+ bb.warn("Recipe %s sets S variable with trailing slash '%s', remove it" % (d.getVar("PN"), d.getVar("S")))
+ if d.getVar("B")[-1] == '/':
+ bb.warn("Recipe %s sets B variable with trailing slash '%s', remove it" % (d.getVar("PN"), d.getVar("B")))
+
+ if os.path.normpath(d.getVar("WORKDIR")) != os.path.normpath(d.getVar("S")):
+ d.appendVar("PSEUDO_IGNORE_PATHS", ",${S}")
+ if os.path.normpath(d.getVar("WORKDIR")) != os.path.normpath(d.getVar("B")):
+ d.appendVar("PSEUDO_IGNORE_PATHS", ",${B}")
+
+ # To add a recipe to the skip list , set:
+ # SKIP_RECIPE[pn] = "message"
+ pn = d.getVar('PN')
+ skip_msg = d.getVarFlag('SKIP_RECIPE', pn)
+ if skip_msg:
+ bb.debug(1, "Skipping %s %s" % (pn, skip_msg))
+ raise bb.parse.SkipRecipe("Recipe will be skipped because: %s" % (skip_msg))
+
+ # Handle PACKAGECONFIG
+ #
+ # These take the form:
+ #
+ # PACKAGECONFIG ??= "<default options>"
+ # PACKAGECONFIG[foo] = "--enable-foo,--disable-foo,foo_depends,foo_runtime_depends,foo_runtime_recommends,foo_conflict_packageconfig"
+ pkgconfigflags = d.getVarFlags("PACKAGECONFIG") or {}
+ if pkgconfigflags:
+ pkgconfig = (d.getVar('PACKAGECONFIG') or "").split()
+ pn = d.getVar("PN")
+
+ mlprefix = d.getVar("MLPREFIX")
+
+ def expandFilter(appends, extension, prefix):
+ appends = bb.utils.explode_deps(d.expand(" ".join(appends)))
+ newappends = []
+ for a in appends:
+ if a.endswith("-native") or ("-cross-" in a):
+ newappends.append(a)
+ elif a.startswith("virtual/"):
+ subs = a.split("/", 1)[1]
+ if subs.startswith(prefix):
+ newappends.append(a + extension)
+ else:
+ newappends.append("virtual/" + prefix + subs + extension)
+ else:
+ if a.startswith(prefix):
+ newappends.append(a + extension)
+ else:
+ newappends.append(prefix + a + extension)
+ return newappends
+
+ def appendVar(varname, appends):
+ if not appends:
+ return
+ if varname.find("DEPENDS") != -1:
+ if bb.data.inherits_class('nativesdk', d) or bb.data.inherits_class('cross-canadian', d) :
+ appends = expandFilter(appends, "", "nativesdk-")
+ elif bb.data.inherits_class('native', d):
+ appends = expandFilter(appends, "-native", "")
+ elif mlprefix:
+ appends = expandFilter(appends, "", mlprefix)
+ varname = d.expand(varname)
+ d.appendVar(varname, " " + " ".join(appends))
+
+ extradeps = []
+ extrardeps = []
+ extrarrecs = []
+ extraconf = []
+ for flag, flagval in sorted(pkgconfigflags.items()):
+ items = flagval.split(",")
+ num = len(items)
+ if num > 6:
+ bb.error("%s: PACKAGECONFIG[%s] Only enable,disable,depend,rdepend,rrecommend,conflict_packageconfig can be specified!"
+ % (d.getVar('PN'), flag))
+
+ if flag in pkgconfig:
+ if num >= 3 and items[2]:
+ extradeps.append(items[2])
+ if num >= 4 and items[3]:
+ extrardeps.append(items[3])
+ if num >= 5 and items[4]:
+ extrarrecs.append(items[4])
+ if num >= 1 and items[0]:
+ extraconf.append(items[0])
+ elif num >= 2 and items[1]:
+ extraconf.append(items[1])
+
+ if num >= 6 and items[5]:
+ conflicts = set(items[5].split())
+ invalid = conflicts.difference(set(pkgconfigflags.keys()))
+ if invalid:
+ bb.error("%s: PACKAGECONFIG[%s] Invalid conflict package config%s '%s' specified."
+ % (d.getVar('PN'), flag, 's' if len(invalid) > 1 else '', ' '.join(invalid)))
+
+ if flag in pkgconfig:
+ intersec = conflicts.intersection(set(pkgconfig))
+ if intersec:
+ bb.fatal("%s: PACKAGECONFIG[%s] Conflict package config%s '%s' set in PACKAGECONFIG."
+ % (d.getVar('PN'), flag, 's' if len(intersec) > 1 else '', ' '.join(intersec)))
+
+ appendVar('DEPENDS', extradeps)
+ appendVar('RDEPENDS:${PN}', extrardeps)
+ appendVar('RRECOMMENDS:${PN}', extrarrecs)
+ appendVar('PACKAGECONFIG_CONFARGS', extraconf)
+
+ pn = d.getVar('PN')
+ license = d.getVar('LICENSE')
+ if license == "INVALID" and pn != "defaultpkgname":
+ bb.fatal('This recipe does not have the LICENSE field set (%s)' % pn)
+
+ if bb.data.inherits_class('license', d):
+ check_license_format(d)
+ unmatched_license_flags = check_license_flags(d)
+ if unmatched_license_flags:
+ if len(unmatched_license_flags) == 1:
+ message = "because it has a restricted license '{0}'. Which is not listed in LICENSE_FLAGS_ACCEPTED".format(unmatched_license_flags[0])
+ else:
+ message = "because it has restricted licenses {0}. Which are not listed in LICENSE_FLAGS_ACCEPTED".format(
+ ", ".join("'{0}'".format(f) for f in unmatched_license_flags))
+ bb.debug(1, "Skipping %s %s" % (pn, message))
+ raise bb.parse.SkipRecipe(message)
+
+ # If we're building a target package we need to use fakeroot (pseudo)
+ # in order to capture permissions, owners, groups and special files
+ if not bb.data.inherits_class('native', d) and not bb.data.inherits_class('cross', d):
+ d.appendVarFlag('do_prepare_recipe_sysroot', 'depends', ' virtual/fakeroot-native:do_populate_sysroot')
+ d.appendVarFlag('do_install', 'depends', ' virtual/fakeroot-native:do_populate_sysroot')
+ d.setVarFlag('do_install', 'fakeroot', '1')
+ d.appendVarFlag('do_package', 'depends', ' virtual/fakeroot-native:do_populate_sysroot')
+ d.setVarFlag('do_package', 'fakeroot', '1')
+ d.setVarFlag('do_package_setscene', 'fakeroot', '1')
+ d.appendVarFlag('do_package_setscene', 'depends', ' virtual/fakeroot-native:do_populate_sysroot')
+ d.setVarFlag('do_devshell', 'fakeroot', '1')
+ d.appendVarFlag('do_devshell', 'depends', ' virtual/fakeroot-native:do_populate_sysroot')
+
+ need_machine = d.getVar('COMPATIBLE_MACHINE')
+ if need_machine and not d.getVar('PARSE_ALL_RECIPES', False):
+ import re
+ compat_machines = (d.getVar('MACHINEOVERRIDES') or "").split(":")
+ for m in compat_machines:
+ if re.match(need_machine, m):
+ break
+ else:
+ raise bb.parse.SkipRecipe("incompatible with machine %s (not in COMPATIBLE_MACHINE)" % d.getVar('MACHINE'))
+
+ source_mirror_fetch = d.getVar('SOURCE_MIRROR_FETCH', False) or d.getVar('PARSE_ALL_RECIPES', False)
+ if not source_mirror_fetch:
+ need_host = d.getVar('COMPATIBLE_HOST')
+ if need_host:
+ import re
+ this_host = d.getVar('HOST_SYS')
+ if not re.match(need_host, this_host):
+ raise bb.parse.SkipRecipe("incompatible with host %s (not in COMPATIBLE_HOST)" % this_host)
+
+ bad_licenses = (d.getVar('INCOMPATIBLE_LICENSE') or "").split()
+
+ check_license = False if pn.startswith("nativesdk-") else True
+ for t in ["-native", "-cross-${TARGET_ARCH}", "-cross-initial-${TARGET_ARCH}",
+ "-crosssdk-${SDK_SYS}", "-crosssdk-initial-${SDK_SYS}",
+ "-cross-canadian-${TRANSLATED_TARGET_ARCH}"]:
+ if pn.endswith(d.expand(t)):
+ check_license = False
+ if pn.startswith("gcc-source-"):
+ check_license = False
+
+ if check_license and bad_licenses:
+ bad_licenses = expand_wildcard_licenses(d, bad_licenses)
+
+ exceptions = (d.getVar("INCOMPATIBLE_LICENSE_EXCEPTIONS") or "").split()
+
+ for lic_exception in exceptions:
+ if ":" in lic_exception:
+ lic_exception = lic_exception.split(":")[1]
+ if lic_exception in oe.license.obsolete_license_list():
+ bb.fatal("Obsolete license %s used in INCOMPATIBLE_LICENSE_EXCEPTIONS" % lic_exception)
+
+ pkgs = d.getVar('PACKAGES').split()
+ skipped_pkgs = {}
+ unskipped_pkgs = []
+ for pkg in pkgs:
+ remaining_bad_licenses = oe.license.apply_pkg_license_exception(pkg, bad_licenses, exceptions)
+
+ incompatible_lic = incompatible_license(d, remaining_bad_licenses, pkg)
+ if incompatible_lic:
+ skipped_pkgs[pkg] = incompatible_lic
+ else:
+ unskipped_pkgs.append(pkg)
+
+ if unskipped_pkgs:
+ for pkg in skipped_pkgs:
+ bb.debug(1, "Skipping the package %s at do_rootfs because of incompatible license(s): %s" % (pkg, ' '.join(skipped_pkgs[pkg])))
+ d.setVar('_exclude_incompatible-' + pkg, ' '.join(skipped_pkgs[pkg]))
+ for pkg in unskipped_pkgs:
+ bb.debug(1, "Including the package %s" % pkg)
+ else:
+ incompatible_lic = incompatible_license(d, bad_licenses)
+ for pkg in skipped_pkgs:
+ incompatible_lic += skipped_pkgs[pkg]
+ incompatible_lic = sorted(list(set(incompatible_lic)))
+
+ if incompatible_lic:
+ bb.debug(1, "Skipping recipe %s because of incompatible license(s): %s" % (pn, ' '.join(incompatible_lic)))
+ raise bb.parse.SkipRecipe("it has incompatible license(s): %s" % ' '.join(incompatible_lic))
+
+ needsrcrev = False
+ srcuri = d.getVar('SRC_URI')
+ for uri_string in srcuri.split():
+ uri = bb.fetch.URI(uri_string)
+ # Also check downloadfilename as the URL path might not be useful for sniffing
+ path = uri.params.get("downloadfilename", uri.path)
+
+ # HTTP/FTP use the wget fetcher
+ if uri.scheme in ("http", "https", "ftp"):
+ d.appendVarFlag('do_fetch', 'depends', ' wget-native:do_populate_sysroot')
+
+ # Svn packages should DEPEND on subversion-native
+ if uri.scheme == "svn":
+ needsrcrev = True
+ d.appendVarFlag('do_fetch', 'depends', ' subversion-native:do_populate_sysroot')
+
+ # Git packages should DEPEND on git-native
+ elif uri.scheme in ("git", "gitsm"):
+ needsrcrev = True
+ d.appendVarFlag('do_fetch', 'depends', ' git-native:do_populate_sysroot')
+
+ # Mercurial packages should DEPEND on mercurial-native
+ elif uri.scheme == "hg":
+ needsrcrev = True
+ d.appendVar("EXTRANATIVEPATH", ' python3-native ')
+ d.appendVarFlag('do_fetch', 'depends', ' mercurial-native:do_populate_sysroot')
+
+ # Perforce packages support SRCREV = "${AUTOREV}"
+ elif uri.scheme == "p4":
+ needsrcrev = True
+
+ # OSC packages should DEPEND on osc-native
+ elif uri.scheme == "osc":
+ d.appendVarFlag('do_fetch', 'depends', ' osc-native:do_populate_sysroot')
+
+ elif uri.scheme == "npm":
+ d.appendVarFlag('do_fetch', 'depends', ' nodejs-native:do_populate_sysroot')
+
+ elif uri.scheme == "repo":
+ needsrcrev = True
+ d.appendVarFlag('do_fetch', 'depends', ' repo-native:do_populate_sysroot')
+
+ # *.lz4 should DEPEND on lz4-native for unpacking
+ if path.endswith('.lz4'):
+ d.appendVarFlag('do_unpack', 'depends', ' lz4-native:do_populate_sysroot')
+
+ # *.zst should DEPEND on zstd-native for unpacking
+ elif path.endswith('.zst'):
+ d.appendVarFlag('do_unpack', 'depends', ' zstd-native:do_populate_sysroot')
+
+ # *.lz should DEPEND on lzip-native for unpacking
+ elif path.endswith('.lz'):
+ d.appendVarFlag('do_unpack', 'depends', ' lzip-native:do_populate_sysroot')
+
+ # *.xz should DEPEND on xz-native for unpacking
+ elif path.endswith('.xz') or path.endswith('.txz'):
+ d.appendVarFlag('do_unpack', 'depends', ' xz-native:do_populate_sysroot')
+
+ # .zip should DEPEND on unzip-native for unpacking
+ elif path.endswith('.zip') or path.endswith('.jar'):
+ d.appendVarFlag('do_unpack', 'depends', ' unzip-native:do_populate_sysroot')
+
+ # Some rpm files may be compressed internally using xz (for example, rpms from Fedora)
+ elif path.endswith('.rpm'):
+ d.appendVarFlag('do_unpack', 'depends', ' xz-native:do_populate_sysroot')
+
+ # *.deb should DEPEND on xz-native for unpacking
+ elif path.endswith('.deb'):
+ d.appendVarFlag('do_unpack', 'depends', ' xz-native:do_populate_sysroot')
+
+ if needsrcrev:
+ d.setVar("SRCPV", "${@bb.fetch2.get_srcrev(d)}")
+
+ # Gather all named SRCREVs to add to the sstate hash calculation
+ # This anonymous python snippet is called multiple times so we
+ # need to be careful to not double up the appends here and cause
+ # the base hash to mismatch the task hash
+ for uri in srcuri.split():
+ parm = bb.fetch.decodeurl(uri)[5]
+ uri_names = parm.get("name", "").split(",")
+ for uri_name in filter(None, uri_names):
+ srcrev_name = "SRCREV_{}".format(uri_name)
+ if srcrev_name not in (d.getVarFlag("do_fetch", "vardeps") or "").split():
+ d.appendVarFlag("do_fetch", "vardeps", " {}".format(srcrev_name))
+
+ set_packagetriplet(d)
+
+ # 'multimachine' handling
+ mach_arch = d.getVar('MACHINE_ARCH')
+ pkg_arch = d.getVar('PACKAGE_ARCH')
+
+ if (pkg_arch == mach_arch):
+ # Already machine specific - nothing further to do
+ return
+
+ #
+ # We always try to scan SRC_URI for urls with machine overrides
+ # unless the package sets SRC_URI_OVERRIDES_PACKAGE_ARCH=0
+ #
+ override = d.getVar('SRC_URI_OVERRIDES_PACKAGE_ARCH')
+ if override != '0':
+ paths = []
+ fpaths = (d.getVar('FILESPATH') or '').split(':')
+ machine = d.getVar('MACHINE')
+ for p in fpaths:
+ if os.path.basename(p) == machine and os.path.isdir(p):
+ paths.append(p)
+
+ if paths:
+ for s in srcuri.split():
+ if not s.startswith("file://"):
+ continue
+ fetcher = bb.fetch2.Fetch([s], d)
+ local = fetcher.localpath(s)
+ for mp in paths:
+ if local.startswith(mp):
+ #bb.note("overriding PACKAGE_ARCH from %s to %s for %s" % (pkg_arch, mach_arch, pn))
+ d.setVar('PACKAGE_ARCH', "${MACHINE_ARCH}")
+ return
+
+ packages = d.getVar('PACKAGES').split()
+ for pkg in packages:
+ pkgarch = d.getVar("PACKAGE_ARCH_%s" % pkg)
+
+ # We could look for != PACKAGE_ARCH here but how to choose
+ # if multiple differences are present?
+ # Look through PACKAGE_ARCHS for the priority order?
+ if pkgarch and pkgarch == mach_arch:
+ d.setVar('PACKAGE_ARCH', "${MACHINE_ARCH}")
+ bb.warn("Recipe %s is marked as only being architecture specific but seems to have machine specific packages?! The recipe may as well mark itself as machine specific directly." % d.getVar("PN"))
+}
+
+addtask cleansstate after do_clean
+python do_cleansstate() {
+ sstate_clean_cachefiles(d)
+}
+addtask cleanall after do_cleansstate
+do_cleansstate[nostamp] = "1"
+
+python do_cleanall() {
+ src_uri = (d.getVar('SRC_URI') or "").split()
+ if not src_uri:
+ return
+
+ try:
+ fetcher = bb.fetch2.Fetch(src_uri, d)
+ fetcher.clean()
+ except bb.fetch2.BBFetchException as e:
+ bb.fatal(str(e))
+}
+do_cleanall[nostamp] = "1"
+
+
+EXPORT_FUNCTIONS do_fetch do_unpack do_configure do_compile do_install do_package
diff --git a/poky/meta/classes-global/buildstats.bbclass b/poky/meta/classes-global/buildstats.bbclass
new file mode 100644
index 0000000..f49a67a
--- /dev/null
+++ b/poky/meta/classes-global/buildstats.bbclass
@@ -0,0 +1,302 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+BUILDSTATS_BASE = "${TMPDIR}/buildstats/"
+
+################################################################################
+# Build statistics gathering.
+#
+# The CPU and Time gathering/tracking functions and bbevent inspiration
+# were written by Christopher Larson.
+#
+################################################################################
+
+def get_buildprocess_cputime(pid):
+ with open("/proc/%d/stat" % pid, "r") as f:
+ fields = f.readline().rstrip().split()
+ # 13: utime, 14: stime, 15: cutime, 16: cstime
+ return sum(int(field) for field in fields[13:16])
+
+def get_process_cputime(pid):
+ import resource
+ with open("/proc/%d/stat" % pid, "r") as f:
+ fields = f.readline().rstrip().split()
+ stats = {
+ 'utime' : fields[13],
+ 'stime' : fields[14],
+ 'cutime' : fields[15],
+ 'cstime' : fields[16],
+ }
+ iostats = {}
+ if os.path.isfile("/proc/%d/io" % pid):
+ with open("/proc/%d/io" % pid, "r") as f:
+ while True:
+ i = f.readline().strip()
+ if not i:
+ break
+ if not ":" in i:
+ # one more extra line is appended (empty or containing "0")
+ # most probably due to race condition in kernel while
+ # updating IO stats
+ break
+ i = i.split(": ")
+ iostats[i[0]] = i[1]
+ resources = resource.getrusage(resource.RUSAGE_SELF)
+ childres = resource.getrusage(resource.RUSAGE_CHILDREN)
+ return stats, iostats, resources, childres
+
+def get_cputime():
+ with open("/proc/stat", "r") as f:
+ fields = f.readline().rstrip().split()[1:]
+ return sum(int(field) for field in fields)
+
+def set_timedata(var, d, server_time):
+ d.setVar(var, server_time)
+
+def get_timedata(var, d, end_time):
+ oldtime = d.getVar(var, False)
+ if oldtime is None:
+ return
+ return end_time - oldtime
+
+def set_buildtimedata(var, d):
+ import time
+ time = time.time()
+ cputime = get_cputime()
+ proctime = get_buildprocess_cputime(os.getpid())
+ d.setVar(var, (time, cputime, proctime))
+
+def get_buildtimedata(var, d):
+ import time
+ timedata = d.getVar(var, False)
+ if timedata is None:
+ return
+ oldtime, oldcpu, oldproc = timedata
+ procdiff = get_buildprocess_cputime(os.getpid()) - oldproc
+ cpudiff = get_cputime() - oldcpu
+ end_time = time.time()
+ timediff = end_time - oldtime
+ if cpudiff > 0:
+ cpuperc = float(procdiff) * 100 / cpudiff
+ else:
+ cpuperc = None
+ return timediff, cpuperc
+
+def write_task_data(status, logfile, e, d):
+ with open(os.path.join(logfile), "a") as f:
+ elapsedtime = get_timedata("__timedata_task", d, e.time)
+ if elapsedtime:
+ f.write(d.expand("${PF}: %s\n" % e.task))
+ f.write(d.expand("Elapsed time: %0.2f seconds\n" % elapsedtime))
+ cpu, iostats, resources, childres = get_process_cputime(os.getpid())
+ if cpu:
+ f.write("utime: %s\n" % cpu['utime'])
+ f.write("stime: %s\n" % cpu['stime'])
+ f.write("cutime: %s\n" % cpu['cutime'])
+ f.write("cstime: %s\n" % cpu['cstime'])
+ for i in iostats:
+ f.write("IO %s: %s\n" % (i, iostats[i]))
+ rusages = ["ru_utime", "ru_stime", "ru_maxrss", "ru_minflt", "ru_majflt", "ru_inblock", "ru_oublock", "ru_nvcsw", "ru_nivcsw"]
+ for i in rusages:
+ f.write("rusage %s: %s\n" % (i, getattr(resources, i)))
+ for i in rusages:
+ f.write("Child rusage %s: %s\n" % (i, getattr(childres, i)))
+ if status == "passed":
+ f.write("Status: PASSED \n")
+ else:
+ f.write("Status: FAILED \n")
+ f.write("Ended: %0.2f \n" % e.time)
+
+def write_host_data(logfile, e, d, type):
+ import subprocess, os, datetime
+ # minimum time allowed for each command to run, in seconds
+ time_threshold = 0.5
+ limit = 10
+ # the total number of commands
+ num_cmds = 0
+ msg = ""
+ if type == "interval":
+ # interval at which data will be logged
+ interval = d.getVar("BB_HEARTBEAT_EVENT", False)
+ if interval is None:
+ bb.warn("buildstats: Collecting host data at intervals failed. Set BB_HEARTBEAT_EVENT=\"<interval>\" in conf/local.conf for the interval at which host data will be logged.")
+ d.setVar("BB_LOG_HOST_STAT_ON_INTERVAL", "0")
+ return
+ interval = int(interval)
+ cmds = d.getVar('BB_LOG_HOST_STAT_CMDS_INTERVAL')
+ msg = "Host Stats: Collecting data at %d second intervals.\n" % interval
+ if cmds is None:
+ d.setVar("BB_LOG_HOST_STAT_ON_INTERVAL", "0")
+ bb.warn("buildstats: Collecting host data at intervals failed. Set BB_LOG_HOST_STAT_CMDS_INTERVAL=\"command1 ; command2 ; ... \" in conf/local.conf.")
+ return
+ if type == "failure":
+ cmds = d.getVar('BB_LOG_HOST_STAT_CMDS_FAILURE')
+ msg = "Host Stats: Collecting data on failure.\n"
+ msg += "Failed at task: " + e.task + "\n"
+ if cmds is None:
+ d.setVar("BB_LOG_HOST_STAT_ON_FAILURE", "0")
+ bb.warn("buildstats: Collecting host data on failure failed. Set BB_LOG_HOST_STAT_CMDS_FAILURE=\"command1 ; command2 ; ... \" in conf/local.conf.")
+ return
+ c_san = []
+ for cmd in cmds.split(";"):
+ if len(cmd) == 0:
+ continue
+ num_cmds += 1
+ c_san.append(cmd)
+ if num_cmds == 0:
+ if type == "interval":
+ d.setVar("BB_LOG_HOST_STAT_ON_INTERVAL", "0")
+ if type == "failure":
+ d.setVar("BB_LOG_HOST_STAT_ON_FAILURE", "0")
+ return
+
+ # return if the interval is not enough to run all commands within the specified BB_HEARTBEAT_EVENT interval
+ if type == "interval":
+ limit = interval / num_cmds
+ if limit <= time_threshold:
+ d.setVar("BB_LOG_HOST_STAT_ON_INTERVAL", "0")
+ bb.warn("buildstats: Collecting host data failed. BB_HEARTBEAT_EVENT interval not enough to run the specified commands. Increase value of BB_HEARTBEAT_EVENT in conf/local.conf.")
+ return
+
+ # set the environment variables
+ path = d.getVar("PATH")
+ opath = d.getVar("BB_ORIGENV", False).getVar("PATH")
+ ospath = os.environ['PATH']
+ os.environ['PATH'] = path + ":" + opath + ":" + ospath
+ with open(logfile, "a") as f:
+ f.write("Event Time: %f\nDate: %s\n" % (e.time, datetime.datetime.now()))
+ f.write("%s" % msg)
+ for c in c_san:
+ try:
+ output = subprocess.check_output(c.split(), stderr=subprocess.STDOUT, timeout=limit).decode('utf-8')
+ except (subprocess.CalledProcessError, subprocess.TimeoutExpired, FileNotFoundError) as err:
+ output = "Error running command: %s\n%s\n" % (c, err)
+ f.write("%s\n%s\n" % (c, output))
+ # reset the environment
+ os.environ['PATH'] = ospath
+
+python run_buildstats () {
+ import bb.build
+ import bb.event
+ import time, subprocess, platform
+
+ bn = d.getVar('BUILDNAME')
+ ########################################################################
+ # bitbake fires HeartbeatEvent even before a build has been
+ # triggered, causing BUILDNAME to be None
+ ########################################################################
+ if bn is not None:
+ bsdir = os.path.join(d.getVar('BUILDSTATS_BASE'), bn)
+ taskdir = os.path.join(bsdir, d.getVar('PF'))
+ if isinstance(e, bb.event.HeartbeatEvent) and bb.utils.to_boolean(d.getVar("BB_LOG_HOST_STAT_ON_INTERVAL")):
+ bb.utils.mkdirhier(bsdir)
+ write_host_data(os.path.join(bsdir, "host_stats_interval"), e, d, "interval")
+
+ if isinstance(e, bb.event.BuildStarted):
+ ########################################################################
+ # If the kernel was not configured to provide I/O statistics, issue
+ # a one time warning.
+ ########################################################################
+ if not os.path.isfile("/proc/%d/io" % os.getpid()):
+ bb.warn("The Linux kernel on your build host was not configured to provide process I/O statistics. (CONFIG_TASK_IO_ACCOUNTING is not set)")
+
+ ########################################################################
+ # at first pass make the buildstats hierarchy and then
+ # set the buildname
+ ########################################################################
+ bb.utils.mkdirhier(bsdir)
+ set_buildtimedata("__timedata_build", d)
+ build_time = os.path.join(bsdir, "build_stats")
+ # write start of build into build_time
+ with open(build_time, "a") as f:
+ host_info = platform.uname()
+ f.write("Host Info: ")
+ for x in host_info:
+ if x:
+ f.write(x + " ")
+ f.write("\n")
+ f.write("Build Started: %0.2f \n" % d.getVar('__timedata_build', False)[0])
+
+ elif isinstance(e, bb.event.BuildCompleted):
+ build_time = os.path.join(bsdir, "build_stats")
+ with open(build_time, "a") as f:
+ ########################################################################
+ # Write build statistics for the build
+ ########################################################################
+ timedata = get_buildtimedata("__timedata_build", d)
+ if timedata:
+ time, cpu = timedata
+ # write end of build and cpu used into build_time
+ f.write("Elapsed time: %0.2f seconds \n" % (time))
+ if cpu:
+ f.write("CPU usage: %0.1f%% \n" % cpu)
+
+ if isinstance(e, bb.build.TaskStarted):
+ set_timedata("__timedata_task", d, e.time)
+ bb.utils.mkdirhier(taskdir)
+ # write into the task event file the name and start time
+ with open(os.path.join(taskdir, e.task), "a") as f:
+ f.write("Event: %s \n" % bb.event.getName(e))
+ f.write("Started: %0.2f \n" % e.time)
+
+ elif isinstance(e, bb.build.TaskSucceeded):
+ write_task_data("passed", os.path.join(taskdir, e.task), e, d)
+ if e.task == "do_rootfs":
+ bs = os.path.join(bsdir, "build_stats")
+ with open(bs, "a") as f:
+ rootfs = d.getVar('IMAGE_ROOTFS')
+ if os.path.isdir(rootfs):
+ try:
+ rootfs_size = subprocess.check_output(["du", "-sh", rootfs],
+ stderr=subprocess.STDOUT).decode('utf-8')
+ f.write("Uncompressed Rootfs size: %s" % rootfs_size)
+ except subprocess.CalledProcessError as err:
+ bb.warn("Failed to get rootfs size: %s" % err.output.decode('utf-8'))
+
+ elif isinstance(e, bb.build.TaskFailed):
+ # Can have a failure before TaskStarted so need to mkdir here too
+ bb.utils.mkdirhier(taskdir)
+ write_task_data("failed", os.path.join(taskdir, e.task), e, d)
+ ########################################################################
+ # Lets make things easier and tell people where the build failed in
+ # build_status. We do this here because BuildCompleted triggers no
+ # matter what the status of the build actually is
+ ########################################################################
+ build_status = os.path.join(bsdir, "build_stats")
+ with open(build_status, "a") as f:
+ f.write(d.expand("Failed at: ${PF} at task: %s \n" % e.task))
+ if bb.utils.to_boolean(d.getVar("BB_LOG_HOST_STAT_ON_FAILURE")):
+ write_host_data(os.path.join(bsdir, "host_stats_%s_failure" % e.task), e, d, "failure")
+}
+
+addhandler run_buildstats
+run_buildstats[eventmask] = "bb.event.BuildStarted bb.event.BuildCompleted bb.event.HeartbeatEvent bb.build.TaskStarted bb.build.TaskSucceeded bb.build.TaskFailed"
+
+python runqueue_stats () {
+ import buildstats
+ from bb import event, runqueue
+ # We should not record any samples before the first task has started,
+ # because that's the first activity shown in the process chart.
+ # Besides, at that point we are sure that the build variables
+ # are available that we need to find the output directory.
+ # The persistent SystemStats is stored in the datastore and
+ # closed when the build is done.
+ system_stats = d.getVar('_buildstats_system_stats', False)
+ if not system_stats and isinstance(e, (bb.runqueue.sceneQueueTaskStarted, bb.runqueue.runQueueTaskStarted)):
+ system_stats = buildstats.SystemStats(d)
+ d.setVar('_buildstats_system_stats', system_stats)
+ if system_stats:
+ # Ensure that we sample at important events.
+ done = isinstance(e, bb.event.BuildCompleted)
+ if system_stats.sample(e, force=done):
+ d.setVar('_buildstats_system_stats', system_stats)
+ if done:
+ system_stats.close()
+ d.delVar('_buildstats_system_stats')
+}
+
+addhandler runqueue_stats
+runqueue_stats[eventmask] = "bb.runqueue.sceneQueueTaskStarted bb.runqueue.runQueueTaskStarted bb.event.HeartbeatEvent bb.event.BuildCompleted bb.event.MonitorDiskEvent"
diff --git a/poky/meta/classes-global/debian.bbclass b/poky/meta/classes-global/debian.bbclass
new file mode 100644
index 0000000..7135d74
--- /dev/null
+++ b/poky/meta/classes-global/debian.bbclass
@@ -0,0 +1,156 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+# Debian package renaming only occurs when a package is built
+# We therefore have to make sure we build all runtime packages
+# before building the current package to make the packages runtime
+# depends are correct
+#
+# Custom library package names can be defined setting
+# DEBIANNAME: + pkgname to the desired name.
+#
+# Better expressed as ensure all RDEPENDS package before we package
+# This means we can't have circular RDEPENDS/RRECOMMENDS
+
+AUTO_LIBNAME_PKGS = "${PACKAGES}"
+
+inherit package
+
+DEBIANRDEP = "do_packagedata"
+do_package_write_ipk[deptask] = "${DEBIANRDEP}"
+do_package_write_deb[deptask] = "${DEBIANRDEP}"
+do_package_write_tar[deptask] = "${DEBIANRDEP}"
+do_package_write_rpm[deptask] = "${DEBIANRDEP}"
+do_package_write_ipk[rdeptask] = "${DEBIANRDEP}"
+do_package_write_deb[rdeptask] = "${DEBIANRDEP}"
+do_package_write_tar[rdeptask] = "${DEBIANRDEP}"
+do_package_write_rpm[rdeptask] = "${DEBIANRDEP}"
+
+python () {
+ if not d.getVar("PACKAGES"):
+ d.setVar("DEBIANRDEP", "")
+}
+
+python debian_package_name_hook () {
+ import glob, copy, stat, errno, re, pathlib, subprocess
+
+ pkgdest = d.getVar("PKGDEST")
+ packages = d.getVar('PACKAGES')
+ so_re = re.compile(r"lib.*\.so")
+
+ def socrunch(s):
+ s = s.lower().replace('_', '-')
+ m = re.match(r"^(.*)(.)\.so\.(.*)$", s)
+ if m is None:
+ return None
+ if m.group(2) in '0123456789':
+ bin = '%s%s-%s' % (m.group(1), m.group(2), m.group(3))
+ else:
+ bin = m.group(1) + m.group(2) + m.group(3)
+ dev = m.group(1) + m.group(2)
+ return (bin, dev)
+
+ def isexec(path):
+ try:
+ s = os.stat(path)
+ except (os.error, AttributeError):
+ return 0
+ return (s[stat.ST_MODE] & stat.S_IEXEC)
+
+ def add_rprovides(pkg, d):
+ newpkg = d.getVar('PKG:' + pkg)
+ if newpkg and newpkg != pkg:
+ provs = (d.getVar('RPROVIDES:' + pkg) or "").split()
+ if pkg not in provs:
+ d.appendVar('RPROVIDES:' + pkg, " " + pkg + " (=" + d.getVar("PKGV") + ")")
+
+ def auto_libname(packages, orig_pkg):
+ p = lambda var: pathlib.PurePath(d.getVar(var))
+ libdirs = (p("base_libdir"), p("libdir"))
+ bindirs = (p("base_bindir"), p("base_sbindir"), p("bindir"), p("sbindir"))
+
+ sonames = []
+ has_bins = 0
+ has_libs = 0
+ for f in pkgfiles[orig_pkg]:
+ # This is .../packages-split/orig_pkg/
+ pkgpath = pathlib.PurePath(pkgdest, orig_pkg)
+ # Strip pkgpath off the full path to a file in the package, re-root
+ # so it is absolute, and then get the parent directory of the file.
+ path = pathlib.PurePath("/") / (pathlib.PurePath(f).relative_to(pkgpath).parent)
+ if path in bindirs:
+ has_bins = 1
+ if path in libdirs:
+ has_libs = 1
+ if so_re.match(os.path.basename(f)):
+ try:
+ cmd = [d.expand("${TARGET_PREFIX}objdump"), "-p", f]
+ output = subprocess.check_output(cmd).decode("utf-8")
+ for m in re.finditer(r"\s+SONAME\s+([^\s]+)", output):
+ if m.group(1) not in sonames:
+ sonames.append(m.group(1))
+ except subprocess.CalledProcessError:
+ pass
+ bb.debug(1, 'LIBNAMES: pkg %s libs %d bins %d sonames %s' % (orig_pkg, has_libs, has_bins, sonames))
+ soname = None
+ if len(sonames) == 1:
+ soname = sonames[0]
+ elif len(sonames) > 1:
+ lead = d.getVar('LEAD_SONAME')
+ if lead:
+ r = re.compile(lead)
+ filtered = []
+ for s in sonames:
+ if r.match(s):
+ filtered.append(s)
+ if len(filtered) == 1:
+ soname = filtered[0]
+ elif len(filtered) > 1:
+ bb.note("Multiple matches (%s) for LEAD_SONAME '%s'" % (", ".join(filtered), lead))
+ else:
+ bb.note("Multiple libraries (%s) found, but LEAD_SONAME '%s' doesn't match any of them" % (", ".join(sonames), lead))
+ else:
+ bb.note("Multiple libraries (%s) found and LEAD_SONAME not defined" % ", ".join(sonames))
+
+ if has_libs and not has_bins and soname:
+ soname_result = socrunch(soname)
+ if soname_result:
+ (pkgname, devname) = soname_result
+ for pkg in packages.split():
+ if (d.getVar('PKG:' + pkg, False) or d.getVar('DEBIAN_NOAUTONAME:' + pkg, False)):
+ add_rprovides(pkg, d)
+ continue
+ debian_pn = d.getVar('DEBIANNAME:' + pkg, False)
+ if debian_pn:
+ newpkg = debian_pn
+ elif pkg == orig_pkg:
+ newpkg = pkgname
+ else:
+ newpkg = pkg.replace(orig_pkg, devname, 1)
+ mlpre=d.getVar('MLPREFIX')
+ if mlpre:
+ if not newpkg.find(mlpre) == 0:
+ newpkg = mlpre + newpkg
+ if newpkg != pkg:
+ bb.note("debian: renaming %s to %s" % (pkg, newpkg))
+ d.setVar('PKG:' + pkg, newpkg)
+ add_rprovides(pkg, d)
+ else:
+ add_rprovides(orig_pkg, d)
+
+ # reversed sort is needed when some package is substring of another
+ # ie in ncurses we get without reverse sort:
+ # DEBUG: LIBNAMES: pkgname libtic5 devname libtic pkg ncurses-libtic orig_pkg ncurses-libtic debian_pn None newpkg libtic5
+ # and later
+ # DEBUG: LIBNAMES: pkgname libtic5 devname libtic pkg ncurses-libticw orig_pkg ncurses-libtic debian_pn None newpkg libticw
+ # so we need to handle ncurses-libticw->libticw5 before ncurses-libtic->libtic5
+ for pkg in sorted((d.getVar('AUTO_LIBNAME_PKGS') or "").split(), reverse=True):
+ auto_libname(packages, pkg)
+}
+
+EXPORT_FUNCTIONS package_name_hook
+
+DEBIAN_NAMES = "1"
diff --git a/poky/meta/classes-global/devshell.bbclass b/poky/meta/classes-global/devshell.bbclass
new file mode 100644
index 0000000..03af56b
--- /dev/null
+++ b/poky/meta/classes-global/devshell.bbclass
@@ -0,0 +1,166 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+inherit terminal
+
+DEVSHELL = "${SHELL}"
+
+PATH:prepend:task-devshell = "${COREBASE}/scripts/git-intercept:"
+
+python do_devshell () {
+ if d.getVarFlag("do_devshell", "manualfakeroot"):
+ d.prependVar("DEVSHELL", "pseudo ")
+ fakeenv = d.getVar("FAKEROOTENV").split()
+ for f in fakeenv:
+ k = f.split("=")
+ d.setVar(k[0], k[1])
+ d.appendVar("OE_TERMINAL_EXPORTS", " " + k[0])
+ d.delVarFlag("do_devshell", "fakeroot")
+
+ oe_terminal(d.getVar('DEVSHELL'), 'OpenEmbedded Developer Shell', d)
+}
+
+addtask devshell after do_patch do_prepare_recipe_sysroot
+
+# The directory that the terminal starts in
+DEVSHELL_STARTDIR ?= "${S}"
+do_devshell[dirs] = "${DEVSHELL_STARTDIR}"
+do_devshell[nostamp] = "1"
+do_devshell[network] = "1"
+
+# devshell and fakeroot/pseudo need careful handling since only the final
+# command should run under fakeroot emulation, any X connection should
+# be done as the normal user. We therfore carefully construct the envionment
+# manually
+python () {
+ if d.getVarFlag("do_devshell", "fakeroot"):
+ # We need to signal our code that we want fakeroot however we
+ # can't manipulate the environment and variables here yet (see YOCTO #4795)
+ d.setVarFlag("do_devshell", "manualfakeroot", "1")
+ d.delVarFlag("do_devshell", "fakeroot")
+}
+
+def pydevshell(d):
+
+ import code
+ import select
+ import signal
+ import termios
+
+ m, s = os.openpty()
+ sname = os.ttyname(s)
+
+ def noechoicanon(fd):
+ old = termios.tcgetattr(fd)
+ old[3] = old[3] &~ termios.ECHO &~ termios.ICANON
+ # &~ termios.ISIG
+ termios.tcsetattr(fd, termios.TCSADRAIN, old)
+
+ # No echo or buffering over the pty
+ noechoicanon(s)
+
+ pid = os.fork()
+ if pid:
+ os.close(m)
+ oe_terminal("oepydevshell-internal.py %s %d" % (sname, pid), 'OpenEmbedded Developer PyShell', d)
+ os._exit(0)
+ else:
+ os.close(s)
+
+ os.dup2(m, sys.stdin.fileno())
+ os.dup2(m, sys.stdout.fileno())
+ os.dup2(m, sys.stderr.fileno())
+
+ bb.utils.nonblockingfd(sys.stdout)
+ bb.utils.nonblockingfd(sys.stderr)
+ bb.utils.nonblockingfd(sys.stdin)
+
+ _context = {
+ "os": os,
+ "bb": bb,
+ "time": time,
+ "d": d,
+ }
+
+ ps1 = "pydevshell> "
+ ps2 = "... "
+ buf = []
+ more = False
+
+ i = code.InteractiveInterpreter(locals=_context)
+ print("OE PyShell (PN = %s)\n" % d.getVar("PN"))
+
+ def prompt(more):
+ if more:
+ prompt = ps2
+ else:
+ prompt = ps1
+ sys.stdout.write(prompt)
+ sys.stdout.flush()
+
+ # Restore Ctrl+C since bitbake masks this
+ def signal_handler(signal, frame):
+ raise KeyboardInterrupt
+ signal.signal(signal.SIGINT, signal_handler)
+
+ child = None
+
+ prompt(more)
+ while True:
+ try:
+ try:
+ (r, _, _) = select.select([sys.stdin], [], [], 1)
+ if not r:
+ continue
+ line = sys.stdin.readline().strip()
+ if not line:
+ prompt(more)
+ continue
+ except EOFError as e:
+ sys.stdout.write("\n")
+ sys.stdout.flush()
+ except (OSError, IOError) as e:
+ if e.errno == 11:
+ continue
+ if e.errno == 5:
+ return
+ raise
+ else:
+ if not child:
+ child = int(line)
+ continue
+ buf.append(line)
+ source = "\n".join(buf)
+ more = i.runsource(source, "<pyshell>")
+ if not more:
+ buf = []
+ sys.stderr.flush()
+ prompt(more)
+ except KeyboardInterrupt:
+ i.write("\nKeyboardInterrupt\n")
+ buf = []
+ more = False
+ prompt(more)
+ except SystemExit:
+ # Easiest way to ensure everything exits
+ os.kill(child, signal.SIGTERM)
+ break
+
+python do_pydevshell() {
+ import signal
+
+ try:
+ pydevshell(d)
+ except SystemExit:
+ # Stop the SIGTERM above causing an error exit code
+ return
+ finally:
+ return
+}
+addtask pydevshell after do_patch
+
+do_pydevshell[nostamp] = "1"
+do_pydevshell[network] = "1"
diff --git a/poky/meta/classes-global/insane.bbclass b/poky/meta/classes-global/insane.bbclass
new file mode 100644
index 0000000..db34b4b
--- /dev/null
+++ b/poky/meta/classes-global/insane.bbclass
@@ -0,0 +1,1454 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+# BB Class inspired by ebuild.sh
+#
+# This class will test files after installation for certain
+# security issues and other kind of issues.
+#
+# Checks we do:
+# -Check the ownership and permissions
+# -Check the RUNTIME path for the $TMPDIR
+# -Check if .la files wrongly point to workdir
+# -Check if .pc files wrongly point to workdir
+# -Check if packages contains .debug directories or .so files
+# where they should be in -dev or -dbg
+# -Check if config.log contains traces to broken autoconf tests
+# -Check invalid characters (non-utf8) on some package metadata
+# -Ensure that binaries in base_[bindir|sbindir|libdir] do not link
+# into exec_prefix
+# -Check that scripts in base_[bindir|sbindir|libdir] do not reference
+# files under exec_prefix
+# -Check if the package name is upper case
+
+# Elect whether a given type of error is a warning or error, they may
+# have been set by other files.
+WARN_QA ?= " libdir xorg-driver-abi buildpaths \
+ textrel incompatible-license files-invalid \
+ infodir build-deps src-uri-bad symlink-to-sysroot multilib \
+ invalid-packageconfig host-user-contaminated uppercase-pn patch-fuzz \
+ mime mime-xdg unlisted-pkg-lics unhandled-features-check \
+ missing-update-alternatives native-last missing-ptest \
+ license-exists license-no-generic license-syntax license-format \
+ license-incompatible license-file-missing obsolete-license \
+ "
+ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig la \
+ perms dep-cmp pkgvarcheck perm-config perm-line perm-link \
+ split-strip packages-list pkgv-undefined var-undefined \
+ version-going-backwards expanded-d invalid-chars \
+ license-checksum dev-elf file-rdeps configure-unsafe \
+ configure-gettext perllocalpod shebang-size \
+ already-stripped installed-vs-shipped ldflags compile-host-path \
+ install-host-path pn-overrides unknown-configure-option \
+ useless-rpaths rpaths staticdev empty-dirs \
+ "
+# Add usrmerge QA check based on distro feature
+ERROR_QA:append = "${@bb.utils.contains('DISTRO_FEATURES', 'usrmerge', ' usrmerge', '', d)}"
+
+FAKEROOT_QA = "host-user-contaminated"
+FAKEROOT_QA[doc] = "QA tests which need to run under fakeroot. If any \
+enabled tests are listed here, the do_package_qa task will run under fakeroot."
+
+ALL_QA = "${WARN_QA} ${ERROR_QA}"
+
+UNKNOWN_CONFIGURE_OPT_IGNORE ?= "--enable-nls --disable-nls --disable-silent-rules --disable-dependency-tracking --with-libtool-sysroot --disable-static"
+
+# This is a list of directories that are expected to be empty.
+QA_EMPTY_DIRS ?= " \
+ /dev/pts \
+ /media \
+ /proc \
+ /run \
+ /tmp \
+ ${localstatedir}/run \
+ ${localstatedir}/volatile \
+"
+# It is possible to specify why a directory is expected to be empty by defining
+# QA_EMPTY_DIRS_RECOMMENDATION:<path>, which will then be included in the error
+# message if the directory is not empty. If it is not specified for a directory,
+# then "but it is expected to be empty" will be used.
+
+def package_qa_clean_path(path, d, pkg=None):
+ """
+ Remove redundant paths from the path for display. If pkg isn't set then
+ TMPDIR is stripped, otherwise PKGDEST/pkg is stripped.
+ """
+ if pkg:
+ path = path.replace(os.path.join(d.getVar("PKGDEST"), pkg), "/")
+ return path.replace(d.getVar("TMPDIR"), "/").replace("//", "/")
+
+QAPATHTEST[shebang-size] = "package_qa_check_shebang_size"
+def package_qa_check_shebang_size(path, name, d, elf, messages):
+ import stat
+ if os.path.islink(path) or stat.S_ISFIFO(os.stat(path).st_mode) or elf:
+ return
+
+ try:
+ with open(path, 'rb') as f:
+ stanza = f.readline(130)
+ except IOError:
+ return
+
+ if stanza.startswith(b'#!'):
+ #Shebang not found
+ try:
+ stanza = stanza.decode("utf-8")
+ except UnicodeDecodeError:
+ #If it is not a text file, it is not a script
+ return
+
+ if len(stanza) > 129:
+ oe.qa.add_message(messages, "shebang-size", "%s: %s maximum shebang size exceeded, the maximum size is 128." % (name, package_qa_clean_path(path, d)))
+ return
+
+QAPATHTEST[libexec] = "package_qa_check_libexec"
+def package_qa_check_libexec(path,name, d, elf, messages):
+
+ # Skip the case where the default is explicitly /usr/libexec
+ libexec = d.getVar('libexecdir')
+ if libexec == "/usr/libexec":
+ return True
+
+ if 'libexec' in path.split(os.path.sep):
+ oe.qa.add_message(messages, "libexec", "%s: %s is using libexec please relocate to %s" % (name, package_qa_clean_path(path, d), libexec))
+ return False
+
+ return True
+
+QAPATHTEST[rpaths] = "package_qa_check_rpath"
+def package_qa_check_rpath(file,name, d, elf, messages):
+ """
+ Check for dangerous RPATHs
+ """
+ if not elf:
+ return
+
+ if os.path.islink(file):
+ return
+
+ bad_dirs = [d.getVar('BASE_WORKDIR'), d.getVar('STAGING_DIR_TARGET')]
+
+ phdrs = elf.run_objdump("-p", d)
+
+ import re
+ rpath_re = re.compile(r"\s+RPATH\s+(.*)")
+ for line in phdrs.split("\n"):
+ m = rpath_re.match(line)
+ if m:
+ rpath = m.group(1)
+ for dir in bad_dirs:
+ if dir in rpath:
+ oe.qa.add_message(messages, "rpaths", "package %s contains bad RPATH %s in file %s" % (name, rpath, file))
+
+QAPATHTEST[useless-rpaths] = "package_qa_check_useless_rpaths"
+def package_qa_check_useless_rpaths(file, name, d, elf, messages):
+ """
+ Check for RPATHs that are useless but not dangerous
+ """
+ def rpath_eq(a, b):
+ return os.path.normpath(a) == os.path.normpath(b)
+
+ if not elf:
+ return
+
+ if os.path.islink(file):
+ return
+
+ libdir = d.getVar("libdir")
+ base_libdir = d.getVar("base_libdir")
+
+ phdrs = elf.run_objdump("-p", d)
+
+ import re
+ rpath_re = re.compile(r"\s+RPATH\s+(.*)")
+ for line in phdrs.split("\n"):
+ m = rpath_re.match(line)
+ if m:
+ rpath = m.group(1)
+ if rpath_eq(rpath, libdir) or rpath_eq(rpath, base_libdir):
+ # The dynamic linker searches both these places anyway. There is no point in
+ # looking there again.
+ oe.qa.add_message(messages, "useless-rpaths", "%s: %s contains probably-redundant RPATH %s" % (name, package_qa_clean_path(file, d, name), rpath))
+
+QAPATHTEST[dev-so] = "package_qa_check_dev"
+def package_qa_check_dev(path, name, d, elf, messages):
+ """
+ Check for ".so" library symlinks in non-dev packages
+ """
+
+ if not name.endswith("-dev") and not name.endswith("-dbg") and not name.endswith("-ptest") and not name.startswith("nativesdk-") and path.endswith(".so") and os.path.islink(path):
+ oe.qa.add_message(messages, "dev-so", "non -dev/-dbg/nativesdk- package %s contains symlink .so '%s'" % \
+ (name, package_qa_clean_path(path, d, name)))
+
+QAPATHTEST[dev-elf] = "package_qa_check_dev_elf"
+def package_qa_check_dev_elf(path, name, d, elf, messages):
+ """
+ Check that -dev doesn't contain real shared libraries. The test has to
+ check that the file is not a link and is an ELF object as some recipes
+ install link-time .so files that are linker scripts.
+ """
+ if name.endswith("-dev") and path.endswith(".so") and not os.path.islink(path) and elf:
+ oe.qa.add_message(messages, "dev-elf", "-dev package %s contains non-symlink .so '%s'" % \
+ (name, package_qa_clean_path(path, d, name)))
+
+QAPATHTEST[staticdev] = "package_qa_check_staticdev"
+def package_qa_check_staticdev(path, name, d, elf, messages):
+ """
+ Check for ".a" library in non-staticdev packages
+ There are a number of exceptions to this rule, -pic packages can contain
+ static libraries, the _nonshared.a belong with their -dev packages and
+ libgcc.a, libgcov.a will be skipped in their packages
+ """
+
+ if not name.endswith("-pic") and not name.endswith("-staticdev") and not name.endswith("-ptest") and path.endswith(".a") and not path.endswith("_nonshared.a") and not '/usr/lib/debug-static/' in path and not '/.debug-static/' in path:
+ oe.qa.add_message(messages, "staticdev", "non -staticdev package contains static .a library: %s path '%s'" % \
+ (name, package_qa_clean_path(path,d, name)))
+
+QAPATHTEST[mime] = "package_qa_check_mime"
+def package_qa_check_mime(path, name, d, elf, messages):
+ """
+ Check if package installs mime types to /usr/share/mime/packages
+ while no inheriting mime.bbclass
+ """
+
+ if d.getVar("datadir") + "/mime/packages" in path and path.endswith('.xml') and not bb.data.inherits_class("mime", d):
+ oe.qa.add_message(messages, "mime", "package contains mime types but does not inherit mime: %s path '%s'" % \
+ (name, package_qa_clean_path(path,d)))
+
+QAPATHTEST[mime-xdg] = "package_qa_check_mime_xdg"
+def package_qa_check_mime_xdg(path, name, d, elf, messages):
+ """
+ Check if package installs desktop file containing MimeType and requires
+ mime-types.bbclass to create /usr/share/applications/mimeinfo.cache
+ """
+
+ if d.getVar("datadir") + "/applications" in path and path.endswith('.desktop') and not bb.data.inherits_class("mime-xdg", d):
+ mime_type_found = False
+ try:
+ with open(path, 'r') as f:
+ for line in f.read().split('\n'):
+ if 'MimeType' in line:
+ mime_type_found = True
+ break;
+ except:
+ # At least libreoffice installs symlinks with absolute paths that are dangling here.
+ # We could implement some magic but for few (one) recipes it is not worth the effort so just warn:
+ wstr = "%s cannot open %s - is it a symlink with absolute path?\n" % (name, package_qa_clean_path(path,d))
+ wstr += "Please check if (linked) file contains key 'MimeType'.\n"
+ pkgname = name
+ if name == d.getVar('PN'):
+ pkgname = '${PN}'
+ wstr += "If yes: add \'inhert mime-xdg\' and \'MIME_XDG_PACKAGES += \"%s\"\' / if no add \'INSANE_SKIP:%s += \"mime-xdg\"\' to recipe." % (pkgname, pkgname)
+ oe.qa.add_message(messages, "mime-xdg", wstr)
+ if mime_type_found:
+ oe.qa.add_message(messages, "mime-xdg", "package contains desktop file with key 'MimeType' but does not inhert mime-xdg: %s path '%s'" % \
+ (name, package_qa_clean_path(path,d)))
+
+def package_qa_check_libdir(d):
+ """
+ Check for wrong library installation paths. For instance, catch
+ recipes installing /lib/bar.so when ${base_libdir}="lib32" or
+ installing in /usr/lib64 when ${libdir}="/usr/lib"
+ """
+ import re
+
+ pkgdest = d.getVar('PKGDEST')
+ base_libdir = d.getVar("base_libdir") + os.sep
+ libdir = d.getVar("libdir") + os.sep
+ libexecdir = d.getVar("libexecdir") + os.sep
+ exec_prefix = d.getVar("exec_prefix") + os.sep
+
+ messages = []
+
+ # The re's are purposely fuzzy, as some there are some .so.x.y.z files
+ # that don't follow the standard naming convention. It checks later
+ # that they are actual ELF files
+ lib_re = re.compile(r"^/lib.+\.so(\..+)?$")
+ exec_re = re.compile(r"^%s.*/lib.+\.so(\..+)?$" % exec_prefix)
+
+ for root, dirs, files in os.walk(pkgdest):
+ if root == pkgdest:
+ # Skip subdirectories for any packages with libdir in INSANE_SKIP
+ skippackages = []
+ for package in dirs:
+ if 'libdir' in (d.getVar('INSANE_SKIP:' + package) or "").split():
+ bb.note("Package %s skipping libdir QA test" % (package))
+ skippackages.append(package)
+ elif d.getVar('PACKAGE_DEBUG_SPLIT_STYLE') == 'debug-file-directory' and package.endswith("-dbg"):
+ bb.note("Package %s skipping libdir QA test for PACKAGE_DEBUG_SPLIT_STYLE equals debug-file-directory" % (package))
+ skippackages.append(package)
+ for package in skippackages:
+ dirs.remove(package)
+ for file in files:
+ full_path = os.path.join(root, file)
+ rel_path = os.path.relpath(full_path, pkgdest)
+ if os.sep in rel_path:
+ package, rel_path = rel_path.split(os.sep, 1)
+ rel_path = os.sep + rel_path
+ if lib_re.match(rel_path):
+ if base_libdir not in rel_path:
+ # make sure it's an actual ELF file
+ elf = oe.qa.ELFFile(full_path)
+ try:
+ elf.open()
+ messages.append("%s: found library in wrong location: %s" % (package, rel_path))
+ except (oe.qa.NotELFFileError):
+ pass
+ if exec_re.match(rel_path):
+ if libdir not in rel_path and libexecdir not in rel_path:
+ # make sure it's an actual ELF file
+ elf = oe.qa.ELFFile(full_path)
+ try:
+ elf.open()
+ messages.append("%s: found library in wrong location: %s" % (package, rel_path))
+ except (oe.qa.NotELFFileError):
+ pass
+
+ if messages:
+ oe.qa.handle_error("libdir", "\n".join(messages), d)
+
+QAPATHTEST[debug-files] = "package_qa_check_dbg"
+def package_qa_check_dbg(path, name, d, elf, messages):
+ """
+ Check for ".debug" files or directories outside of the dbg package
+ """
+
+ if not "-dbg" in name and not "-ptest" in name:
+ if '.debug' in path.split(os.path.sep):
+ oe.qa.add_message(messages, "debug-files", "non debug package contains .debug directory: %s path %s" % \
+ (name, package_qa_clean_path(path,d)))
+
+QAPATHTEST[arch] = "package_qa_check_arch"
+def package_qa_check_arch(path,name,d, elf, messages):
+ """
+ Check if archs are compatible
+ """
+ import re, oe.elf
+
+ if not elf:
+ return
+
+ target_os = d.getVar('HOST_OS')
+ target_arch = d.getVar('HOST_ARCH')
+ provides = d.getVar('PROVIDES')
+ bpn = d.getVar('BPN')
+
+ if target_arch == "allarch":
+ pn = d.getVar('PN')
+ oe.qa.add_message(messages, "arch", pn + ": Recipe inherits the allarch class, but has packaged architecture-specific binaries")
+ return
+
+ # FIXME: Cross package confuse this check, so just skip them
+ for s in ['cross', 'nativesdk', 'cross-canadian']:
+ if bb.data.inherits_class(s, d):
+ return
+
+ # avoid following links to /usr/bin (e.g. on udev builds)
+ # we will check the files pointed to anyway...
+ if os.path.islink(path):
+ return
+
+ #if this will throw an exception, then fix the dict above
+ (machine, osabi, abiversion, littleendian, bits) \
+ = oe.elf.machine_dict(d)[target_os][target_arch]
+
+ # Check the architecture and endiannes of the binary
+ is_32 = (("virtual/kernel" in provides) or bb.data.inherits_class("module", d)) and \
+ (target_os == "linux-gnux32" or target_os == "linux-muslx32" or \
+ target_os == "linux-gnu_ilp32" or re.match(r'mips64.*32', d.getVar('DEFAULTTUNE')))
+ is_bpf = (oe.qa.elf_machine_to_string(elf.machine()) == "BPF")
+ if not ((machine == elf.machine()) or is_32 or is_bpf):
+ oe.qa.add_message(messages, "arch", "Architecture did not match (%s, expected %s) in %s" % \
+ (oe.qa.elf_machine_to_string(elf.machine()), oe.qa.elf_machine_to_string(machine), package_qa_clean_path(path, d, name)))
+ elif not ((bits == elf.abiSize()) or is_32 or is_bpf):
+ oe.qa.add_message(messages, "arch", "Bit size did not match (%d, expected %d) in %s" % \
+ (elf.abiSize(), bits, package_qa_clean_path(path, d, name)))
+ elif not ((littleendian == elf.isLittleEndian()) or is_bpf):
+ oe.qa.add_message(messages, "arch", "Endiannes did not match (%d, expected %d) in %s" % \
+ (elf.isLittleEndian(), littleendian, package_qa_clean_path(path,d, name)))
+
+QAPATHTEST[desktop] = "package_qa_check_desktop"
+def package_qa_check_desktop(path, name, d, elf, messages):
+ """
+ Run all desktop files through desktop-file-validate.
+ """
+ if path.endswith(".desktop"):
+ desktop_file_validate = os.path.join(d.getVar('STAGING_BINDIR_NATIVE'),'desktop-file-validate')
+ output = os.popen("%s %s" % (desktop_file_validate, path))
+ # This only produces output on errors
+ for l in output:
+ oe.qa.add_message(messages, "desktop", "Desktop file issue: " + l.strip())
+
+QAPATHTEST[textrel] = "package_qa_textrel"
+def package_qa_textrel(path, name, d, elf, messages):
+ """
+ Check if the binary contains relocations in .text
+ """
+
+ if not elf:
+ return
+
+ if os.path.islink(path):
+ return
+
+ phdrs = elf.run_objdump("-p", d)
+ sane = True
+
+ import re
+ textrel_re = re.compile(r"\s+TEXTREL\s+")
+ for line in phdrs.split("\n"):
+ if textrel_re.match(line):
+ sane = False
+ break
+
+ if not sane:
+ path = package_qa_clean_path(path, d, name)
+ oe.qa.add_message(messages, "textrel", "%s: ELF binary %s has relocations in .text" % (name, path))
+
+QAPATHTEST[ldflags] = "package_qa_hash_style"
+def package_qa_hash_style(path, name, d, elf, messages):
+ """
+ Check if the binary has the right hash style...
+ """
+
+ if not elf:
+ return
+
+ if os.path.islink(path):
+ return
+
+ gnu_hash = "--hash-style=gnu" in d.getVar('LDFLAGS')
+ if not gnu_hash:
+ gnu_hash = "--hash-style=both" in d.getVar('LDFLAGS')
+ if not gnu_hash:
+ return
+
+ sane = False
+ has_syms = False
+
+ phdrs = elf.run_objdump("-p", d)
+
+ # If this binary has symbols, we expect it to have GNU_HASH too.
+ for line in phdrs.split("\n"):
+ if "SYMTAB" in line:
+ has_syms = True
+ if "GNU_HASH" in line or "MIPS_XHASH" in line:
+ sane = True
+ if ("[mips32]" in line or "[mips64]" in line) and d.getVar('TCLIBC') == "musl":
+ sane = True
+ if has_syms and not sane:
+ path = package_qa_clean_path(path, d, name)
+ oe.qa.add_message(messages, "ldflags", "File %s in package %s doesn't have GNU_HASH (didn't pass LDFLAGS?)" % (path, name))
+
+
+QAPATHTEST[buildpaths] = "package_qa_check_buildpaths"
+def package_qa_check_buildpaths(path, name, d, elf, messages):
+ """
+ Check for build paths inside target files and error if paths are not
+ explicitly ignored.
+ """
+ import stat
+
+ # Ignore symlinks/devs/fifos
+ mode = os.lstat(path).st_mode
+ if stat.S_ISLNK(mode) or stat.S_ISBLK(mode) or stat.S_ISFIFO(mode) or stat.S_ISCHR(mode) or stat.S_ISSOCK(mode):
+ return
+
+ tmpdir = bytes(d.getVar('TMPDIR'), encoding="utf-8")
+ with open(path, 'rb') as f:
+ file_content = f.read()
+ if tmpdir in file_content:
+ trimmed = path.replace(os.path.join (d.getVar("PKGDEST"), name), "")
+ oe.qa.add_message(messages, "buildpaths", "File %s in package %s contains reference to TMPDIR" % (trimmed, name))
+
+
+QAPATHTEST[xorg-driver-abi] = "package_qa_check_xorg_driver_abi"
+def package_qa_check_xorg_driver_abi(path, name, d, elf, messages):
+ """
+ Check that all packages containing Xorg drivers have ABI dependencies
+ """
+
+ # Skip dev, dbg or nativesdk packages
+ if name.endswith("-dev") or name.endswith("-dbg") or name.startswith("nativesdk-"):
+ return
+
+ driverdir = d.expand("${libdir}/xorg/modules/drivers/")
+ if driverdir in path and path.endswith(".so"):
+ mlprefix = d.getVar('MLPREFIX') or ''
+ for rdep in bb.utils.explode_deps(d.getVar('RDEPENDS:' + name) or ""):
+ if rdep.startswith("%sxorg-abi-" % mlprefix):
+ return
+ oe.qa.add_message(messages, "xorg-driver-abi", "Package %s contains Xorg driver (%s) but no xorg-abi- dependencies" % (name, os.path.basename(path)))
+
+QAPATHTEST[infodir] = "package_qa_check_infodir"
+def package_qa_check_infodir(path, name, d, elf, messages):
+ """
+ Check that /usr/share/info/dir isn't shipped in a particular package
+ """
+ infodir = d.expand("${infodir}/dir")
+
+ if infodir in path:
+ oe.qa.add_message(messages, "infodir", "The /usr/share/info/dir file is not meant to be shipped in a particular package.")
+
+QAPATHTEST[symlink-to-sysroot] = "package_qa_check_symlink_to_sysroot"
+def package_qa_check_symlink_to_sysroot(path, name, d, elf, messages):
+ """
+ Check that the package doesn't contain any absolute symlinks to the sysroot.
+ """
+ if os.path.islink(path):
+ target = os.readlink(path)
+ if os.path.isabs(target):
+ tmpdir = d.getVar('TMPDIR')
+ if target.startswith(tmpdir):
+ trimmed = path.replace(os.path.join (d.getVar("PKGDEST"), name), "")
+ oe.qa.add_message(messages, "symlink-to-sysroot", "Symlink %s in %s points to TMPDIR" % (trimmed, name))
+
+# Check license variables
+do_populate_lic[postfuncs] += "populate_lic_qa_checksum"
+python populate_lic_qa_checksum() {
+ """
+ Check for changes in the license files.
+ """
+
+ lic_files = d.getVar('LIC_FILES_CHKSUM') or ''
+ lic = d.getVar('LICENSE')
+ pn = d.getVar('PN')
+
+ if lic == "CLOSED":
+ return
+
+ if not lic_files and d.getVar('SRC_URI'):
+ oe.qa.handle_error("license-checksum", pn + ": Recipe file fetches files and does not have license file information (LIC_FILES_CHKSUM)", d)
+
+ srcdir = d.getVar('S')
+ corebase_licensefile = d.getVar('COREBASE') + "/LICENSE"
+ for url in lic_files.split():
+ try:
+ (type, host, path, user, pswd, parm) = bb.fetch.decodeurl(url)
+ except bb.fetch.MalformedUrl:
+ oe.qa.handle_error("license-checksum", pn + ": LIC_FILES_CHKSUM contains an invalid URL: " + url, d)
+ continue
+ srclicfile = os.path.join(srcdir, path)
+ if not os.path.isfile(srclicfile):
+ oe.qa.handle_error("license-checksum", pn + ": LIC_FILES_CHKSUM points to an invalid file: " + srclicfile, d)
+ continue
+
+ if (srclicfile == corebase_licensefile):
+ bb.warn("${COREBASE}/LICENSE is not a valid license file, please use '${COMMON_LICENSE_DIR}/MIT' for a MIT License file in LIC_FILES_CHKSUM. This will become an error in the future")
+
+ recipemd5 = parm.get('md5', '')
+ beginline, endline = 0, 0
+ if 'beginline' in parm:
+ beginline = int(parm['beginline'])
+ if 'endline' in parm:
+ endline = int(parm['endline'])
+
+ if (not beginline) and (not endline):
+ md5chksum = bb.utils.md5_file(srclicfile)
+ with open(srclicfile, 'r', errors='replace') as f:
+ license = f.read().splitlines()
+ else:
+ with open(srclicfile, 'rb') as f:
+ import hashlib
+ lineno = 0
+ license = []
+ m = hashlib.new('MD5', usedforsecurity=False)
+ for line in f:
+ lineno += 1
+ if (lineno >= beginline):
+ if ((lineno <= endline) or not endline):
+ m.update(line)
+ license.append(line.decode('utf-8', errors='replace').rstrip())
+ else:
+ break
+ md5chksum = m.hexdigest()
+ if recipemd5 == md5chksum:
+ bb.note (pn + ": md5 checksum matched for ", url)
+ else:
+ if recipemd5:
+ msg = pn + ": The LIC_FILES_CHKSUM does not match for " + url
+ msg = msg + "\n" + pn + ": The new md5 checksum is " + md5chksum
+ max_lines = int(d.getVar('QA_MAX_LICENSE_LINES') or 20)
+ if not license or license[-1] != '':
+ # Ensure that our license text ends with a line break
+ # (will be added with join() below).
+ license.append('')
+ remove = len(license) - max_lines
+ if remove > 0:
+ start = max_lines // 2
+ end = start + remove - 1
+ del license[start:end]
+ license.insert(start, '...')
+ msg = msg + "\n" + pn + ": Here is the selected license text:" + \
+ "\n" + \
+ "{:v^70}".format(" beginline=%d " % beginline if beginline else "") + \
+ "\n" + "\n".join(license) + \
+ "{:^^70}".format(" endline=%d " % endline if endline else "")
+ if beginline:
+ if endline:
+ srcfiledesc = "%s (lines %d through to %d)" % (srclicfile, beginline, endline)
+ else:
+ srcfiledesc = "%s (beginning on line %d)" % (srclicfile, beginline)
+ elif endline:
+ srcfiledesc = "%s (ending on line %d)" % (srclicfile, endline)
+ else:
+ srcfiledesc = srclicfile
+ msg = msg + "\n" + pn + ": Check if the license information has changed in %s to verify that the LICENSE value \"%s\" remains valid" % (srcfiledesc, lic)
+
+ else:
+ msg = pn + ": LIC_FILES_CHKSUM is not specified for " + url
+ msg = msg + "\n" + pn + ": The md5 checksum is " + md5chksum
+ oe.qa.handle_error("license-checksum", msg, d)
+
+ oe.qa.exit_if_errors(d)
+}
+
+def qa_check_staged(path,d):
+ """
+ Check staged la and pc files for common problems like references to the work
+ directory.
+
+ As this is run after every stage we should be able to find the one
+ responsible for the errors easily even if we look at every .pc and .la file.
+ """
+
+ tmpdir = d.getVar('TMPDIR')
+ workdir = os.path.join(tmpdir, "work")
+ recipesysroot = d.getVar("RECIPE_SYSROOT")
+
+ if bb.data.inherits_class("native", d) or bb.data.inherits_class("cross", d):
+ pkgconfigcheck = workdir
+ else:
+ pkgconfigcheck = tmpdir
+
+ skip = (d.getVar('INSANE_SKIP') or "").split()
+ skip_la = False
+ if 'la' in skip:
+ bb.note("Recipe %s skipping qa checking: la" % d.getVar('PN'))
+ skip_la = True
+
+ skip_pkgconfig = False
+ if 'pkgconfig' in skip:
+ bb.note("Recipe %s skipping qa checking: pkgconfig" % d.getVar('PN'))
+ skip_pkgconfig = True
+
+ skip_shebang_size = False
+ if 'shebang-size' in skip:
+ bb.note("Recipe %s skipping qa checkking: shebang-size" % d.getVar('PN'))
+ skip_shebang_size = True
+
+ # find all .la and .pc files
+ # read the content
+ # and check for stuff that looks wrong
+ for root, dirs, files in os.walk(path):
+ for file in files:
+ path = os.path.join(root,file)
+ if file.endswith(".la") and not skip_la:
+ with open(path) as f:
+ file_content = f.read()
+ file_content = file_content.replace(recipesysroot, "")
+ if workdir in file_content:
+ error_msg = "%s failed sanity test (workdir) in path %s" % (file,root)
+ oe.qa.handle_error("la", error_msg, d)
+ elif file.endswith(".pc") and not skip_pkgconfig:
+ with open(path) as f:
+ file_content = f.read()
+ file_content = file_content.replace(recipesysroot, "")
+ if pkgconfigcheck in file_content:
+ error_msg = "%s failed sanity test (tmpdir) in path %s" % (file,root)
+ oe.qa.handle_error("pkgconfig", error_msg, d)
+
+ if not skip_shebang_size:
+ errors = {}
+ package_qa_check_shebang_size(path, "", d, None, errors)
+ for e in errors:
+ oe.qa.handle_error(e, errors[e], d)
+
+
+# Run all package-wide warnfuncs and errorfuncs
+def package_qa_package(warnfuncs, errorfuncs, package, d):
+ warnings = {}
+ errors = {}
+
+ for func in warnfuncs:
+ func(package, d, warnings)
+ for func in errorfuncs:
+ func(package, d, errors)
+
+ for w in warnings:
+ oe.qa.handle_error(w, warnings[w], d)
+ for e in errors:
+ oe.qa.handle_error(e, errors[e], d)
+
+ return len(errors) == 0
+
+# Run all recipe-wide warnfuncs and errorfuncs
+def package_qa_recipe(warnfuncs, errorfuncs, pn, d):
+ warnings = {}
+ errors = {}
+
+ for func in warnfuncs:
+ func(pn, d, warnings)
+ for func in errorfuncs:
+ func(pn, d, errors)
+
+ for w in warnings:
+ oe.qa.handle_error(w, warnings[w], d)
+ for e in errors:
+ oe.qa.handle_error(e, errors[e], d)
+
+ return len(errors) == 0
+
+def prepopulate_objdump_p(elf, d):
+ output = elf.run_objdump("-p", d)
+ return (elf.name, output)
+
+# Walk over all files in a directory and call func
+def package_qa_walk(warnfuncs, errorfuncs, package, d):
+ #if this will throw an exception, then fix the dict above
+ target_os = d.getVar('HOST_OS')
+ target_arch = d.getVar('HOST_ARCH')
+
+ warnings = {}
+ errors = {}
+ elves = {}
+ for path in pkgfiles[package]:
+ elf = None
+ if os.path.isfile(path):
+ elf = oe.qa.ELFFile(path)
+ try:
+ elf.open()
+ elf.close()
+ except oe.qa.NotELFFileError:
+ elf = None
+ if elf:
+ elves[path] = elf
+
+ results = oe.utils.multiprocess_launch(prepopulate_objdump_p, elves.values(), d, extraargs=(d,))
+ for item in results:
+ elves[item[0]].set_objdump("-p", item[1])
+
+ for path in pkgfiles[package]:
+ if path in elves:
+ elves[path].open()
+ for func in warnfuncs:
+ func(path, package, d, elves.get(path), warnings)
+ for func in errorfuncs:
+ func(path, package, d, elves.get(path), errors)
+ if path in elves:
+ elves[path].close()
+
+ for w in warnings:
+ oe.qa.handle_error(w, warnings[w], d)
+ for e in errors:
+ oe.qa.handle_error(e, errors[e], d)
+
+def package_qa_check_rdepends(pkg, pkgdest, skip, taskdeps, packages, d):
+ # Don't do this check for kernel/module recipes, there aren't too many debug/development
+ # packages and you can get false positives e.g. on kernel-module-lirc-dev
+ if bb.data.inherits_class("kernel", d) or bb.data.inherits_class("module-base", d):
+ return
+
+ if not "-dbg" in pkg and not "packagegroup-" in pkg and not "-image" in pkg:
+ localdata = bb.data.createCopy(d)
+ localdata.setVar('OVERRIDES', localdata.getVar('OVERRIDES') + ':' + pkg)
+
+ # Now check the RDEPENDS
+ rdepends = bb.utils.explode_deps(localdata.getVar('RDEPENDS') or "")
+
+ # Now do the sanity check!!!
+ if "build-deps" not in skip:
+ for rdepend in rdepends:
+ if "-dbg" in rdepend and "debug-deps" not in skip:
+ error_msg = "%s rdepends on %s" % (pkg,rdepend)
+ oe.qa.handle_error("debug-deps", error_msg, d)
+ if (not "-dev" in pkg and not "-staticdev" in pkg) and rdepend.endswith("-dev") and "dev-deps" not in skip:
+ error_msg = "%s rdepends on %s" % (pkg, rdepend)
+ oe.qa.handle_error("dev-deps", error_msg, d)
+ if rdepend not in packages:
+ rdep_data = oe.packagedata.read_subpkgdata(rdepend, d)
+ if rdep_data and 'PN' in rdep_data and rdep_data['PN'] in taskdeps:
+ continue
+ if not rdep_data or not 'PN' in rdep_data:
+ pkgdata_dir = d.getVar("PKGDATA_DIR")
+ try:
+ possibles = os.listdir("%s/runtime-rprovides/%s/" % (pkgdata_dir, rdepend))
+ except OSError:
+ possibles = []
+ for p in possibles:
+ rdep_data = oe.packagedata.read_subpkgdata(p, d)
+ if rdep_data and 'PN' in rdep_data and rdep_data['PN'] in taskdeps:
+ break
+ if rdep_data and 'PN' in rdep_data and rdep_data['PN'] in taskdeps:
+ continue
+ if rdep_data and 'PN' in rdep_data:
+ error_msg = "%s rdepends on %s, but it isn't a build dependency, missing %s in DEPENDS or PACKAGECONFIG?" % (pkg, rdepend, rdep_data['PN'])
+ else:
+ error_msg = "%s rdepends on %s, but it isn't a build dependency?" % (pkg, rdepend)
+ oe.qa.handle_error("build-deps", error_msg, d)
+
+ if "file-rdeps" not in skip:
+ ignored_file_rdeps = set(['/bin/sh', '/usr/bin/env', 'rtld(GNU_HASH)'])
+ if bb.data.inherits_class('nativesdk', d):
+ ignored_file_rdeps |= set(['/bin/bash', '/usr/bin/perl', 'perl'])
+ # For Saving the FILERDEPENDS
+ filerdepends = {}
+ rdep_data = oe.packagedata.read_subpkgdata(pkg, d)
+ for key in rdep_data:
+ if key.startswith("FILERDEPENDS:"):
+ for subkey in bb.utils.explode_deps(rdep_data[key]):
+ if subkey not in ignored_file_rdeps and \
+ not subkey.startswith('perl('):
+ # We already know it starts with FILERDEPENDS_
+ filerdepends[subkey] = key[13:]
+
+ if filerdepends:
+ done = rdepends[:]
+ # Add the rprovides of itself
+ if pkg not in done:
+ done.insert(0, pkg)
+
+ # The python is not a package, but python-core provides it, so
+ # skip checking /usr/bin/python if python is in the rdeps, in
+ # case there is a RDEPENDS:pkg = "python" in the recipe.
+ for py in [ d.getVar('MLPREFIX') + "python", "python" ]:
+ if py in done:
+ filerdepends.pop("/usr/bin/python",None)
+ done.remove(py)
+ for rdep in done:
+ # The file dependencies may contain package names, e.g.,
+ # perl
+ filerdepends.pop(rdep,None)
+
+ # For Saving the FILERPROVIDES, RPROVIDES and FILES_INFO
+ rdep_data = oe.packagedata.read_subpkgdata(rdep, d)
+ for key in rdep_data:
+ if key.startswith("FILERPROVIDES:") or key.startswith("RPROVIDES:"):
+ for subkey in bb.utils.explode_deps(rdep_data[key]):
+ filerdepends.pop(subkey,None)
+ # Add the files list to the rprovides
+ if key.startswith("FILES_INFO:"):
+ # Use eval() to make it as a dict
+ for subkey in eval(rdep_data[key]):
+ filerdepends.pop(subkey,None)
+ if not filerdepends:
+ # Break if all the file rdepends are met
+ break
+ if filerdepends:
+ for key in filerdepends:
+ error_msg = "%s contained in package %s requires %s, but no providers found in RDEPENDS:%s?" % \
+ (filerdepends[key].replace(":%s" % pkg, "").replace("@underscore@", "_"), pkg, key, pkg)
+ oe.qa.handle_error("file-rdeps", error_msg, d)
+package_qa_check_rdepends[vardepsexclude] = "OVERRIDES"
+
+def package_qa_check_deps(pkg, pkgdest, d):
+
+ localdata = bb.data.createCopy(d)
+ localdata.setVar('OVERRIDES', pkg)
+
+ def check_valid_deps(var):
+ try:
+ rvar = bb.utils.explode_dep_versions2(localdata.getVar(var) or "")
+ except ValueError as e:
+ bb.fatal("%s:%s: %s" % (var, pkg, e))
+ for dep in rvar:
+ for v in rvar[dep]:
+ if v and not v.startswith(('< ', '= ', '> ', '<= ', '>=')):
+ error_msg = "%s:%s is invalid: %s (%s) only comparisons <, =, >, <=, and >= are allowed" % (var, pkg, dep, v)
+ oe.qa.handle_error("dep-cmp", error_msg, d)
+
+ check_valid_deps('RDEPENDS')
+ check_valid_deps('RRECOMMENDS')
+ check_valid_deps('RSUGGESTS')
+ check_valid_deps('RPROVIDES')
+ check_valid_deps('RREPLACES')
+ check_valid_deps('RCONFLICTS')
+
+QAPKGTEST[usrmerge] = "package_qa_check_usrmerge"
+def package_qa_check_usrmerge(pkg, d, messages):
+
+ pkgdest = d.getVar('PKGDEST')
+ pkg_dir = pkgdest + os.sep + pkg + os.sep
+ merged_dirs = ['bin', 'sbin', 'lib'] + d.getVar('MULTILIB_VARIANTS').split()
+ for f in merged_dirs:
+ if os.path.exists(pkg_dir + f) and not os.path.islink(pkg_dir + f):
+ msg = "%s package is not obeying usrmerge distro feature. /%s should be relocated to /usr." % (pkg, f)
+ oe.qa.add_message(messages, "usrmerge", msg)
+ return False
+ return True
+
+QAPKGTEST[perllocalpod] = "package_qa_check_perllocalpod"
+def package_qa_check_perllocalpod(pkg, d, messages):
+ """
+ Check that the recipe didn't ship a perlocal.pod file, which shouldn't be
+ installed in a distribution package. cpan.bbclass sets NO_PERLLOCAL=1 to
+ handle this for most recipes.
+ """
+ import glob
+ pkgd = oe.path.join(d.getVar('PKGDEST'), pkg)
+ podpath = oe.path.join(pkgd, d.getVar("libdir"), "perl*", "*", "*", "perllocal.pod")
+
+ matches = glob.glob(podpath)
+ if matches:
+ matches = [package_qa_clean_path(path, d, pkg) for path in matches]
+ msg = "%s contains perllocal.pod (%s), should not be installed" % (pkg, " ".join(matches))
+ oe.qa.add_message(messages, "perllocalpod", msg)
+
+QAPKGTEST[expanded-d] = "package_qa_check_expanded_d"
+def package_qa_check_expanded_d(package, d, messages):
+ """
+ Check for the expanded D (${D}) value in pkg_* and FILES
+ variables, warn the user to use it correctly.
+ """
+ sane = True
+ expanded_d = d.getVar('D')
+
+ for var in 'FILES','pkg_preinst', 'pkg_postinst', 'pkg_prerm', 'pkg_postrm':
+ bbvar = d.getVar(var + ":" + package) or ""
+ if expanded_d in bbvar:
+ if var == 'FILES':
+ oe.qa.add_message(messages, "expanded-d", "FILES in %s recipe should not contain the ${D} variable as it references the local build directory not the target filesystem, best solution is to remove the ${D} reference" % package)
+ sane = False
+ else:
+ oe.qa.add_message(messages, "expanded-d", "%s in %s recipe contains ${D}, it should be replaced by $D instead" % (var, package))
+ sane = False
+ return sane
+
+QAPKGTEST[unlisted-pkg-lics] = "package_qa_check_unlisted_pkg_lics"
+def package_qa_check_unlisted_pkg_lics(package, d, messages):
+ """
+ Check that all licenses for a package are among the licenses for the recipe.
+ """
+ pkg_lics = d.getVar('LICENSE:' + package)
+ if not pkg_lics:
+ return True
+
+ recipe_lics_set = oe.license.list_licenses(d.getVar('LICENSE'))
+ package_lics = oe.license.list_licenses(pkg_lics)
+ unlisted = package_lics - recipe_lics_set
+ if unlisted:
+ oe.qa.add_message(messages, "unlisted-pkg-lics",
+ "LICENSE:%s includes licenses (%s) that are not "
+ "listed in LICENSE" % (package, ' '.join(unlisted)))
+ return False
+ obsolete = set(oe.license.obsolete_license_list()) & package_lics - recipe_lics_set
+ if obsolete:
+ oe.qa.add_message(messages, "obsolete-license",
+ "LICENSE:%s includes obsolete licenses %s" % (package, ' '.join(obsolete)))
+ return False
+ return True
+
+QAPKGTEST[empty-dirs] = "package_qa_check_empty_dirs"
+def package_qa_check_empty_dirs(pkg, d, messages):
+ """
+ Check for the existence of files in directories that are expected to be
+ empty.
+ """
+
+ pkgd = oe.path.join(d.getVar('PKGDEST'), pkg)
+ for dir in (d.getVar('QA_EMPTY_DIRS') or "").split():
+ empty_dir = oe.path.join(pkgd, dir)
+ if os.path.exists(empty_dir) and os.listdir(empty_dir):
+ recommendation = (d.getVar('QA_EMPTY_DIRS_RECOMMENDATION:' + dir) or
+ "but it is expected to be empty")
+ msg = "%s installs files in %s, %s" % (pkg, dir, recommendation)
+ oe.qa.add_message(messages, "empty-dirs", msg)
+
+def package_qa_check_encoding(keys, encode, d):
+ def check_encoding(key, enc):
+ sane = True
+ value = d.getVar(key)
+ if value:
+ try:
+ s = value.encode(enc)
+ except UnicodeDecodeError as e:
+ error_msg = "%s has non %s characters" % (key,enc)
+ sane = False
+ oe.qa.handle_error("invalid-chars", error_msg, d)
+ return sane
+
+ for key in keys:
+ sane = check_encoding(key, encode)
+ if not sane:
+ break
+
+HOST_USER_UID := "${@os.getuid()}"
+HOST_USER_GID := "${@os.getgid()}"
+
+QAPATHTEST[host-user-contaminated] = "package_qa_check_host_user"
+def package_qa_check_host_user(path, name, d, elf, messages):
+ """Check for paths outside of /home which are owned by the user running bitbake."""
+
+ if not os.path.lexists(path):
+ return
+
+ dest = d.getVar('PKGDEST')
+ pn = d.getVar('PN')
+ home = os.path.join(dest, name, 'home')
+ if path == home or path.startswith(home + os.sep):
+ return
+
+ try:
+ stat = os.lstat(path)
+ except OSError as exc:
+ import errno
+ if exc.errno != errno.ENOENT:
+ raise
+ else:
+ check_uid = int(d.getVar('HOST_USER_UID'))
+ if stat.st_uid == check_uid:
+ oe.qa.add_message(messages, "host-user-contaminated", "%s: %s is owned by uid %d, which is the same as the user running bitbake. This may be due to host contamination" % (pn, package_qa_clean_path(path, d, name), check_uid))
+ return False
+
+ check_gid = int(d.getVar('HOST_USER_GID'))
+ if stat.st_gid == check_gid:
+ oe.qa.add_message(messages, "host-user-contaminated", "%s: %s is owned by gid %d, which is the same as the user running bitbake. This may be due to host contamination" % (pn, package_qa_clean_path(path, d, name), check_gid))
+ return False
+ return True
+
+QARECIPETEST[unhandled-features-check] = "package_qa_check_unhandled_features_check"
+def package_qa_check_unhandled_features_check(pn, d, messages):
+ if not bb.data.inherits_class('features_check', d):
+ var_set = False
+ for kind in ['DISTRO', 'MACHINE', 'COMBINED']:
+ for var in ['ANY_OF_' + kind + '_FEATURES', 'REQUIRED_' + kind + '_FEATURES', 'CONFLICT_' + kind + '_FEATURES']:
+ if d.getVar(var) is not None or d.hasOverrides(var):
+ var_set = True
+ if var_set:
+ oe.qa.handle_error("unhandled-features-check", "%s: recipe doesn't inherit features_check" % pn, d)
+
+QARECIPETEST[missing-update-alternatives] = "package_qa_check_missing_update_alternatives"
+def package_qa_check_missing_update_alternatives(pn, d, messages):
+ # Look at all packages and find out if any of those sets ALTERNATIVE variable
+ # without inheriting update-alternatives class
+ for pkg in (d.getVar('PACKAGES') or '').split():
+ if d.getVar('ALTERNATIVE:%s' % pkg) and not bb.data.inherits_class('update-alternatives', d):
+ oe.qa.handle_error("missing-update-alternatives", "%s: recipe defines ALTERNATIVE:%s but doesn't inherit update-alternatives. This might fail during do_rootfs later!" % (pn, pkg), d)
+
+# The PACKAGE FUNC to scan each package
+python do_package_qa () {
+ import subprocess
+ import oe.packagedata
+
+ bb.note("DO PACKAGE QA")
+
+ main_lic = d.getVar('LICENSE')
+
+ # Check for obsolete license references in main LICENSE (packages are checked below for any changes)
+ main_licenses = oe.license.list_licenses(d.getVar('LICENSE'))
+ obsolete = set(oe.license.obsolete_license_list()) & main_licenses
+ if obsolete:
+ oe.qa.handle_error("obsolete-license", "Recipe LICENSE includes obsolete licenses %s" % ' '.join(obsolete), d)
+
+ bb.build.exec_func("read_subpackage_metadata", d)
+
+ # Check non UTF-8 characters on recipe's metadata
+ package_qa_check_encoding(['DESCRIPTION', 'SUMMARY', 'LICENSE', 'SECTION'], 'utf-8', d)
+
+ logdir = d.getVar('T')
+ pn = d.getVar('PN')
+
+ # Scan the packages...
+ pkgdest = d.getVar('PKGDEST')
+ packages = set((d.getVar('PACKAGES') or '').split())
+
+ global pkgfiles
+ pkgfiles = {}
+ for pkg in packages:
+ pkgfiles[pkg] = []
+ pkgdir = os.path.join(pkgdest, pkg)
+ for walkroot, dirs, files in os.walk(pkgdir):
+ # Don't walk into top-level CONTROL or DEBIAN directories as these
+ # are temporary directories created by do_package.
+ if walkroot == pkgdir:
+ for control in ("CONTROL", "DEBIAN"):
+ if control in dirs:
+ dirs.remove(control)
+ for file in files:
+ pkgfiles[pkg].append(os.path.join(walkroot, file))
+
+ # no packages should be scanned
+ if not packages:
+ return
+
+ import re
+ # The package name matches the [a-z0-9.+-]+ regular expression
+ pkgname_pattern = re.compile(r"^[a-z0-9.+-]+$")
+
+ taskdepdata = d.getVar("BB_TASKDEPDATA", False)
+ taskdeps = set()
+ for dep in taskdepdata:
+ taskdeps.add(taskdepdata[dep][0])
+
+ def parse_test_matrix(matrix_name):
+ testmatrix = d.getVarFlags(matrix_name) or {}
+ g = globals()
+ warnchecks = []
+ for w in (d.getVar("WARN_QA") or "").split():
+ if w in skip:
+ continue
+ if w in testmatrix and testmatrix[w] in g:
+ warnchecks.append(g[testmatrix[w]])
+
+ errorchecks = []
+ for e in (d.getVar("ERROR_QA") or "").split():
+ if e in skip:
+ continue
+ if e in testmatrix and testmatrix[e] in g:
+ errorchecks.append(g[testmatrix[e]])
+ return warnchecks, errorchecks
+
+ for package in packages:
+ skip = set((d.getVar('INSANE_SKIP') or "").split() +
+ (d.getVar('INSANE_SKIP:' + package) or "").split())
+ if skip:
+ bb.note("Package %s skipping QA tests: %s" % (package, str(skip)))
+
+ bb.note("Checking Package: %s" % package)
+ # Check package name
+ if not pkgname_pattern.match(package):
+ oe.qa.handle_error("pkgname",
+ "%s doesn't match the [a-z0-9.+-]+ regex" % package, d)
+
+ warn_checks, error_checks = parse_test_matrix("QAPATHTEST")
+ package_qa_walk(warn_checks, error_checks, package, d)
+
+ warn_checks, error_checks = parse_test_matrix("QAPKGTEST")
+ package_qa_package(warn_checks, error_checks, package, d)
+
+ package_qa_check_rdepends(package, pkgdest, skip, taskdeps, packages, d)
+ package_qa_check_deps(package, pkgdest, d)
+
+ warn_checks, error_checks = parse_test_matrix("QARECIPETEST")
+ package_qa_recipe(warn_checks, error_checks, pn, d)
+
+ if 'libdir' in d.getVar("ALL_QA").split():
+ package_qa_check_libdir(d)
+
+ oe.qa.exit_if_errors(d)
+}
+
+# binutils is used for most checks, so need to set as dependency
+# POPULATESYSROOTDEPS is defined in staging class.
+do_package_qa[depends] += "${POPULATESYSROOTDEPS}"
+do_package_qa[vardeps] = "${@bb.utils.contains('ERROR_QA', 'empty-dirs', 'QA_EMPTY_DIRS', '', d)}"
+do_package_qa[vardepsexclude] = "BB_TASKDEPDATA"
+do_package_qa[rdeptask] = "do_packagedata"
+addtask do_package_qa after do_packagedata do_package before do_build
+
+# Add the package specific INSANE_SKIPs to the sstate dependencies
+python() {
+ pkgs = (d.getVar('PACKAGES') or '').split()
+ for pkg in pkgs:
+ d.appendVarFlag("do_package_qa", "vardeps", " INSANE_SKIP:{}".format(pkg))
+}
+
+SSTATETASKS += "do_package_qa"
+do_package_qa[sstate-inputdirs] = ""
+do_package_qa[sstate-outputdirs] = ""
+python do_package_qa_setscene () {
+ sstate_setscene(d)
+}
+addtask do_package_qa_setscene
+
+python do_qa_sysroot() {
+ bb.note("QA checking do_populate_sysroot")
+ sysroot_destdir = d.expand('${SYSROOT_DESTDIR}')
+ for sysroot_dir in d.expand('${SYSROOT_DIRS}').split():
+ qa_check_staged(sysroot_destdir + sysroot_dir, d)
+ oe.qa.exit_with_message_if_errors("do_populate_sysroot for this recipe installed files with QA issues", d)
+}
+do_populate_sysroot[postfuncs] += "do_qa_sysroot"
+
+python do_qa_patch() {
+ import subprocess
+
+ ###########################################################################
+ # Check patch.log for fuzz warnings
+ #
+ # Further information on why we check for patch fuzz warnings:
+ # http://lists.openembedded.org/pipermail/openembedded-core/2018-March/148675.html
+ # https://bugzilla.yoctoproject.org/show_bug.cgi?id=10450
+ ###########################################################################
+
+ logdir = d.getVar('T')
+ patchlog = os.path.join(logdir,"log.do_patch")
+
+ if os.path.exists(patchlog):
+ fuzzheader = '--- Patch fuzz start ---'
+ fuzzfooter = '--- Patch fuzz end ---'
+ statement = "grep -e '%s' %s > /dev/null" % (fuzzheader, patchlog)
+ if subprocess.call(statement, shell=True) == 0:
+ msg = "Fuzz detected:\n\n"
+ fuzzmsg = ""
+ inFuzzInfo = False
+ f = open(patchlog, "r")
+ for line in f:
+ if fuzzheader in line:
+ inFuzzInfo = True
+ fuzzmsg = ""
+ elif fuzzfooter in line:
+ fuzzmsg = fuzzmsg.replace('\n\n', '\n')
+ msg += fuzzmsg
+ msg += "\n"
+ inFuzzInfo = False
+ elif inFuzzInfo and not 'Now at patch' in line:
+ fuzzmsg += line
+ f.close()
+ msg += "The context lines in the patches can be updated with devtool:\n"
+ msg += "\n"
+ msg += " devtool modify %s\n" % d.getVar('PN')
+ msg += " devtool finish --force-patch-refresh %s <layer_path>\n\n" % d.getVar('PN')
+ msg += "Don't forget to review changes done by devtool!\n"
+ if bb.utils.filter('ERROR_QA', 'patch-fuzz', d):
+ bb.error(msg)
+ elif bb.utils.filter('WARN_QA', 'patch-fuzz', d):
+ bb.warn(msg)
+ msg = "Patch log indicates that patches do not apply cleanly."
+ oe.qa.handle_error("patch-fuzz", msg, d)
+
+ # Check if the patch contains a correctly formatted and spelled Upstream-Status
+ import re
+ from oe import patch
+
+ coremeta_path = os.path.join(d.getVar('COREBASE'), 'meta', '')
+ for url in patch.src_patches(d):
+ (_, _, fullpath, _, _, _) = bb.fetch.decodeurl(url)
+
+ # skip patches not in oe-core
+ if not os.path.abspath(fullpath).startswith(coremeta_path):
+ continue
+
+ kinda_status_re = re.compile(r"^.*upstream.*status.*$", re.IGNORECASE | re.MULTILINE)
+ strict_status_re = re.compile(r"^Upstream-Status: (Pending|Submitted|Denied|Accepted|Inappropriate|Backport|Inactive-Upstream)( .+)?$", re.MULTILINE)
+ guidelines = "https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines#Patch_Header_Recommendations:_Upstream-Status"
+
+ with open(fullpath, encoding='utf-8', errors='ignore') as f:
+ file_content = f.read()
+ match_kinda = kinda_status_re.search(file_content)
+ match_strict = strict_status_re.search(file_content)
+
+ if not match_strict:
+ if match_kinda:
+ bb.error("Malformed Upstream-Status in patch\n%s\nPlease correct according to %s :\n%s" % (fullpath, guidelines, match_kinda.group(0)))
+ else:
+ bb.error("Missing Upstream-Status in patch\n%s\nPlease add according to %s ." % (fullpath, guidelines))
+}
+
+python do_qa_configure() {
+ import subprocess
+
+ ###########################################################################
+ # Check config.log for cross compile issues
+ ###########################################################################
+
+ configs = []
+ workdir = d.getVar('WORKDIR')
+
+ skip = (d.getVar('INSANE_SKIP') or "").split()
+ skip_configure_unsafe = False
+ if 'configure-unsafe' in skip:
+ bb.note("Recipe %s skipping qa checking: configure-unsafe" % d.getVar('PN'))
+ skip_configure_unsafe = True
+
+ if bb.data.inherits_class('autotools', d) and not skip_configure_unsafe:
+ bb.note("Checking autotools environment for common misconfiguration")
+ for root, dirs, files in os.walk(workdir):
+ statement = "grep -q -F -e 'is unsafe for cross-compilation' %s" % \
+ os.path.join(root,"config.log")
+ if "config.log" in files:
+ if subprocess.call(statement, shell=True) == 0:
+ error_msg = """This autoconf log indicates errors, it looked at host include and/or library paths while determining system capabilities.
+Rerun configure task after fixing this."""
+ oe.qa.handle_error("configure-unsafe", error_msg, d)
+
+ if "configure.ac" in files:
+ configs.append(os.path.join(root,"configure.ac"))
+ if "configure.in" in files:
+ configs.append(os.path.join(root, "configure.in"))
+
+ ###########################################################################
+ # Check gettext configuration and dependencies are correct
+ ###########################################################################
+
+ skip_configure_gettext = False
+ if 'configure-gettext' in skip:
+ bb.note("Recipe %s skipping qa checking: configure-gettext" % d.getVar('PN'))
+ skip_configure_gettext = True
+
+ cnf = d.getVar('EXTRA_OECONF') or ""
+ if not ("gettext" in d.getVar('P') or "gcc-runtime" in d.getVar('P') or \
+ "--disable-nls" in cnf or skip_configure_gettext):
+ ml = d.getVar("MLPREFIX") or ""
+ if bb.data.inherits_class('cross-canadian', d):
+ gt = "nativesdk-gettext"
+ else:
+ gt = "gettext-native"
+ deps = bb.utils.explode_deps(d.getVar('DEPENDS') or "")
+ if gt not in deps:
+ for config in configs:
+ gnu = "grep \"^[[:space:]]*AM_GNU_GETTEXT\" %s >/dev/null" % config
+ if subprocess.call(gnu, shell=True) == 0:
+ error_msg = "AM_GNU_GETTEXT used but no inherit gettext"
+ oe.qa.handle_error("configure-gettext", error_msg, d)
+
+ ###########################################################################
+ # Check unrecognised configure options (with a white list)
+ ###########################################################################
+ if bb.data.inherits_class("autotools", d):
+ bb.note("Checking configure output for unrecognised options")
+ try:
+ if bb.data.inherits_class("autotools", d):
+ flag = "WARNING: unrecognized options:"
+ log = os.path.join(d.getVar('B'), 'config.log')
+ output = subprocess.check_output(['grep', '-F', flag, log]).decode("utf-8").replace(', ', ' ').replace('"', '')
+ options = set()
+ for line in output.splitlines():
+ options |= set(line.partition(flag)[2].split())
+ ignore_opts = set(d.getVar("UNKNOWN_CONFIGURE_OPT_IGNORE").split())
+ options -= ignore_opts
+ if options:
+ pn = d.getVar('PN')
+ error_msg = pn + ": configure was passed unrecognised options: " + " ".join(options)
+ oe.qa.handle_error("unknown-configure-option", error_msg, d)
+ except subprocess.CalledProcessError:
+ pass
+
+ # Check invalid PACKAGECONFIG
+ pkgconfig = (d.getVar("PACKAGECONFIG") or "").split()
+ if pkgconfig:
+ pkgconfigflags = d.getVarFlags("PACKAGECONFIG") or {}
+ for pconfig in pkgconfig:
+ if pconfig not in pkgconfigflags:
+ pn = d.getVar('PN')
+ error_msg = "%s: invalid PACKAGECONFIG: %s" % (pn, pconfig)
+ oe.qa.handle_error("invalid-packageconfig", error_msg, d)
+
+ oe.qa.exit_if_errors(d)
+}
+
+def unpack_check_src_uri(pn, d):
+ import re
+
+ skip = (d.getVar('INSANE_SKIP') or "").split()
+ if 'src-uri-bad' in skip:
+ bb.note("Recipe %s skipping qa checking: src-uri-bad" % d.getVar('PN'))
+ return
+
+ if "${PN}" in d.getVar("SRC_URI", False):
+ oe.qa.handle_error("src-uri-bad", "%s: SRC_URI uses PN not BPN" % pn, d)
+
+ for url in d.getVar("SRC_URI").split():
+ # Search for github and gitlab URLs that pull unstable archives (comment for future greppers)
+ if re.search(r"git(hu|la)b\.com/.+/.+/archive/.+", url):
+ oe.qa.handle_error("src-uri-bad", "%s: SRC_URI uses unstable GitHub/GitLab archives, convert recipe to use git protocol" % pn, d)
+
+python do_qa_unpack() {
+ src_uri = d.getVar('SRC_URI')
+ s_dir = d.getVar('S')
+ if src_uri and not os.path.exists(s_dir):
+ bb.warn('%s: the directory %s (%s) pointed to by the S variable doesn\'t exist - please set S within the recipe to point to where the source has been unpacked to' % (d.getVar('PN'), d.getVar('S', False), s_dir))
+
+ unpack_check_src_uri(d.getVar('PN'), d)
+}
+
+# Check for patch fuzz
+do_patch[postfuncs] += "do_qa_patch "
+
+# Check broken config.log files, for packages requiring Gettext which
+# don't have it in DEPENDS.
+#addtask qa_configure after do_configure before do_compile
+do_configure[postfuncs] += "do_qa_configure "
+
+# Check does S exist.
+do_unpack[postfuncs] += "do_qa_unpack"
+
+python () {
+ import re
+
+ tests = d.getVar('ALL_QA').split()
+ if "desktop" in tests:
+ d.appendVar("PACKAGE_DEPENDS", " desktop-file-utils-native")
+
+ ###########################################################################
+ # Check various variables
+ ###########################################################################
+
+ # Checking ${FILESEXTRAPATHS}
+ extrapaths = (d.getVar("FILESEXTRAPATHS") or "")
+ if '__default' not in extrapaths.split(":"):
+ msg = "FILESEXTRAPATHS-variable, must always use :prepend (or :append)\n"
+ msg += "type of assignment, and don't forget the colon.\n"
+ msg += "Please assign it with the format of:\n"
+ msg += " FILESEXTRAPATHS:append := \":${THISDIR}/Your_Files_Path\" or\n"
+ msg += " FILESEXTRAPATHS:prepend := \"${THISDIR}/Your_Files_Path:\"\n"
+ msg += "in your bbappend file\n\n"
+ msg += "Your incorrect assignment is:\n"
+ msg += "%s\n" % extrapaths
+ bb.warn(msg)
+
+ overrides = d.getVar('OVERRIDES').split(':')
+ pn = d.getVar('PN')
+ if pn in overrides:
+ msg = 'Recipe %s has PN of "%s" which is in OVERRIDES, this can result in unexpected behaviour.' % (d.getVar("FILE"), pn)
+ oe.qa.handle_error("pn-overrides", msg, d)
+ prog = re.compile(r'[A-Z]')
+ if prog.search(pn):
+ oe.qa.handle_error("uppercase-pn", 'PN: %s is upper case, this can result in unexpected behavior.' % pn, d)
+
+ # Some people mistakenly use DEPENDS:${PN} instead of DEPENDS and wonder
+ # why it doesn't work.
+ if (d.getVar(d.expand('DEPENDS:${PN}'))):
+ oe.qa.handle_error("pkgvarcheck", "recipe uses DEPENDS:${PN}, should use DEPENDS", d)
+
+ issues = []
+ if (d.getVar('PACKAGES') or "").split():
+ for dep in (d.getVar('QADEPENDS') or "").split():
+ d.appendVarFlag('do_package_qa', 'depends', " %s:do_populate_sysroot" % dep)
+ for var in 'RDEPENDS', 'RRECOMMENDS', 'RSUGGESTS', 'RCONFLICTS', 'RPROVIDES', 'RREPLACES', 'FILES', 'pkg_preinst', 'pkg_postinst', 'pkg_prerm', 'pkg_postrm', 'ALLOW_EMPTY':
+ if d.getVar(var, False):
+ issues.append(var)
+
+ fakeroot_tests = d.getVar('FAKEROOT_QA').split()
+ if set(tests) & set(fakeroot_tests):
+ d.setVarFlag('do_package_qa', 'fakeroot', '1')
+ d.appendVarFlag('do_package_qa', 'depends', ' virtual/fakeroot-native:do_populate_sysroot')
+ else:
+ d.setVarFlag('do_package_qa', 'rdeptask', '')
+ for i in issues:
+ oe.qa.handle_error("pkgvarcheck", "%s: Variable %s is set as not being package specific, please fix this." % (d.getVar("FILE"), i), d)
+
+ if 'native-last' not in (d.getVar('INSANE_SKIP') or "").split():
+ for native_class in ['native', 'nativesdk']:
+ if bb.data.inherits_class(native_class, d):
+
+ inherited_classes = d.getVar('__inherit_cache', False) or []
+ needle = "/" + native_class
+
+ bbclassextend = (d.getVar('BBCLASSEXTEND') or '').split()
+ # BBCLASSEXTEND items are always added in the end
+ skip_classes = bbclassextend
+ if bb.data.inherits_class('native', d) or 'native' in bbclassextend:
+ # native also inherits nopackages and relocatable bbclasses
+ skip_classes.extend(['nopackages', 'relocatable'])
+
+ broken_order = []
+ for class_item in reversed(inherited_classes):
+ if needle not in class_item:
+ for extend_item in skip_classes:
+ if '/%s.bbclass' % extend_item in class_item:
+ break
+ else:
+ pn = d.getVar('PN')
+ broken_order.append(os.path.basename(class_item))
+ else:
+ break
+ if broken_order:
+ oe.qa.handle_error("native-last", "%s: native/nativesdk class is not inherited last, this can result in unexpected behaviour. "
+ "Classes inherited after native/nativesdk: %s" % (pn, " ".join(broken_order)), d)
+
+ oe.qa.exit_if_errors(d)
+}
diff --git a/poky/meta/classes-global/license.bbclass b/poky/meta/classes-global/license.bbclass
new file mode 100644
index 0000000..560acb8
--- /dev/null
+++ b/poky/meta/classes-global/license.bbclass
@@ -0,0 +1,426 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+# Populates LICENSE_DIRECTORY as set in distro config with the license files as set by
+# LIC_FILES_CHKSUM.
+# TODO:
+# - There is a real issue revolving around license naming standards.
+
+LICENSE_DIRECTORY ??= "${DEPLOY_DIR}/licenses"
+LICSSTATEDIR = "${WORKDIR}/license-destdir/"
+
+# Create extra package with license texts and add it to RRECOMMENDS:${PN}
+LICENSE_CREATE_PACKAGE[type] = "boolean"
+LICENSE_CREATE_PACKAGE ??= "0"
+LICENSE_PACKAGE_SUFFIX ??= "-lic"
+LICENSE_FILES_DIRECTORY ??= "${datadir}/licenses/"
+
+addtask populate_lic after do_patch before do_build
+do_populate_lic[dirs] = "${LICSSTATEDIR}/${PN}"
+do_populate_lic[cleandirs] = "${LICSSTATEDIR}"
+
+python do_populate_lic() {
+ """
+ Populate LICENSE_DIRECTORY with licenses.
+ """
+ lic_files_paths = find_license_files(d)
+
+ # The base directory we wrangle licenses to
+ destdir = os.path.join(d.getVar('LICSSTATEDIR'), d.getVar('PN'))
+ copy_license_files(lic_files_paths, destdir)
+ info = get_recipe_info(d)
+ with open(os.path.join(destdir, "recipeinfo"), "w") as f:
+ for key in sorted(info.keys()):
+ f.write("%s: %s\n" % (key, info[key]))
+ oe.qa.exit_if_errors(d)
+}
+
+PSEUDO_IGNORE_PATHS .= ",${@','.join(((d.getVar('COMMON_LICENSE_DIR') or '') + ' ' + (d.getVar('LICENSE_PATH') or '') + ' ' + d.getVar('COREBASE') + '/meta/COPYING').split())}"
+# it would be better to copy them in do_install:append, but find_license_filesa is python
+python perform_packagecopy:prepend () {
+ enabled = oe.data.typed_value('LICENSE_CREATE_PACKAGE', d)
+ if d.getVar('CLASSOVERRIDE') == 'class-target' and enabled:
+ lic_files_paths = find_license_files(d)
+
+ # LICENSE_FILES_DIRECTORY starts with '/' so os.path.join cannot be used to join D and LICENSE_FILES_DIRECTORY
+ destdir = d.getVar('D') + os.path.join(d.getVar('LICENSE_FILES_DIRECTORY'), d.getVar('PN'))
+ copy_license_files(lic_files_paths, destdir)
+ add_package_and_files(d)
+}
+perform_packagecopy[vardeps] += "LICENSE_CREATE_PACKAGE"
+
+def get_recipe_info(d):
+ info = {}
+ info["PV"] = d.getVar("PV")
+ info["PR"] = d.getVar("PR")
+ info["LICENSE"] = d.getVar("LICENSE")
+ return info
+
+def add_package_and_files(d):
+ packages = d.getVar('PACKAGES')
+ files = d.getVar('LICENSE_FILES_DIRECTORY')
+ pn = d.getVar('PN')
+ pn_lic = "%s%s" % (pn, d.getVar('LICENSE_PACKAGE_SUFFIX', False))
+ if pn_lic in packages.split():
+ bb.warn("%s package already existed in %s." % (pn_lic, pn))
+ else:
+ # first in PACKAGES to be sure that nothing else gets LICENSE_FILES_DIRECTORY
+ d.setVar('PACKAGES', "%s %s" % (pn_lic, packages))
+ d.setVar('FILES:' + pn_lic, files)
+
+def copy_license_files(lic_files_paths, destdir):
+ import shutil
+ import errno
+
+ bb.utils.mkdirhier(destdir)
+ for (basename, path, beginline, endline) in lic_files_paths:
+ try:
+ src = path
+ dst = os.path.join(destdir, basename)
+ if os.path.exists(dst):
+ os.remove(dst)
+ if os.path.islink(src):
+ src = os.path.realpath(src)
+ canlink = os.access(src, os.W_OK) and (os.stat(src).st_dev == os.stat(destdir).st_dev) and beginline is None and endline is None
+ if canlink:
+ try:
+ os.link(src, dst)
+ except OSError as err:
+ if err.errno == errno.EXDEV:
+ # Copy license files if hardlink is not possible even if st_dev is the
+ # same on source and destination (docker container with device-mapper?)
+ canlink = False
+ else:
+ raise
+ # Only chown if we did hardlink and we're running under pseudo
+ if canlink and os.environ.get('PSEUDO_DISABLED') == '0':
+ os.chown(dst,0,0)
+ if not canlink:
+ begin_idx = max(0, int(beginline) - 1) if beginline is not None else None
+ end_idx = max(0, int(endline)) if endline is not None else None
+ if begin_idx is None and end_idx is None:
+ shutil.copyfile(src, dst)
+ else:
+ with open(src, 'rb') as src_f:
+ with open(dst, 'wb') as dst_f:
+ dst_f.write(b''.join(src_f.readlines()[begin_idx:end_idx]))
+
+ except Exception as e:
+ bb.warn("Could not copy license file %s to %s: %s" % (src, dst, e))
+
+def find_license_files(d):
+ """
+ Creates list of files used in LIC_FILES_CHKSUM and generic LICENSE files.
+ """
+ import shutil
+ import oe.license
+ from collections import defaultdict, OrderedDict
+
+ # All the license files for the package
+ lic_files = d.getVar('LIC_FILES_CHKSUM') or ""
+ pn = d.getVar('PN')
+ # The license files are located in S/LIC_FILE_CHECKSUM.
+ srcdir = d.getVar('S')
+ # Directory we store the generic licenses as set in the distro configuration
+ generic_directory = d.getVar('COMMON_LICENSE_DIR')
+ # List of basename, path tuples
+ lic_files_paths = []
+ # hash for keep track generic lics mappings
+ non_generic_lics = {}
+ # Entries from LIC_FILES_CHKSUM
+ lic_chksums = {}
+ license_source_dirs = []
+ license_source_dirs.append(generic_directory)
+ try:
+ additional_lic_dirs = d.getVar('LICENSE_PATH').split()
+ for lic_dir in additional_lic_dirs:
+ license_source_dirs.append(lic_dir)
+ except:
+ pass
+
+ class FindVisitor(oe.license.LicenseVisitor):
+ def visit_Str(self, node):
+ #
+ # Until I figure out what to do with
+ # the two modifiers I support (or greater = +
+ # and "with exceptions" being *
+ # we'll just strip out the modifier and put
+ # the base license.
+ find_license(node.s.replace("+", "").replace("*", ""))
+ self.generic_visit(node)
+
+ def visit_Constant(self, node):
+ find_license(node.value.replace("+", "").replace("*", ""))
+ self.generic_visit(node)
+
+ def find_license(license_type):
+ try:
+ bb.utils.mkdirhier(gen_lic_dest)
+ except:
+ pass
+ spdx_generic = None
+ license_source = None
+ # If the generic does not exist we need to check to see if there is an SPDX mapping to it,
+ # unless NO_GENERIC_LICENSE is set.
+ for lic_dir in license_source_dirs:
+ if not os.path.isfile(os.path.join(lic_dir, license_type)):
+ if d.getVarFlag('SPDXLICENSEMAP', license_type) != None:
+ # Great, there is an SPDXLICENSEMAP. We can copy!
+ bb.debug(1, "We need to use a SPDXLICENSEMAP for %s" % (license_type))
+ spdx_generic = d.getVarFlag('SPDXLICENSEMAP', license_type)
+ license_source = lic_dir
+ break
+ elif os.path.isfile(os.path.join(lic_dir, license_type)):
+ spdx_generic = license_type
+ license_source = lic_dir
+ break
+
+ non_generic_lic = d.getVarFlag('NO_GENERIC_LICENSE', license_type)
+ if spdx_generic and license_source:
+ # we really should copy to generic_ + spdx_generic, however, that ends up messing the manifest
+ # audit up. This should be fixed in emit_pkgdata (or, we actually got and fix all the recipes)
+
+ lic_files_paths.append(("generic_" + license_type, os.path.join(license_source, spdx_generic),
+ None, None))
+
+ # The user may attempt to use NO_GENERIC_LICENSE for a generic license which doesn't make sense
+ # and should not be allowed, warn the user in this case.
+ if d.getVarFlag('NO_GENERIC_LICENSE', license_type):
+ oe.qa.handle_error("license-no-generic",
+ "%s: %s is a generic license, please don't use NO_GENERIC_LICENSE for it." % (pn, license_type), d)
+
+ elif non_generic_lic and non_generic_lic in lic_chksums:
+ # if NO_GENERIC_LICENSE is set, we copy the license files from the fetched source
+ # of the package rather than the license_source_dirs.
+ lic_files_paths.append(("generic_" + license_type,
+ os.path.join(srcdir, non_generic_lic), None, None))
+ non_generic_lics[non_generic_lic] = license_type
+ else:
+ # Explicitly avoid the CLOSED license because this isn't generic
+ if license_type != 'CLOSED':
+ # And here is where we warn people that their licenses are lousy
+ oe.qa.handle_error("license-exists",
+ "%s: No generic license file exists for: %s in any provider" % (pn, license_type), d)
+ pass
+
+ if not generic_directory:
+ bb.fatal("COMMON_LICENSE_DIR is unset. Please set this in your distro config")
+
+ for url in lic_files.split():
+ try:
+ (method, host, path, user, pswd, parm) = bb.fetch.decodeurl(url)
+ if method != "file" or not path:
+ raise bb.fetch.MalformedUrl()
+ except bb.fetch.MalformedUrl:
+ bb.fatal("%s: LIC_FILES_CHKSUM contains an invalid URL: %s" % (d.getVar('PF'), url))
+ # We want the license filename and path
+ chksum = parm.get('md5', None)
+ beginline = parm.get('beginline')
+ endline = parm.get('endline')
+ lic_chksums[path] = (chksum, beginline, endline)
+
+ v = FindVisitor()
+ try:
+ v.visit_string(d.getVar('LICENSE'))
+ except oe.license.InvalidLicense as exc:
+ bb.fatal('%s: %s' % (d.getVar('PF'), exc))
+ except SyntaxError:
+ oe.qa.handle_error("license-syntax",
+ "%s: Failed to parse it's LICENSE field." % (d.getVar('PF')), d)
+ # Add files from LIC_FILES_CHKSUM to list of license files
+ lic_chksum_paths = defaultdict(OrderedDict)
+ for path, data in sorted(lic_chksums.items()):
+ lic_chksum_paths[os.path.basename(path)][data] = (os.path.join(srcdir, path), data[1], data[2])
+ for basename, files in lic_chksum_paths.items():
+ if len(files) == 1:
+ # Don't copy again a LICENSE already handled as non-generic
+ if basename in non_generic_lics:
+ continue
+ data = list(files.values())[0]
+ lic_files_paths.append(tuple([basename] + list(data)))
+ else:
+ # If there are multiple different license files with identical
+ # basenames we rename them to <file>.0, <file>.1, ...
+ for i, data in enumerate(files.values()):
+ lic_files_paths.append(tuple(["%s.%d" % (basename, i)] + list(data)))
+
+ return lic_files_paths
+
+def return_spdx(d, license):
+ """
+ This function returns the spdx mapping of a license if it exists.
+ """
+ return d.getVarFlag('SPDXLICENSEMAP', license)
+
+def canonical_license(d, license):
+ """
+ Return the canonical (SPDX) form of the license if available (so GPLv3
+ becomes GPL-3.0-only) or the passed license if there is no canonical form.
+ """
+ return d.getVarFlag('SPDXLICENSEMAP', license) or license
+
+def expand_wildcard_licenses(d, wildcard_licenses):
+ """
+ There are some common wildcard values users may want to use. Support them
+ here.
+ """
+ licenses = set(wildcard_licenses)
+ mapping = {
+ "AGPL-3.0*" : ["AGPL-3.0-only", "AGPL-3.0-or-later"],
+ "GPL-3.0*" : ["GPL-3.0-only", "GPL-3.0-or-later"],
+ "LGPL-3.0*" : ["LGPL-3.0-only", "LGPL-3.0-or-later"],
+ }
+ for k in mapping:
+ if k in wildcard_licenses:
+ licenses.remove(k)
+ for item in mapping[k]:
+ licenses.add(item)
+
+ for l in licenses:
+ if l in oe.license.obsolete_license_list():
+ bb.fatal("Error, %s is an obsolete license, please use an SPDX reference in INCOMPATIBLE_LICENSE" % l)
+ if "*" in l:
+ bb.fatal("Error, %s is an invalid license wildcard entry" % l)
+
+ return list(licenses)
+
+def incompatible_license_contains(license, truevalue, falsevalue, d):
+ license = canonical_license(d, license)
+ bad_licenses = (d.getVar('INCOMPATIBLE_LICENSE') or "").split()
+ bad_licenses = expand_wildcard_licenses(d, bad_licenses)
+ return truevalue if license in bad_licenses else falsevalue
+
+def incompatible_pkg_license(d, dont_want_licenses, license):
+ # Handles an "or" or two license sets provided by
+ # flattened_licenses(), pick one that works if possible.
+ def choose_lic_set(a, b):
+ return a if all(oe.license.license_ok(canonical_license(d, lic),
+ dont_want_licenses) for lic in a) else b
+
+ try:
+ licenses = oe.license.flattened_licenses(license, choose_lic_set)
+ except oe.license.LicenseError as exc:
+ bb.fatal('%s: %s' % (d.getVar('P'), exc))
+
+ incompatible_lic = []
+ for l in licenses:
+ license = canonical_license(d, l)
+ if not oe.license.license_ok(license, dont_want_licenses):
+ incompatible_lic.append(license)
+
+ return sorted(incompatible_lic)
+
+def incompatible_license(d, dont_want_licenses, package=None):
+ """
+ This function checks if a recipe has only incompatible licenses. It also
+ take into consideration 'or' operand. dont_want_licenses should be passed
+ as canonical (SPDX) names.
+ """
+ import oe.license
+ license = d.getVar("LICENSE:%s" % package) if package else None
+ if not license:
+ license = d.getVar('LICENSE')
+
+ return incompatible_pkg_license(d, dont_want_licenses, license)
+
+def check_license_flags(d):
+ """
+ This function checks if a recipe has any LICENSE_FLAGS that
+ aren't acceptable.
+
+ If it does, it returns the all LICENSE_FLAGS missing from the list
+ of acceptable license flags, or all of the LICENSE_FLAGS if there
+ is no list of acceptable flags.
+
+ If everything is is acceptable, it returns None.
+ """
+
+ def license_flag_matches(flag, acceptlist, pn):
+ """
+ Return True if flag matches something in acceptlist, None if not.
+
+ Before we test a flag against the acceptlist, we append _${PN}
+ to it. We then try to match that string against the
+ acceptlist. This covers the normal case, where we expect
+ LICENSE_FLAGS to be a simple string like 'commercial', which
+ the user typically matches exactly in the acceptlist by
+ explicitly appending the package name e.g 'commercial_foo'.
+ If we fail the match however, we then split the flag across
+ '_' and append each fragment and test until we either match or
+ run out of fragments.
+ """
+ flag_pn = ("%s_%s" % (flag, pn))
+ for candidate in acceptlist:
+ if flag_pn == candidate:
+ return True
+
+ flag_cur = ""
+ flagments = flag_pn.split("_")
+ flagments.pop() # we've already tested the full string
+ for flagment in flagments:
+ if flag_cur:
+ flag_cur += "_"
+ flag_cur += flagment
+ for candidate in acceptlist:
+ if flag_cur == candidate:
+ return True
+ return False
+
+ def all_license_flags_match(license_flags, acceptlist):
+ """ Return all unmatched flags, None if all flags match """
+ pn = d.getVar('PN')
+ split_acceptlist = acceptlist.split()
+ flags = []
+ for flag in license_flags.split():
+ if not license_flag_matches(flag, split_acceptlist, pn):
+ flags.append(flag)
+ return flags if flags else None
+
+ license_flags = d.getVar('LICENSE_FLAGS')
+ if license_flags:
+ acceptlist = d.getVar('LICENSE_FLAGS_ACCEPTED')
+ if not acceptlist:
+ return license_flags.split()
+ unmatched_flags = all_license_flags_match(license_flags, acceptlist)
+ if unmatched_flags:
+ return unmatched_flags
+ return None
+
+def check_license_format(d):
+ """
+ This function checks if LICENSE is well defined,
+ Validate operators in LICENSES.
+ No spaces are allowed between LICENSES.
+ """
+ pn = d.getVar('PN')
+ licenses = d.getVar('LICENSE')
+ from oe.license import license_operator, license_operator_chars, license_pattern
+
+ elements = list(filter(lambda x: x.strip(), license_operator.split(licenses)))
+ for pos, element in enumerate(elements):
+ if license_pattern.match(element):
+ if pos > 0 and license_pattern.match(elements[pos - 1]):
+ oe.qa.handle_error('license-format',
+ '%s: LICENSE value "%s" has an invalid format - license names ' \
+ 'must be separated by the following characters to indicate ' \
+ 'the license selection: %s' %
+ (pn, licenses, license_operator_chars), d)
+ elif not license_operator.match(element):
+ oe.qa.handle_error('license-format',
+ '%s: LICENSE value "%s" has an invalid separator "%s" that is not ' \
+ 'in the valid list of separators (%s)' %
+ (pn, licenses, element, license_operator_chars), d)
+
+SSTATETASKS += "do_populate_lic"
+do_populate_lic[sstate-inputdirs] = "${LICSSTATEDIR}"
+do_populate_lic[sstate-outputdirs] = "${LICENSE_DIRECTORY}/"
+
+IMAGE_CLASSES:append = " license_image"
+
+python do_populate_lic_setscene () {
+ sstate_setscene(d)
+}
+addtask do_populate_lic_setscene
diff --git a/poky/meta/classes-global/logging.bbclass b/poky/meta/classes-global/logging.bbclass
new file mode 100644
index 0000000..ce03abf
--- /dev/null
+++ b/poky/meta/classes-global/logging.bbclass
@@ -0,0 +1,107 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+# The following logging mechanisms are to be used in bash functions of recipes.
+# They are intended to map one to one in intention and output format with the
+# python recipe logging functions of a similar naming convention: bb.plain(),
+# bb.note(), etc.
+
+LOGFIFO = "${T}/fifo.${@os.getpid()}"
+
+# Print the output exactly as it is passed in. Typically used for output of
+# tasks that should be seen on the console. Use sparingly.
+# Output: logs console
+bbplain() {
+ if [ -p ${LOGFIFO} ] ; then
+ printf "%b\0" "bbplain $*" > ${LOGFIFO}
+ else
+ echo "$*"
+ fi
+}
+
+# Notify the user of a noteworthy condition.
+# Output: logs
+bbnote() {
+ if [ -p ${LOGFIFO} ] ; then
+ printf "%b\0" "bbnote $*" > ${LOGFIFO}
+ else
+ echo "NOTE: $*"
+ fi
+}
+
+# Print a warning to the log. Warnings are non-fatal, and do not
+# indicate a build failure.
+# Output: logs console
+bbwarn() {
+ if [ -p ${LOGFIFO} ] ; then
+ printf "%b\0" "bbwarn $*" > ${LOGFIFO}
+ else
+ echo "WARNING: $*"
+ fi
+}
+
+# Print an error to the log. Errors are non-fatal in that the build can
+# continue, but they do indicate a build failure.
+# Output: logs console
+bberror() {
+ if [ -p ${LOGFIFO} ] ; then
+ printf "%b\0" "bberror $*" > ${LOGFIFO}
+ else
+ echo "ERROR: $*"
+ fi
+}
+
+# Print a fatal error to the log. Fatal errors indicate build failure
+# and halt the build, exiting with an error code.
+# Output: logs console
+bbfatal() {
+ if [ -p ${LOGFIFO} ] ; then
+ printf "%b\0" "bbfatal $*" > ${LOGFIFO}
+ else
+ echo "ERROR: $*"
+ fi
+ exit 1
+}
+
+# Like bbfatal, except prevents the suppression of the error log by
+# bitbake's UI.
+# Output: logs console
+bbfatal_log() {
+ if [ -p ${LOGFIFO} ] ; then
+ printf "%b\0" "bbfatal_log $*" > ${LOGFIFO}
+ else
+ echo "ERROR: $*"
+ fi
+ exit 1
+}
+
+# Print debug messages. These are appropriate for progress checkpoint
+# messages to the logs. Depending on the debug log level, they may also
+# go to the console.
+# Output: logs console
+# Usage: bbdebug 1 "first level debug message"
+# bbdebug 2 "second level debug message"
+bbdebug() {
+ USAGE='Usage: bbdebug [123] "message"'
+ if [ $# -lt 2 ]; then
+ bbfatal "$USAGE"
+ fi
+
+ # Strip off the debug level and ensure it is an integer
+ DBGLVL=$1; shift
+ NONDIGITS=$(echo "$DBGLVL" | tr -d "[:digit:]")
+ if [ "$NONDIGITS" ]; then
+ bbfatal "$USAGE"
+ fi
+
+ # All debug output is printed to the logs
+ if [ -p ${LOGFIFO} ] ; then
+ printf "%b\0" "bbdebug $DBGLVL $*" > ${LOGFIFO}
+ else
+ echo "DEBUG: $*"
+ fi
+}
+
diff --git a/poky/meta/classes-global/mirrors.bbclass b/poky/meta/classes-global/mirrors.bbclass
new file mode 100644
index 0000000..9643b31
--- /dev/null
+++ b/poky/meta/classes-global/mirrors.bbclass
@@ -0,0 +1,95 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+MIRRORS += "\
+${DEBIAN_MIRROR} http://snapshot.debian.org/archive/debian/20180310T215105Z/pool \
+${DEBIAN_MIRROR} http://snapshot.debian.org/archive/debian-archive/20120328T092752Z/debian/pool \
+${DEBIAN_MIRROR} http://snapshot.debian.org/archive/debian-archive/20110127T084257Z/debian/pool \
+${DEBIAN_MIRROR} http://snapshot.debian.org/archive/debian-archive/20090802T004153Z/debian/pool \
+${DEBIAN_MIRROR} http://ftp.de.debian.org/debian/pool \
+${DEBIAN_MIRROR} http://ftp.au.debian.org/debian/pool \
+${DEBIAN_MIRROR} http://ftp.cl.debian.org/debian/pool \
+${DEBIAN_MIRROR} http://ftp.hr.debian.org/debian/pool \
+${DEBIAN_MIRROR} http://ftp.fi.debian.org/debian/pool \
+${DEBIAN_MIRROR} http://ftp.hk.debian.org/debian/pool \
+${DEBIAN_MIRROR} http://ftp.hu.debian.org/debian/pool \
+${DEBIAN_MIRROR} http://ftp.ie.debian.org/debian/pool \
+${DEBIAN_MIRROR} http://ftp.it.debian.org/debian/pool \
+${DEBIAN_MIRROR} http://ftp.jp.debian.org/debian/pool \
+${DEBIAN_MIRROR} http://ftp.no.debian.org/debian/pool \
+${DEBIAN_MIRROR} http://ftp.pl.debian.org/debian/pool \
+${DEBIAN_MIRROR} http://ftp.ro.debian.org/debian/pool \
+${DEBIAN_MIRROR} http://ftp.si.debian.org/debian/pool \
+${DEBIAN_MIRROR} http://ftp.es.debian.org/debian/pool \
+${DEBIAN_MIRROR} http://ftp.se.debian.org/debian/pool \
+${DEBIAN_MIRROR} http://ftp.tr.debian.org/debian/pool \
+${GNU_MIRROR} https://mirrors.kernel.org/gnu \
+${KERNELORG_MIRROR} http://www.kernel.org/pub \
+${GNUPG_MIRROR} ftp://ftp.gnupg.org/gcrypt \
+${GNUPG_MIRROR} ftp://ftp.franken.de/pub/crypt/mirror/ftp.gnupg.org/gcrypt \
+${GNUPG_MIRROR} ftp://mirrors.dotsrc.org/gcrypt \
+ftp://dante.ctan.org/tex-archive ftp://ftp.fu-berlin.de/tex/CTAN \
+ftp://dante.ctan.org/tex-archive http://sunsite.sut.ac.jp/pub/archives/ctan/ \
+ftp://dante.ctan.org/tex-archive http://ctan.unsw.edu.au/ \
+ftp://ftp.gnutls.org/gcrypt/gnutls ${GNUPG_MIRROR}/gnutls \
+http://ftp.info-zip.org/pub/infozip/src/ ftp://sunsite.icm.edu.pl/pub/unix/archiving/info-zip/src/ \
+http://www.mirrorservice.org/sites/lsof.itap.purdue.edu/pub/tools/unix/lsof/ http://www.mirrorservice.org/sites/lsof.itap.purdue.edu/pub/tools/unix/lsof/OLD/ \
+${APACHE_MIRROR} http://www.us.apache.org/dist \
+${APACHE_MIRROR} http://archive.apache.org/dist \
+http://downloads.sourceforge.net/watchdog/ http://fossies.org/linux/misc/ \
+${SAVANNAH_GNU_MIRROR} http://download-mirror.savannah.gnu.org/releases \
+${SAVANNAH_NONGNU_MIRROR} http://download-mirror.savannah.nongnu.org/releases \
+ftp://sourceware.org/pub http://mirrors.kernel.org/sourceware \
+ftp://sourceware.org/pub http://gd.tuwien.ac.at/gnu/sourceware \
+ftp://sourceware.org/pub http://ftp.gwdg.de/pub/linux/sources.redhat.com/sourceware \
+cvs://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \
+svn://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \
+git://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \
+gitsm://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \
+hg://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \
+bzr://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \
+p4://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \
+osc://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \
+https?://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \
+ftp://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \
+npm://.*/?.* http://downloads.yoctoproject.org/mirror/sources/ \
+cvs://.*/.* http://sources.openembedded.org/ \
+svn://.*/.* http://sources.openembedded.org/ \
+git://.*/.* http://sources.openembedded.org/ \
+gitsm://.*/.* http://sources.openembedded.org/ \
+hg://.*/.* http://sources.openembedded.org/ \
+bzr://.*/.* http://sources.openembedded.org/ \
+p4://.*/.* http://sources.openembedded.org/ \
+osc://.*/.* http://sources.openembedded.org/ \
+https?://.*/.* http://sources.openembedded.org/ \
+ftp://.*/.* http://sources.openembedded.org/ \
+npm://.*/?.* http://sources.openembedded.org/ \
+${CPAN_MIRROR} http://cpan.metacpan.org/ \
+${CPAN_MIRROR} http://search.cpan.org/CPAN/ \
+https?://downloads.yoctoproject.org/releases/uninative/ https://mirrors.kernel.org/yocto/uninative/ \
+https?://downloads.yoctoproject.org/mirror/sources/ https://mirrors.kernel.org/yocto-sources/ \
+"
+
+# Use MIRRORS to provide git repo fallbacks using the https protocol, for cases
+# where git native protocol fetches may fail due to local firewall rules, etc.
+
+MIRRORS += "\
+git://salsa.debian.org/.* git://salsa.debian.org/PATH;protocol=https \
+git://git.gnome.org/.* git://gitlab.gnome.org/GNOME/PATH;protocol=https \
+git://.*/.* git://HOST/PATH;protocol=https \
+git://.*/.* git://HOST/git/PATH;protocol=https \
+"
+
+# Switch glibc and binutils recipes to use shallow clones as they're large and this
+# improves user experience whilst allowing the flexibility of git urls in the recipes
+BB_GIT_SHALLOW:pn-binutils = "1"
+BB_GIT_SHALLOW:pn-binutils-cross-${TARGET_ARCH} = "1"
+BB_GIT_SHALLOW:pn-binutils-cross-canadian-${TRANSLATED_TARGET_ARCH} = "1"
+BB_GIT_SHALLOW:pn-binutils-cross-testsuite = "1"
+BB_GIT_SHALLOW:pn-binutils-crosssdk-${SDK_SYS} = "1"
+BB_GIT_SHALLOW:pn-glibc = "1"
+PREMIRRORS += "git://sourceware.org/git/glibc.git https://downloads.yoctoproject.org/mirror/sources/ \
+ git://sourceware.org/git/binutils-gdb.git https://downloads.yoctoproject.org/mirror/sources/"
diff --git a/poky/meta/classes-global/package.bbclass b/poky/meta/classes-global/package.bbclass
new file mode 100644
index 0000000..2d985d8
--- /dev/null
+++ b/poky/meta/classes-global/package.bbclass
@@ -0,0 +1,2546 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+#
+# Packaging process
+#
+# Executive summary: This class iterates over the functions listed in PACKAGEFUNCS
+# Taking D and splitting it up into the packages listed in PACKAGES, placing the
+# resulting output in PKGDEST.
+#
+# There are the following default steps but PACKAGEFUNCS can be extended:
+#
+# a) package_convert_pr_autoinc - convert AUTOINC in PKGV to ${PRSERV_PV_AUTOINC}
+#
+# b) perform_packagecopy - Copy D into PKGD
+#
+# c) package_do_split_locales - Split out the locale files, updates FILES and PACKAGES
+#
+# d) split_and_strip_files - split the files into runtime and debug and strip them.
+# Debug files include debug info split, and associated sources that end up in -dbg packages
+#
+# e) fixup_perms - Fix up permissions in the package before we split it.
+#
+# f) populate_packages - Split the files in PKGD into separate packages in PKGDEST/<pkgname>
+# Also triggers the binary stripping code to put files in -dbg packages.
+#
+# g) package_do_filedeps - Collect perfile run-time dependency metadata
+# The data is stores in FILER{PROVIDES,DEPENDS}_file_pkg variables with
+# a list of affected files in FILER{PROVIDES,DEPENDS}FLIST_pkg
+#
+# h) package_do_shlibs - Look at the shared libraries generated and autotmatically add any
+# dependencies found. Also stores the package name so anyone else using this library
+# knows which package to depend on.
+#
+# i) package_do_pkgconfig - Keep track of which packages need and provide which .pc files
+#
+# j) read_shlibdeps - Reads the stored shlibs information into the metadata
+#
+# k) package_depchains - Adds automatic dependencies to -dbg and -dev packages
+#
+# l) emit_pkgdata - saves the packaging data into PKGDATA_DIR for use in later
+# packaging steps
+
+inherit packagedata
+inherit chrpath
+inherit package_pkgdata
+inherit insane
+
+PKGD = "${WORKDIR}/package"
+PKGDEST = "${WORKDIR}/packages-split"
+
+LOCALE_SECTION ?= ''
+
+ALL_MULTILIB_PACKAGE_ARCHS = "${@all_multilib_tune_values(d, 'PACKAGE_ARCHS')}"
+
+# rpm is used for the per-file dependency identification
+# dwarfsrcfiles is used to determine the list of debug source files
+PACKAGE_DEPENDS += "rpm-native dwarfsrcfiles-native"
+
+
+# If your postinstall can execute at rootfs creation time rather than on
+# target but depends on a native/cross tool in order to execute, you need to
+# list that tool in PACKAGE_WRITE_DEPS. Target package dependencies belong
+# in the package dependencies as normal, this is just for native/cross support
+# tools at rootfs build time.
+PACKAGE_WRITE_DEPS ??= ""
+
+def legitimize_package_name(s):
+ """
+ Make sure package names are legitimate strings
+ """
+ import re
+
+ def fixutf(m):
+ cp = m.group(1)
+ if cp:
+ return ('\\u%s' % cp).encode('latin-1').decode('unicode_escape')
+
+ # Handle unicode codepoints encoded as <U0123>, as in glibc locale files.
+ s = re.sub(r'<U([0-9A-Fa-f]{1,4})>', fixutf, s)
+
+ # Remaining package name validity fixes
+ return s.lower().replace('_', '-').replace('@', '+').replace(',', '+').replace('/', '-')
+
+def do_split_packages(d, root, file_regex, output_pattern, description, postinst=None, recursive=False, hook=None, extra_depends=None, aux_files_pattern=None, postrm=None, allow_dirs=False, prepend=False, match_path=False, aux_files_pattern_verbatim=None, allow_links=False, summary=None):
+ """
+ Used in .bb files to split up dynamically generated subpackages of a
+ given package, usually plugins or modules.
+
+ Arguments:
+ root -- the path in which to search
+ file_regex -- regular expression to match searched files. Use
+ parentheses () to mark the part of this expression
+ that should be used to derive the module name (to be
+ substituted where %s is used in other function
+ arguments as noted below)
+ output_pattern -- pattern to use for the package names. Must include %s.
+ description -- description to set for each package. Must include %s.
+ postinst -- postinstall script to use for all packages (as a
+ string)
+ recursive -- True to perform a recursive search - default False
+ hook -- a hook function to be called for every match. The
+ function will be called with the following arguments
+ (in the order listed):
+ f: full path to the file/directory match
+ pkg: the package name
+ file_regex: as above
+ output_pattern: as above
+ modulename: the module name derived using file_regex
+ extra_depends -- extra runtime dependencies (RDEPENDS) to be set for
+ all packages. The default value of None causes a
+ dependency on the main package (${PN}) - if you do
+ not want this, pass '' for this parameter.
+ aux_files_pattern -- extra item(s) to be added to FILES for each
+ package. Can be a single string item or a list of
+ strings for multiple items. Must include %s.
+ postrm -- postrm script to use for all packages (as a string)
+ allow_dirs -- True allow directories to be matched - default False
+ prepend -- if True, prepend created packages to PACKAGES instead
+ of the default False which appends them
+ match_path -- match file_regex on the whole relative path to the
+ root rather than just the file name
+ aux_files_pattern_verbatim -- extra item(s) to be added to FILES for
+ each package, using the actual derived module name
+ rather than converting it to something legal for a
+ package name. Can be a single string item or a list
+ of strings for multiple items. Must include %s.
+ allow_links -- True to allow symlinks to be matched - default False
+ summary -- Summary to set for each package. Must include %s;
+ defaults to description if not set.
+
+ """
+
+ dvar = d.getVar('PKGD')
+ root = d.expand(root)
+ output_pattern = d.expand(output_pattern)
+ extra_depends = d.expand(extra_depends)
+
+ # If the root directory doesn't exist, don't error out later but silently do
+ # no splitting.
+ if not os.path.exists(dvar + root):
+ return []
+
+ ml = d.getVar("MLPREFIX")
+ if ml:
+ if not output_pattern.startswith(ml):
+ output_pattern = ml + output_pattern
+
+ newdeps = []
+ for dep in (extra_depends or "").split():
+ if dep.startswith(ml):
+ newdeps.append(dep)
+ else:
+ newdeps.append(ml + dep)
+ if newdeps:
+ extra_depends = " ".join(newdeps)
+
+
+ packages = d.getVar('PACKAGES').split()
+ split_packages = set()
+
+ if postinst:
+ postinst = '#!/bin/sh\n' + postinst + '\n'
+ if postrm:
+ postrm = '#!/bin/sh\n' + postrm + '\n'
+ if not recursive:
+ objs = os.listdir(dvar + root)
+ else:
+ objs = []
+ for walkroot, dirs, files in os.walk(dvar + root):
+ for file in files:
+ relpath = os.path.join(walkroot, file).replace(dvar + root + '/', '', 1)
+ if relpath:
+ objs.append(relpath)
+
+ if extra_depends == None:
+ extra_depends = d.getVar("PN")
+
+ if not summary:
+ summary = description
+
+ for o in sorted(objs):
+ import re, stat
+ if match_path:
+ m = re.match(file_regex, o)
+ else:
+ m = re.match(file_regex, os.path.basename(o))
+
+ if not m:
+ continue
+ f = os.path.join(dvar + root, o)
+ mode = os.lstat(f).st_mode
+ if not (stat.S_ISREG(mode) or (allow_links and stat.S_ISLNK(mode)) or (allow_dirs and stat.S_ISDIR(mode))):
+ continue
+ on = legitimize_package_name(m.group(1))
+ pkg = output_pattern % on
+ split_packages.add(pkg)
+ if not pkg in packages:
+ if prepend:
+ packages = [pkg] + packages
+ else:
+ packages.append(pkg)
+ oldfiles = d.getVar('FILES:' + pkg)
+ newfile = os.path.join(root, o)
+ # These names will be passed through glob() so if the filename actually
+ # contains * or ? (rare, but possible) we need to handle that specially
+ newfile = newfile.replace('*', '[*]')
+ newfile = newfile.replace('?', '[?]')
+ if not oldfiles:
+ the_files = [newfile]
+ if aux_files_pattern:
+ if type(aux_files_pattern) is list:
+ for fp in aux_files_pattern:
+ the_files.append(fp % on)
+ else:
+ the_files.append(aux_files_pattern % on)
+ if aux_files_pattern_verbatim:
+ if type(aux_files_pattern_verbatim) is list:
+ for fp in aux_files_pattern_verbatim:
+ the_files.append(fp % m.group(1))
+ else:
+ the_files.append(aux_files_pattern_verbatim % m.group(1))
+ d.setVar('FILES:' + pkg, " ".join(the_files))
+ else:
+ d.setVar('FILES:' + pkg, oldfiles + " " + newfile)
+ if extra_depends != '':
+ d.appendVar('RDEPENDS:' + pkg, ' ' + extra_depends)
+ if not d.getVar('DESCRIPTION:' + pkg):
+ d.setVar('DESCRIPTION:' + pkg, description % on)
+ if not d.getVar('SUMMARY:' + pkg):
+ d.setVar('SUMMARY:' + pkg, summary % on)
+ if postinst:
+ d.setVar('pkg_postinst:' + pkg, postinst)
+ if postrm:
+ d.setVar('pkg_postrm:' + pkg, postrm)
+ if callable(hook):
+ hook(f, pkg, file_regex, output_pattern, m.group(1))
+
+ d.setVar('PACKAGES', ' '.join(packages))
+ return list(split_packages)
+
+PACKAGE_DEPENDS += "file-native"
+
+python () {
+ if d.getVar('PACKAGES') != '':
+ deps = ""
+ for dep in (d.getVar('PACKAGE_DEPENDS') or "").split():
+ deps += " %s:do_populate_sysroot" % dep
+ if d.getVar('PACKAGE_MINIDEBUGINFO') == '1':
+ deps += ' xz-native:do_populate_sysroot'
+ d.appendVarFlag('do_package', 'depends', deps)
+
+ # shlibs requires any DEPENDS to have already packaged for the *.list files
+ d.appendVarFlag('do_package', 'deptask', " do_packagedata")
+}
+
+# Get a list of files from file vars by searching files under current working directory
+# The list contains symlinks, directories and normal files.
+def files_from_filevars(filevars):
+ import os,glob
+ cpath = oe.cachedpath.CachedPath()
+ files = []
+ for f in filevars:
+ if os.path.isabs(f):
+ f = '.' + f
+ if not f.startswith("./"):
+ f = './' + f
+ globbed = glob.glob(f)
+ if globbed:
+ if [ f ] != globbed:
+ files += globbed
+ continue
+ files.append(f)
+
+ symlink_paths = []
+ for ind, f in enumerate(files):
+ # Handle directory symlinks. Truncate path to the lowest level symlink
+ parent = ''
+ for dirname in f.split('/')[:-1]:
+ parent = os.path.join(parent, dirname)
+ if dirname == '.':
+ continue
+ if cpath.islink(parent):
+ bb.warn("FILES contains file '%s' which resides under a "
+ "directory symlink. Please fix the recipe and use the "
+ "real path for the file." % f[1:])
+ symlink_paths.append(f)
+ files[ind] = parent
+ f = parent
+ break
+
+ if not cpath.islink(f):
+ if cpath.isdir(f):
+ newfiles = [ os.path.join(f,x) for x in os.listdir(f) ]
+ if newfiles:
+ files += newfiles
+
+ return files, symlink_paths
+
+# Called in package_<rpm,ipk,deb>.bbclass to get the correct list of configuration files
+def get_conffiles(pkg, d):
+ pkgdest = d.getVar('PKGDEST')
+ root = os.path.join(pkgdest, pkg)
+ cwd = os.getcwd()
+ os.chdir(root)
+
+ conffiles = d.getVar('CONFFILES:%s' % pkg);
+ if conffiles == None:
+ conffiles = d.getVar('CONFFILES')
+ if conffiles == None:
+ conffiles = ""
+ conffiles = conffiles.split()
+ conf_orig_list = files_from_filevars(conffiles)[0]
+
+ # Remove links and directories from conf_orig_list to get conf_list which only contains normal files
+ conf_list = []
+ for f in conf_orig_list:
+ if os.path.isdir(f):
+ continue
+ if os.path.islink(f):
+ continue
+ if not os.path.exists(f):
+ continue
+ conf_list.append(f)
+
+ # Remove the leading './'
+ for i in range(0, len(conf_list)):
+ conf_list[i] = conf_list[i][1:]
+
+ os.chdir(cwd)
+ return conf_list
+
+def checkbuildpath(file, d):
+ tmpdir = d.getVar('TMPDIR')
+ with open(file) as f:
+ file_content = f.read()
+ if tmpdir in file_content:
+ return True
+
+ return False
+
+def parse_debugsources_from_dwarfsrcfiles_output(dwarfsrcfiles_output):
+ debugfiles = {}
+
+ for line in dwarfsrcfiles_output.splitlines():
+ if line.startswith("\t"):
+ debugfiles[os.path.normpath(line.split()[0])] = ""
+
+ return debugfiles.keys()
+
+def source_info(file, d, fatal=True):
+ import subprocess
+
+ cmd = ["dwarfsrcfiles", file]
+ try:
+ output = subprocess.check_output(cmd, universal_newlines=True, stderr=subprocess.STDOUT)
+ retval = 0
+ except subprocess.CalledProcessError as exc:
+ output = exc.output
+ retval = exc.returncode
+
+ # 255 means a specific file wasn't fully parsed to get the debug file list, which is not a fatal failure
+ if retval != 0 and retval != 255:
+ msg = "dwarfsrcfiles failed with exit code %s (cmd was %s)%s" % (retval, cmd, ":\n%s" % output if output else "")
+ if fatal:
+ bb.fatal(msg)
+ bb.note(msg)
+
+ debugsources = parse_debugsources_from_dwarfsrcfiles_output(output)
+
+ return list(debugsources)
+
+def splitdebuginfo(file, dvar, dv, d):
+ # Function to split a single file into two components, one is the stripped
+ # target system binary, the other contains any debugging information. The
+ # two files are linked to reference each other.
+ #
+ # return a mapping of files:debugsources
+
+ import stat
+ import subprocess
+
+ src = file[len(dvar):]
+ dest = dv["libdir"] + os.path.dirname(src) + dv["dir"] + "/" + os.path.basename(src) + dv["append"]
+ debugfile = dvar + dest
+ sources = []
+
+ if file.endswith(".ko") and file.find("/lib/modules/") != -1:
+ if oe.package.is_kernel_module_signed(file):
+ bb.debug(1, "Skip strip on signed module %s" % file)
+ return (file, sources)
+
+ # Split the file...
+ bb.utils.mkdirhier(os.path.dirname(debugfile))
+ #bb.note("Split %s -> %s" % (file, debugfile))
+ # Only store off the hard link reference if we successfully split!
+
+ dvar = d.getVar('PKGD')
+ objcopy = d.getVar("OBJCOPY")
+
+ newmode = None
+ if not os.access(file, os.W_OK) or os.access(file, os.R_OK):
+ origmode = os.stat(file)[stat.ST_MODE]
+ newmode = origmode | stat.S_IWRITE | stat.S_IREAD
+ os.chmod(file, newmode)
+
+ # We need to extract the debug src information here...
+ if dv["srcdir"]:
+ sources = source_info(file, d)
+
+ bb.utils.mkdirhier(os.path.dirname(debugfile))
+
+ subprocess.check_output([objcopy, '--only-keep-debug', file, debugfile], stderr=subprocess.STDOUT)
+
+ # Set the debuglink to have the view of the file path on the target
+ subprocess.check_output([objcopy, '--add-gnu-debuglink', debugfile, file], stderr=subprocess.STDOUT)
+
+ if newmode:
+ os.chmod(file, origmode)
+
+ return (file, sources)
+
+def splitstaticdebuginfo(file, dvar, dv, d):
+ # Unlike the function above, there is no way to split a static library
+ # two components. So to get similar results we will copy the unmodified
+ # static library (containing the debug symbols) into a new directory.
+ # We will then strip (preserving symbols) the static library in the
+ # typical location.
+ #
+ # return a mapping of files:debugsources
+
+ import stat
+
+ src = file[len(dvar):]
+ dest = dv["staticlibdir"] + os.path.dirname(src) + dv["staticdir"] + "/" + os.path.basename(src) + dv["staticappend"]
+ debugfile = dvar + dest
+ sources = []
+
+ # Copy the file...
+ bb.utils.mkdirhier(os.path.dirname(debugfile))
+ #bb.note("Copy %s -> %s" % (file, debugfile))
+
+ dvar = d.getVar('PKGD')
+
+ newmode = None
+ if not os.access(file, os.W_OK) or os.access(file, os.R_OK):
+ origmode = os.stat(file)[stat.ST_MODE]
+ newmode = origmode | stat.S_IWRITE | stat.S_IREAD
+ os.chmod(file, newmode)
+
+ # We need to extract the debug src information here...
+ if dv["srcdir"]:
+ sources = source_info(file, d)
+
+ bb.utils.mkdirhier(os.path.dirname(debugfile))
+
+ # Copy the unmodified item to the debug directory
+ shutil.copy2(file, debugfile)
+
+ if newmode:
+ os.chmod(file, origmode)
+
+ return (file, sources)
+
+def inject_minidebuginfo(file, dvar, dv, d):
+ # Extract just the symbols from debuginfo into minidebuginfo,
+ # compress it with xz and inject it back into the binary in a .gnu_debugdata section.
+ # https://sourceware.org/gdb/onlinedocs/gdb/MiniDebugInfo.html
+
+ import subprocess
+
+ readelf = d.getVar('READELF')
+ nm = d.getVar('NM')
+ objcopy = d.getVar('OBJCOPY')
+
+ minidebuginfodir = d.expand('${WORKDIR}/minidebuginfo')
+
+ src = file[len(dvar):]
+ dest = dv["libdir"] + os.path.dirname(src) + dv["dir"] + "/" + os.path.basename(src) + dv["append"]
+ debugfile = dvar + dest
+ minidebugfile = minidebuginfodir + src + '.minidebug'
+ bb.utils.mkdirhier(os.path.dirname(minidebugfile))
+
+ # If we didn't produce debuginfo for any reason, we can't produce minidebuginfo either
+ # so skip it.
+ if not os.path.exists(debugfile):
+ bb.debug(1, 'ELF file {} has no debuginfo, skipping minidebuginfo injection'.format(file))
+ return
+
+ # Find non-allocated PROGBITS, NOTE, and NOBITS sections in the debuginfo.
+ # We will exclude all of these from minidebuginfo to save space.
+ remove_section_names = []
+ for line in subprocess.check_output([readelf, '-W', '-S', debugfile], universal_newlines=True).splitlines():
+ fields = line.split()
+ if len(fields) < 8:
+ continue
+ name = fields[0]
+ type = fields[1]
+ flags = fields[7]
+ # .debug_ sections will be removed by objcopy -S so no need to explicitly remove them
+ if name.startswith('.debug_'):
+ continue
+ if 'A' not in flags and type in ['PROGBITS', 'NOTE', 'NOBITS']:
+ remove_section_names.append(name)
+
+ # List dynamic symbols in the binary. We can exclude these from minidebuginfo
+ # because they are always present in the binary.
+ dynsyms = set()
+ for line in subprocess.check_output([nm, '-D', file, '--format=posix', '--defined-only'], universal_newlines=True).splitlines():
+ dynsyms.add(line.split()[0])
+
+ # Find all function symbols from debuginfo which aren't in the dynamic symbols table.
+ # These are the ones we want to keep in minidebuginfo.
+ keep_symbols_file = minidebugfile + '.symlist'
+ found_any_symbols = False
+ with open(keep_symbols_file, 'w') as f:
+ for line in subprocess.check_output([nm, debugfile, '--format=sysv', '--defined-only'], universal_newlines=True).splitlines():
+ fields = line.split('|')
+ if len(fields) < 7:
+ continue
+ name = fields[0].strip()
+ type = fields[3].strip()
+ if type == 'FUNC' and name not in dynsyms:
+ f.write('{}\n'.format(name))
+ found_any_symbols = True
+
+ if not found_any_symbols:
+ bb.debug(1, 'ELF file {} contains no symbols, skipping minidebuginfo injection'.format(file))
+ return
+
+ bb.utils.remove(minidebugfile)
+ bb.utils.remove(minidebugfile + '.xz')
+
+ subprocess.check_call([objcopy, '-S'] +
+ ['--remove-section={}'.format(s) for s in remove_section_names] +
+ ['--keep-symbols={}'.format(keep_symbols_file), debugfile, minidebugfile])
+
+ subprocess.check_call(['xz', '--keep', minidebugfile])
+
+ subprocess.check_call([objcopy, '--add-section', '.gnu_debugdata={}.xz'.format(minidebugfile), file])
+
+def copydebugsources(debugsrcdir, sources, d):
+ # The debug src information written out to sourcefile is further processed
+ # and copied to the destination here.
+
+ import stat
+ import subprocess
+
+ if debugsrcdir and sources:
+ sourcefile = d.expand("${WORKDIR}/debugsources.list")
+ bb.utils.remove(sourcefile)
+
+ # filenames are null-separated - this is an artefact of the previous use
+ # of rpm's debugedit, which was writing them out that way, and the code elsewhere
+ # is still assuming that.
+ debuglistoutput = '\0'.join(sources) + '\0'
+ with open(sourcefile, 'a') as sf:
+ sf.write(debuglistoutput)
+
+ dvar = d.getVar('PKGD')
+ strip = d.getVar("STRIP")
+ objcopy = d.getVar("OBJCOPY")
+ workdir = d.getVar("WORKDIR")
+ sdir = d.getVar("S")
+ cflags = d.expand("${CFLAGS}")
+
+ prefixmap = {}
+ for flag in cflags.split():
+ if not flag.startswith("-fdebug-prefix-map"):
+ continue
+ if "recipe-sysroot" in flag:
+ continue
+ flag = flag.split("=")
+ prefixmap[flag[1]] = flag[2]
+
+ nosuchdir = []
+ basepath = dvar
+ for p in debugsrcdir.split("/"):
+ basepath = basepath + "/" + p
+ if not cpath.exists(basepath):
+ nosuchdir.append(basepath)
+ bb.utils.mkdirhier(basepath)
+ cpath.updatecache(basepath)
+
+ for pmap in prefixmap:
+ # Ignore files from the recipe sysroots (target and native)
+ cmd = "LC_ALL=C ; sort -z -u '%s' | egrep -v -z '((<internal>|<built-in>)$|/.*recipe-sysroot.*/)' | " % sourcefile
+ # We need to ignore files that are not actually ours
+ # we do this by only paying attention to items from this package
+ cmd += "fgrep -zw '%s' | " % prefixmap[pmap]
+ # Remove prefix in the source paths
+ cmd += "sed 's#%s/##g' | " % (prefixmap[pmap])
+ cmd += "(cd '%s' ; cpio -pd0mlL --no-preserve-owner '%s%s' 2>/dev/null)" % (pmap, dvar, prefixmap[pmap])
+
+ try:
+ subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
+ except subprocess.CalledProcessError:
+ # Can "fail" if internal headers/transient sources are attempted
+ pass
+ # cpio seems to have a bug with -lL together and symbolic links are just copied, not dereferenced.
+ # Work around this by manually finding and copying any symbolic links that made it through.
+ cmd = "find %s%s -type l -print0 -delete | sed s#%s%s/##g | (cd '%s' ; cpio -pd0mL --no-preserve-owner '%s%s')" % \
+ (dvar, prefixmap[pmap], dvar, prefixmap[pmap], pmap, dvar, prefixmap[pmap])
+ subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
+
+ # debugsources.list may be polluted from the host if we used externalsrc,
+ # cpio uses copy-pass and may have just created a directory structure
+ # matching the one from the host, if thats the case move those files to
+ # debugsrcdir to avoid host contamination.
+ # Empty dir structure will be deleted in the next step.
+
+ # Same check as above for externalsrc
+ if workdir not in sdir:
+ if os.path.exists(dvar + debugsrcdir + sdir):
+ cmd = "mv %s%s%s/* %s%s" % (dvar, debugsrcdir, sdir, dvar,debugsrcdir)
+ subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
+
+ # The copy by cpio may have resulted in some empty directories! Remove these
+ cmd = "find %s%s -empty -type d -delete" % (dvar, debugsrcdir)
+ subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
+
+ # Also remove debugsrcdir if its empty
+ for p in nosuchdir[::-1]:
+ if os.path.exists(p) and not os.listdir(p):
+ os.rmdir(p)
+
+#
+# Package data handling routines
+#
+
+def get_package_mapping (pkg, basepkg, d, depversions=None):
+ import oe.packagedata
+
+ data = oe.packagedata.read_subpkgdata(pkg, d)
+ key = "PKG:%s" % pkg
+
+ if key in data:
+ if bb.data.inherits_class('allarch', d) and bb.data.inherits_class('packagegroup', d) and pkg != data[key]:
+ bb.error("An allarch packagegroup shouldn't depend on packages which are dynamically renamed (%s to %s)" % (pkg, data[key]))
+ # Have to avoid undoing the write_extra_pkgs(global_variants...)
+ if bb.data.inherits_class('allarch', d) and not d.getVar('MULTILIB_VARIANTS') \
+ and data[key] == basepkg:
+ return pkg
+ if depversions == []:
+ # Avoid returning a mapping if the renamed package rprovides its original name
+ rprovkey = "RPROVIDES:%s" % pkg
+ if rprovkey in data:
+ if pkg in bb.utils.explode_dep_versions2(data[rprovkey]):
+ bb.note("%s rprovides %s, not replacing the latter" % (data[key], pkg))
+ return pkg
+ # Do map to rewritten package name
+ return data[key]
+
+ return pkg
+
+def get_package_additional_metadata (pkg_type, d):
+ base_key = "PACKAGE_ADD_METADATA"
+ for key in ("%s_%s" % (base_key, pkg_type.upper()), base_key):
+ if d.getVar(key, False) is None:
+ continue
+ d.setVarFlag(key, "type", "list")
+ if d.getVarFlag(key, "separator") is None:
+ d.setVarFlag(key, "separator", "\\n")
+ metadata_fields = [field.strip() for field in oe.data.typed_value(key, d)]
+ return "\n".join(metadata_fields).strip()
+
+def runtime_mapping_rename (varname, pkg, d):
+ #bb.note("%s before: %s" % (varname, d.getVar(varname)))
+
+ new_depends = {}
+ deps = bb.utils.explode_dep_versions2(d.getVar(varname) or "")
+ for depend, depversions in deps.items():
+ new_depend = get_package_mapping(depend, pkg, d, depversions)
+ if depend != new_depend:
+ bb.note("package name mapping done: %s -> %s" % (depend, new_depend))
+ new_depends[new_depend] = deps[depend]
+
+ d.setVar(varname, bb.utils.join_deps(new_depends, commasep=False))
+
+ #bb.note("%s after: %s" % (varname, d.getVar(varname)))
+
+#
+# Used by do_packagedata (and possibly other routines post do_package)
+#
+
+PRSERV_ACTIVE = "${@bool(d.getVar("PRSERV_HOST"))}"
+PRSERV_ACTIVE[vardepvalue] = "${PRSERV_ACTIVE}"
+package_get_auto_pr[vardepsexclude] = "BB_TASKDEPDATA"
+package_get_auto_pr[vardeps] += "PRSERV_ACTIVE"
+python package_get_auto_pr() {
+ import oe.prservice
+
+ def get_do_package_hash(pn):
+ if d.getVar("BB_RUNTASK") != "do_package":
+ taskdepdata = d.getVar("BB_TASKDEPDATA", False)
+ for dep in taskdepdata:
+ if taskdepdata[dep][1] == "do_package" and taskdepdata[dep][0] == pn:
+ return taskdepdata[dep][6]
+ return None
+
+ # Support per recipe PRSERV_HOST
+ pn = d.getVar('PN')
+ host = d.getVar("PRSERV_HOST_" + pn)
+ if not (host is None):
+ d.setVar("PRSERV_HOST", host)
+
+ pkgv = d.getVar("PKGV")
+
+ # PR Server not active, handle AUTOINC
+ if not d.getVar('PRSERV_HOST'):
+ d.setVar("PRSERV_PV_AUTOINC", "0")
+ return
+
+ auto_pr = None
+ pv = d.getVar("PV")
+ version = d.getVar("PRAUTOINX")
+ pkgarch = d.getVar("PACKAGE_ARCH")
+ checksum = get_do_package_hash(pn)
+
+ # If do_package isn't in the dependencies, we can't get the checksum...
+ if not checksum:
+ bb.warn('Task %s requested do_package unihash, but it was not available.' % d.getVar('BB_RUNTASK'))
+ #taskdepdata = d.getVar("BB_TASKDEPDATA", False)
+ #for dep in taskdepdata:
+ # bb.warn('%s:%s = %s' % (taskdepdata[dep][0], taskdepdata[dep][1], taskdepdata[dep][6]))
+ return
+
+ if d.getVar('PRSERV_LOCKDOWN'):
+ auto_pr = d.getVar('PRAUTO_' + version + '_' + pkgarch) or d.getVar('PRAUTO_' + version) or None
+ if auto_pr is None:
+ bb.fatal("Can NOT get PRAUTO from lockdown exported file")
+ d.setVar('PRAUTO',str(auto_pr))
+ return
+
+ try:
+ conn = oe.prservice.prserv_make_conn(d)
+ if conn is not None:
+ if "AUTOINC" in pkgv:
+ srcpv = bb.fetch2.get_srcrev(d)
+ base_ver = "AUTOINC-%s" % version[:version.find(srcpv)]
+ value = conn.getPR(base_ver, pkgarch, srcpv)
+ d.setVar("PRSERV_PV_AUTOINC", str(value))
+
+ auto_pr = conn.getPR(version, pkgarch, checksum)
+ conn.close()
+ except Exception as e:
+ bb.fatal("Can NOT get PRAUTO, exception %s" % str(e))
+ if auto_pr is None:
+ bb.fatal("Can NOT get PRAUTO from remote PR service")
+ d.setVar('PRAUTO',str(auto_pr))
+}
+
+#
+# Package functions suitable for inclusion in PACKAGEFUNCS
+#
+
+python package_convert_pr_autoinc() {
+ pkgv = d.getVar("PKGV")
+
+ # Adjust pkgv as necessary...
+ if 'AUTOINC' in pkgv:
+ d.setVar("PKGV", pkgv.replace("AUTOINC", "${PRSERV_PV_AUTOINC}"))
+
+ # Change PRSERV_PV_AUTOINC and EXTENDPRAUTO usage to special values
+ d.setVar('PRSERV_PV_AUTOINC', '@PRSERV_PV_AUTOINC@')
+ d.setVar('EXTENDPRAUTO', '@EXTENDPRAUTO@')
+}
+
+LOCALEBASEPN ??= "${PN}"
+
+python package_do_split_locales() {
+ if (d.getVar('PACKAGE_NO_LOCALE') == '1'):
+ bb.debug(1, "package requested not splitting locales")
+ return
+
+ packages = (d.getVar('PACKAGES') or "").split()
+
+ datadir = d.getVar('datadir')
+ if not datadir:
+ bb.note("datadir not defined")
+ return
+
+ dvar = d.getVar('PKGD')
+ pn = d.getVar('LOCALEBASEPN')
+
+ if pn + '-locale' in packages:
+ packages.remove(pn + '-locale')
+
+ localedir = os.path.join(dvar + datadir, 'locale')
+
+ if not cpath.isdir(localedir):
+ bb.debug(1, "No locale files in this package")
+ return
+
+ locales = os.listdir(localedir)
+
+ summary = d.getVar('SUMMARY') or pn
+ description = d.getVar('DESCRIPTION') or ""
+ locale_section = d.getVar('LOCALE_SECTION')
+ mlprefix = d.getVar('MLPREFIX') or ""
+ for l in sorted(locales):
+ ln = legitimize_package_name(l)
+ pkg = pn + '-locale-' + ln
+ packages.append(pkg)
+ d.setVar('FILES:' + pkg, os.path.join(datadir, 'locale', l))
+ d.setVar('RRECOMMENDS:' + pkg, '%svirtual-locale-%s' % (mlprefix, ln))
+ d.setVar('RPROVIDES:' + pkg, '%s-locale %s%s-translation' % (pn, mlprefix, ln))
+ d.setVar('SUMMARY:' + pkg, '%s - %s translations' % (summary, l))
+ d.setVar('DESCRIPTION:' + pkg, '%s This package contains language translation files for the %s locale.' % (description, l))
+ if locale_section:
+ d.setVar('SECTION:' + pkg, locale_section)
+
+ d.setVar('PACKAGES', ' '.join(packages))
+
+ # Disabled by RP 18/06/07
+ # Wildcards aren't supported in debian
+ # They break with ipkg since glibc-locale* will mean that
+ # glibc-localedata-translit* won't install as a dependency
+ # for some other package which breaks meta-toolchain
+ # Probably breaks since virtual-locale- isn't provided anywhere
+ #rdep = (d.getVar('RDEPENDS:%s' % pn) or "").split()
+ #rdep.append('%s-locale*' % pn)
+ #d.setVar('RDEPENDS:%s' % pn, ' '.join(rdep))
+}
+
+python perform_packagecopy () {
+ import subprocess
+ import shutil
+
+ dest = d.getVar('D')
+ dvar = d.getVar('PKGD')
+
+ # Start by package population by taking a copy of the installed
+ # files to operate on
+ # Preserve sparse files and hard links
+ cmd = 'tar --exclude=./sysroot-only -cf - -C %s -p -S . | tar -xf - -C %s' % (dest, dvar)
+ subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
+
+ # replace RPATHs for the nativesdk binaries, to make them relocatable
+ if bb.data.inherits_class('nativesdk', d) or bb.data.inherits_class('cross-canadian', d):
+ rpath_replace (dvar, d)
+}
+perform_packagecopy[cleandirs] = "${PKGD}"
+perform_packagecopy[dirs] = "${PKGD}"
+
+# We generate a master list of directories to process, we start by
+# seeding this list with reasonable defaults, then load from
+# the fs-perms.txt files
+python fixup_perms () {
+ import pwd, grp
+
+ # init using a string with the same format as a line as documented in
+ # the fs-perms.txt file
+ # <path> <mode> <uid> <gid> <walk> <fmode> <fuid> <fgid>
+ # <path> link <link target>
+ #
+ # __str__ can be used to print out an entry in the input format
+ #
+ # if fs_perms_entry.path is None:
+ # an error occurred
+ # if fs_perms_entry.link, you can retrieve:
+ # fs_perms_entry.path = path
+ # fs_perms_entry.link = target of link
+ # if not fs_perms_entry.link, you can retrieve:
+ # fs_perms_entry.path = path
+ # fs_perms_entry.mode = expected dir mode or None
+ # fs_perms_entry.uid = expected uid or -1
+ # fs_perms_entry.gid = expected gid or -1
+ # fs_perms_entry.walk = 'true' or something else
+ # fs_perms_entry.fmode = expected file mode or None
+ # fs_perms_entry.fuid = expected file uid or -1
+ # fs_perms_entry_fgid = expected file gid or -1
+ class fs_perms_entry():
+ def __init__(self, line):
+ lsplit = line.split()
+ if len(lsplit) == 3 and lsplit[1].lower() == "link":
+ self._setlink(lsplit[0], lsplit[2])
+ elif len(lsplit) == 8:
+ self._setdir(lsplit[0], lsplit[1], lsplit[2], lsplit[3], lsplit[4], lsplit[5], lsplit[6], lsplit[7])
+ else:
+ msg = "Fixup Perms: invalid config line %s" % line
+ oe.qa.handle_error("perm-config", msg, d)
+ self.path = None
+ self.link = None
+
+ def _setdir(self, path, mode, uid, gid, walk, fmode, fuid, fgid):
+ self.path = os.path.normpath(path)
+ self.link = None
+ self.mode = self._procmode(mode)
+ self.uid = self._procuid(uid)
+ self.gid = self._procgid(gid)
+ self.walk = walk.lower()
+ self.fmode = self._procmode(fmode)
+ self.fuid = self._procuid(fuid)
+ self.fgid = self._procgid(fgid)
+
+ def _setlink(self, path, link):
+ self.path = os.path.normpath(path)
+ self.link = link
+
+ def _procmode(self, mode):
+ if not mode or (mode and mode == "-"):
+ return None
+ else:
+ return int(mode,8)
+
+ # Note uid/gid -1 has special significance in os.lchown
+ def _procuid(self, uid):
+ if uid is None or uid == "-":
+ return -1
+ elif uid.isdigit():
+ return int(uid)
+ else:
+ return pwd.getpwnam(uid).pw_uid
+
+ def _procgid(self, gid):
+ if gid is None or gid == "-":
+ return -1
+ elif gid.isdigit():
+ return int(gid)
+ else:
+ return grp.getgrnam(gid).gr_gid
+
+ # Use for debugging the entries
+ def __str__(self):
+ if self.link:
+ return "%s link %s" % (self.path, self.link)
+ else:
+ mode = "-"
+ if self.mode:
+ mode = "0%o" % self.mode
+ fmode = "-"
+ if self.fmode:
+ fmode = "0%o" % self.fmode
+ uid = self._mapugid(self.uid)
+ gid = self._mapugid(self.gid)
+ fuid = self._mapugid(self.fuid)
+ fgid = self._mapugid(self.fgid)
+ return "%s %s %s %s %s %s %s %s" % (self.path, mode, uid, gid, self.walk, fmode, fuid, fgid)
+
+ def _mapugid(self, id):
+ if id is None or id == -1:
+ return "-"
+ else:
+ return "%d" % id
+
+ # Fix the permission, owner and group of path
+ def fix_perms(path, mode, uid, gid, dir):
+ if mode and not os.path.islink(path):
+ #bb.note("Fixup Perms: chmod 0%o %s" % (mode, dir))
+ os.chmod(path, mode)
+ # -1 is a special value that means don't change the uid/gid
+ # if they are BOTH -1, don't bother to lchown
+ if not (uid == -1 and gid == -1):
+ #bb.note("Fixup Perms: lchown %d:%d %s" % (uid, gid, dir))
+ os.lchown(path, uid, gid)
+
+ # Return a list of configuration files based on either the default
+ # files/fs-perms.txt or the contents of FILESYSTEM_PERMS_TABLES
+ # paths are resolved via BBPATH
+ def get_fs_perms_list(d):
+ str = ""
+ bbpath = d.getVar('BBPATH')
+ fs_perms_tables = d.getVar('FILESYSTEM_PERMS_TABLES') or ""
+ for conf_file in fs_perms_tables.split():
+ confpath = bb.utils.which(bbpath, conf_file)
+ if confpath:
+ str += " %s" % bb.utils.which(bbpath, conf_file)
+ else:
+ bb.warn("cannot find %s specified in FILESYSTEM_PERMS_TABLES" % conf_file)
+ return str
+
+
+
+ dvar = d.getVar('PKGD')
+
+ fs_perms_table = {}
+ fs_link_table = {}
+
+ # By default all of the standard directories specified in
+ # bitbake.conf will get 0755 root:root.
+ target_path_vars = [ 'base_prefix',
+ 'prefix',
+ 'exec_prefix',
+ 'base_bindir',
+ 'base_sbindir',
+ 'base_libdir',
+ 'datadir',
+ 'sysconfdir',
+ 'servicedir',
+ 'sharedstatedir',
+ 'localstatedir',
+ 'infodir',
+ 'mandir',
+ 'docdir',
+ 'bindir',
+ 'sbindir',
+ 'libexecdir',
+ 'libdir',
+ 'includedir',
+ 'oldincludedir' ]
+
+ for path in target_path_vars:
+ dir = d.getVar(path) or ""
+ if dir == "":
+ continue
+ fs_perms_table[dir] = fs_perms_entry(d.expand("%s 0755 root root false - - -" % (dir)))
+
+ # Now we actually load from the configuration files
+ for conf in get_fs_perms_list(d).split():
+ if not os.path.exists(conf):
+ continue
+ with open(conf) as f:
+ for line in f:
+ if line.startswith('#'):
+ continue
+ lsplit = line.split()
+ if len(lsplit) == 0:
+ continue
+ if len(lsplit) != 8 and not (len(lsplit) == 3 and lsplit[1].lower() == "link"):
+ msg = "Fixup perms: %s invalid line: %s" % (conf, line)
+ oe.qa.handle_error("perm-line", msg, d)
+ continue
+ entry = fs_perms_entry(d.expand(line))
+ if entry and entry.path:
+ if entry.link:
+ fs_link_table[entry.path] = entry
+ if entry.path in fs_perms_table:
+ fs_perms_table.pop(entry.path)
+ else:
+ fs_perms_table[entry.path] = entry
+ if entry.path in fs_link_table:
+ fs_link_table.pop(entry.path)
+
+ # Debug -- list out in-memory table
+ #for dir in fs_perms_table:
+ # bb.note("Fixup Perms: %s: %s" % (dir, str(fs_perms_table[dir])))
+ #for link in fs_link_table:
+ # bb.note("Fixup Perms: %s: %s" % (link, str(fs_link_table[link])))
+
+ # We process links first, so we can go back and fixup directory ownership
+ # for any newly created directories
+ # Process in sorted order so /run gets created before /run/lock, etc.
+ for entry in sorted(fs_link_table.values(), key=lambda x: x.link):
+ link = entry.link
+ dir = entry.path
+ origin = dvar + dir
+ if not (cpath.exists(origin) and cpath.isdir(origin) and not cpath.islink(origin)):
+ continue
+
+ if link[0] == "/":
+ target = dvar + link
+ ptarget = link
+ else:
+ target = os.path.join(os.path.dirname(origin), link)
+ ptarget = os.path.join(os.path.dirname(dir), link)
+ if os.path.exists(target):
+ msg = "Fixup Perms: Unable to correct directory link, target already exists: %s -> %s" % (dir, ptarget)
+ oe.qa.handle_error("perm-link", msg, d)
+ continue
+
+ # Create path to move directory to, move it, and then setup the symlink
+ bb.utils.mkdirhier(os.path.dirname(target))
+ #bb.note("Fixup Perms: Rename %s -> %s" % (dir, ptarget))
+ bb.utils.rename(origin, target)
+ #bb.note("Fixup Perms: Link %s -> %s" % (dir, link))
+ os.symlink(link, origin)
+
+ for dir in fs_perms_table:
+ origin = dvar + dir
+ if not (cpath.exists(origin) and cpath.isdir(origin)):
+ continue
+
+ fix_perms(origin, fs_perms_table[dir].mode, fs_perms_table[dir].uid, fs_perms_table[dir].gid, dir)
+
+ if fs_perms_table[dir].walk == 'true':
+ for root, dirs, files in os.walk(origin):
+ for dr in dirs:
+ each_dir = os.path.join(root, dr)
+ fix_perms(each_dir, fs_perms_table[dir].mode, fs_perms_table[dir].uid, fs_perms_table[dir].gid, dir)
+ for f in files:
+ each_file = os.path.join(root, f)
+ fix_perms(each_file, fs_perms_table[dir].fmode, fs_perms_table[dir].fuid, fs_perms_table[dir].fgid, dir)
+}
+
+def package_debug_vars(d):
+ # We default to '.debug' style
+ if d.getVar('PACKAGE_DEBUG_SPLIT_STYLE') == 'debug-file-directory':
+ # Single debug-file-directory style debug info
+ debug_vars = {
+ "append": ".debug",
+ "staticappend": "",
+ "dir": "",
+ "staticdir": "",
+ "libdir": "/usr/lib/debug",
+ "staticlibdir": "/usr/lib/debug-static",
+ "srcdir": "/usr/src/debug",
+ }
+ elif d.getVar('PACKAGE_DEBUG_SPLIT_STYLE') == 'debug-without-src':
+ # Original OE-core, a.k.a. ".debug", style debug info, but without sources in /usr/src/debug
+ debug_vars = {
+ "append": "",
+ "staticappend": "",
+ "dir": "/.debug",
+ "staticdir": "/.debug-static",
+ "libdir": "",
+ "staticlibdir": "",
+ "srcdir": "",
+ }
+ elif d.getVar('PACKAGE_DEBUG_SPLIT_STYLE') == 'debug-with-srcpkg':
+ debug_vars = {
+ "append": "",
+ "staticappend": "",
+ "dir": "/.debug",
+ "staticdir": "/.debug-static",
+ "libdir": "",
+ "staticlibdir": "",
+ "srcdir": "/usr/src/debug",
+ }
+ else:
+ # Original OE-core, a.k.a. ".debug", style debug info
+ debug_vars = {
+ "append": "",
+ "staticappend": "",
+ "dir": "/.debug",
+ "staticdir": "/.debug-static",
+ "libdir": "",
+ "staticlibdir": "",
+ "srcdir": "/usr/src/debug",
+ }
+
+ return debug_vars
+
+python split_and_strip_files () {
+ import stat, errno
+ import subprocess
+
+ dvar = d.getVar('PKGD')
+ pn = d.getVar('PN')
+ hostos = d.getVar('HOST_OS')
+
+ oldcwd = os.getcwd()
+ os.chdir(dvar)
+
+ dv = package_debug_vars(d)
+
+ #
+ # First lets figure out all of the files we may have to process ... do this only once!
+ #
+ elffiles = {}
+ symlinks = {}
+ staticlibs = []
+ inodes = {}
+ libdir = os.path.abspath(dvar + os.sep + d.getVar("libdir"))
+ baselibdir = os.path.abspath(dvar + os.sep + d.getVar("base_libdir"))
+ skipfiles = (d.getVar("INHIBIT_PACKAGE_STRIP_FILES") or "").split()
+ if (d.getVar('INHIBIT_PACKAGE_STRIP') != '1' or \
+ d.getVar('INHIBIT_PACKAGE_DEBUG_SPLIT') != '1'):
+ checkelf = {}
+ checkelflinks = {}
+ for root, dirs, files in cpath.walk(dvar):
+ for f in files:
+ file = os.path.join(root, f)
+
+ # Skip debug files
+ if dv["append"] and file.endswith(dv["append"]):
+ continue
+ if dv["dir"] and dv["dir"] in os.path.dirname(file[len(dvar):]):
+ continue
+
+ if file in skipfiles:
+ continue
+
+ if oe.package.is_static_lib(file):
+ staticlibs.append(file)
+ continue
+
+ try:
+ ltarget = cpath.realpath(file, dvar, False)
+ s = cpath.lstat(ltarget)
+ except OSError as e:
+ (err, strerror) = e.args
+ if err != errno.ENOENT:
+ raise
+ # Skip broken symlinks
+ continue
+ if not s:
+ continue
+ # Check its an executable
+ if (s[stat.ST_MODE] & stat.S_IXUSR) or (s[stat.ST_MODE] & stat.S_IXGRP) \
+ or (s[stat.ST_MODE] & stat.S_IXOTH) \
+ or ((file.startswith(libdir) or file.startswith(baselibdir)) \
+ and (".so" in f or ".node" in f)) \
+ or (f.startswith('vmlinux') or ".ko" in f):
+
+ if cpath.islink(file):
+ checkelflinks[file] = ltarget
+ continue
+ # Use a reference of device ID and inode number to identify files
+ file_reference = "%d_%d" % (s.st_dev, s.st_ino)
+ checkelf[file] = (file, file_reference)
+
+ results = oe.utils.multiprocess_launch(oe.package.is_elf, checkelflinks.values(), d)
+ results_map = {}
+ for (ltarget, elf_file) in results:
+ results_map[ltarget] = elf_file
+ for file in checkelflinks:
+ ltarget = checkelflinks[file]
+ # If it's a symlink, and points to an ELF file, we capture the readlink target
+ if results_map[ltarget]:
+ target = os.readlink(file)
+ #bb.note("Sym: %s (%d)" % (ltarget, results_map[ltarget]))
+ symlinks[file] = target
+
+ results = oe.utils.multiprocess_launch(oe.package.is_elf, checkelf.keys(), d)
+
+ # Sort results by file path. This ensures that the files are always
+ # processed in the same order, which is important to make sure builds
+ # are reproducible when dealing with hardlinks
+ results.sort(key=lambda x: x[0])
+
+ for (file, elf_file) in results:
+ # It's a file (or hardlink), not a link
+ # ...but is it ELF, and is it already stripped?
+ if elf_file & 1:
+ if elf_file & 2:
+ if 'already-stripped' in (d.getVar('INSANE_SKIP:' + pn) or "").split():
+ bb.note("Skipping file %s from %s for already-stripped QA test" % (file[len(dvar):], pn))
+ else:
+ msg = "File '%s' from %s was already stripped, this will prevent future debugging!" % (file[len(dvar):], pn)
+ oe.qa.handle_error("already-stripped", msg, d)
+ continue
+
+ # At this point we have an unstripped elf file. We need to:
+ # a) Make sure any file we strip is not hardlinked to anything else outside this tree
+ # b) Only strip any hardlinked file once (no races)
+ # c) Track any hardlinks between files so that we can reconstruct matching debug file hardlinks
+
+ # Use a reference of device ID and inode number to identify files
+ file_reference = checkelf[file][1]
+ if file_reference in inodes:
+ os.unlink(file)
+ os.link(inodes[file_reference][0], file)
+ inodes[file_reference].append(file)
+ else:
+ inodes[file_reference] = [file]
+ # break hardlink
+ bb.utils.break_hardlinks(file)
+ elffiles[file] = elf_file
+ # Modified the file so clear the cache
+ cpath.updatecache(file)
+
+ def strip_pkgd_prefix(f):
+ nonlocal dvar
+
+ if f.startswith(dvar):
+ return f[len(dvar):]
+
+ return f
+
+ #
+ # First lets process debug splitting
+ #
+ if (d.getVar('INHIBIT_PACKAGE_DEBUG_SPLIT') != '1'):
+ results = oe.utils.multiprocess_launch(splitdebuginfo, list(elffiles), d, extraargs=(dvar, dv, d))
+
+ if dv["srcdir"] and not hostos.startswith("mingw"):
+ if (d.getVar('PACKAGE_DEBUG_STATIC_SPLIT') == '1'):
+ results = oe.utils.multiprocess_launch(splitstaticdebuginfo, staticlibs, d, extraargs=(dvar, dv, d))
+ else:
+ for file in staticlibs:
+ results.append( (file,source_info(file, d)) )
+
+ d.setVar("PKGDEBUGSOURCES", {strip_pkgd_prefix(f): sorted(s) for f, s in results})
+
+ sources = set()
+ for r in results:
+ sources.update(r[1])
+
+ # Hardlink our debug symbols to the other hardlink copies
+ for ref in inodes:
+ if len(inodes[ref]) == 1:
+ continue
+
+ target = inodes[ref][0][len(dvar):]
+ for file in inodes[ref][1:]:
+ src = file[len(dvar):]
+ dest = dv["libdir"] + os.path.dirname(src) + dv["dir"] + "/" + os.path.basename(target) + dv["append"]
+ fpath = dvar + dest
+ ftarget = dvar + dv["libdir"] + os.path.dirname(target) + dv["dir"] + "/" + os.path.basename(target) + dv["append"]
+ bb.utils.mkdirhier(os.path.dirname(fpath))
+ # Only one hardlink of separated debug info file in each directory
+ if not os.access(fpath, os.R_OK):
+ #bb.note("Link %s -> %s" % (fpath, ftarget))
+ os.link(ftarget, fpath)
+
+ # Create symlinks for all cases we were able to split symbols
+ for file in symlinks:
+ src = file[len(dvar):]
+ dest = dv["libdir"] + os.path.dirname(src) + dv["dir"] + "/" + os.path.basename(src) + dv["append"]
+ fpath = dvar + dest
+ # Skip it if the target doesn't exist
+ try:
+ s = os.stat(fpath)
+ except OSError as e:
+ (err, strerror) = e.args
+ if err != errno.ENOENT:
+ raise
+ continue
+
+ ltarget = symlinks[file]
+ lpath = os.path.dirname(ltarget)
+ lbase = os.path.basename(ltarget)
+ ftarget = ""
+ if lpath and lpath != ".":
+ ftarget += lpath + dv["dir"] + "/"
+ ftarget += lbase + dv["append"]
+ if lpath.startswith(".."):
+ ftarget = os.path.join("..", ftarget)
+ bb.utils.mkdirhier(os.path.dirname(fpath))
+ #bb.note("Symlink %s -> %s" % (fpath, ftarget))
+ os.symlink(ftarget, fpath)
+
+ # Process the dv["srcdir"] if requested...
+ # This copies and places the referenced sources for later debugging...
+ copydebugsources(dv["srcdir"], sources, d)
+ #
+ # End of debug splitting
+ #
+
+ #
+ # Now lets go back over things and strip them
+ #
+ if (d.getVar('INHIBIT_PACKAGE_STRIP') != '1'):
+ strip = d.getVar("STRIP")
+ sfiles = []
+ for file in elffiles:
+ elf_file = int(elffiles[file])
+ #bb.note("Strip %s" % file)
+ sfiles.append((file, elf_file, strip))
+ if (d.getVar('PACKAGE_STRIP_STATIC') == '1' or d.getVar('PACKAGE_DEBUG_STATIC_SPLIT') == '1'):
+ for f in staticlibs:
+ sfiles.append((f, 16, strip))
+
+ oe.utils.multiprocess_launch(oe.package.runstrip, sfiles, d)
+
+ # Build "minidebuginfo" and reinject it back into the stripped binaries
+ if d.getVar('PACKAGE_MINIDEBUGINFO') == '1':
+ oe.utils.multiprocess_launch(inject_minidebuginfo, list(elffiles), d,
+ extraargs=(dvar, dv, d))
+
+ #
+ # End of strip
+ #
+ os.chdir(oldcwd)
+}
+
+python populate_packages () {
+ import glob, re
+
+ workdir = d.getVar('WORKDIR')
+ outdir = d.getVar('DEPLOY_DIR')
+ dvar = d.getVar('PKGD')
+ packages = d.getVar('PACKAGES').split()
+ pn = d.getVar('PN')
+
+ bb.utils.mkdirhier(outdir)
+ os.chdir(dvar)
+
+ autodebug = not (d.getVar("NOAUTOPACKAGEDEBUG") or False)
+
+ split_source_package = (d.getVar('PACKAGE_DEBUG_SPLIT_STYLE') == 'debug-with-srcpkg')
+
+ # If debug-with-srcpkg mode is enabled then add the source package if it
+ # doesn't exist and add the source file contents to the source package.
+ if split_source_package:
+ src_package_name = ('%s-src' % d.getVar('PN'))
+ if not src_package_name in packages:
+ packages.append(src_package_name)
+ d.setVar('FILES:%s' % src_package_name, '/usr/src/debug')
+
+ # Sanity check PACKAGES for duplicates
+ # Sanity should be moved to sanity.bbclass once we have the infrastructure
+ package_dict = {}
+
+ for i, pkg in enumerate(packages):
+ if pkg in package_dict:
+ msg = "%s is listed in PACKAGES multiple times, this leads to packaging errors." % pkg
+ oe.qa.handle_error("packages-list", msg, d)
+ # Ensure the source package gets the chance to pick up the source files
+ # before the debug package by ordering it first in PACKAGES. Whether it
+ # actually picks up any source files is controlled by
+ # PACKAGE_DEBUG_SPLIT_STYLE.
+ elif pkg.endswith("-src"):
+ package_dict[pkg] = (10, i)
+ elif autodebug and pkg.endswith("-dbg"):
+ package_dict[pkg] = (30, i)
+ else:
+ package_dict[pkg] = (50, i)
+ packages = sorted(package_dict.keys(), key=package_dict.get)
+ d.setVar('PACKAGES', ' '.join(packages))
+ pkgdest = d.getVar('PKGDEST')
+
+ seen = []
+
+ # os.mkdir masks the permissions with umask so we have to unset it first
+ oldumask = os.umask(0)
+
+ debug = []
+ for root, dirs, files in cpath.walk(dvar):
+ dir = root[len(dvar):]
+ if not dir:
+ dir = os.sep
+ for f in (files + dirs):
+ path = "." + os.path.join(dir, f)
+ if "/.debug/" in path or "/.debug-static/" in path or path.endswith("/.debug"):
+ debug.append(path)
+
+ for pkg in packages:
+ root = os.path.join(pkgdest, pkg)
+ bb.utils.mkdirhier(root)
+
+ filesvar = d.getVar('FILES:%s' % pkg) or ""
+ if "//" in filesvar:
+ msg = "FILES variable for package %s contains '//' which is invalid. Attempting to fix this but you should correct the metadata.\n" % pkg
+ oe.qa.handle_error("files-invalid", msg, d)
+ filesvar.replace("//", "/")
+
+ origfiles = filesvar.split()
+ files, symlink_paths = files_from_filevars(origfiles)
+
+ if autodebug and pkg.endswith("-dbg"):
+ files.extend(debug)
+
+ for file in files:
+ if (not cpath.islink(file)) and (not cpath.exists(file)):
+ continue
+ if file in seen:
+ continue
+ seen.append(file)
+
+ def mkdir(src, dest, p):
+ src = os.path.join(src, p)
+ dest = os.path.join(dest, p)
+ fstat = cpath.stat(src)
+ os.mkdir(dest)
+ os.chmod(dest, fstat.st_mode)
+ os.chown(dest, fstat.st_uid, fstat.st_gid)
+ if p not in seen:
+ seen.append(p)
+ cpath.updatecache(dest)
+
+ def mkdir_recurse(src, dest, paths):
+ if cpath.exists(dest + '/' + paths):
+ return
+ while paths.startswith("./"):
+ paths = paths[2:]
+ p = "."
+ for c in paths.split("/"):
+ p = os.path.join(p, c)
+ if not cpath.exists(os.path.join(dest, p)):
+ mkdir(src, dest, p)
+
+ if cpath.isdir(file) and not cpath.islink(file):
+ mkdir_recurse(dvar, root, file)
+ continue
+
+ mkdir_recurse(dvar, root, os.path.dirname(file))
+ fpath = os.path.join(root,file)
+ if not cpath.islink(file):
+ os.link(file, fpath)
+ continue
+ ret = bb.utils.copyfile(file, fpath)
+ if ret is False or ret == 0:
+ bb.fatal("File population failed")
+
+ # Check if symlink paths exist
+ for file in symlink_paths:
+ if not os.path.exists(os.path.join(root,file)):
+ bb.fatal("File '%s' cannot be packaged into '%s' because its "
+ "parent directory structure does not exist. One of "
+ "its parent directories is a symlink whose target "
+ "directory is not included in the package." %
+ (file, pkg))
+
+ os.umask(oldumask)
+ os.chdir(workdir)
+
+ # Handle excluding packages with incompatible licenses
+ package_list = []
+ for pkg in packages:
+ licenses = d.getVar('_exclude_incompatible-' + pkg)
+ if licenses:
+ msg = "Excluding %s from packaging as it has incompatible license(s): %s" % (pkg, licenses)
+ oe.qa.handle_error("incompatible-license", msg, d)
+ else:
+ package_list.append(pkg)
+ d.setVar('PACKAGES', ' '.join(package_list))
+
+ unshipped = []
+ for root, dirs, files in cpath.walk(dvar):
+ dir = root[len(dvar):]
+ if not dir:
+ dir = os.sep
+ for f in (files + dirs):
+ path = os.path.join(dir, f)
+ if ('.' + path) not in seen:
+ unshipped.append(path)
+
+ if unshipped != []:
+ msg = pn + ": Files/directories were installed but not shipped in any package:"
+ if "installed-vs-shipped" in (d.getVar('INSANE_SKIP:' + pn) or "").split():
+ bb.note("Package %s skipping QA tests: installed-vs-shipped" % pn)
+ else:
+ for f in unshipped:
+ msg = msg + "\n " + f
+ msg = msg + "\nPlease set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.\n"
+ msg = msg + "%s: %d installed and not shipped files." % (pn, len(unshipped))
+ oe.qa.handle_error("installed-vs-shipped", msg, d)
+}
+populate_packages[dirs] = "${D}"
+
+python package_fixsymlinks () {
+ import errno
+ pkgdest = d.getVar('PKGDEST')
+ packages = d.getVar("PACKAGES", False).split()
+
+ dangling_links = {}
+ pkg_files = {}
+ for pkg in packages:
+ dangling_links[pkg] = []
+ pkg_files[pkg] = []
+ inst_root = os.path.join(pkgdest, pkg)
+ for path in pkgfiles[pkg]:
+ rpath = path[len(inst_root):]
+ pkg_files[pkg].append(rpath)
+ rtarget = cpath.realpath(path, inst_root, True, assume_dir = True)
+ if not cpath.lexists(rtarget):
+ dangling_links[pkg].append(os.path.normpath(rtarget[len(inst_root):]))
+
+ newrdepends = {}
+ for pkg in dangling_links:
+ for l in dangling_links[pkg]:
+ found = False
+ bb.debug(1, "%s contains dangling link %s" % (pkg, l))
+ for p in packages:
+ if l in pkg_files[p]:
+ found = True
+ bb.debug(1, "target found in %s" % p)
+ if p == pkg:
+ break
+ if pkg not in newrdepends:
+ newrdepends[pkg] = []
+ newrdepends[pkg].append(p)
+ break
+ if found == False:
+ bb.note("%s contains dangling symlink to %s" % (pkg, l))
+
+ for pkg in newrdepends:
+ rdepends = bb.utils.explode_dep_versions2(d.getVar('RDEPENDS:' + pkg) or "")
+ for p in newrdepends[pkg]:
+ if p not in rdepends:
+ rdepends[p] = []
+ d.setVar('RDEPENDS:' + pkg, bb.utils.join_deps(rdepends, commasep=False))
+}
+
+
+python package_package_name_hook() {
+ """
+ A package_name_hook function can be used to rewrite the package names by
+ changing PKG. For an example, see debian.bbclass.
+ """
+ pass
+}
+
+EXPORT_FUNCTIONS package_name_hook
+
+
+PKGDESTWORK = "${WORKDIR}/pkgdata"
+
+PKGDATA_VARS = "PN PE PV PR PKGE PKGV PKGR LICENSE DESCRIPTION SUMMARY RDEPENDS RPROVIDES RRECOMMENDS RSUGGESTS RREPLACES RCONFLICTS SECTION PKG ALLOW_EMPTY FILES CONFFILES FILES_INFO PACKAGE_ADD_METADATA pkg_postinst pkg_postrm pkg_preinst pkg_prerm"
+
+python emit_pkgdata() {
+ from glob import glob
+ import json
+ import bb.compress.zstd
+
+ def process_postinst_on_target(pkg, mlprefix):
+ pkgval = d.getVar('PKG:%s' % pkg)
+ if pkgval is None:
+ pkgval = pkg
+
+ defer_fragment = """
+if [ -n "$D" ]; then
+ $INTERCEPT_DIR/postinst_intercept delay_to_first_boot %s mlprefix=%s
+ exit 0
+fi
+""" % (pkgval, mlprefix)
+
+ postinst = d.getVar('pkg_postinst:%s' % pkg)
+ postinst_ontarget = d.getVar('pkg_postinst_ontarget:%s' % pkg)
+
+ if postinst_ontarget:
+ bb.debug(1, 'adding deferred pkg_postinst_ontarget() to pkg_postinst() for %s' % pkg)
+ if not postinst:
+ postinst = '#!/bin/sh\n'
+ postinst += defer_fragment
+ postinst += postinst_ontarget
+ d.setVar('pkg_postinst:%s' % pkg, postinst)
+
+ def add_set_e_to_scriptlets(pkg):
+ for scriptlet_name in ('pkg_preinst', 'pkg_postinst', 'pkg_prerm', 'pkg_postrm'):
+ scriptlet = d.getVar('%s:%s' % (scriptlet_name, pkg))
+ if scriptlet:
+ scriptlet_split = scriptlet.split('\n')
+ if scriptlet_split[0].startswith("#!"):
+ scriptlet = scriptlet_split[0] + "\nset -e\n" + "\n".join(scriptlet_split[1:])
+ else:
+ scriptlet = "set -e\n" + "\n".join(scriptlet_split[0:])
+ d.setVar('%s:%s' % (scriptlet_name, pkg), scriptlet)
+
+ def write_if_exists(f, pkg, var):
+ def encode(str):
+ import codecs
+ c = codecs.getencoder("unicode_escape")
+ return c(str)[0].decode("latin1")
+
+ val = d.getVar('%s:%s' % (var, pkg))
+ if val:
+ f.write('%s:%s: %s\n' % (var, pkg, encode(val)))
+ return val
+ val = d.getVar('%s' % (var))
+ if val:
+ f.write('%s: %s\n' % (var, encode(val)))
+ return val
+
+ def write_extra_pkgs(variants, pn, packages, pkgdatadir):
+ for variant in variants:
+ with open("%s/%s-%s" % (pkgdatadir, variant, pn), 'w') as fd:
+ fd.write("PACKAGES: %s\n" % ' '.join(
+ map(lambda pkg: '%s-%s' % (variant, pkg), packages.split())))
+
+ def write_extra_runtime_pkgs(variants, packages, pkgdatadir):
+ for variant in variants:
+ for pkg in packages.split():
+ ml_pkg = "%s-%s" % (variant, pkg)
+ subdata_file = "%s/runtime/%s" % (pkgdatadir, ml_pkg)
+ with open(subdata_file, 'w') as fd:
+ fd.write("PKG:%s: %s" % (ml_pkg, pkg))
+
+ packages = d.getVar('PACKAGES')
+ pkgdest = d.getVar('PKGDEST')
+ pkgdatadir = d.getVar('PKGDESTWORK')
+
+ data_file = pkgdatadir + d.expand("/${PN}")
+ with open(data_file, 'w') as fd:
+ fd.write("PACKAGES: %s\n" % packages)
+
+ pkgdebugsource = d.getVar("PKGDEBUGSOURCES") or []
+
+ pn = d.getVar('PN')
+ global_variants = (d.getVar('MULTILIB_GLOBAL_VARIANTS') or "").split()
+ variants = (d.getVar('MULTILIB_VARIANTS') or "").split()
+
+ if bb.data.inherits_class('kernel', d) or bb.data.inherits_class('module-base', d):
+ write_extra_pkgs(variants, pn, packages, pkgdatadir)
+
+ if bb.data.inherits_class('allarch', d) and not variants \
+ and not bb.data.inherits_class('packagegroup', d):
+ write_extra_pkgs(global_variants, pn, packages, pkgdatadir)
+
+ workdir = d.getVar('WORKDIR')
+
+ for pkg in packages.split():
+ pkgval = d.getVar('PKG:%s' % pkg)
+ if pkgval is None:
+ pkgval = pkg
+ d.setVar('PKG:%s' % pkg, pkg)
+
+ extended_data = {
+ "files_info": {}
+ }
+
+ pkgdestpkg = os.path.join(pkgdest, pkg)
+ files = {}
+ files_extra = {}
+ total_size = 0
+ seen = set()
+ for f in pkgfiles[pkg]:
+ fpath = os.sep + os.path.relpath(f, pkgdestpkg)
+
+ fstat = os.lstat(f)
+ files[fpath] = fstat.st_size
+
+ extended_data["files_info"].setdefault(fpath, {})
+ extended_data["files_info"][fpath]['size'] = fstat.st_size
+
+ if fstat.st_ino not in seen:
+ seen.add(fstat.st_ino)
+ total_size += fstat.st_size
+
+ if fpath in pkgdebugsource:
+ extended_data["files_info"][fpath]['debugsrc'] = pkgdebugsource[fpath]
+ del pkgdebugsource[fpath]
+
+ d.setVar('FILES_INFO:' + pkg , json.dumps(files, sort_keys=True))
+
+ process_postinst_on_target(pkg, d.getVar("MLPREFIX"))
+ add_set_e_to_scriptlets(pkg)
+
+ subdata_file = pkgdatadir + "/runtime/%s" % pkg
+ with open(subdata_file, 'w') as sf:
+ for var in (d.getVar('PKGDATA_VARS') or "").split():
+ val = write_if_exists(sf, pkg, var)
+
+ write_if_exists(sf, pkg, 'FILERPROVIDESFLIST')
+ for dfile in sorted((d.getVar('FILERPROVIDESFLIST:' + pkg) or "").split()):
+ write_if_exists(sf, pkg, 'FILERPROVIDES:' + dfile)
+
+ write_if_exists(sf, pkg, 'FILERDEPENDSFLIST')
+ for dfile in sorted((d.getVar('FILERDEPENDSFLIST:' + pkg) or "").split()):
+ write_if_exists(sf, pkg, 'FILERDEPENDS:' + dfile)
+
+ sf.write('%s:%s: %d\n' % ('PKGSIZE', pkg, total_size))
+
+ subdata_extended_file = pkgdatadir + "/extended/%s.json.zstd" % pkg
+ num_threads = int(d.getVar("BB_NUMBER_THREADS"))
+ with bb.compress.zstd.open(subdata_extended_file, "wt", encoding="utf-8", num_threads=num_threads) as f:
+ json.dump(extended_data, f, sort_keys=True, separators=(",", ":"))
+
+ # Symlinks needed for rprovides lookup
+ rprov = d.getVar('RPROVIDES:%s' % pkg) or d.getVar('RPROVIDES')
+ if rprov:
+ for p in bb.utils.explode_deps(rprov):
+ subdata_sym = pkgdatadir + "/runtime-rprovides/%s/%s" % (p, pkg)
+ bb.utils.mkdirhier(os.path.dirname(subdata_sym))
+ oe.path.symlink("../../runtime/%s" % pkg, subdata_sym, True)
+
+ allow_empty = d.getVar('ALLOW_EMPTY:%s' % pkg)
+ if not allow_empty:
+ allow_empty = d.getVar('ALLOW_EMPTY')
+ root = "%s/%s" % (pkgdest, pkg)
+ os.chdir(root)
+ g = glob('*')
+ if g or allow_empty == "1":
+ # Symlinks needed for reverse lookups (from the final package name)
+ subdata_sym = pkgdatadir + "/runtime-reverse/%s" % pkgval
+ oe.path.symlink("../runtime/%s" % pkg, subdata_sym, True)
+
+ packagedfile = pkgdatadir + '/runtime/%s.packaged' % pkg
+ open(packagedfile, 'w').close()
+
+ if bb.data.inherits_class('kernel', d) or bb.data.inherits_class('module-base', d):
+ write_extra_runtime_pkgs(variants, packages, pkgdatadir)
+
+ if bb.data.inherits_class('allarch', d) and not variants \
+ and not bb.data.inherits_class('packagegroup', d):
+ write_extra_runtime_pkgs(global_variants, packages, pkgdatadir)
+
+}
+emit_pkgdata[dirs] = "${PKGDESTWORK}/runtime ${PKGDESTWORK}/runtime-reverse ${PKGDESTWORK}/runtime-rprovides ${PKGDESTWORK}/extended"
+emit_pkgdata[vardepsexclude] = "BB_NUMBER_THREADS"
+
+ldconfig_postinst_fragment() {
+if [ x"$D" = "x" ]; then
+ if [ -x /sbin/ldconfig ]; then /sbin/ldconfig ; fi
+fi
+}
+
+RPMDEPS = "${STAGING_LIBDIR_NATIVE}/rpm/rpmdeps --alldeps --define '__font_provides %{nil}'"
+
+# Collect perfile run-time dependency metadata
+# Output:
+# FILERPROVIDESFLIST:pkg - list of all files w/ deps
+# FILERPROVIDES:filepath:pkg - per file dep
+#
+# FILERDEPENDSFLIST:pkg - list of all files w/ deps
+# FILERDEPENDS:filepath:pkg - per file dep
+
+python package_do_filedeps() {
+ if d.getVar('SKIP_FILEDEPS') == '1':
+ return
+
+ pkgdest = d.getVar('PKGDEST')
+ packages = d.getVar('PACKAGES')
+ rpmdeps = d.getVar('RPMDEPS')
+
+ def chunks(files, n):
+ return [files[i:i+n] for i in range(0, len(files), n)]
+
+ pkglist = []
+ for pkg in packages.split():
+ if d.getVar('SKIP_FILEDEPS:' + pkg) == '1':
+ continue
+ if pkg.endswith('-dbg') or pkg.endswith('-doc') or pkg.find('-locale-') != -1 or pkg.find('-localedata-') != -1 or pkg.find('-gconv-') != -1 or pkg.find('-charmap-') != -1 or pkg.startswith('kernel-module-') or pkg.endswith('-src'):
+ continue
+ for files in chunks(pkgfiles[pkg], 100):
+ pkglist.append((pkg, files, rpmdeps, pkgdest))
+
+ processed = oe.utils.multiprocess_launch(oe.package.filedeprunner, pkglist, d)
+
+ provides_files = {}
+ requires_files = {}
+
+ for result in processed:
+ (pkg, provides, requires) = result
+
+ if pkg not in provides_files:
+ provides_files[pkg] = []
+ if pkg not in requires_files:
+ requires_files[pkg] = []
+
+ for file in sorted(provides):
+ provides_files[pkg].append(file)
+ key = "FILERPROVIDES:" + file + ":" + pkg
+ d.appendVar(key, " " + " ".join(provides[file]))
+
+ for file in sorted(requires):
+ requires_files[pkg].append(file)
+ key = "FILERDEPENDS:" + file + ":" + pkg
+ d.appendVar(key, " " + " ".join(requires[file]))
+
+ for pkg in requires_files:
+ d.setVar("FILERDEPENDSFLIST:" + pkg, " ".join(sorted(requires_files[pkg])))
+ for pkg in provides_files:
+ d.setVar("FILERPROVIDESFLIST:" + pkg, " ".join(sorted(provides_files[pkg])))
+}
+
+SHLIBSDIRS = "${WORKDIR_PKGDATA}/${MLPREFIX}shlibs2"
+SHLIBSWORKDIR = "${PKGDESTWORK}/${MLPREFIX}shlibs2"
+
+python package_do_shlibs() {
+ import itertools
+ import re, pipes
+ import subprocess
+
+ exclude_shlibs = d.getVar('EXCLUDE_FROM_SHLIBS', False)
+ if exclude_shlibs:
+ bb.note("not generating shlibs")
+ return
+
+ lib_re = re.compile(r"^.*\.so")
+ libdir_re = re.compile(r".*/%s$" % d.getVar('baselib'))
+
+ packages = d.getVar('PACKAGES')
+
+ shlib_pkgs = []
+ exclusion_list = d.getVar("EXCLUDE_PACKAGES_FROM_SHLIBS")
+ if exclusion_list:
+ for pkg in packages.split():
+ if pkg not in exclusion_list.split():
+ shlib_pkgs.append(pkg)
+ else:
+ bb.note("not generating shlibs for %s" % pkg)
+ else:
+ shlib_pkgs = packages.split()
+
+ hostos = d.getVar('HOST_OS')
+
+ workdir = d.getVar('WORKDIR')
+
+ ver = d.getVar('PKGV')
+ if not ver:
+ msg = "PKGV not defined"
+ oe.qa.handle_error("pkgv-undefined", msg, d)
+ return
+
+ pkgdest = d.getVar('PKGDEST')
+
+ shlibswork_dir = d.getVar('SHLIBSWORKDIR')
+
+ def linux_so(file, pkg, pkgver, d):
+ needs_ldconfig = False
+ needed = set()
+ sonames = set()
+ renames = []
+ ldir = os.path.dirname(file).replace(pkgdest + "/" + pkg, '')
+ cmd = d.getVar('OBJDUMP') + " -p " + pipes.quote(file) + " 2>/dev/null"
+ fd = os.popen(cmd)
+ lines = fd.readlines()
+ fd.close()
+ rpath = tuple()
+ for l in lines:
+ m = re.match(r"\s+RPATH\s+([^\s]*)", l)
+ if m:
+ rpaths = m.group(1).replace("$ORIGIN", ldir).split(":")
+ rpath = tuple(map(os.path.normpath, rpaths))
+ for l in lines:
+ m = re.match(r"\s+NEEDED\s+([^\s]*)", l)
+ if m:
+ dep = m.group(1)
+ if dep not in needed:
+ needed.add((dep, file, rpath))
+ m = re.match(r"\s+SONAME\s+([^\s]*)", l)
+ if m:
+ this_soname = m.group(1)
+ prov = (this_soname, ldir, pkgver)
+ if not prov in sonames:
+ # if library is private (only used by package) then do not build shlib for it
+ import fnmatch
+ if not private_libs or len([i for i in private_libs if fnmatch.fnmatch(this_soname, i)]) == 0:
+ sonames.add(prov)
+ if libdir_re.match(os.path.dirname(file)):
+ needs_ldconfig = True
+ if needs_ldconfig and snap_symlinks and (os.path.basename(file) != this_soname):
+ renames.append((file, os.path.join(os.path.dirname(file), this_soname)))
+ return (needs_ldconfig, needed, sonames, renames)
+
+ def darwin_so(file, needed, sonames, renames, pkgver):
+ if not os.path.exists(file):
+ return
+ ldir = os.path.dirname(file).replace(pkgdest + "/" + pkg, '')
+
+ def get_combinations(base):
+ #
+ # Given a base library name, find all combinations of this split by "." and "-"
+ #
+ combos = []
+ options = base.split(".")
+ for i in range(1, len(options) + 1):
+ combos.append(".".join(options[0:i]))
+ options = base.split("-")
+ for i in range(1, len(options) + 1):
+ combos.append("-".join(options[0:i]))
+ return combos
+
+ if (file.endswith('.dylib') or file.endswith('.so')) and not pkg.endswith('-dev') and not pkg.endswith('-dbg') and not pkg.endswith('-src'):
+ # Drop suffix
+ name = os.path.basename(file).rsplit(".",1)[0]
+ # Find all combinations
+ combos = get_combinations(name)
+ for combo in combos:
+ if not combo in sonames:
+ prov = (combo, ldir, pkgver)
+ sonames.add(prov)
+ if file.endswith('.dylib') or file.endswith('.so'):
+ rpath = []
+ p = subprocess.Popen([d.expand("${HOST_PREFIX}otool"), '-l', file], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ out, err = p.communicate()
+ # If returned successfully, process stdout for results
+ if p.returncode == 0:
+ for l in out.split("\n"):
+ l = l.strip()
+ if l.startswith('path '):
+ rpath.append(l.split()[1])
+
+ p = subprocess.Popen([d.expand("${HOST_PREFIX}otool"), '-L', file], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ out, err = p.communicate()
+ # If returned successfully, process stdout for results
+ if p.returncode == 0:
+ for l in out.split("\n"):
+ l = l.strip()
+ if not l or l.endswith(":"):
+ continue
+ if "is not an object file" in l:
+ continue
+ name = os.path.basename(l.split()[0]).rsplit(".", 1)[0]
+ if name and name not in needed[pkg]:
+ needed[pkg].add((name, file, tuple()))
+
+ def mingw_dll(file, needed, sonames, renames, pkgver):
+ if not os.path.exists(file):
+ return
+
+ if file.endswith(".dll"):
+ # assume all dlls are shared objects provided by the package
+ sonames.add((os.path.basename(file), os.path.dirname(file).replace(pkgdest + "/" + pkg, ''), pkgver))
+
+ if (file.endswith(".dll") or file.endswith(".exe")):
+ # use objdump to search for "DLL Name: .*\.dll"
+ p = subprocess.Popen([d.expand("${HOST_PREFIX}objdump"), "-p", file], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ out, err = p.communicate()
+ # process the output, grabbing all .dll names
+ if p.returncode == 0:
+ for m in re.finditer(r"DLL Name: (.*?\.dll)$", out.decode(), re.MULTILINE | re.IGNORECASE):
+ dllname = m.group(1)
+ if dllname:
+ needed[pkg].add((dllname, file, tuple()))
+
+ if d.getVar('PACKAGE_SNAP_LIB_SYMLINKS') == "1":
+ snap_symlinks = True
+ else:
+ snap_symlinks = False
+
+ needed = {}
+
+ shlib_provider = oe.package.read_shlib_providers(d)
+
+ for pkg in shlib_pkgs:
+ private_libs = d.getVar('PRIVATE_LIBS:' + pkg) or d.getVar('PRIVATE_LIBS') or ""
+ private_libs = private_libs.split()
+ needs_ldconfig = False
+ bb.debug(2, "calculating shlib provides for %s" % pkg)
+
+ pkgver = d.getVar('PKGV:' + pkg)
+ if not pkgver:
+ pkgver = d.getVar('PV_' + pkg)
+ if not pkgver:
+ pkgver = ver
+
+ needed[pkg] = set()
+ sonames = set()
+ renames = []
+ linuxlist = []
+ for file in pkgfiles[pkg]:
+ soname = None
+ if cpath.islink(file):
+ continue
+ if hostos == "darwin" or hostos == "darwin8":
+ darwin_so(file, needed, sonames, renames, pkgver)
+ elif hostos.startswith("mingw"):
+ mingw_dll(file, needed, sonames, renames, pkgver)
+ elif os.access(file, os.X_OK) or lib_re.match(file):
+ linuxlist.append(file)
+
+ if linuxlist:
+ results = oe.utils.multiprocess_launch(linux_so, linuxlist, d, extraargs=(pkg, pkgver, d))
+ for r in results:
+ ldconfig = r[0]
+ needed[pkg] |= r[1]
+ sonames |= r[2]
+ renames.extend(r[3])
+ needs_ldconfig = needs_ldconfig or ldconfig
+
+ for (old, new) in renames:
+ bb.note("Renaming %s to %s" % (old, new))
+ bb.utils.rename(old, new)
+ pkgfiles[pkg].remove(old)
+
+ shlibs_file = os.path.join(shlibswork_dir, pkg + ".list")
+ if len(sonames):
+ with open(shlibs_file, 'w') as fd:
+ for s in sorted(sonames):
+ if s[0] in shlib_provider and s[1] in shlib_provider[s[0]]:
+ (old_pkg, old_pkgver) = shlib_provider[s[0]][s[1]]
+ if old_pkg != pkg:
+ bb.warn('%s-%s was registered as shlib provider for %s, changing it to %s-%s because it was built later' % (old_pkg, old_pkgver, s[0], pkg, pkgver))
+ bb.debug(1, 'registering %s-%s as shlib provider for %s' % (pkg, pkgver, s[0]))
+ fd.write(s[0] + ':' + s[1] + ':' + s[2] + '\n')
+ if s[0] not in shlib_provider:
+ shlib_provider[s[0]] = {}
+ shlib_provider[s[0]][s[1]] = (pkg, pkgver)
+ if needs_ldconfig:
+ bb.debug(1, 'adding ldconfig call to postinst for %s' % pkg)
+ postinst = d.getVar('pkg_postinst:%s' % pkg)
+ if not postinst:
+ postinst = '#!/bin/sh\n'
+ postinst += d.getVar('ldconfig_postinst_fragment')
+ d.setVar('pkg_postinst:%s' % pkg, postinst)
+ bb.debug(1, 'LIBNAMES: pkg %s sonames %s' % (pkg, sonames))
+
+ assumed_libs = d.getVar('ASSUME_SHLIBS')
+ if assumed_libs:
+ libdir = d.getVar("libdir")
+ for e in assumed_libs.split():
+ l, dep_pkg = e.split(":")
+ lib_ver = None
+ dep_pkg = dep_pkg.rsplit("_", 1)
+ if len(dep_pkg) == 2:
+ lib_ver = dep_pkg[1]
+ dep_pkg = dep_pkg[0]
+ if l not in shlib_provider:
+ shlib_provider[l] = {}
+ shlib_provider[l][libdir] = (dep_pkg, lib_ver)
+
+ libsearchpath = [d.getVar('libdir'), d.getVar('base_libdir')]
+
+ for pkg in shlib_pkgs:
+ bb.debug(2, "calculating shlib requirements for %s" % pkg)
+
+ private_libs = d.getVar('PRIVATE_LIBS:' + pkg) or d.getVar('PRIVATE_LIBS') or ""
+ private_libs = private_libs.split()
+
+ deps = list()
+ for n in needed[pkg]:
+ # if n is in private libraries, don't try to search provider for it
+ # this could cause problem in case some abc.bb provides private
+ # /opt/abc/lib/libfoo.so.1 and contains /usr/bin/abc depending on system library libfoo.so.1
+ # but skipping it is still better alternative than providing own
+ # version and then adding runtime dependency for the same system library
+ import fnmatch
+ if private_libs and len([i for i in private_libs if fnmatch.fnmatch(n[0], i)]) > 0:
+ bb.debug(2, '%s: Dependency %s covered by PRIVATE_LIBS' % (pkg, n[0]))
+ continue
+ if n[0] in shlib_provider.keys():
+ shlib_provider_map = shlib_provider[n[0]]
+ matches = set()
+ for p in itertools.chain(list(n[2]), sorted(shlib_provider_map.keys()), libsearchpath):
+ if p in shlib_provider_map:
+ matches.add(p)
+ if len(matches) > 1:
+ matchpkgs = ', '.join([shlib_provider_map[match][0] for match in matches])
+ bb.error("%s: Multiple shlib providers for %s: %s (used by files: %s)" % (pkg, n[0], matchpkgs, n[1]))
+ elif len(matches) == 1:
+ (dep_pkg, ver_needed) = shlib_provider_map[matches.pop()]
+
+ bb.debug(2, '%s: Dependency %s requires package %s (used by files: %s)' % (pkg, n[0], dep_pkg, n[1]))
+
+ if dep_pkg == pkg:
+ continue
+
+ if ver_needed:
+ dep = "%s (>= %s)" % (dep_pkg, ver_needed)
+ else:
+ dep = dep_pkg
+ if not dep in deps:
+ deps.append(dep)
+ continue
+ bb.note("Couldn't find shared library provider for %s, used by files: %s" % (n[0], n[1]))
+
+ deps_file = os.path.join(pkgdest, pkg + ".shlibdeps")
+ if os.path.exists(deps_file):
+ os.remove(deps_file)
+ if deps:
+ with open(deps_file, 'w') as fd:
+ for dep in sorted(deps):
+ fd.write(dep + '\n')
+}
+
+python package_do_pkgconfig () {
+ import re
+
+ packages = d.getVar('PACKAGES')
+ workdir = d.getVar('WORKDIR')
+ pkgdest = d.getVar('PKGDEST')
+
+ shlibs_dirs = d.getVar('SHLIBSDIRS').split()
+ shlibswork_dir = d.getVar('SHLIBSWORKDIR')
+
+ pc_re = re.compile(r'(.*)\.pc$')
+ var_re = re.compile(r'(.*)=(.*)')
+ field_re = re.compile(r'(.*): (.*)')
+
+ pkgconfig_provided = {}
+ pkgconfig_needed = {}
+ for pkg in packages.split():
+ pkgconfig_provided[pkg] = []
+ pkgconfig_needed[pkg] = []
+ for file in sorted(pkgfiles[pkg]):
+ m = pc_re.match(file)
+ if m:
+ pd = bb.data.init()
+ name = m.group(1)
+ pkgconfig_provided[pkg].append(os.path.basename(name))
+ if not os.access(file, os.R_OK):
+ continue
+ with open(file, 'r') as f:
+ lines = f.readlines()
+ for l in lines:
+ m = var_re.match(l)
+ if m:
+ name = m.group(1)
+ val = m.group(2)
+ pd.setVar(name, pd.expand(val))
+ continue
+ m = field_re.match(l)
+ if m:
+ hdr = m.group(1)
+ exp = pd.expand(m.group(2))
+ if hdr == 'Requires':
+ pkgconfig_needed[pkg] += exp.replace(',', ' ').split()
+
+ for pkg in packages.split():
+ pkgs_file = os.path.join(shlibswork_dir, pkg + ".pclist")
+ if pkgconfig_provided[pkg] != []:
+ with open(pkgs_file, 'w') as f:
+ for p in sorted(pkgconfig_provided[pkg]):
+ f.write('%s\n' % p)
+
+ # Go from least to most specific since the last one found wins
+ for dir in reversed(shlibs_dirs):
+ if not os.path.exists(dir):
+ continue
+ for file in sorted(os.listdir(dir)):
+ m = re.match(r'^(.*)\.pclist$', file)
+ if m:
+ pkg = m.group(1)
+ with open(os.path.join(dir, file)) as fd:
+ lines = fd.readlines()
+ pkgconfig_provided[pkg] = []
+ for l in lines:
+ pkgconfig_provided[pkg].append(l.rstrip())
+
+ for pkg in packages.split():
+ deps = []
+ for n in pkgconfig_needed[pkg]:
+ found = False
+ for k in pkgconfig_provided.keys():
+ if n in pkgconfig_provided[k]:
+ if k != pkg and not (k in deps):
+ deps.append(k)
+ found = True
+ if found == False:
+ bb.note("couldn't find pkgconfig module '%s' in any package" % n)
+ deps_file = os.path.join(pkgdest, pkg + ".pcdeps")
+ if len(deps):
+ with open(deps_file, 'w') as fd:
+ for dep in deps:
+ fd.write(dep + '\n')
+}
+
+def read_libdep_files(d):
+ pkglibdeps = {}
+ packages = d.getVar('PACKAGES').split()
+ for pkg in packages:
+ pkglibdeps[pkg] = {}
+ for extension in ".shlibdeps", ".pcdeps", ".clilibdeps":
+ depsfile = d.expand("${PKGDEST}/" + pkg + extension)
+ if os.access(depsfile, os.R_OK):
+ with open(depsfile) as fd:
+ lines = fd.readlines()
+ for l in lines:
+ l.rstrip()
+ deps = bb.utils.explode_dep_versions2(l)
+ for dep in deps:
+ if not dep in pkglibdeps[pkg]:
+ pkglibdeps[pkg][dep] = deps[dep]
+ return pkglibdeps
+
+python read_shlibdeps () {
+ pkglibdeps = read_libdep_files(d)
+
+ packages = d.getVar('PACKAGES').split()
+ for pkg in packages:
+ rdepends = bb.utils.explode_dep_versions2(d.getVar('RDEPENDS:' + pkg) or "")
+ for dep in sorted(pkglibdeps[pkg]):
+ # Add the dep if it's not already there, or if no comparison is set
+ if dep not in rdepends:
+ rdepends[dep] = []
+ for v in pkglibdeps[pkg][dep]:
+ if v not in rdepends[dep]:
+ rdepends[dep].append(v)
+ d.setVar('RDEPENDS:' + pkg, bb.utils.join_deps(rdepends, commasep=False))
+}
+
+python package_depchains() {
+ """
+ For a given set of prefix and postfix modifiers, make those packages
+ RRECOMMENDS on the corresponding packages for its RDEPENDS.
+
+ Example: If package A depends upon package B, and A's .bb emits an
+ A-dev package, this would make A-dev Recommends: B-dev.
+
+ If only one of a given suffix is specified, it will take the RRECOMMENDS
+ based on the RDEPENDS of *all* other packages. If more than one of a given
+ suffix is specified, its will only use the RDEPENDS of the single parent
+ package.
+ """
+
+ packages = d.getVar('PACKAGES')
+ postfixes = (d.getVar('DEPCHAIN_POST') or '').split()
+ prefixes = (d.getVar('DEPCHAIN_PRE') or '').split()
+
+ def pkg_adddeprrecs(pkg, base, suffix, getname, depends, d):
+
+ #bb.note('depends for %s is %s' % (base, depends))
+ rreclist = bb.utils.explode_dep_versions2(d.getVar('RRECOMMENDS:' + pkg) or "")
+
+ for depend in sorted(depends):
+ if depend.find('-native') != -1 or depend.find('-cross') != -1 or depend.startswith('virtual/'):
+ #bb.note("Skipping %s" % depend)
+ continue
+ if depend.endswith('-dev'):
+ depend = depend[:-4]
+ if depend.endswith('-dbg'):
+ depend = depend[:-4]
+ pkgname = getname(depend, suffix)
+ #bb.note("Adding %s for %s" % (pkgname, depend))
+ if pkgname not in rreclist and pkgname != pkg:
+ rreclist[pkgname] = []
+
+ #bb.note('setting: RRECOMMENDS:%s=%s' % (pkg, ' '.join(rreclist)))
+ d.setVar('RRECOMMENDS:%s' % pkg, bb.utils.join_deps(rreclist, commasep=False))
+
+ def pkg_addrrecs(pkg, base, suffix, getname, rdepends, d):
+
+ #bb.note('rdepends for %s is %s' % (base, rdepends))
+ rreclist = bb.utils.explode_dep_versions2(d.getVar('RRECOMMENDS:' + pkg) or "")
+
+ for depend in sorted(rdepends):
+ if depend.find('virtual-locale-') != -1:
+ #bb.note("Skipping %s" % depend)
+ continue
+ if depend.endswith('-dev'):
+ depend = depend[:-4]
+ if depend.endswith('-dbg'):
+ depend = depend[:-4]
+ pkgname = getname(depend, suffix)
+ #bb.note("Adding %s for %s" % (pkgname, depend))
+ if pkgname not in rreclist and pkgname != pkg:
+ rreclist[pkgname] = []
+
+ #bb.note('setting: RRECOMMENDS:%s=%s' % (pkg, ' '.join(rreclist)))
+ d.setVar('RRECOMMENDS:%s' % pkg, bb.utils.join_deps(rreclist, commasep=False))
+
+ def add_dep(list, dep):
+ if dep not in list:
+ list.append(dep)
+
+ depends = []
+ for dep in bb.utils.explode_deps(d.getVar('DEPENDS') or ""):
+ add_dep(depends, dep)
+
+ rdepends = []
+ for pkg in packages.split():
+ for dep in bb.utils.explode_deps(d.getVar('RDEPENDS:' + pkg) or ""):
+ add_dep(rdepends, dep)
+
+ #bb.note('rdepends is %s' % rdepends)
+
+ def post_getname(name, suffix):
+ return '%s%s' % (name, suffix)
+ def pre_getname(name, suffix):
+ return '%s%s' % (suffix, name)
+
+ pkgs = {}
+ for pkg in packages.split():
+ for postfix in postfixes:
+ if pkg.endswith(postfix):
+ if not postfix in pkgs:
+ pkgs[postfix] = {}
+ pkgs[postfix][pkg] = (pkg[:-len(postfix)], post_getname)
+
+ for prefix in prefixes:
+ if pkg.startswith(prefix):
+ if not prefix in pkgs:
+ pkgs[prefix] = {}
+ pkgs[prefix][pkg] = (pkg[:-len(prefix)], pre_getname)
+
+ if "-dbg" in pkgs:
+ pkglibdeps = read_libdep_files(d)
+ pkglibdeplist = []
+ for pkg in pkglibdeps:
+ for k in pkglibdeps[pkg]:
+ add_dep(pkglibdeplist, k)
+ dbgdefaultdeps = ((d.getVar('DEPCHAIN_DBGDEFAULTDEPS') == '1') or (bb.data.inherits_class('packagegroup', d)))
+
+ for suffix in pkgs:
+ for pkg in pkgs[suffix]:
+ if d.getVarFlag('RRECOMMENDS:' + pkg, 'nodeprrecs'):
+ continue
+ (base, func) = pkgs[suffix][pkg]
+ if suffix == "-dev":
+ pkg_adddeprrecs(pkg, base, suffix, func, depends, d)
+ elif suffix == "-dbg":
+ if not dbgdefaultdeps:
+ pkg_addrrecs(pkg, base, suffix, func, pkglibdeplist, d)
+ continue
+ if len(pkgs[suffix]) == 1:
+ pkg_addrrecs(pkg, base, suffix, func, rdepends, d)
+ else:
+ rdeps = []
+ for dep in bb.utils.explode_deps(d.getVar('RDEPENDS:' + base) or ""):
+ add_dep(rdeps, dep)
+ pkg_addrrecs(pkg, base, suffix, func, rdeps, d)
+}
+
+# Since bitbake can't determine which variables are accessed during package
+# iteration, we need to list them here:
+PACKAGEVARS = "FILES RDEPENDS RRECOMMENDS SUMMARY DESCRIPTION RSUGGESTS RPROVIDES RCONFLICTS PKG ALLOW_EMPTY pkg_postinst pkg_postrm pkg_postinst_ontarget INITSCRIPT_NAME INITSCRIPT_PARAMS DEBIAN_NOAUTONAME ALTERNATIVE PKGE PKGV PKGR USERADD_PARAM GROUPADD_PARAM CONFFILES SYSTEMD_SERVICE LICENSE SECTION pkg_preinst pkg_prerm RREPLACES GROUPMEMS_PARAM SYSTEMD_AUTO_ENABLE SKIP_FILEDEPS PRIVATE_LIBS PACKAGE_ADD_METADATA"
+
+def gen_packagevar(d, pkgvars="PACKAGEVARS"):
+ ret = []
+ pkgs = (d.getVar("PACKAGES") or "").split()
+ vars = (d.getVar(pkgvars) or "").split()
+ for v in vars:
+ ret.append(v)
+ for p in pkgs:
+ for v in vars:
+ ret.append(v + ":" + p)
+
+ # Ensure that changes to INCOMPATIBLE_LICENSE re-run do_package for
+ # affected recipes.
+ ret.append('_exclude_incompatible-%s' % p)
+ return " ".join(ret)
+
+PACKAGE_PREPROCESS_FUNCS ?= ""
+# Functions for setting up PKGD
+PACKAGEBUILDPKGD ?= " \
+ package_prepare_pkgdata \
+ perform_packagecopy \
+ ${PACKAGE_PREPROCESS_FUNCS} \
+ split_and_strip_files \
+ fixup_perms \
+ "
+# Functions which split PKGD up into separate packages
+PACKAGESPLITFUNCS ?= " \
+ package_do_split_locales \
+ populate_packages"
+# Functions which process metadata based on split packages
+PACKAGEFUNCS += " \
+ package_fixsymlinks \
+ package_name_hook \
+ package_do_filedeps \
+ package_do_shlibs \
+ package_do_pkgconfig \
+ read_shlibdeps \
+ package_depchains \
+ emit_pkgdata"
+
+python do_package () {
+ # Change the following version to cause sstate to invalidate the package
+ # cache. This is useful if an item this class depends on changes in a
+ # way that the output of this class changes. rpmdeps is a good example
+ # as any change to rpmdeps requires this to be rerun.
+ # PACKAGE_BBCLASS_VERSION = "4"
+
+ # Init cachedpath
+ global cpath
+ cpath = oe.cachedpath.CachedPath()
+
+ ###########################################################################
+ # Sanity test the setup
+ ###########################################################################
+
+ packages = (d.getVar('PACKAGES') or "").split()
+ if len(packages) < 1:
+ bb.debug(1, "No packages to build, skipping do_package")
+ return
+
+ workdir = d.getVar('WORKDIR')
+ outdir = d.getVar('DEPLOY_DIR')
+ dest = d.getVar('D')
+ dvar = d.getVar('PKGD')
+ pn = d.getVar('PN')
+
+ if not workdir or not outdir or not dest or not dvar or not pn:
+ msg = "WORKDIR, DEPLOY_DIR, D, PN and PKGD all must be defined, unable to package"
+ oe.qa.handle_error("var-undefined", msg, d)
+ return
+
+ bb.build.exec_func("package_convert_pr_autoinc", d)
+
+ ###########################################################################
+ # Optimisations
+ ###########################################################################
+
+ # Continually expanding complex expressions is inefficient, particularly
+ # when we write to the datastore and invalidate the expansion cache. This
+ # code pre-expands some frequently used variables
+
+ def expandVar(x, d):
+ d.setVar(x, d.getVar(x))
+
+ for x in 'PN', 'PV', 'BPN', 'TARGET_SYS', 'EXTENDPRAUTO':
+ expandVar(x, d)
+
+ ###########################################################################
+ # Setup PKGD (from D)
+ ###########################################################################
+
+ for f in (d.getVar('PACKAGEBUILDPKGD') or '').split():
+ bb.build.exec_func(f, d)
+
+ ###########################################################################
+ # Split up PKGD into PKGDEST
+ ###########################################################################
+
+ cpath = oe.cachedpath.CachedPath()
+
+ for f in (d.getVar('PACKAGESPLITFUNCS') or '').split():
+ bb.build.exec_func(f, d)
+
+ ###########################################################################
+ # Process PKGDEST
+ ###########################################################################
+
+ # Build global list of files in each split package
+ global pkgfiles
+ pkgfiles = {}
+ packages = d.getVar('PACKAGES').split()
+ pkgdest = d.getVar('PKGDEST')
+ for pkg in packages:
+ pkgfiles[pkg] = []
+ for walkroot, dirs, files in cpath.walk(pkgdest + "/" + pkg):
+ for file in files:
+ pkgfiles[pkg].append(walkroot + os.sep + file)
+
+ for f in (d.getVar('PACKAGEFUNCS') or '').split():
+ bb.build.exec_func(f, d)
+
+ oe.qa.exit_if_errors(d)
+}
+
+do_package[dirs] = "${SHLIBSWORKDIR} ${D}"
+do_package[vardeps] += "${PACKAGEBUILDPKGD} ${PACKAGESPLITFUNCS} ${PACKAGEFUNCS} ${@gen_packagevar(d)}"
+addtask package after do_install
+
+SSTATETASKS += "do_package"
+do_package[cleandirs] = "${PKGDEST} ${PKGDESTWORK}"
+do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST} ${PKGDESTWORK}"
+do_package_setscene[dirs] = "${STAGING_DIR}"
+
+python do_package_setscene () {
+ sstate_setscene(d)
+}
+addtask do_package_setscene
+
+# Copy from PKGDESTWORK to tempdirectory as tempdirectory can be cleaned at both
+# do_package_setscene and do_packagedata_setscene leading to races
+python do_packagedata () {
+ bb.build.exec_func("package_get_auto_pr", d)
+
+ src = d.expand("${PKGDESTWORK}")
+ dest = d.expand("${WORKDIR}/pkgdata-pdata-input")
+ oe.path.copyhardlinktree(src, dest)
+
+ bb.build.exec_func("packagedata_translate_pr_autoinc", d)
+}
+do_packagedata[cleandirs] += "${WORKDIR}/pkgdata-pdata-input"
+
+# Translate the EXTENDPRAUTO and AUTOINC to the final values
+packagedata_translate_pr_autoinc() {
+ find ${WORKDIR}/pkgdata-pdata-input -type f | xargs --no-run-if-empty \
+ sed -e 's,@PRSERV_PV_AUTOINC@,${PRSERV_PV_AUTOINC},g' \
+ -e 's,@EXTENDPRAUTO@,${EXTENDPRAUTO},g' -i
+}
+
+addtask packagedata before do_build after do_package
+
+SSTATETASKS += "do_packagedata"
+do_packagedata[sstate-inputdirs] = "${WORKDIR}/pkgdata-pdata-input"
+do_packagedata[sstate-outputdirs] = "${PKGDATA_DIR}"
+do_packagedata[stamp-extra-info] = "${MACHINE_ARCH}"
+
+python do_packagedata_setscene () {
+ sstate_setscene(d)
+}
+addtask do_packagedata_setscene
+
+#
+# Helper functions for the package writing classes
+#
+
+def mapping_rename_hook(d):
+ """
+ Rewrite variables to account for package renaming in things
+ like debian.bbclass or manual PKG variable name changes
+ """
+ pkg = d.getVar("PKG")
+ runtime_mapping_rename("RDEPENDS", pkg, d)
+ runtime_mapping_rename("RRECOMMENDS", pkg, d)
+ runtime_mapping_rename("RSUGGESTS", pkg, d)
diff --git a/poky/meta/classes-global/package_deb.bbclass b/poky/meta/classes-global/package_deb.bbclass
new file mode 100644
index 0000000..ec7e10d
--- /dev/null
+++ b/poky/meta/classes-global/package_deb.bbclass
@@ -0,0 +1,329 @@
+#
+# Copyright 2006-2008 OpenedHand Ltd.
+#
+# SPDX-License-Identifier: MIT
+#
+
+inherit package
+
+IMAGE_PKGTYPE ?= "deb"
+
+DPKG_BUILDCMD ??= "dpkg-deb"
+
+DPKG_ARCH ?= "${@debian_arch_map(d.getVar('TARGET_ARCH'), d.getVar('TUNE_FEATURES'))}"
+DPKG_ARCH[vardepvalue] = "${DPKG_ARCH}"
+
+PKGWRITEDIRDEB = "${WORKDIR}/deploy-debs"
+
+APTCONF_TARGET = "${WORKDIR}"
+
+APT_ARGS = "${@['', '--no-install-recommends'][d.getVar("NO_RECOMMENDATIONS") == "1"]}"
+
+def debian_arch_map(arch, tune):
+ tune_features = tune.split()
+ if arch == "allarch":
+ return "all"
+ if arch in ["i586", "i686"]:
+ return "i386"
+ if arch == "x86_64":
+ if "mx32" in tune_features:
+ return "x32"
+ return "amd64"
+ if arch.startswith("mips"):
+ endian = ["el", ""]["bigendian" in tune_features]
+ if "n64" in tune_features:
+ return "mips64" + endian
+ if "n32" in tune_features:
+ return "mipsn32" + endian
+ return "mips" + endian
+ if arch == "powerpc":
+ return arch + ["", "spe"]["spe" in tune_features]
+ if arch == "aarch64":
+ return "arm64"
+ if arch == "arm":
+ return arch + ["el", "hf"]["callconvention-hard" in tune_features]
+ return arch
+
+python do_package_deb () {
+ packages = d.getVar('PACKAGES')
+ if not packages:
+ bb.debug(1, "PACKAGES not defined, nothing to package")
+ return
+
+ tmpdir = d.getVar('TMPDIR')
+ if os.access(os.path.join(tmpdir, "stamps", "DEB_PACKAGE_INDEX_CLEAN"),os.R_OK):
+ os.unlink(os.path.join(tmpdir, "stamps", "DEB_PACKAGE_INDEX_CLEAN"))
+
+ oe.utils.multiprocess_launch(deb_write_pkg, packages.split(), d, extraargs=(d,))
+}
+do_package_deb[vardeps] += "deb_write_pkg"
+do_package_deb[vardepsexclude] = "BB_NUMBER_THREADS"
+
+def deb_write_pkg(pkg, d):
+ import re, copy
+ import textwrap
+ import subprocess
+ import collections
+ import codecs
+
+ outdir = d.getVar('PKGWRITEDIRDEB')
+ pkgdest = d.getVar('PKGDEST')
+
+ def cleanupcontrol(root):
+ for p in ['CONTROL', 'DEBIAN']:
+ p = os.path.join(root, p)
+ if os.path.exists(p):
+ bb.utils.prunedir(p)
+
+ localdata = bb.data.createCopy(d)
+ root = "%s/%s" % (pkgdest, pkg)
+
+ lf = bb.utils.lockfile(root + ".lock")
+ try:
+
+ localdata.setVar('ROOT', '')
+ localdata.setVar('ROOT_%s' % pkg, root)
+ pkgname = localdata.getVar('PKG:%s' % pkg)
+ if not pkgname:
+ pkgname = pkg
+ localdata.setVar('PKG', pkgname)
+
+ localdata.setVar('OVERRIDES', d.getVar("OVERRIDES", False) + ":" + pkg)
+
+ basedir = os.path.join(os.path.dirname(root))
+
+ pkgoutdir = os.path.join(outdir, localdata.getVar('PACKAGE_ARCH'))
+ bb.utils.mkdirhier(pkgoutdir)
+
+ os.chdir(root)
+ cleanupcontrol(root)
+ from glob import glob
+ g = glob('*')
+ if not g and localdata.getVar('ALLOW_EMPTY', False) != "1":
+ bb.note("Not creating empty archive for %s-%s-%s" % (pkg, localdata.getVar('PKGV'), localdata.getVar('PKGR')))
+ return
+
+ controldir = os.path.join(root, 'DEBIAN')
+ bb.utils.mkdirhier(controldir)
+ os.chmod(controldir, 0o755)
+
+ ctrlfile = codecs.open(os.path.join(controldir, 'control'), 'w', 'utf-8')
+
+ fields = []
+ pe = d.getVar('PKGE')
+ if pe and int(pe) > 0:
+ fields.append(["Version: %s:%s-%s\n", ['PKGE', 'PKGV', 'PKGR']])
+ else:
+ fields.append(["Version: %s-%s\n", ['PKGV', 'PKGR']])
+ fields.append(["Description: %s\n", ['DESCRIPTION']])
+ fields.append(["Section: %s\n", ['SECTION']])
+ fields.append(["Priority: %s\n", ['PRIORITY']])
+ fields.append(["Maintainer: %s\n", ['MAINTAINER']])
+ fields.append(["Architecture: %s\n", ['DPKG_ARCH']])
+ fields.append(["OE: %s\n", ['PN']])
+ fields.append(["PackageArch: %s\n", ['PACKAGE_ARCH']])
+ if d.getVar('HOMEPAGE'):
+ fields.append(["Homepage: %s\n", ['HOMEPAGE']])
+
+ # Package, Version, Maintainer, Description - mandatory
+ # Section, Priority, Essential, Architecture, Source, Depends, Pre-Depends, Recommends, Suggests, Conflicts, Replaces, Provides - Optional
+
+
+ def pullData(l, d):
+ l2 = []
+ for i in l:
+ data = d.getVar(i)
+ if data is None:
+ raise KeyError(i)
+ if i == 'DPKG_ARCH' and d.getVar('PACKAGE_ARCH') == 'all':
+ data = 'all'
+ elif i == 'PACKAGE_ARCH' or i == 'DPKG_ARCH':
+ # The params in deb package control don't allow character
+ # `_', so change the arch's `_' to `-'. Such as `x86_64'
+ # -->`x86-64'
+ data = data.replace('_', '-')
+ l2.append(data)
+ return l2
+
+ ctrlfile.write("Package: %s\n" % pkgname)
+ if d.getVar('PACKAGE_ARCH') == "all":
+ ctrlfile.write("Multi-Arch: foreign\n")
+ # check for required fields
+ for (c, fs) in fields:
+ # Special behavior for description...
+ if 'DESCRIPTION' in fs:
+ summary = localdata.getVar('SUMMARY') or localdata.getVar('DESCRIPTION') or "."
+ ctrlfile.write('Description: %s\n' % summary)
+ description = localdata.getVar('DESCRIPTION') or "."
+ description = textwrap.dedent(description).strip()
+ if '\\n' in description:
+ # Manually indent
+ for t in description.split('\\n'):
+ ctrlfile.write(' %s\n' % (t.strip() or '.'))
+ else:
+ # Auto indent
+ ctrlfile.write('%s\n' % textwrap.fill(description.strip(), width=74, initial_indent=' ', subsequent_indent=' '))
+
+ else:
+ ctrlfile.write(c % tuple(pullData(fs, localdata)))
+
+ # more fields
+
+ custom_fields_chunk = get_package_additional_metadata("deb", localdata)
+ if custom_fields_chunk:
+ ctrlfile.write(custom_fields_chunk)
+ ctrlfile.write("\n")
+
+ mapping_rename_hook(localdata)
+
+ def debian_cmp_remap(var):
+ # dpkg does not allow for '(', ')' or ':' in a dependency name
+ # Replace any instances of them with '__'
+ #
+ # In debian '>' and '<' do not mean what it appears they mean
+ # '<' = less or equal
+ # '>' = greater or equal
+ # adjust these to the '<<' and '>>' equivalents
+ # Also, "=" specifiers only work if they have the PR in, so 1.2.3 != 1.2.3-r0
+ # so to avoid issues, map this to ">= 1.2.3 << 1.2.3.0"
+ for dep in list(var.keys()):
+ if '(' in dep or '/' in dep:
+ newdep = re.sub(r'[(:)/]', '__', dep)
+ if newdep.startswith("__"):
+ newdep = "A" + newdep
+ if newdep != dep:
+ var[newdep] = var[dep]
+ del var[dep]
+ for dep in var:
+ for i, v in enumerate(var[dep]):
+ if (v or "").startswith("< "):
+ var[dep][i] = var[dep][i].replace("< ", "<< ")
+ elif (v or "").startswith("> "):
+ var[dep][i] = var[dep][i].replace("> ", ">> ")
+ elif (v or "").startswith("= ") and "-r" not in v:
+ ver = var[dep][i].replace("= ", "")
+ var[dep][i] = var[dep][i].replace("= ", ">= ")
+ var[dep].append("<< " + ver + ".0")
+
+ rdepends = bb.utils.explode_dep_versions2(localdata.getVar("RDEPENDS") or "")
+ debian_cmp_remap(rdepends)
+ for dep in list(rdepends.keys()):
+ if dep == pkg:
+ del rdepends[dep]
+ continue
+ if '*' in dep:
+ del rdepends[dep]
+ rrecommends = bb.utils.explode_dep_versions2(localdata.getVar("RRECOMMENDS") or "")
+ debian_cmp_remap(rrecommends)
+ for dep in list(rrecommends.keys()):
+ if '*' in dep:
+ del rrecommends[dep]
+ rsuggests = bb.utils.explode_dep_versions2(localdata.getVar("RSUGGESTS") or "")
+ debian_cmp_remap(rsuggests)
+ # Deliberately drop version information here, not wanted/supported by deb
+ rprovides = dict.fromkeys(bb.utils.explode_dep_versions2(localdata.getVar("RPROVIDES") or ""), [])
+ # Remove file paths if any from rprovides, debian does not support custom providers
+ for key in list(rprovides.keys()):
+ if key.startswith('/'):
+ del rprovides[key]
+ rprovides = collections.OrderedDict(sorted(rprovides.items(), key=lambda x: x[0]))
+ debian_cmp_remap(rprovides)
+ rreplaces = bb.utils.explode_dep_versions2(localdata.getVar("RREPLACES") or "")
+ debian_cmp_remap(rreplaces)
+ rconflicts = bb.utils.explode_dep_versions2(localdata.getVar("RCONFLICTS") or "")
+ debian_cmp_remap(rconflicts)
+ if rdepends:
+ ctrlfile.write("Depends: %s\n" % bb.utils.join_deps(rdepends))
+ if rsuggests:
+ ctrlfile.write("Suggests: %s\n" % bb.utils.join_deps(rsuggests))
+ if rrecommends:
+ ctrlfile.write("Recommends: %s\n" % bb.utils.join_deps(rrecommends))
+ if rprovides:
+ ctrlfile.write("Provides: %s\n" % bb.utils.join_deps(rprovides))
+ if rreplaces:
+ ctrlfile.write("Replaces: %s\n" % bb.utils.join_deps(rreplaces))
+ if rconflicts:
+ ctrlfile.write("Conflicts: %s\n" % bb.utils.join_deps(rconflicts))
+ ctrlfile.close()
+
+ for script in ["preinst", "postinst", "prerm", "postrm"]:
+ scriptvar = localdata.getVar('pkg_%s' % script)
+ if not scriptvar:
+ continue
+ scriptvar = scriptvar.strip()
+ scriptfile = open(os.path.join(controldir, script), 'w')
+
+ if scriptvar.startswith("#!"):
+ pos = scriptvar.find("\n") + 1
+ scriptfile.write(scriptvar[:pos])
+ else:
+ pos = 0
+ scriptfile.write("#!/bin/sh\n")
+
+ # Prevent the prerm/postrm scripts from being run during an upgrade
+ if script in ('prerm', 'postrm'):
+ scriptfile.write('[ "$1" != "upgrade" ] || exit 0\n')
+
+ scriptfile.write(scriptvar[pos:])
+ scriptfile.write('\n')
+ scriptfile.close()
+ os.chmod(os.path.join(controldir, script), 0o755)
+
+ conffiles_str = ' '.join(get_conffiles(pkg, d))
+ if conffiles_str:
+ conffiles = open(os.path.join(controldir, 'conffiles'), 'w')
+ for f in conffiles_str.split():
+ if os.path.exists(oe.path.join(root, f)):
+ conffiles.write('%s\n' % f)
+ conffiles.close()
+
+ os.chdir(basedir)
+ subprocess.check_output("PATH=\"%s\" %s -b %s %s" % (localdata.getVar("PATH"), localdata.getVar("DPKG_BUILDCMD"),
+ root, pkgoutdir),
+ stderr=subprocess.STDOUT,
+ shell=True)
+
+ finally:
+ cleanupcontrol(root)
+ bb.utils.unlockfile(lf)
+
+# Otherwise allarch packages may change depending on override configuration
+deb_write_pkg[vardepsexclude] = "OVERRIDES"
+
+# Have to list any variables referenced as X_<pkg> that aren't in pkgdata here
+DEBEXTRAVARS = "PKGV PKGR PKGV DESCRIPTION SECTION PRIORITY MAINTAINER DPKG_ARCH PN HOMEPAGE PACKAGE_ADD_METADATA_DEB"
+do_package_write_deb[vardeps] += "${@gen_packagevar(d, 'DEBEXTRAVARS')}"
+
+SSTATETASKS += "do_package_write_deb"
+do_package_write_deb[sstate-inputdirs] = "${PKGWRITEDIRDEB}"
+do_package_write_deb[sstate-outputdirs] = "${DEPLOY_DIR_DEB}"
+
+python do_package_write_deb_setscene () {
+ tmpdir = d.getVar('TMPDIR')
+
+ if os.access(os.path.join(tmpdir, "stamps", "DEB_PACKAGE_INDEX_CLEAN"),os.R_OK):
+ os.unlink(os.path.join(tmpdir, "stamps", "DEB_PACKAGE_INDEX_CLEAN"))
+
+ sstate_setscene(d)
+}
+addtask do_package_write_deb_setscene
+
+python () {
+ if d.getVar('PACKAGES') != '':
+ deps = ' dpkg-native:do_populate_sysroot virtual/fakeroot-native:do_populate_sysroot'
+ d.appendVarFlag('do_package_write_deb', 'depends', deps)
+ d.setVarFlag('do_package_write_deb', 'fakeroot', "1")
+}
+
+python do_package_write_deb () {
+ bb.build.exec_func("read_subpackage_metadata", d)
+ bb.build.exec_func("do_package_deb", d)
+}
+do_package_write_deb[dirs] = "${PKGWRITEDIRDEB}"
+do_package_write_deb[cleandirs] = "${PKGWRITEDIRDEB}"
+do_package_write_deb[depends] += "${@oe.utils.build_depends_string(d.getVar('PACKAGE_WRITE_DEPS'), 'do_populate_sysroot')}"
+addtask package_write_deb after do_packagedata do_package do_deploy_source_date_epoch before do_build
+do_build[rdeptask] += "do_package_write_deb"
+
+PACKAGEINDEXDEPS += "dpkg-native:do_populate_sysroot"
+PACKAGEINDEXDEPS += "apt-native:do_populate_sysroot"
diff --git a/poky/meta/classes-global/package_ipk.bbclass b/poky/meta/classes-global/package_ipk.bbclass
new file mode 100644
index 0000000..c43592a
--- /dev/null
+++ b/poky/meta/classes-global/package_ipk.bbclass
@@ -0,0 +1,292 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+inherit package
+
+IMAGE_PKGTYPE ?= "ipk"
+
+IPKGCONF_TARGET = "${WORKDIR}/opkg.conf"
+IPKGCONF_SDK = "${WORKDIR}/opkg-sdk.conf"
+IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
+
+PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
+
+# Program to be used to build opkg packages
+OPKGBUILDCMD ??= 'opkg-build -Z xz -a "${XZ_DEFAULTS}"'
+
+OPKG_ARGS += "--force_postinstall --prefer-arch-to-version"
+OPKG_ARGS += "${@['', '--no-install-recommends'][d.getVar("NO_RECOMMENDATIONS") == "1"]}"
+OPKG_ARGS += "${@['', '--add-exclude ' + ' --add-exclude '.join((d.getVar('PACKAGE_EXCLUDE') or "").split())][(d.getVar("PACKAGE_EXCLUDE") or "").strip() != ""]}"
+
+OPKGLIBDIR ??= "${localstatedir}/lib"
+
+python do_package_ipk () {
+ workdir = d.getVar('WORKDIR')
+ outdir = d.getVar('PKGWRITEDIRIPK')
+ tmpdir = d.getVar('TMPDIR')
+ pkgdest = d.getVar('PKGDEST')
+ if not workdir or not outdir or not tmpdir:
+ bb.error("Variables incorrectly set, unable to package")
+ return
+
+ packages = d.getVar('PACKAGES')
+ if not packages or packages == '':
+ bb.debug(1, "No packages; nothing to do")
+ return
+
+ # We're about to add new packages so the index needs to be checked
+ # so remove the appropriate stamp file.
+ if os.access(os.path.join(tmpdir, "stamps", "IPK_PACKAGE_INDEX_CLEAN"), os.R_OK):
+ os.unlink(os.path.join(tmpdir, "stamps", "IPK_PACKAGE_INDEX_CLEAN"))
+
+ oe.utils.multiprocess_launch(ipk_write_pkg, packages.split(), d, extraargs=(d,))
+}
+do_package_ipk[vardeps] += "ipk_write_pkg"
+do_package_ipk[vardepsexclude] = "BB_NUMBER_THREADS"
+
+def ipk_write_pkg(pkg, d):
+ import re, copy
+ import subprocess
+ import textwrap
+ import collections
+ import glob
+
+ def cleanupcontrol(root):
+ for p in ['CONTROL', 'DEBIAN']:
+ p = os.path.join(root, p)
+ if os.path.exists(p):
+ bb.utils.prunedir(p)
+
+ outdir = d.getVar('PKGWRITEDIRIPK')
+ pkgdest = d.getVar('PKGDEST')
+ recipesource = os.path.basename(d.getVar('FILE'))
+
+ localdata = bb.data.createCopy(d)
+ root = "%s/%s" % (pkgdest, pkg)
+
+ lf = bb.utils.lockfile(root + ".lock")
+ try:
+ localdata.setVar('ROOT', '')
+ localdata.setVar('ROOT_%s' % pkg, root)
+ pkgname = localdata.getVar('PKG:%s' % pkg)
+ if not pkgname:
+ pkgname = pkg
+ localdata.setVar('PKG', pkgname)
+
+ localdata.setVar('OVERRIDES', d.getVar("OVERRIDES", False) + ":" + pkg)
+
+ basedir = os.path.join(os.path.dirname(root))
+ arch = localdata.getVar('PACKAGE_ARCH')
+
+ if localdata.getVar('IPK_HIERARCHICAL_FEED', False) == "1":
+ # Spread packages across subdirectories so each isn't too crowded
+ if pkgname.startswith('lib'):
+ pkg_prefix = 'lib' + pkgname[3]
+ else:
+ pkg_prefix = pkgname[0]
+
+ # Keep -dbg, -dev, -doc, -staticdev, -locale and -locale-* packages
+ # together. These package suffixes are taken from the definitions of
+ # PACKAGES and PACKAGES_DYNAMIC in meta/conf/bitbake.conf
+ if pkgname[-4:] in ('-dbg', '-dev', '-doc'):
+ pkg_subdir = pkgname[:-4]
+ elif pkgname.endswith('-staticdev'):
+ pkg_subdir = pkgname[:-10]
+ elif pkgname.endswith('-locale'):
+ pkg_subdir = pkgname[:-7]
+ elif '-locale-' in pkgname:
+ pkg_subdir = pkgname[:pkgname.find('-locale-')]
+ else:
+ pkg_subdir = pkgname
+
+ pkgoutdir = "%s/%s/%s/%s" % (outdir, arch, pkg_prefix, pkg_subdir)
+ else:
+ pkgoutdir = "%s/%s" % (outdir, arch)
+
+ bb.utils.mkdirhier(pkgoutdir)
+ os.chdir(root)
+ cleanupcontrol(root)
+ g = glob.glob('*')
+ if not g and localdata.getVar('ALLOW_EMPTY', False) != "1":
+ bb.note("Not creating empty archive for %s-%s-%s" % (pkg, localdata.getVar('PKGV'), localdata.getVar('PKGR')))
+ return
+
+ controldir = os.path.join(root, 'CONTROL')
+ bb.utils.mkdirhier(controldir)
+ ctrlfile = open(os.path.join(controldir, 'control'), 'w')
+
+ fields = []
+ pe = d.getVar('PKGE')
+ if pe and int(pe) > 0:
+ fields.append(["Version: %s:%s-%s\n", ['PKGE', 'PKGV', 'PKGR']])
+ else:
+ fields.append(["Version: %s-%s\n", ['PKGV', 'PKGR']])
+ fields.append(["Description: %s\n", ['DESCRIPTION']])
+ fields.append(["Section: %s\n", ['SECTION']])
+ fields.append(["Priority: %s\n", ['PRIORITY']])
+ fields.append(["Maintainer: %s\n", ['MAINTAINER']])
+ fields.append(["License: %s\n", ['LICENSE']])
+ fields.append(["Architecture: %s\n", ['PACKAGE_ARCH']])
+ fields.append(["OE: %s\n", ['PN']])
+ if d.getVar('HOMEPAGE'):
+ fields.append(["Homepage: %s\n", ['HOMEPAGE']])
+
+ def pullData(l, d):
+ l2 = []
+ for i in l:
+ l2.append(d.getVar(i))
+ return l2
+
+ ctrlfile.write("Package: %s\n" % pkgname)
+ # check for required fields
+ for (c, fs) in fields:
+ for f in fs:
+ if localdata.getVar(f, False) is None:
+ raise KeyError(f)
+ # Special behavior for description...
+ if 'DESCRIPTION' in fs:
+ summary = localdata.getVar('SUMMARY') or localdata.getVar('DESCRIPTION') or "."
+ ctrlfile.write('Description: %s\n' % summary)
+ description = localdata.getVar('DESCRIPTION') or "."
+ description = textwrap.dedent(description).strip()
+ if '\\n' in description:
+ # Manually indent: multiline description includes a leading space
+ for t in description.split('\\n'):
+ ctrlfile.write(' %s\n' % (t.strip() or ' .'))
+ else:
+ # Auto indent
+ ctrlfile.write('%s\n' % textwrap.fill(description, width=74, initial_indent=' ', subsequent_indent=' '))
+ else:
+ ctrlfile.write(c % tuple(pullData(fs, localdata)))
+
+ custom_fields_chunk = get_package_additional_metadata("ipk", localdata)
+ if custom_fields_chunk is not None:
+ ctrlfile.write(custom_fields_chunk)
+ ctrlfile.write("\n")
+
+ mapping_rename_hook(localdata)
+
+ def debian_cmp_remap(var):
+ # In debian '>' and '<' do not mean what it appears they mean
+ # '<' = less or equal
+ # '>' = greater or equal
+ # adjust these to the '<<' and '>>' equivalents
+ # Also, "=" specifiers only work if they have the PR in, so 1.2.3 != 1.2.3-r0
+ # so to avoid issues, map this to ">= 1.2.3 << 1.2.3.0"
+ for dep in var:
+ for i, v in enumerate(var[dep]):
+ if (v or "").startswith("< "):
+ var[dep][i] = var[dep][i].replace("< ", "<< ")
+ elif (v or "").startswith("> "):
+ var[dep][i] = var[dep][i].replace("> ", ">> ")
+ elif (v or "").startswith("= ") and "-r" not in v:
+ ver = var[dep][i].replace("= ", "")
+ var[dep][i] = var[dep][i].replace("= ", ">= ")
+ var[dep].append("<< " + ver + ".0")
+
+ rdepends = bb.utils.explode_dep_versions2(localdata.getVar("RDEPENDS") or "")
+ debian_cmp_remap(rdepends)
+ rrecommends = bb.utils.explode_dep_versions2(localdata.getVar("RRECOMMENDS") or "")
+ debian_cmp_remap(rrecommends)
+ rsuggests = bb.utils.explode_dep_versions2(localdata.getVar("RSUGGESTS") or "")
+ debian_cmp_remap(rsuggests)
+ # Deliberately drop version information here, not wanted/supported by ipk
+ rprovides = dict.fromkeys(bb.utils.explode_dep_versions2(localdata.getVar("RPROVIDES") or ""), [])
+ rprovides = collections.OrderedDict(sorted(rprovides.items(), key=lambda x: x[0]))
+ debian_cmp_remap(rprovides)
+ rreplaces = bb.utils.explode_dep_versions2(localdata.getVar("RREPLACES") or "")
+ debian_cmp_remap(rreplaces)
+ rconflicts = bb.utils.explode_dep_versions2(localdata.getVar("RCONFLICTS") or "")
+ debian_cmp_remap(rconflicts)
+
+ if rdepends:
+ ctrlfile.write("Depends: %s\n" % bb.utils.join_deps(rdepends))
+ if rsuggests:
+ ctrlfile.write("Suggests: %s\n" % bb.utils.join_deps(rsuggests))
+ if rrecommends:
+ ctrlfile.write("Recommends: %s\n" % bb.utils.join_deps(rrecommends))
+ if rprovides:
+ ctrlfile.write("Provides: %s\n" % bb.utils.join_deps(rprovides))
+ if rreplaces:
+ ctrlfile.write("Replaces: %s\n" % bb.utils.join_deps(rreplaces))
+ if rconflicts:
+ ctrlfile.write("Conflicts: %s\n" % bb.utils.join_deps(rconflicts))
+ ctrlfile.write("Source: %s\n" % recipesource)
+ ctrlfile.close()
+
+ for script in ["preinst", "postinst", "prerm", "postrm"]:
+ scriptvar = localdata.getVar('pkg_%s' % script)
+ if not scriptvar:
+ continue
+ scriptfile = open(os.path.join(controldir, script), 'w')
+ scriptfile.write(scriptvar)
+ scriptfile.close()
+ os.chmod(os.path.join(controldir, script), 0o755)
+
+ conffiles_str = ' '.join(get_conffiles(pkg, d))
+ if conffiles_str:
+ conffiles = open(os.path.join(controldir, 'conffiles'), 'w')
+ for f in conffiles_str.split():
+ if os.path.exists(oe.path.join(root, f)):
+ conffiles.write('%s\n' % f)
+ conffiles.close()
+
+ os.chdir(basedir)
+ subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
+ d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir),
+ stderr=subprocess.STDOUT,
+ shell=True)
+
+ if d.getVar('IPK_SIGN_PACKAGES') == '1':
+ ipkver = "%s-%s" % (localdata.getVar('PKGV'), localdata.getVar('PKGR'))
+ ipk_to_sign = "%s/%s_%s_%s.ipk" % (pkgoutdir, pkgname, ipkver, localdata.getVar('PACKAGE_ARCH'))
+ sign_ipk(d, ipk_to_sign)
+
+ finally:
+ cleanupcontrol(root)
+ bb.utils.unlockfile(lf)
+
+# Have to list any variables referenced as X_<pkg> that aren't in pkgdata here
+IPKEXTRAVARS = "PRIORITY MAINTAINER PACKAGE_ARCH HOMEPAGE PACKAGE_ADD_METADATA_IPK"
+ipk_write_pkg[vardeps] += "${@gen_packagevar(d, 'IPKEXTRAVARS')}"
+
+# Otherwise allarch packages may change depending on override configuration
+ipk_write_pkg[vardepsexclude] = "OVERRIDES"
+
+
+SSTATETASKS += "do_package_write_ipk"
+do_package_write_ipk[sstate-inputdirs] = "${PKGWRITEDIRIPK}"
+do_package_write_ipk[sstate-outputdirs] = "${DEPLOY_DIR_IPK}"
+
+python do_package_write_ipk_setscene () {
+ tmpdir = d.getVar('TMPDIR')
+
+ if os.access(os.path.join(tmpdir, "stamps", "IPK_PACKAGE_INDEX_CLEAN"), os.R_OK):
+ os.unlink(os.path.join(tmpdir, "stamps", "IPK_PACKAGE_INDEX_CLEAN"))
+
+ sstate_setscene(d)
+}
+addtask do_package_write_ipk_setscene
+
+python () {
+ if d.getVar('PACKAGES') != '':
+ deps = ' opkg-utils-native:do_populate_sysroot virtual/fakeroot-native:do_populate_sysroot xz-native:do_populate_sysroot'
+ d.appendVarFlag('do_package_write_ipk', 'depends', deps)
+ d.setVarFlag('do_package_write_ipk', 'fakeroot', "1")
+}
+
+python do_package_write_ipk () {
+ bb.build.exec_func("read_subpackage_metadata", d)
+ bb.build.exec_func("do_package_ipk", d)
+}
+do_package_write_ipk[dirs] = "${PKGWRITEDIRIPK}"
+do_package_write_ipk[cleandirs] = "${PKGWRITEDIRIPK}"
+do_package_write_ipk[depends] += "${@oe.utils.build_depends_string(d.getVar('PACKAGE_WRITE_DEPS'), 'do_populate_sysroot')}"
+addtask package_write_ipk after do_packagedata do_package do_deploy_source_date_epoch before do_build
+do_build[rdeptask] += "do_package_write_ipk"
+
+PACKAGEINDEXDEPS += "opkg-utils-native:do_populate_sysroot"
+PACKAGEINDEXDEPS += "opkg-native:do_populate_sysroot"
diff --git a/poky/meta/classes-global/package_pkgdata.bbclass b/poky/meta/classes-global/package_pkgdata.bbclass
new file mode 100644
index 0000000..f653bd9
--- /dev/null
+++ b/poky/meta/classes-global/package_pkgdata.bbclass
@@ -0,0 +1,173 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+WORKDIR_PKGDATA = "${WORKDIR}/pkgdata-sysroot"
+
+def package_populate_pkgdata_dir(pkgdatadir, d):
+ import glob
+
+ postinsts = []
+ seendirs = set()
+ stagingdir = d.getVar("PKGDATA_DIR")
+ pkgarchs = ['${MACHINE_ARCH}']
+ pkgarchs = pkgarchs + list(reversed(d.getVar("PACKAGE_EXTRA_ARCHS").split()))
+ pkgarchs.append('allarch')
+
+ bb.utils.mkdirhier(pkgdatadir)
+ for pkgarch in pkgarchs:
+ for manifest in glob.glob(d.expand("${SSTATE_MANIFESTS}/manifest-%s-*.packagedata" % pkgarch)):
+ with open(manifest, "r") as f:
+ for l in f:
+ l = l.strip()
+ dest = l.replace(stagingdir, "")
+ if l.endswith("/"):
+ staging_copydir(l, pkgdatadir, dest, seendirs)
+ continue
+ try:
+ staging_copyfile(l, pkgdatadir, dest, postinsts, seendirs)
+ except FileExistsError:
+ continue
+
+python package_prepare_pkgdata() {
+ import copy
+ import glob
+
+ taskdepdata = d.getVar("BB_TASKDEPDATA", False)
+ mytaskname = d.getVar("BB_RUNTASK")
+ if mytaskname.endswith("_setscene"):
+ mytaskname = mytaskname.replace("_setscene", "")
+ workdir = d.getVar("WORKDIR")
+ pn = d.getVar("PN")
+ stagingdir = d.getVar("PKGDATA_DIR")
+ pkgdatadir = d.getVar("WORKDIR_PKGDATA")
+
+ # Detect bitbake -b usage
+ nodeps = d.getVar("BB_LIMITEDDEPS") or False
+ if nodeps:
+ staging_package_populate_pkgdata_dir(pkgdatadir, d)
+ return
+
+ start = None
+ configuredeps = []
+ for dep in taskdepdata:
+ data = taskdepdata[dep]
+ if data[1] == mytaskname and data[0] == pn:
+ start = dep
+ break
+ if start is None:
+ bb.fatal("Couldn't find ourself in BB_TASKDEPDATA?")
+
+ # We need to figure out which sysroot files we need to expose to this task.
+ # This needs to match what would get restored from sstate, which is controlled
+ # ultimately by calls from bitbake to setscene_depvalid().
+ # That function expects a setscene dependency tree. We build a dependency tree
+ # condensed to inter-sstate task dependencies, similar to that used by setscene
+ # tasks. We can then call into setscene_depvalid() and decide
+ # which dependencies we can "see" and should expose in the recipe specific sysroot.
+ setscenedeps = copy.deepcopy(taskdepdata)
+
+ start = set([start])
+
+ sstatetasks = d.getVar("SSTATETASKS").split()
+ # Add recipe specific tasks referenced by setscene_depvalid()
+ sstatetasks.append("do_stash_locale")
+
+ # If start is an sstate task (like do_package) we need to add in its direct dependencies
+ # else the code below won't recurse into them.
+ for dep in set(start):
+ for dep2 in setscenedeps[dep][3]:
+ start.add(dep2)
+ start.remove(dep)
+
+ # Create collapsed do_populate_sysroot -> do_populate_sysroot tree
+ for dep in taskdepdata:
+ data = setscenedeps[dep]
+ if data[1] not in sstatetasks:
+ for dep2 in setscenedeps:
+ data2 = setscenedeps[dep2]
+ if dep in data2[3]:
+ data2[3].update(setscenedeps[dep][3])
+ data2[3].remove(dep)
+ if dep in start:
+ start.update(setscenedeps[dep][3])
+ start.remove(dep)
+ del setscenedeps[dep]
+
+ # Remove circular references
+ for dep in setscenedeps:
+ if dep in setscenedeps[dep][3]:
+ setscenedeps[dep][3].remove(dep)
+
+ # Direct dependencies should be present and can be depended upon
+ for dep in set(start):
+ if setscenedeps[dep][1] == "do_packagedata":
+ if dep not in configuredeps:
+ configuredeps.append(dep)
+
+ msgbuf = []
+ # Call into setscene_depvalid for each sub-dependency and only copy sysroot files
+ # for ones that would be restored from sstate.
+ done = list(start)
+ next = list(start)
+ while next:
+ new = []
+ for dep in next:
+ data = setscenedeps[dep]
+ for datadep in data[3]:
+ if datadep in done:
+ continue
+ taskdeps = {}
+ taskdeps[dep] = setscenedeps[dep][:2]
+ taskdeps[datadep] = setscenedeps[datadep][:2]
+ retval = setscene_depvalid(datadep, taskdeps, [], d, msgbuf)
+ done.append(datadep)
+ new.append(datadep)
+ if retval:
+ msgbuf.append("Skipping setscene dependency %s" % datadep)
+ continue
+ if datadep not in configuredeps and setscenedeps[datadep][1] == "do_packagedata":
+ configuredeps.append(datadep)
+ msgbuf.append("Adding dependency on %s" % setscenedeps[datadep][0])
+ else:
+ msgbuf.append("Following dependency on %s" % setscenedeps[datadep][0])
+ next = new
+
+ # This logging is too verbose for day to day use sadly
+ #bb.debug(2, "\n".join(msgbuf))
+
+ seendirs = set()
+ postinsts = []
+ multilibs = {}
+ manifests = {}
+
+ msg_adding = []
+
+ for dep in configuredeps:
+ c = setscenedeps[dep][0]
+ msg_adding.append(c)
+
+ manifest, d2 = oe.sstatesig.find_sstate_manifest(c, setscenedeps[dep][2], "packagedata", d, multilibs)
+ destsysroot = pkgdatadir
+
+ if manifest:
+ targetdir = destsysroot
+ with open(manifest, "r") as f:
+ manifests[dep] = manifest
+ for l in f:
+ l = l.strip()
+ dest = targetdir + l.replace(stagingdir, "")
+ if l.endswith("/"):
+ staging_copydir(l, targetdir, dest, seendirs)
+ continue
+ staging_copyfile(l, targetdir, dest, postinsts, seendirs)
+
+ bb.note("Installed into pkgdata-sysroot: %s" % str(msg_adding))
+
+}
+package_prepare_pkgdata[cleandirs] = "${WORKDIR_PKGDATA}"
+package_prepare_pkgdata[vardepsexclude] += "MACHINE_ARCH PACKAGE_EXTRA_ARCHS SDK_ARCH BUILD_ARCH SDK_OS BB_TASKDEPDATA SSTATETASKS"
+
+
diff --git a/poky/meta/classes-global/package_rpm.bbclass b/poky/meta/classes-global/package_rpm.bbclass
new file mode 100644
index 0000000..81a2060
--- /dev/null
+++ b/poky/meta/classes-global/package_rpm.bbclass
@@ -0,0 +1,755 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+inherit package
+
+IMAGE_PKGTYPE ?= "rpm"
+
+RPM="rpm"
+RPMBUILD="rpmbuild"
+
+PKGWRITEDIRRPM = "${WORKDIR}/deploy-rpms"
+
+# Maintaining the perfile dependencies has singificant overhead when writing the
+# packages. When set, this value merges them for efficiency.
+MERGEPERFILEDEPS = "1"
+
+# Filter dependencies based on a provided function.
+def filter_deps(var, f):
+ import collections
+
+ depends_dict = bb.utils.explode_dep_versions2(var)
+ newdeps_dict = collections.OrderedDict()
+ for dep in depends_dict:
+ if f(dep):
+ newdeps_dict[dep] = depends_dict[dep]
+ return bb.utils.join_deps(newdeps_dict, commasep=False)
+
+# Filter out absolute paths (typically /bin/sh and /usr/bin/env) and any perl
+# dependencies for nativesdk packages.
+def filter_nativesdk_deps(srcname, var):
+ if var and srcname.startswith("nativesdk-"):
+ var = filter_deps(var, lambda dep: not dep.startswith('/') and dep != 'perl' and not dep.startswith('perl('))
+ return var
+
+# Construct per file dependencies file
+def write_rpm_perfiledata(srcname, d):
+ workdir = d.getVar('WORKDIR')
+ packages = d.getVar('PACKAGES')
+ pkgd = d.getVar('PKGD')
+
+ def dump_filerdeps(varname, outfile, d):
+ outfile.write("#!/usr/bin/env python3\n\n")
+ outfile.write("# Dependency table\n")
+ outfile.write('deps = {\n')
+ for pkg in packages.split():
+ dependsflist_key = 'FILE' + varname + 'FLIST' + ":" + pkg
+ dependsflist = (d.getVar(dependsflist_key) or "")
+ for dfile in dependsflist.split():
+ key = "FILE" + varname + ":" + dfile + ":" + pkg
+ deps = filter_nativesdk_deps(srcname, d.getVar(key) or "")
+ depends_dict = bb.utils.explode_dep_versions(deps)
+ file = dfile.replace("@underscore@", "_")
+ file = file.replace("@closebrace@", "]")
+ file = file.replace("@openbrace@", "[")
+ file = file.replace("@tab@", "\t")
+ file = file.replace("@space@", " ")
+ file = file.replace("@at@", "@")
+ outfile.write('"' + pkgd + file + '" : "')
+ for dep in depends_dict:
+ ver = depends_dict[dep]
+ if dep and ver:
+ ver = ver.replace("(","")
+ ver = ver.replace(")","")
+ outfile.write(dep + " " + ver + " ")
+ else:
+ outfile.write(dep + " ")
+ outfile.write('",\n')
+ outfile.write('}\n\n')
+ outfile.write("import sys\n")
+ outfile.write("while 1:\n")
+ outfile.write("\tline = sys.stdin.readline().strip()\n")
+ outfile.write("\tif not line:\n")
+ outfile.write("\t\tsys.exit(0)\n")
+ outfile.write("\tif line in deps:\n")
+ outfile.write("\t\tprint(deps[line] + '\\n')\n")
+
+ # OE-core dependencies a.k.a. RPM requires
+ outdepends = workdir + "/" + srcname + ".requires"
+
+ dependsfile = open(outdepends, 'w')
+
+ dump_filerdeps('RDEPENDS', dependsfile, d)
+
+ dependsfile.close()
+ os.chmod(outdepends, 0o755)
+
+ # OE-core / RPM Provides
+ outprovides = workdir + "/" + srcname + ".provides"
+
+ providesfile = open(outprovides, 'w')
+
+ dump_filerdeps('RPROVIDES', providesfile, d)
+
+ providesfile.close()
+ os.chmod(outprovides, 0o755)
+
+ return (outdepends, outprovides)
+
+
+python write_specfile () {
+ import oe.packagedata
+
+ # append information for logs and patches to %prep
+ def add_prep(d,spec_files_bottom):
+ if d.getVarFlag('ARCHIVER_MODE', 'srpm') == '1' and bb.data.inherits_class('archiver', d):
+ spec_files_bottom.append('%%prep -n %s' % d.getVar('PN') )
+ spec_files_bottom.append('%s' % "echo \"include logs and patches, Please check them in SOURCES\"")
+ spec_files_bottom.append('')
+
+ # append the name of tarball to key word 'SOURCE' in xxx.spec.
+ def tail_source(d):
+ if d.getVarFlag('ARCHIVER_MODE', 'srpm') == '1' and bb.data.inherits_class('archiver', d):
+ ar_outdir = d.getVar('ARCHIVER_OUTDIR')
+ if not os.path.exists(ar_outdir):
+ return
+ source_list = os.listdir(ar_outdir)
+ source_number = 0
+ for source in source_list:
+ # do_deploy_archives may have already run (from sstate) meaning a .src.rpm may already
+ # exist in ARCHIVER_OUTDIR so skip if present.
+ if source.endswith(".src.rpm"):
+ continue
+ # The rpmbuild doesn't need the root permission, but it needs
+ # to know the file's user and group name, the only user and
+ # group in fakeroot is "root" when working in fakeroot.
+ f = os.path.join(ar_outdir, source)
+ os.chown(f, 0, 0)
+ spec_preamble_top.append('Source%s: %s' % (source_number, source))
+ source_number += 1
+
+ # In RPM, dependencies are of the format: pkg <>= Epoch:Version-Release
+ # This format is similar to OE, however there are restrictions on the
+ # characters that can be in a field. In the Version field, "-"
+ # characters are not allowed. "-" is allowed in the Release field.
+ #
+ # We translate the "-" in the version to a "+", by loading the PKGV
+ # from the dependent recipe, replacing the - with a +, and then using
+ # that value to do a replace inside of this recipe's dependencies.
+ # This preserves the "-" separator between the version and release, as
+ # well as any "-" characters inside of the release field.
+ #
+ # All of this has to happen BEFORE the mapping_rename_hook as
+ # after renaming we cannot look up the dependencies in the packagedata
+ # store.
+ def translate_vers(varname, d):
+ depends = d.getVar(varname)
+ if depends:
+ depends_dict = bb.utils.explode_dep_versions2(depends)
+ newdeps_dict = {}
+ for dep in depends_dict:
+ verlist = []
+ for ver in depends_dict[dep]:
+ if '-' in ver:
+ subd = oe.packagedata.read_subpkgdata_dict(dep, d)
+ if 'PKGV' in subd:
+ pv = subd['PV']
+ pkgv = subd['PKGV']
+ reppv = pkgv.replace('-', '+')
+ ver = ver.replace(pv, reppv).replace(pkgv, reppv)
+ if 'PKGR' in subd:
+ # Make sure PKGR rather than PR in ver
+ pr = '-' + subd['PR']
+ pkgr = '-' + subd['PKGR']
+ if pkgr not in ver:
+ ver = ver.replace(pr, pkgr)
+ verlist.append(ver)
+ else:
+ verlist.append(ver)
+ newdeps_dict[dep] = verlist
+ depends = bb.utils.join_deps(newdeps_dict)
+ d.setVar(varname, depends.strip())
+
+ # We need to change the style the dependency from BB to RPM
+ # This needs to happen AFTER the mapping_rename_hook
+ def print_deps(variable, tag, array, d):
+ depends = variable
+ if depends:
+ depends_dict = bb.utils.explode_dep_versions2(depends)
+ for dep in depends_dict:
+ for ver in depends_dict[dep]:
+ ver = ver.replace('(', '')
+ ver = ver.replace(')', '')
+ array.append("%s: %s %s" % (tag, dep, ver))
+ if not len(depends_dict[dep]):
+ array.append("%s: %s" % (tag, dep))
+
+ def walk_files(walkpath, target, conffiles, dirfiles):
+ # We can race against the ipk/deb backends which create CONTROL or DEBIAN directories
+ # when packaging. We just ignore these files which are created in
+ # packages-split/ and not package/
+ # We have the odd situation where the CONTROL/DEBIAN directory can be removed in the middle of
+ # of the walk, the isdir() test would then fail and the walk code would assume its a file
+ # hence we check for the names in files too.
+ for rootpath, dirs, files in os.walk(walkpath):
+ path = rootpath.replace(walkpath, "")
+ if path.endswith("DEBIAN") or path.endswith("CONTROL"):
+ continue
+ path = path.replace("%", "%%%%%%%%")
+
+ # Treat all symlinks to directories as normal files.
+ # os.walk() lists them as directories.
+ def move_to_files(dir):
+ if os.path.islink(os.path.join(rootpath, dir)):
+ files.append(dir)
+ return True
+ else:
+ return False
+ dirs[:] = [dir for dir in dirs if not move_to_files(dir)]
+
+ # Directory handling can happen in two ways, either DIRFILES is not set at all
+ # in which case we fall back to the older behaviour of packages owning all their
+ # directories
+ if dirfiles is None:
+ for dir in dirs:
+ if dir == "CONTROL" or dir == "DEBIAN":
+ continue
+ dir = dir.replace("%", "%%%%%%%%")
+ # All packages own the directories their files are in...
+ target.append('%dir "' + path + '/' + dir + '"')
+ else:
+ # packages own only empty directories or explict directory.
+ # This will prevent the overlapping of security permission.
+ if path and not files and not dirs:
+ target.append('%dir "' + path + '"')
+ elif path and path in dirfiles:
+ target.append('%dir "' + path + '"')
+
+ for file in files:
+ if file == "CONTROL" or file == "DEBIAN":
+ continue
+ file = file.replace("%", "%%%%%%%%")
+ if conffiles.count(path + '/' + file):
+ target.append('%config "' + path + '/' + file + '"')
+ else:
+ target.append('"' + path + '/' + file + '"')
+
+ # Prevent the prerm/postrm scripts from being run during an upgrade
+ def wrap_uninstall(scriptvar):
+ scr = scriptvar.strip()
+ if scr.startswith("#!"):
+ pos = scr.find("\n") + 1
+ else:
+ pos = 0
+ scr = scr[:pos] + 'if [ "$1" = "0" ] ; then\n' + scr[pos:] + '\nfi'
+ return scr
+
+ def get_perfile(varname, pkg, d):
+ deps = []
+ dependsflist_key = 'FILE' + varname + 'FLIST' + ":" + pkg
+ dependsflist = (d.getVar(dependsflist_key) or "")
+ for dfile in dependsflist.split():
+ key = "FILE" + varname + ":" + dfile + ":" + pkg
+ depends = d.getVar(key)
+ if depends:
+ deps.append(depends)
+ return " ".join(deps)
+
+ def append_description(spec_preamble, text):
+ """
+ Add the description to the spec file.
+ """
+ import textwrap
+ dedent_text = textwrap.dedent(text).strip()
+ # Bitbake saves "\n" as "\\n"
+ if '\\n' in dedent_text:
+ for t in dedent_text.split('\\n'):
+ spec_preamble.append(t.strip())
+ else:
+ spec_preamble.append('%s' % textwrap.fill(dedent_text, width=75))
+
+ packages = d.getVar('PACKAGES')
+ if not packages or packages == '':
+ bb.debug(1, "No packages; nothing to do")
+ return
+
+ pkgdest = d.getVar('PKGDEST')
+ if not pkgdest:
+ bb.fatal("No PKGDEST")
+
+ outspecfile = d.getVar('OUTSPECFILE')
+ if not outspecfile:
+ bb.fatal("No OUTSPECFILE")
+
+ # Construct the SPEC file...
+ srcname = d.getVar('PN')
+ localdata = bb.data.createCopy(d)
+ localdata.setVar('OVERRIDES', d.getVar("OVERRIDES", False) + ":" + srcname)
+ srcsummary = (localdata.getVar('SUMMARY') or localdata.getVar('DESCRIPTION') or ".")
+ srcversion = localdata.getVar('PKGV').replace('-', '+')
+ srcrelease = localdata.getVar('PKGR')
+ srcepoch = (localdata.getVar('PKGE') or "")
+ srclicense = localdata.getVar('LICENSE')
+ srcsection = localdata.getVar('SECTION')
+ srcmaintainer = localdata.getVar('MAINTAINER')
+ srchomepage = localdata.getVar('HOMEPAGE')
+ srcdescription = localdata.getVar('DESCRIPTION') or "."
+ srccustomtagschunk = get_package_additional_metadata("rpm", localdata)
+
+ srcdepends = d.getVar('DEPENDS')
+ srcrdepends = ""
+ srcrrecommends = ""
+ srcrsuggests = ""
+ srcrprovides = ""
+ srcrreplaces = ""
+ srcrconflicts = ""
+ srcrobsoletes = ""
+
+ srcrpreinst = []
+ srcrpostinst = []
+ srcrprerm = []
+ srcrpostrm = []
+
+ spec_preamble_top = []
+ spec_preamble_bottom = []
+
+ spec_scriptlets_top = []
+ spec_scriptlets_bottom = []
+
+ spec_files_top = []
+ spec_files_bottom = []
+
+ perfiledeps = (d.getVar("MERGEPERFILEDEPS") or "0") == "0"
+ extra_pkgdata = (d.getVar("RPM_EXTRA_PKGDATA") or "0") == "1"
+
+ for pkg in packages.split():
+ localdata = bb.data.createCopy(d)
+
+ root = "%s/%s" % (pkgdest, pkg)
+
+ localdata.setVar('ROOT', '')
+ localdata.setVar('ROOT_%s' % pkg, root)
+ pkgname = localdata.getVar('PKG:%s' % pkg)
+ if not pkgname:
+ pkgname = pkg
+ localdata.setVar('PKG', pkgname)
+
+ localdata.setVar('OVERRIDES', d.getVar("OVERRIDES", False) + ":" + pkg)
+
+ conffiles = get_conffiles(pkg, d)
+ dirfiles = localdata.getVar('DIRFILES')
+ if dirfiles is not None:
+ dirfiles = dirfiles.split()
+
+ splitname = pkgname
+
+ splitsummary = (localdata.getVar('SUMMARY') or localdata.getVar('DESCRIPTION') or ".")
+ splitversion = (localdata.getVar('PKGV') or "").replace('-', '+')
+ splitrelease = (localdata.getVar('PKGR') or "")
+ splitepoch = (localdata.getVar('PKGE') or "")
+ splitlicense = (localdata.getVar('LICENSE') or "")
+ splitsection = (localdata.getVar('SECTION') or "")
+ splitdescription = (localdata.getVar('DESCRIPTION') or ".")
+ splitcustomtagschunk = get_package_additional_metadata("rpm", localdata)
+
+ translate_vers('RDEPENDS', localdata)
+ translate_vers('RRECOMMENDS', localdata)
+ translate_vers('RSUGGESTS', localdata)
+ translate_vers('RPROVIDES', localdata)
+ translate_vers('RREPLACES', localdata)
+ translate_vers('RCONFLICTS', localdata)
+
+ # Map the dependencies into their final form
+ mapping_rename_hook(localdata)
+
+ splitrdepends = localdata.getVar('RDEPENDS') or ""
+ splitrrecommends = localdata.getVar('RRECOMMENDS') or ""
+ splitrsuggests = localdata.getVar('RSUGGESTS') or ""
+ splitrprovides = localdata.getVar('RPROVIDES') or ""
+ splitrreplaces = localdata.getVar('RREPLACES') or ""
+ splitrconflicts = localdata.getVar('RCONFLICTS') or ""
+ splitrobsoletes = ""
+
+ splitrpreinst = localdata.getVar('pkg_preinst')
+ splitrpostinst = localdata.getVar('pkg_postinst')
+ splitrprerm = localdata.getVar('pkg_prerm')
+ splitrpostrm = localdata.getVar('pkg_postrm')
+
+
+ if not perfiledeps:
+ # Add in summary of per file dependencies
+ splitrdepends = splitrdepends + " " + get_perfile('RDEPENDS', pkg, d)
+ splitrprovides = splitrprovides + " " + get_perfile('RPROVIDES', pkg, d)
+
+ splitrdepends = filter_nativesdk_deps(srcname, splitrdepends)
+
+ # Gather special src/first package data
+ if srcname == splitname:
+ archiving = d.getVarFlag('ARCHIVER_MODE', 'srpm') == '1' and \
+ bb.data.inherits_class('archiver', d)
+ if archiving and srclicense != splitlicense:
+ bb.warn("The SRPM produced may not have the correct overall source license in the License tag. This is due to the LICENSE for the primary package and SRPM conflicting.")
+
+ srclicense = splitlicense
+ srcrdepends = splitrdepends
+ srcrrecommends = splitrrecommends
+ srcrsuggests = splitrsuggests
+ srcrprovides = splitrprovides
+ srcrreplaces = splitrreplaces
+ srcrconflicts = splitrconflicts
+
+ srcrpreinst = splitrpreinst
+ srcrpostinst = splitrpostinst
+ srcrprerm = splitrprerm
+ srcrpostrm = splitrpostrm
+
+ file_list = []
+ walk_files(root, file_list, conffiles, dirfiles)
+ if not file_list and localdata.getVar('ALLOW_EMPTY', False) != "1":
+ bb.note("Not creating empty RPM package for %s" % splitname)
+ else:
+ spec_files_top.append('%files')
+ if extra_pkgdata:
+ package_rpm_extra_pkgdata(splitname, spec_files_top, localdata)
+ spec_files_top.append('%defattr(-,-,-,-)')
+ if file_list:
+ bb.note("Creating RPM package for %s" % splitname)
+ spec_files_top.extend(file_list)
+ else:
+ bb.note("Creating empty RPM package for %s" % splitname)
+ spec_files_top.append('')
+ continue
+
+ # Process subpackage data
+ spec_preamble_bottom.append('%%package -n %s' % splitname)
+ spec_preamble_bottom.append('Summary: %s' % splitsummary)
+ if srcversion != splitversion:
+ spec_preamble_bottom.append('Version: %s' % splitversion)
+ if srcrelease != splitrelease:
+ spec_preamble_bottom.append('Release: %s' % splitrelease)
+ if srcepoch != splitepoch:
+ spec_preamble_bottom.append('Epoch: %s' % splitepoch)
+ spec_preamble_bottom.append('License: %s' % splitlicense)
+ spec_preamble_bottom.append('Group: %s' % splitsection)
+
+ if srccustomtagschunk != splitcustomtagschunk:
+ spec_preamble_bottom.append(splitcustomtagschunk)
+
+ # Replaces == Obsoletes && Provides
+ robsoletes = bb.utils.explode_dep_versions2(splitrobsoletes)
+ rprovides = bb.utils.explode_dep_versions2(splitrprovides)
+ rreplaces = bb.utils.explode_dep_versions2(splitrreplaces)
+ for dep in rreplaces:
+ if not dep in robsoletes:
+ robsoletes[dep] = rreplaces[dep]
+ if not dep in rprovides:
+ rprovides[dep] = rreplaces[dep]
+ splitrobsoletes = bb.utils.join_deps(robsoletes, commasep=False)
+ splitrprovides = bb.utils.join_deps(rprovides, commasep=False)
+
+ print_deps(splitrdepends, "Requires", spec_preamble_bottom, d)
+ if splitrpreinst:
+ print_deps(splitrdepends, "Requires(pre)", spec_preamble_bottom, d)
+ if splitrpostinst:
+ print_deps(splitrdepends, "Requires(post)", spec_preamble_bottom, d)
+ if splitrprerm:
+ print_deps(splitrdepends, "Requires(preun)", spec_preamble_bottom, d)
+ if splitrpostrm:
+ print_deps(splitrdepends, "Requires(postun)", spec_preamble_bottom, d)
+
+ print_deps(splitrrecommends, "Recommends", spec_preamble_bottom, d)
+ print_deps(splitrsuggests, "Suggests", spec_preamble_bottom, d)
+ print_deps(splitrprovides, "Provides", spec_preamble_bottom, d)
+ print_deps(splitrobsoletes, "Obsoletes", spec_preamble_bottom, d)
+ print_deps(splitrconflicts, "Conflicts", spec_preamble_bottom, d)
+
+ spec_preamble_bottom.append('')
+
+ spec_preamble_bottom.append('%%description -n %s' % splitname)
+ append_description(spec_preamble_bottom, splitdescription)
+
+ spec_preamble_bottom.append('')
+
+ # Now process scriptlets
+ if splitrpreinst:
+ spec_scriptlets_bottom.append('%%pre -n %s' % splitname)
+ spec_scriptlets_bottom.append('# %s - preinst' % splitname)
+ spec_scriptlets_bottom.append(splitrpreinst)
+ spec_scriptlets_bottom.append('')
+ if splitrpostinst:
+ spec_scriptlets_bottom.append('%%post -n %s' % splitname)
+ spec_scriptlets_bottom.append('# %s - postinst' % splitname)
+ spec_scriptlets_bottom.append(splitrpostinst)
+ spec_scriptlets_bottom.append('')
+ if splitrprerm:
+ spec_scriptlets_bottom.append('%%preun -n %s' % splitname)
+ spec_scriptlets_bottom.append('# %s - prerm' % splitname)
+ scriptvar = wrap_uninstall(splitrprerm)
+ spec_scriptlets_bottom.append(scriptvar)
+ spec_scriptlets_bottom.append('')
+ if splitrpostrm:
+ spec_scriptlets_bottom.append('%%postun -n %s' % splitname)
+ spec_scriptlets_bottom.append('# %s - postrm' % splitname)
+ scriptvar = wrap_uninstall(splitrpostrm)
+ spec_scriptlets_bottom.append(scriptvar)
+ spec_scriptlets_bottom.append('')
+
+ # Now process files
+ file_list = []
+ walk_files(root, file_list, conffiles, dirfiles)
+ if not file_list and localdata.getVar('ALLOW_EMPTY', False) != "1":
+ bb.note("Not creating empty RPM package for %s" % splitname)
+ else:
+ spec_files_bottom.append('%%files -n %s' % splitname)
+ if extra_pkgdata:
+ package_rpm_extra_pkgdata(splitname, spec_files_bottom, localdata)
+ spec_files_bottom.append('%defattr(-,-,-,-)')
+ if file_list:
+ bb.note("Creating RPM package for %s" % splitname)
+ spec_files_bottom.extend(file_list)
+ else:
+ bb.note("Creating empty RPM package for %s" % splitname)
+ spec_files_bottom.append('')
+
+ del localdata
+
+ add_prep(d,spec_files_bottom)
+ spec_preamble_top.append('Summary: %s' % srcsummary)
+ spec_preamble_top.append('Name: %s' % srcname)
+ spec_preamble_top.append('Version: %s' % srcversion)
+ spec_preamble_top.append('Release: %s' % srcrelease)
+ if srcepoch and srcepoch.strip() != "":
+ spec_preamble_top.append('Epoch: %s' % srcepoch)
+ spec_preamble_top.append('License: %s' % srclicense)
+ spec_preamble_top.append('Group: %s' % srcsection)
+ spec_preamble_top.append('Packager: %s' % srcmaintainer)
+ if srchomepage:
+ spec_preamble_top.append('URL: %s' % srchomepage)
+ if srccustomtagschunk:
+ spec_preamble_top.append(srccustomtagschunk)
+ tail_source(d)
+
+ # Replaces == Obsoletes && Provides
+ robsoletes = bb.utils.explode_dep_versions2(srcrobsoletes)
+ rprovides = bb.utils.explode_dep_versions2(srcrprovides)
+ rreplaces = bb.utils.explode_dep_versions2(srcrreplaces)
+ for dep in rreplaces:
+ if not dep in robsoletes:
+ robsoletes[dep] = rreplaces[dep]
+ if not dep in rprovides:
+ rprovides[dep] = rreplaces[dep]
+ srcrobsoletes = bb.utils.join_deps(robsoletes, commasep=False)
+ srcrprovides = bb.utils.join_deps(rprovides, commasep=False)
+
+ print_deps(srcdepends, "BuildRequires", spec_preamble_top, d)
+ print_deps(srcrdepends, "Requires", spec_preamble_top, d)
+ if srcrpreinst:
+ print_deps(srcrdepends, "Requires(pre)", spec_preamble_top, d)
+ if srcrpostinst:
+ print_deps(srcrdepends, "Requires(post)", spec_preamble_top, d)
+ if srcrprerm:
+ print_deps(srcrdepends, "Requires(preun)", spec_preamble_top, d)
+ if srcrpostrm:
+ print_deps(srcrdepends, "Requires(postun)", spec_preamble_top, d)
+
+ print_deps(srcrrecommends, "Recommends", spec_preamble_top, d)
+ print_deps(srcrsuggests, "Suggests", spec_preamble_top, d)
+ print_deps(srcrprovides, "Provides", spec_preamble_top, d)
+ print_deps(srcrobsoletes, "Obsoletes", spec_preamble_top, d)
+ print_deps(srcrconflicts, "Conflicts", spec_preamble_top, d)
+
+ spec_preamble_top.append('')
+
+ spec_preamble_top.append('%description')
+ append_description(spec_preamble_top, srcdescription)
+
+ spec_preamble_top.append('')
+
+ if srcrpreinst:
+ spec_scriptlets_top.append('%pre')
+ spec_scriptlets_top.append('# %s - preinst' % srcname)
+ spec_scriptlets_top.append(srcrpreinst)
+ spec_scriptlets_top.append('')
+ if srcrpostinst:
+ spec_scriptlets_top.append('%post')
+ spec_scriptlets_top.append('# %s - postinst' % srcname)
+ spec_scriptlets_top.append(srcrpostinst)
+ spec_scriptlets_top.append('')
+ if srcrprerm:
+ spec_scriptlets_top.append('%preun')
+ spec_scriptlets_top.append('# %s - prerm' % srcname)
+ scriptvar = wrap_uninstall(srcrprerm)
+ spec_scriptlets_top.append(scriptvar)
+ spec_scriptlets_top.append('')
+ if srcrpostrm:
+ spec_scriptlets_top.append('%postun')
+ spec_scriptlets_top.append('# %s - postrm' % srcname)
+ scriptvar = wrap_uninstall(srcrpostrm)
+ spec_scriptlets_top.append(scriptvar)
+ spec_scriptlets_top.append('')
+
+ # Write the SPEC file
+ specfile = open(outspecfile, 'w')
+
+ # RPMSPEC_PREAMBLE is a way to add arbitrary text to the top
+ # of the generated spec file
+ external_preamble = d.getVar("RPMSPEC_PREAMBLE")
+ if external_preamble:
+ specfile.write(external_preamble + "\n")
+
+ for line in spec_preamble_top:
+ specfile.write(line + "\n")
+
+ for line in spec_preamble_bottom:
+ specfile.write(line + "\n")
+
+ for line in spec_scriptlets_top:
+ specfile.write(line + "\n")
+
+ for line in spec_scriptlets_bottom:
+ specfile.write(line + "\n")
+
+ for line in spec_files_top:
+ specfile.write(line + "\n")
+
+ for line in spec_files_bottom:
+ specfile.write(line + "\n")
+
+ specfile.close()
+}
+# Otherwise allarch packages may change depending on override configuration
+write_specfile[vardepsexclude] = "OVERRIDES"
+
+# Have to list any variables referenced as X_<pkg> that aren't in pkgdata here
+RPMEXTRAVARS = "PACKAGE_ADD_METADATA_RPM"
+write_specfile[vardeps] += "${@gen_packagevar(d, 'RPMEXTRAVARS')}"
+
+python do_package_rpm () {
+ workdir = d.getVar('WORKDIR')
+ tmpdir = d.getVar('TMPDIR')
+ pkgd = d.getVar('PKGD')
+ pkgdest = d.getVar('PKGDEST')
+ if not workdir or not pkgd or not tmpdir:
+ bb.error("Variables incorrectly set, unable to package")
+ return
+
+ packages = d.getVar('PACKAGES')
+ if not packages or packages == '':
+ bb.debug(1, "No packages; nothing to do")
+ return
+
+ # Construct the spec file...
+ # If the spec file already exist, and has not been stored into
+ # pseudo's files.db, it maybe cause rpmbuild src.rpm fail,
+ # so remove it before doing rpmbuild src.rpm.
+ srcname = d.getVar('PN')
+ outspecfile = workdir + "/" + srcname + ".spec"
+ if os.path.isfile(outspecfile):
+ os.remove(outspecfile)
+ d.setVar('OUTSPECFILE', outspecfile)
+ bb.build.exec_func('write_specfile', d)
+
+ perfiledeps = (d.getVar("MERGEPERFILEDEPS") or "0") == "0"
+ if perfiledeps:
+ outdepends, outprovides = write_rpm_perfiledata(srcname, d)
+
+ # Setup the rpmbuild arguments...
+ rpmbuild = d.getVar('RPMBUILD')
+ targetsys = d.getVar('TARGET_SYS')
+ targetvendor = d.getVar('HOST_VENDOR')
+
+ # Too many places in dnf stack assume that arch-independent packages are "noarch".
+ # Let's not fight against this.
+ package_arch = (d.getVar('PACKAGE_ARCH') or "").replace("-", "_")
+ if package_arch == "all":
+ package_arch = "noarch"
+
+ sdkpkgsuffix = (d.getVar('SDKPKGSUFFIX') or "nativesdk").replace("-", "_")
+ d.setVar('PACKAGE_ARCH_EXTEND', package_arch)
+ pkgwritedir = d.expand('${PKGWRITEDIRRPM}/${PACKAGE_ARCH_EXTEND}')
+ d.setVar('RPM_PKGWRITEDIR', pkgwritedir)
+ bb.debug(1, 'PKGWRITEDIR: %s' % d.getVar('RPM_PKGWRITEDIR'))
+ pkgarch = d.expand('${PACKAGE_ARCH_EXTEND}${HOST_VENDOR}-linux')
+ bb.utils.mkdirhier(pkgwritedir)
+ os.chmod(pkgwritedir, 0o755)
+
+ cmd = rpmbuild
+ cmd = cmd + " --noclean --nodeps --short-circuit --target " + pkgarch + " --buildroot " + pkgd
+ cmd = cmd + " --define '_topdir " + workdir + "' --define '_rpmdir " + pkgwritedir + "'"
+ cmd = cmd + " --define '_builddir " + d.getVar('B') + "'"
+ cmd = cmd + " --define '_build_name_fmt %%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm'"
+ cmd = cmd + " --define '_use_internal_dependency_generator 0'"
+ cmd = cmd + " --define '_binaries_in_noarch_packages_terminate_build 0'"
+ cmd = cmd + " --define '_build_id_links none'"
+ cmd = cmd + " --define '_binary_payload w19T%d.zstdio'" % int(d.getVar("ZSTD_THREADS"))
+ cmd = cmd + " --define '_source_payload w19T%d.zstdio'" % int(d.getVar("ZSTD_THREADS"))
+ cmd = cmd + " --define 'clamp_mtime_to_source_date_epoch 1'"
+ cmd = cmd + " --define 'use_source_date_epoch_as_buildtime 1'"
+ cmd = cmd + " --define '_buildhost reproducible'"
+ cmd = cmd + " --define '__font_provides %{nil}'"
+ if perfiledeps:
+ cmd = cmd + " --define '__find_requires " + outdepends + "'"
+ cmd = cmd + " --define '__find_provides " + outprovides + "'"
+ else:
+ cmd = cmd + " --define '__find_requires %{nil}'"
+ cmd = cmd + " --define '__find_provides %{nil}'"
+ cmd = cmd + " --define '_unpackaged_files_terminate_build 0'"
+ cmd = cmd + " --define 'debug_package %{nil}'"
+ cmd = cmd + " --define '_tmppath " + workdir + "'"
+ if d.getVarFlag('ARCHIVER_MODE', 'srpm') == '1' and bb.data.inherits_class('archiver', d):
+ cmd = cmd + " --define '_sourcedir " + d.getVar('ARCHIVER_OUTDIR') + "'"
+ cmdsrpm = cmd + " --define '_srcrpmdir " + d.getVar('ARCHIVER_RPMOUTDIR') + "'"
+ cmdsrpm = cmdsrpm + " -bs " + outspecfile
+ # Build the .src.rpm
+ d.setVar('SBUILDSPEC', cmdsrpm + "\n")
+ d.setVarFlag('SBUILDSPEC', 'func', '1')
+ bb.build.exec_func('SBUILDSPEC', d)
+ cmd = cmd + " -bb " + outspecfile
+
+ # rpm 4 creates various empty directories in _topdir, let's clean them up
+ cleanupcmd = "rm -rf %s/BUILDROOT %s/SOURCES %s/SPECS %s/SRPMS" % (workdir, workdir, workdir, workdir)
+
+ # Build the rpm package!
+ d.setVar('BUILDSPEC', cmd + "\n" + cleanupcmd + "\n")
+ d.setVarFlag('BUILDSPEC', 'func', '1')
+ bb.build.exec_func('BUILDSPEC', d)
+
+ if d.getVar('RPM_SIGN_PACKAGES') == '1':
+ bb.build.exec_func("sign_rpm", d)
+}
+
+python () {
+ if d.getVar('PACKAGES') != '':
+ deps = ' rpm-native:do_populate_sysroot virtual/fakeroot-native:do_populate_sysroot'
+ d.appendVarFlag('do_package_write_rpm', 'depends', deps)
+ d.setVarFlag('do_package_write_rpm', 'fakeroot', '1')
+}
+
+SSTATETASKS += "do_package_write_rpm"
+do_package_write_rpm[sstate-inputdirs] = "${PKGWRITEDIRRPM}"
+do_package_write_rpm[sstate-outputdirs] = "${DEPLOY_DIR_RPM}"
+# Take a shared lock, we can write multiple packages at the same time...
+# but we need to stop the rootfs/solver from running while we do...
+do_package_write_rpm[sstate-lockfile-shared] += "${DEPLOY_DIR_RPM}/rpm.lock"
+
+python do_package_write_rpm_setscene () {
+ sstate_setscene(d)
+}
+addtask do_package_write_rpm_setscene
+
+python do_package_write_rpm () {
+ bb.build.exec_func("read_subpackage_metadata", d)
+ bb.build.exec_func("do_package_rpm", d)
+}
+
+do_package_write_rpm[dirs] = "${PKGWRITEDIRRPM}"
+do_package_write_rpm[cleandirs] = "${PKGWRITEDIRRPM}"
+do_package_write_rpm[depends] += "${@oe.utils.build_depends_string(d.getVar('PACKAGE_WRITE_DEPS'), 'do_populate_sysroot')}"
+addtask package_write_rpm after do_packagedata do_package do_deploy_source_date_epoch before do_build
+do_build[rdeptask] += "do_package_write_rpm"
+
+PACKAGEINDEXDEPS += "rpm-native:do_populate_sysroot"
+PACKAGEINDEXDEPS += "createrepo-c-native:do_populate_sysroot"
diff --git a/poky/meta/classes-global/package_tar.bbclass b/poky/meta/classes-global/package_tar.bbclass
new file mode 100644
index 0000000..de995f9
--- /dev/null
+++ b/poky/meta/classes-global/package_tar.bbclass
@@ -0,0 +1,77 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+inherit package
+
+IMAGE_PKGTYPE ?= "tar"
+
+python do_package_tar () {
+ import subprocess
+
+ oldcwd = os.getcwd()
+
+ workdir = d.getVar('WORKDIR')
+ if not workdir:
+ bb.error("WORKDIR not defined, unable to package")
+ return
+
+ outdir = d.getVar('DEPLOY_DIR_TAR')
+ if not outdir:
+ bb.error("DEPLOY_DIR_TAR not defined, unable to package")
+ return
+
+ dvar = d.getVar('D')
+ if not dvar:
+ bb.error("D not defined, unable to package")
+ return
+
+ packages = d.getVar('PACKAGES')
+ if not packages:
+ bb.debug(1, "PACKAGES not defined, nothing to package")
+ return
+
+ pkgdest = d.getVar('PKGDEST')
+
+ bb.utils.mkdirhier(outdir)
+ bb.utils.mkdirhier(dvar)
+
+ for pkg in packages.split():
+ localdata = bb.data.createCopy(d)
+ root = "%s/%s" % (pkgdest, pkg)
+
+ overrides = localdata.getVar('OVERRIDES', False)
+ localdata.setVar('OVERRIDES', '%s:%s' % (overrides, pkg))
+
+ bb.utils.mkdirhier(root)
+ basedir = os.path.dirname(root)
+ tarfn = localdata.expand("${DEPLOY_DIR_TAR}/${PKG}-${PKGV}-${PKGR}.tar.gz")
+ os.chdir(root)
+ dlist = os.listdir(root)
+ if not dlist:
+ bb.note("Not creating empty archive for %s-%s-%s" % (pkg, localdata.getVar('PKGV'), localdata.getVar('PKGR')))
+ continue
+ args = "tar -cz --exclude=CONTROL --exclude=DEBIAN -f".split()
+ ret = subprocess.call(args + [tarfn] + dlist)
+ if ret != 0:
+ bb.error("Creation of tar %s failed." % tarfn)
+
+ os.chdir(oldcwd)
+}
+
+python () {
+ if d.getVar('PACKAGES') != '':
+ deps = ' tar-native:do_populate_sysroot virtual/fakeroot-native:do_populate_sysroot'
+ d.appendVarFlag('do_package_write_tar', 'depends', deps)
+ d.setVarFlag('do_package_write_tar', 'fakeroot', "1")
+}
+
+
+python do_package_write_tar () {
+ bb.build.exec_func("read_subpackage_metadata", d)
+ bb.build.exec_func("do_package_tar", d)
+}
+do_package_write_tar[dirs] = "${D}"
+addtask package_write_tar before do_build after do_packagedata do_package
diff --git a/poky/meta/classes-global/packagedata.bbclass b/poky/meta/classes-global/packagedata.bbclass
new file mode 100644
index 0000000..9f72c01
--- /dev/null
+++ b/poky/meta/classes-global/packagedata.bbclass
@@ -0,0 +1,40 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+python read_subpackage_metadata () {
+ import oe.packagedata
+
+ vars = {
+ "PN" : d.getVar('PN'),
+ "PE" : d.getVar('PE'),
+ "PV" : d.getVar('PV'),
+ "PR" : d.getVar('PR'),
+ }
+
+ data = oe.packagedata.read_pkgdata(vars["PN"], d)
+
+ for key in data.keys():
+ d.setVar(key, data[key])
+
+ for pkg in d.getVar('PACKAGES').split():
+ sdata = oe.packagedata.read_subpkgdata(pkg, d)
+ for key in sdata.keys():
+ if key in vars:
+ if sdata[key] != vars[key]:
+ if key == "PN":
+ bb.fatal("Recipe %s is trying to create package %s which was already written by recipe %s. This will cause corruption, please resolve this and only provide the package from one recipe or the other or only build one of the recipes." % (vars[key], pkg, sdata[key]))
+ bb.fatal("Recipe %s is trying to change %s from '%s' to '%s'. This will cause do_package_write_* failures since the incorrect data will be used and they will be unable to find the right workdir." % (vars["PN"], key, vars[key], sdata[key]))
+ continue
+ #
+ # If we set unsuffixed variables here there is a chance they could clobber override versions
+ # of that variable, e.g. DESCRIPTION could clobber DESCRIPTION:<pkgname>
+ # We therefore don't clobber for the unsuffixed variable versions
+ #
+ if key.endswith(":" + pkg):
+ d.setVar(key, sdata[key])
+ else:
+ d.setVar(key, sdata[key], parsing=True)
+}
diff --git a/poky/meta/classes-global/patch.bbclass b/poky/meta/classes-global/patch.bbclass
new file mode 100644
index 0000000..e3157c7
--- /dev/null
+++ b/poky/meta/classes-global/patch.bbclass
@@ -0,0 +1,171 @@
+# Copyright (C) 2006 OpenedHand LTD
+#
+# SPDX-License-Identifier: MIT
+
+# Point to an empty file so any user's custom settings don't break things
+QUILTRCFILE ?= "${STAGING_ETCDIR_NATIVE}/quiltrc"
+
+PATCHDEPENDENCY = "${PATCHTOOL}-native:do_populate_sysroot"
+
+# There is a bug in patch 2.7.3 and earlier where index lines
+# in patches can change file modes when they shouldn't:
+# http://git.savannah.gnu.org/cgit/patch.git/patch/?id=82b800c9552a088a241457948219d25ce0a407a4
+# This leaks into debug sources in particular. Add the dependency
+# to target recipes to avoid this problem until we can rely on 2.7.4 or later.
+PATCHDEPENDENCY:append:class-target = " patch-replacement-native:do_populate_sysroot"
+
+PATCH_GIT_USER_NAME ?= "OpenEmbedded"
+PATCH_GIT_USER_EMAIL ?= "oe.patch@oe"
+
+inherit terminal
+
+python () {
+ if d.getVar('PATCHTOOL') == 'git' and d.getVar('PATCH_COMMIT_FUNCTIONS') == '1':
+ extratasks = bb.build.tasksbetween('do_unpack', 'do_patch', d)
+ try:
+ extratasks.remove('do_unpack')
+ except ValueError:
+ # For some recipes do_unpack doesn't exist, ignore it
+ pass
+
+ d.appendVarFlag('do_patch', 'prefuncs', ' patch_task_patch_prefunc')
+ for task in extratasks:
+ d.appendVarFlag(task, 'postfuncs', ' patch_task_postfunc')
+}
+
+python patch_task_patch_prefunc() {
+ # Prefunc for do_patch
+ srcsubdir = d.getVar('S')
+
+ workdir = os.path.abspath(d.getVar('WORKDIR'))
+ testsrcdir = os.path.abspath(srcsubdir)
+ if (testsrcdir + os.sep).startswith(workdir + os.sep):
+ # Double-check that either workdir or S or some directory in-between is a git repository
+ found = False
+ while testsrcdir != workdir:
+ if os.path.exists(os.path.join(testsrcdir, '.git')):
+ found = True
+ break
+ if testsrcdir == workdir:
+ break
+ testsrcdir = os.path.dirname(testsrcdir)
+ if not found:
+ bb.fatal('PATCHTOOL = "git" set for source tree that is not a git repository. Refusing to continue as that may result in commits being made in your metadata repository.')
+
+ patchdir = os.path.join(srcsubdir, 'patches')
+ if os.path.exists(patchdir):
+ if os.listdir(patchdir):
+ d.setVar('PATCH_HAS_PATCHES_DIR', '1')
+ else:
+ os.rmdir(patchdir)
+}
+
+python patch_task_postfunc() {
+ # Prefunc for task functions between do_unpack and do_patch
+ import oe.patch
+ import shutil
+ func = d.getVar('BB_RUNTASK')
+ srcsubdir = d.getVar('S')
+
+ if os.path.exists(srcsubdir):
+ if func == 'do_patch':
+ haspatches = (d.getVar('PATCH_HAS_PATCHES_DIR') == '1')
+ patchdir = os.path.join(srcsubdir, 'patches')
+ if os.path.exists(patchdir):
+ shutil.rmtree(patchdir)
+ if haspatches:
+ stdout, _ = bb.process.run('git status --porcelain patches', cwd=srcsubdir)
+ if stdout:
+ bb.process.run('git checkout patches', cwd=srcsubdir)
+ stdout, _ = bb.process.run('git status --porcelain .', cwd=srcsubdir)
+ if stdout:
+ useroptions = []
+ oe.patch.GitApplyTree.gitCommandUserOptions(useroptions, d=d)
+ bb.process.run('git add .; git %s commit -a -m "Committing changes from %s\n\n%s"' % (' '.join(useroptions), func, oe.patch.GitApplyTree.ignore_commit_prefix + ' - from %s' % func), cwd=srcsubdir)
+}
+
+def src_patches(d, all=False, expand=True):
+ import oe.patch
+ return oe.patch.src_patches(d, all, expand)
+
+def should_apply(parm, d):
+ """Determine if we should apply the given patch"""
+ import oe.patch
+ return oe.patch.should_apply(parm, d)
+
+should_apply[vardepsexclude] = "DATE SRCDATE"
+
+python patch_do_patch() {
+ import oe.patch
+
+ patchsetmap = {
+ "patch": oe.patch.PatchTree,
+ "quilt": oe.patch.QuiltTree,
+ "git": oe.patch.GitApplyTree,
+ }
+
+ cls = patchsetmap[d.getVar('PATCHTOOL') or 'quilt']
+
+ resolvermap = {
+ "noop": oe.patch.NOOPResolver,
+ "user": oe.patch.UserResolver,
+ }
+
+ rcls = resolvermap[d.getVar('PATCHRESOLVE') or 'user']
+
+ classes = {}
+
+ s = d.getVar('S')
+
+ os.putenv('PATH', d.getVar('PATH'))
+
+ # We must use one TMPDIR per process so that the "patch" processes
+ # don't generate the same temp file name.
+
+ import tempfile
+ process_tmpdir = tempfile.mkdtemp()
+ os.environ['TMPDIR'] = process_tmpdir
+
+ for patch in src_patches(d):
+ _, _, local, _, _, parm = bb.fetch.decodeurl(patch)
+
+ if "patchdir" in parm:
+ patchdir = parm["patchdir"]
+ if not os.path.isabs(patchdir):
+ patchdir = os.path.join(s, patchdir)
+ if not os.path.isdir(patchdir):
+ bb.fatal("Target directory '%s' not found, patchdir '%s' is incorrect in patch file '%s'" %
+ (patchdir, parm["patchdir"], parm['patchname']))
+ else:
+ patchdir = s
+
+ if not patchdir in classes:
+ patchset = cls(patchdir, d)
+ resolver = rcls(patchset, oe_terminal)
+ classes[patchdir] = (patchset, resolver)
+ patchset.Clean()
+ else:
+ patchset, resolver = classes[patchdir]
+
+ bb.note("Applying patch '%s' (%s)" % (parm['patchname'], oe.path.format_display(local, d)))
+ try:
+ patchset.Import({"file":local, "strippath": parm['striplevel']}, True)
+ except Exception as exc:
+ bb.utils.remove(process_tmpdir, True)
+ bb.fatal("Importing patch '%s' with striplevel '%s'\n%s" % (parm['patchname'], parm['striplevel'], repr(exc).replace("\\n", "\n")))
+ try:
+ resolver.Resolve()
+ except bb.BBHandledException as e:
+ bb.utils.remove(process_tmpdir, True)
+ bb.fatal("Applying patch '%s' on target directory '%s'\n%s" % (parm['patchname'], patchdir, repr(e).replace("\\n", "\n")))
+
+ bb.utils.remove(process_tmpdir, True)
+ del os.environ['TMPDIR']
+}
+patch_do_patch[vardepsexclude] = "PATCHRESOLVE"
+
+addtask patch after do_unpack
+do_patch[dirs] = "${WORKDIR}"
+do_patch[depends] = "${PATCHDEPENDENCY}"
+
+EXPORT_FUNCTIONS do_patch
diff --git a/poky/meta/classes-global/sanity.bbclass b/poky/meta/classes-global/sanity.bbclass
new file mode 100644
index 0000000..4a403a2
--- /dev/null
+++ b/poky/meta/classes-global/sanity.bbclass
@@ -0,0 +1,1029 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+#
+# Sanity check the users setup for common misconfigurations
+#
+
+SANITY_REQUIRED_UTILITIES ?= "patch diffstat git bzip2 tar \
+ gzip gawk chrpath wget cpio perl file which"
+
+def bblayers_conf_file(d):
+ return os.path.join(d.getVar('TOPDIR'), 'conf/bblayers.conf')
+
+def sanity_conf_read(fn):
+ with open(fn, 'r') as f:
+ lines = f.readlines()
+ return lines
+
+def sanity_conf_find_line(pattern, lines):
+ import re
+ return next(((index, line)
+ for index, line in enumerate(lines)
+ if re.search(pattern, line)), (None, None))
+
+def sanity_conf_update(fn, lines, version_var_name, new_version):
+ index, line = sanity_conf_find_line(r"^%s" % version_var_name, lines)
+ lines[index] = '%s = "%d"\n' % (version_var_name, new_version)
+ with open(fn, "w") as f:
+ f.write(''.join(lines))
+
+# Functions added to this variable MUST throw a NotImplementedError exception unless
+# they successfully changed the config version in the config file. Exceptions
+# are used since exec_func doesn't handle return values.
+BBLAYERS_CONF_UPDATE_FUNCS += " \
+ conf/bblayers.conf:LCONF_VERSION:LAYER_CONF_VERSION:oecore_update_bblayers \
+ conf/local.conf:CONF_VERSION:LOCALCONF_VERSION:oecore_update_localconf \
+ conf/site.conf:SCONF_VERSION:SITE_CONF_VERSION:oecore_update_siteconf \
+"
+
+SANITY_DIFF_TOOL ?= "meld"
+
+SANITY_LOCALCONF_SAMPLE ?= "${COREBASE}/meta*/conf/local.conf.sample"
+python oecore_update_localconf() {
+ # Check we are using a valid local.conf
+ current_conf = d.getVar('CONF_VERSION')
+ conf_version = d.getVar('LOCALCONF_VERSION')
+
+ failmsg = """Your version of local.conf was generated from an older/newer version of
+local.conf.sample and there have been updates made to this file. Please compare the two
+files and merge any changes before continuing.
+
+Matching the version numbers will remove this message.
+
+\"${SANITY_DIFF_TOOL} conf/local.conf ${SANITY_LOCALCONF_SAMPLE}\"
+
+is a good way to visualise the changes."""
+ failmsg = d.expand(failmsg)
+
+ raise NotImplementedError(failmsg)
+}
+
+SANITY_SITECONF_SAMPLE ?= "${COREBASE}/meta*/conf/site.conf.sample"
+python oecore_update_siteconf() {
+ # If we have a site.conf, check it's valid
+ current_sconf = d.getVar('SCONF_VERSION')
+ sconf_version = d.getVar('SITE_CONF_VERSION')
+
+ failmsg = """Your version of site.conf was generated from an older version of
+site.conf.sample and there have been updates made to this file. Please compare the two
+files and merge any changes before continuing.
+
+Matching the version numbers will remove this message.
+
+\"${SANITY_DIFF_TOOL} conf/site.conf ${SANITY_SITECONF_SAMPLE}\"
+
+is a good way to visualise the changes."""
+ failmsg = d.expand(failmsg)
+
+ raise NotImplementedError(failmsg)
+}
+
+SANITY_BBLAYERCONF_SAMPLE ?= "${COREBASE}/meta*/conf/bblayers.conf.sample"
+python oecore_update_bblayers() {
+ # bblayers.conf is out of date, so see if we can resolve that
+
+ current_lconf = int(d.getVar('LCONF_VERSION'))
+ lconf_version = int(d.getVar('LAYER_CONF_VERSION'))
+
+ failmsg = """Your version of bblayers.conf has the wrong LCONF_VERSION (has ${LCONF_VERSION}, expecting ${LAYER_CONF_VERSION}).
+Please compare your file against bblayers.conf.sample and merge any changes before continuing.
+"${SANITY_DIFF_TOOL} conf/bblayers.conf ${SANITY_BBLAYERCONF_SAMPLE}"
+
+is a good way to visualise the changes."""
+ failmsg = d.expand(failmsg)
+
+ if not current_lconf:
+ raise NotImplementedError(failmsg)
+
+ lines = []
+
+ if current_lconf < 4:
+ raise NotImplementedError(failmsg)
+
+ bblayers_fn = bblayers_conf_file(d)
+ lines = sanity_conf_read(bblayers_fn)
+
+ if current_lconf == 4 and lconf_version > 4:
+ topdir_var = '$' + '{TOPDIR}'
+ index, bbpath_line = sanity_conf_find_line('BBPATH', lines)
+ if bbpath_line:
+ start = bbpath_line.find('"')
+ if start != -1 and (len(bbpath_line) != (start + 1)):
+ if bbpath_line[start + 1] == '"':
+ lines[index] = (bbpath_line[:start + 1] +
+ topdir_var + bbpath_line[start + 1:])
+ else:
+ if not topdir_var in bbpath_line:
+ lines[index] = (bbpath_line[:start + 1] +
+ topdir_var + ':' + bbpath_line[start + 1:])
+ else:
+ raise NotImplementedError(failmsg)
+ else:
+ index, bbfiles_line = sanity_conf_find_line('BBFILES', lines)
+ if bbfiles_line:
+ lines.insert(index, 'BBPATH = "' + topdir_var + '"\n')
+ else:
+ raise NotImplementedError(failmsg)
+
+ current_lconf += 1
+ sanity_conf_update(bblayers_fn, lines, 'LCONF_VERSION', current_lconf)
+ bb.note("Your conf/bblayers.conf has been automatically updated.")
+ return
+
+ elif current_lconf == 5 and lconf_version > 5:
+ # Null update, to avoid issues with people switching between poky and other distros
+ current_lconf = 6
+ sanity_conf_update(bblayers_fn, lines, 'LCONF_VERSION', current_lconf)
+ bb.note("Your conf/bblayers.conf has been automatically updated.")
+ return
+
+ status.addresult()
+
+ elif current_lconf == 6 and lconf_version > 6:
+ # Handle rename of meta-yocto -> meta-poky
+ # This marks the start of separate version numbers but code is needed in OE-Core
+ # for the migration, one last time.
+ layers = d.getVar('BBLAYERS').split()
+ layers = [ os.path.basename(path) for path in layers ]
+ if 'meta-yocto' in layers:
+ found = False
+ while True:
+ index, meta_yocto_line = sanity_conf_find_line(r'.*meta-yocto[\'"\s\n]', lines)
+ if meta_yocto_line:
+ lines[index] = meta_yocto_line.replace('meta-yocto', 'meta-poky')
+ found = True
+ else:
+ break
+ if not found:
+ raise NotImplementedError(failmsg)
+ index, meta_yocto_line = sanity_conf_find_line('LCONF_VERSION.*\n', lines)
+ if meta_yocto_line:
+ lines[index] = 'POKY_BBLAYERS_CONF_VERSION = "1"\n'
+ else:
+ raise NotImplementedError(failmsg)
+ with open(bblayers_fn, "w") as f:
+ f.write(''.join(lines))
+ bb.note("Your conf/bblayers.conf has been automatically updated.")
+ return
+ current_lconf += 1
+ sanity_conf_update(bblayers_fn, lines, 'LCONF_VERSION', current_lconf)
+ bb.note("Your conf/bblayers.conf has been automatically updated.")
+ return
+
+ raise NotImplementedError(failmsg)
+}
+
+def raise_sanity_error(msg, d, network_error=False):
+ if d.getVar("SANITY_USE_EVENTS") == "1":
+ try:
+ bb.event.fire(bb.event.SanityCheckFailed(msg, network_error), d)
+ except TypeError:
+ bb.event.fire(bb.event.SanityCheckFailed(msg), d)
+ return
+
+ bb.fatal(""" OE-core's config sanity checker detected a potential misconfiguration.
+ Either fix the cause of this error or at your own risk disable the checker (see sanity.conf).
+ Following is the list of potential problems / advisories:
+
+ %s""" % msg)
+
+# Check a single tune for validity.
+def check_toolchain_tune(data, tune, multilib):
+ tune_errors = []
+ if not tune:
+ return "No tuning found for %s multilib." % multilib
+ localdata = bb.data.createCopy(data)
+ if multilib != "default":
+ # Apply the overrides so we can look at the details.
+ overrides = localdata.getVar("OVERRIDES", False) + ":virtclass-multilib-" + multilib
+ localdata.setVar("OVERRIDES", overrides)
+ bb.debug(2, "Sanity-checking tuning '%s' (%s) features:" % (tune, multilib))
+ features = (localdata.getVar("TUNE_FEATURES:tune-%s" % tune) or "").split()
+ if not features:
+ return "Tuning '%s' has no defined features, and cannot be used." % tune
+ valid_tunes = localdata.getVarFlags('TUNEVALID') or {}
+ conflicts = localdata.getVarFlags('TUNECONFLICTS') or {}
+ # [doc] is the documentation for the variable, not a real feature
+ if 'doc' in valid_tunes:
+ del valid_tunes['doc']
+ if 'doc' in conflicts:
+ del conflicts['doc']
+ for feature in features:
+ if feature in conflicts:
+ for conflict in conflicts[feature].split():
+ if conflict in features:
+ tune_errors.append("Feature '%s' conflicts with '%s'." %
+ (feature, conflict))
+ if feature in valid_tunes:
+ bb.debug(2, " %s: %s" % (feature, valid_tunes[feature]))
+ else:
+ tune_errors.append("Feature '%s' is not defined." % feature)
+ if tune_errors:
+ return "Tuning '%s' has the following errors:\n" % tune + '\n'.join(tune_errors)
+
+def check_toolchain(data):
+ tune_error_set = []
+ deftune = data.getVar("DEFAULTTUNE")
+ tune_errors = check_toolchain_tune(data, deftune, 'default')
+ if tune_errors:
+ tune_error_set.append(tune_errors)
+
+ multilibs = (data.getVar("MULTILIB_VARIANTS") or "").split()
+ global_multilibs = (data.getVar("MULTILIB_GLOBAL_VARIANTS") or "").split()
+
+ if multilibs:
+ seen_libs = []
+ seen_tunes = []
+ for lib in multilibs:
+ if lib in seen_libs:
+ tune_error_set.append("The multilib '%s' appears more than once." % lib)
+ else:
+ seen_libs.append(lib)
+ if not lib in global_multilibs:
+ tune_error_set.append("Multilib %s is not present in MULTILIB_GLOBAL_VARIANTS" % lib)
+ tune = data.getVar("DEFAULTTUNE:virtclass-multilib-%s" % lib)
+ if tune in seen_tunes:
+ tune_error_set.append("The tuning '%s' appears in more than one multilib." % tune)
+ else:
+ seen_libs.append(tune)
+ if tune == deftune:
+ tune_error_set.append("Multilib '%s' (%s) is also the default tuning." % (lib, deftune))
+ else:
+ tune_errors = check_toolchain_tune(data, tune, lib)
+ if tune_errors:
+ tune_error_set.append(tune_errors)
+ if tune_error_set:
+ return "Toolchain tunings invalid:\n" + '\n'.join(tune_error_set) + "\n"
+
+ return ""
+
+def check_conf_exists(fn, data):
+ bbpath = []
+ fn = data.expand(fn)
+ vbbpath = data.getVar("BBPATH", False)
+ if vbbpath:
+ bbpath += vbbpath.split(":")
+ for p in bbpath:
+ currname = os.path.join(data.expand(p), fn)
+ if os.access(currname, os.R_OK):
+ return True
+ return False
+
+def check_create_long_filename(filepath, pathname):
+ import string, random
+ testfile = os.path.join(filepath, ''.join(random.choice(string.ascii_letters) for x in range(200)))
+ try:
+ if not os.path.exists(filepath):
+ bb.utils.mkdirhier(filepath)
+ f = open(testfile, "w")
+ f.close()
+ os.remove(testfile)
+ except IOError as e:
+ import errno
+ err, strerror = e.args
+ if err == errno.ENAMETOOLONG:
+ return "Failed to create a file with a long name in %s. Please use a filesystem that does not unreasonably limit filename length.\n" % pathname
+ else:
+ return "Failed to create a file in %s: %s.\n" % (pathname, strerror)
+ except OSError as e:
+ errno, strerror = e.args
+ return "Failed to create %s directory in which to run long name sanity check: %s.\n" % (pathname, strerror)
+ return ""
+
+def check_path_length(filepath, pathname, limit):
+ if len(filepath) > limit:
+ return "The length of %s is longer than %s, this would cause unexpected errors, please use a shorter path.\n" % (pathname, limit)
+ return ""
+
+def get_filesystem_id(path):
+ import subprocess
+ try:
+ return subprocess.check_output(["stat", "-f", "-c", "%t", path]).decode('utf-8').strip()
+ except subprocess.CalledProcessError:
+ bb.warn("Can't get filesystem id of: %s" % path)
+ return None
+
+# Check that the path isn't located on nfs.
+def check_not_nfs(path, name):
+ # The nfs' filesystem id is 6969
+ if get_filesystem_id(path) == "6969":
+ return "The %s: %s can't be located on nfs.\n" % (name, path)
+ return ""
+
+# Check that the path is on a case-sensitive file system
+def check_case_sensitive(path, name):
+ import tempfile
+ with tempfile.NamedTemporaryFile(prefix='TmP', dir=path) as tmp_file:
+ if os.path.exists(tmp_file.name.lower()):
+ return "The %s (%s) can't be on a case-insensitive file system.\n" % (name, path)
+ return ""
+
+# Check that path isn't a broken symlink
+def check_symlink(lnk, data):
+ if os.path.islink(lnk) and not os.path.exists(lnk):
+ raise_sanity_error("%s is a broken symlink." % lnk, data)
+
+def check_connectivity(d):
+ # URI's to check can be set in the CONNECTIVITY_CHECK_URIS variable
+ # using the same syntax as for SRC_URI. If the variable is not set
+ # the check is skipped
+ test_uris = (d.getVar('CONNECTIVITY_CHECK_URIS') or "").split()
+ retval = ""
+
+ bbn = d.getVar('BB_NO_NETWORK')
+ if bbn not in (None, '0', '1'):
+ return 'BB_NO_NETWORK should be "0" or "1", but it is "%s"' % bbn
+
+ # Only check connectivity if network enabled and the
+ # CONNECTIVITY_CHECK_URIS are set
+ network_enabled = not (bbn == '1')
+ check_enabled = len(test_uris)
+ if check_enabled and network_enabled:
+ # Take a copy of the data store and unset MIRRORS and PREMIRRORS
+ data = bb.data.createCopy(d)
+ data.delVar('PREMIRRORS')
+ data.delVar('MIRRORS')
+ try:
+ fetcher = bb.fetch2.Fetch(test_uris, data)
+ fetcher.checkstatus()
+ except Exception as err:
+ # Allow the message to be configured so that users can be
+ # pointed to a support mechanism.
+ msg = data.getVar('CONNECTIVITY_CHECK_MSG') or ""
+ if len(msg) == 0:
+ msg = "%s.\n" % err
+ msg += " Please ensure your host's network is configured correctly.\n"
+ msg += " Please ensure CONNECTIVITY_CHECK_URIS is correct and specified URIs are available.\n"
+ msg += " If your ISP or network is blocking the above URL,\n"
+ msg += " try with another domain name, for example by setting:\n"
+ msg += " CONNECTIVITY_CHECK_URIS = \"https://www.example.com/\""
+ msg += " You could also set BB_NO_NETWORK = \"1\" to disable network\n"
+ msg += " access if all required sources are on local disk.\n"
+ retval = msg
+
+ return retval
+
+def check_supported_distro(sanity_data):
+ from fnmatch import fnmatch
+
+ tested_distros = sanity_data.getVar('SANITY_TESTED_DISTROS')
+ if not tested_distros:
+ return
+
+ try:
+ distro = oe.lsb.distro_identifier()
+ except Exception:
+ distro = None
+
+ if not distro:
+ bb.warn('Host distribution could not be determined; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.')
+
+ for supported in [x.strip() for x in tested_distros.split('\\n')]:
+ if fnmatch(distro, supported):
+ return
+
+ bb.warn('Host distribution "%s" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.' % distro)
+
+# Checks we should only make if MACHINE is set correctly
+def check_sanity_validmachine(sanity_data):
+ messages = ""
+
+ # Check TUNE_ARCH is set
+ if sanity_data.getVar('TUNE_ARCH') == 'INVALID':
+ messages = messages + 'TUNE_ARCH is unset. Please ensure your MACHINE configuration includes a valid tune configuration file which will set this correctly.\n'
+
+ # Check TARGET_OS is set
+ if sanity_data.getVar('TARGET_OS') == 'INVALID':
+ messages = messages + 'Please set TARGET_OS directly, or choose a MACHINE or DISTRO that does so.\n'
+
+ # Check that we don't have duplicate entries in PACKAGE_ARCHS & that TUNE_PKGARCH is in PACKAGE_ARCHS
+ pkgarchs = sanity_data.getVar('PACKAGE_ARCHS')
+ tunepkg = sanity_data.getVar('TUNE_PKGARCH')
+ defaulttune = sanity_data.getVar('DEFAULTTUNE')
+ tunefound = False
+ seen = {}
+ dups = []
+
+ for pa in pkgarchs.split():
+ if seen.get(pa, 0) == 1:
+ dups.append(pa)
+ else:
+ seen[pa] = 1
+ if pa == tunepkg:
+ tunefound = True
+
+ if len(dups):
+ messages = messages + "Error, the PACKAGE_ARCHS variable contains duplicates. The following archs are listed more than once: %s" % " ".join(dups)
+
+ if tunefound == False:
+ messages = messages + "Error, the PACKAGE_ARCHS variable (%s) for DEFAULTTUNE (%s) does not contain TUNE_PKGARCH (%s)." % (pkgarchs, defaulttune, tunepkg)
+
+ return messages
+
+# Patch before 2.7 can't handle all the features in git-style diffs. Some
+# patches may incorrectly apply, and others won't apply at all.
+def check_patch_version(sanity_data):
+ import re, subprocess
+
+ try:
+ result = subprocess.check_output(["patch", "--version"], stderr=subprocess.STDOUT).decode('utf-8')
+ version = re.search(r"[0-9.]+", result.splitlines()[0]).group()
+ if bb.utils.vercmp_string_op(version, "2.7", "<"):
+ return "Your version of patch is older than 2.7 and has bugs which will break builds. Please install a newer version of patch.\n"
+ else:
+ return None
+ except subprocess.CalledProcessError as e:
+ return "Unable to execute patch --version, exit code %d:\n%s\n" % (e.returncode, e.output)
+
+# Glibc needs make 4.0 or later, we may as well match at this point
+def check_make_version(sanity_data):
+ import subprocess
+
+ try:
+ result = subprocess.check_output(['make', '--version'], stderr=subprocess.STDOUT).decode('utf-8')
+ except subprocess.CalledProcessError as e:
+ return "Unable to execute make --version, exit code %d\n%s\n" % (e.returncode, e.output)
+ version = result.split()[2]
+ if bb.utils.vercmp_string_op(version, "4.0", "<"):
+ return "Please install a make version of 4.0 or later.\n"
+
+ if bb.utils.vercmp_string_op(version, "4.2.1", "=="):
+ distro = oe.lsb.distro_identifier()
+ if "ubuntu" in distro or "debian" in distro or "linuxmint" in distro:
+ return None
+ return "make version 4.2.1 is known to have issues on Centos/OpenSUSE and other non-Ubuntu systems. Please use a buildtools-make-tarball or a newer version of make.\n"
+ return None
+
+
+# Check if we're running on WSL (Windows Subsystem for Linux).
+# WSLv1 is known not to work but WSLv2 should work properly as
+# long as the VHDX file is optimized often, let the user know
+# upfront.
+# More information on installing WSLv2 at:
+# https://docs.microsoft.com/en-us/windows/wsl/wsl2-install
+def check_wsl(d):
+ with open("/proc/version", "r") as f:
+ verdata = f.readlines()
+ for l in verdata:
+ if "Microsoft" in l:
+ return "OpenEmbedded doesn't work under WSLv1, please upgrade to WSLv2 if you want to run builds on Windows"
+ elif "microsoft" in l:
+ bb.warn("You are running bitbake under WSLv2, this works properly but you should optimize your VHDX file eventually to avoid running out of storage space")
+ return None
+
+# Require at least gcc version 7.5.
+#
+# This can be fixed on CentOS-7 with devtoolset-6+
+# https://www.softwarecollections.org/en/scls/rhscl/devtoolset-6/
+#
+# A less invasive fix is with scripts/install-buildtools (or with user
+# built buildtools-extended-tarball)
+#
+def check_gcc_version(sanity_data):
+ import subprocess
+
+ build_cc, version = oe.utils.get_host_compiler_version(sanity_data)
+ if build_cc.strip() == "gcc":
+ if bb.utils.vercmp_string_op(version, "7.5", "<"):
+ return "Your version of gcc is older than 7.5 and will break builds. Please install a newer version of gcc (you could use the project's buildtools-extended-tarball or use scripts/install-buildtools).\n"
+ return None
+
+# Tar version 1.24 and onwards handle overwriting symlinks correctly
+# but earlier versions do not; this needs to work properly for sstate
+# Version 1.28 is needed so opkg-build works correctly when reproducibile builds are enabled
+def check_tar_version(sanity_data):
+ import subprocess
+ try:
+ result = subprocess.check_output(["tar", "--version"], stderr=subprocess.STDOUT).decode('utf-8')
+ except subprocess.CalledProcessError as e:
+ return "Unable to execute tar --version, exit code %d\n%s\n" % (e.returncode, e.output)
+ version = result.split()[3]
+ if bb.utils.vercmp_string_op(version, "1.28", "<"):
+ return "Your version of tar is older than 1.28 and does not have the support needed to enable reproducible builds. Please install a newer version of tar (you could use the project's buildtools-tarball from our last release or use scripts/install-buildtools).\n"
+ return None
+
+# We use git parameters and functionality only found in 1.7.8 or later
+# The kernel tools assume git >= 1.8.3.1 (verified needed > 1.7.9.5) see #6162
+# The git fetcher also had workarounds for git < 1.7.9.2 which we've dropped
+def check_git_version(sanity_data):
+ import subprocess
+ try:
+ result = subprocess.check_output(["git", "--version"], stderr=subprocess.DEVNULL).decode('utf-8')
+ except subprocess.CalledProcessError as e:
+ return "Unable to execute git --version, exit code %d\n%s\n" % (e.returncode, e.output)
+ version = result.split()[2]
+ if bb.utils.vercmp_string_op(version, "1.8.3.1", "<"):
+ return "Your version of git is older than 1.8.3.1 and has bugs which will break builds. Please install a newer version of git.\n"
+ return None
+
+# Check the required perl modules which may not be installed by default
+def check_perl_modules(sanity_data):
+ import subprocess
+ ret = ""
+ modules = ( "Text::ParseWords", "Thread::Queue", "Data::Dumper" )
+ errresult = ''
+ for m in modules:
+ try:
+ subprocess.check_output(["perl", "-e", "use %s" % m])
+ except subprocess.CalledProcessError as e:
+ errresult += bytes.decode(e.output)
+ ret += "%s " % m
+ if ret:
+ return "Required perl module(s) not found: %s\n\n%s\n" % (ret, errresult)
+ return None
+
+def sanity_check_conffiles(d):
+ funcs = d.getVar('BBLAYERS_CONF_UPDATE_FUNCS').split()
+ for func in funcs:
+ conffile, current_version, required_version, func = func.split(":")
+ if check_conf_exists(conffile, d) and d.getVar(current_version) is not None and \
+ d.getVar(current_version) != d.getVar(required_version):
+ try:
+ bb.build.exec_func(func, d)
+ except NotImplementedError as e:
+ bb.fatal(str(e))
+ d.setVar("BB_INVALIDCONF", True)
+
+def drop_v14_cross_builds(d):
+ import glob
+ indexes = glob.glob(d.expand("${SSTATE_MANIFESTS}/index-${BUILD_ARCH}_*"))
+ for i in indexes:
+ with open(i, "r") as f:
+ lines = f.readlines()
+ for l in reversed(lines):
+ try:
+ (stamp, manifest, workdir) = l.split()
+ except ValueError:
+ bb.fatal("Invalid line '%s' in sstate manifest '%s'" % (l, i))
+ for m in glob.glob(manifest + ".*"):
+ if m.endswith(".postrm"):
+ continue
+ sstate_clean_manifest(m, d)
+ bb.utils.remove(stamp + "*")
+ bb.utils.remove(workdir, recurse = True)
+
+def sanity_handle_abichanges(status, d):
+ #
+ # Check the 'ABI' of TMPDIR
+ #
+ import subprocess
+
+ current_abi = d.getVar('OELAYOUT_ABI')
+ abifile = d.getVar('SANITY_ABIFILE')
+ if os.path.exists(abifile):
+ with open(abifile, "r") as f:
+ abi = f.read().strip()
+ if not abi.isdigit():
+ with open(abifile, "w") as f:
+ f.write(current_abi)
+ elif int(abi) <= 11 and current_abi == "12":
+ status.addresult("The layout of TMPDIR changed for Recipe Specific Sysroots.\nConversion doesn't make sense and this change will rebuild everything so please delete TMPDIR (%s).\n" % d.getVar("TMPDIR"))
+ elif int(abi) <= 13 and current_abi == "14":
+ status.addresult("TMPDIR changed to include path filtering from the pseudo database.\nIt is recommended to use a clean TMPDIR with the new pseudo path filtering so TMPDIR (%s) would need to be removed to continue.\n" % d.getVar("TMPDIR"))
+ elif int(abi) == 14 and current_abi == "15":
+ drop_v14_cross_builds(d)
+ with open(abifile, "w") as f:
+ f.write(current_abi)
+ elif (abi != current_abi):
+ # Code to convert from one ABI to another could go here if possible.
+ status.addresult("Error, TMPDIR has changed its layout version number (%s to %s) and you need to either rebuild, revert or adjust it at your own risk.\n" % (abi, current_abi))
+ else:
+ with open(abifile, "w") as f:
+ f.write(current_abi)
+
+def check_sanity_sstate_dir_change(sstate_dir, data):
+ # Sanity checks to be done when the value of SSTATE_DIR changes
+
+ # Check that SSTATE_DIR isn't on a filesystem with limited filename length (eg. eCryptFS)
+ testmsg = ""
+ if sstate_dir != "":
+ testmsg = check_create_long_filename(sstate_dir, "SSTATE_DIR")
+ # If we don't have permissions to SSTATE_DIR, suggest the user set it as an SSTATE_MIRRORS
+ try:
+ err = testmsg.split(': ')[1].strip()
+ if err == "Permission denied.":
+ testmsg = testmsg + "You could try using %s in SSTATE_MIRRORS rather than as an SSTATE_CACHE.\n" % (sstate_dir)
+ except IndexError:
+ pass
+ return testmsg
+
+def check_sanity_version_change(status, d):
+ # Sanity checks to be done when SANITY_VERSION or NATIVELSBSTRING changes
+ # In other words, these tests run once in a given build directory and then
+ # never again until the sanity version or host distrubution id/version changes.
+
+ # Check the python install is complete. Examples that are often removed in
+ # minimal installations: glib-2.0-natives requries # xml.parsers.expat and icu
+ # requires distutils.sysconfig.
+ try:
+ import xml.parsers.expat
+ import distutils.sysconfig
+ except ImportError as e:
+ status.addresult('Your Python 3 is not a full install. Please install the module %s (see the Getting Started guide for further information).\n' % e.name)
+
+ status.addresult(check_gcc_version(d))
+ status.addresult(check_make_version(d))
+ status.addresult(check_patch_version(d))
+ status.addresult(check_tar_version(d))
+ status.addresult(check_git_version(d))
+ status.addresult(check_perl_modules(d))
+ status.addresult(check_wsl(d))
+
+ missing = ""
+
+ if not check_app_exists("${MAKE}", d):
+ missing = missing + "GNU make,"
+
+ if not check_app_exists('${BUILD_CC}', d):
+ missing = missing + "C Compiler (%s)," % d.getVar("BUILD_CC")
+
+ if not check_app_exists('${BUILD_CXX}', d):
+ missing = missing + "C++ Compiler (%s)," % d.getVar("BUILD_CXX")
+
+ required_utilities = d.getVar('SANITY_REQUIRED_UTILITIES')
+
+ for util in required_utilities.split():
+ if not check_app_exists(util, d):
+ missing = missing + "%s," % util
+
+ if missing:
+ missing = missing.rstrip(',')
+ status.addresult("Please install the following missing utilities: %s\n" % missing)
+
+ assume_provided = d.getVar('ASSUME_PROVIDED').split()
+ # Check user doesn't have ASSUME_PROVIDED = instead of += in local.conf
+ if "diffstat-native" not in assume_provided:
+ status.addresult('Please use ASSUME_PROVIDED +=, not ASSUME_PROVIDED = in your local.conf\n')
+
+ # Check that TMPDIR isn't on a filesystem with limited filename length (eg. eCryptFS)
+ import stat
+ tmpdir = d.getVar('TMPDIR')
+ status.addresult(check_create_long_filename(tmpdir, "TMPDIR"))
+ tmpdirmode = os.stat(tmpdir).st_mode
+ if (tmpdirmode & stat.S_ISGID):
+ status.addresult("TMPDIR is setgid, please don't build in a setgid directory")
+ if (tmpdirmode & stat.S_ISUID):
+ status.addresult("TMPDIR is setuid, please don't build in a setuid directory")
+
+ # Check that a user isn't building in a path in PSEUDO_IGNORE_PATHS
+ pseudoignorepaths = d.getVar('PSEUDO_IGNORE_PATHS', expand=True).split(",")
+ workdir = d.getVar('WORKDIR', expand=True)
+ for i in pseudoignorepaths:
+ if i and workdir.startswith(i):
+ status.addresult("You are building in a path included in PSEUDO_IGNORE_PATHS " + str(i) + " please locate the build outside this path.\n")
+
+ # Check if PSEUDO_IGNORE_PATHS and and paths under pseudo control overlap
+ pseudoignorepaths = d.getVar('PSEUDO_IGNORE_PATHS', expand=True).split(",")
+ pseudo_control_dir = "${D},${PKGD},${PKGDEST},${IMAGEROOTFS},${SDK_OUTPUT}"
+ pseudocontroldir = d.expand(pseudo_control_dir).split(",")
+ for i in pseudoignorepaths:
+ for j in pseudocontroldir:
+ if i and j:
+ if j.startswith(i):
+ status.addresult("A path included in PSEUDO_IGNORE_PATHS " + str(i) + " and the path " + str(j) + " overlap and this will break pseudo permission and ownership tracking. Please set the path " + str(j) + " to a different directory which does not overlap with pseudo controlled directories. \n")
+
+ # Some third-party software apparently relies on chmod etc. being suid root (!!)
+ import stat
+ suid_check_bins = "chown chmod mknod".split()
+ for bin_cmd in suid_check_bins:
+ bin_path = bb.utils.which(os.environ["PATH"], bin_cmd)
+ if bin_path:
+ bin_stat = os.stat(bin_path)
+ if bin_stat.st_uid == 0 and bin_stat.st_mode & stat.S_ISUID:
+ status.addresult('%s has the setuid bit set. This interferes with pseudo and may cause other issues that break the build process.\n' % bin_path)
+
+ # Check that we can fetch from various network transports
+ netcheck = check_connectivity(d)
+ status.addresult(netcheck)
+ if netcheck:
+ status.network_error = True
+
+ nolibs = d.getVar('NO32LIBS')
+ if not nolibs:
+ lib32path = '/lib'
+ if os.path.exists('/lib64') and ( os.path.islink('/lib64') or os.path.islink('/lib') ):
+ lib32path = '/lib32'
+
+ if os.path.exists('%s/libc.so.6' % lib32path) and not os.path.exists('/usr/include/gnu/stubs-32.h'):
+ status.addresult("You have a 32-bit libc, but no 32-bit headers. You must install the 32-bit libc headers.\n")
+
+ bbpaths = d.getVar('BBPATH').split(":")
+ if ("." in bbpaths or "./" in bbpaths or "" in bbpaths):
+ status.addresult("BBPATH references the current directory, either through " \
+ "an empty entry, a './' or a '.'.\n\t This is unsafe and means your "\
+ "layer configuration is adding empty elements to BBPATH.\n\t "\
+ "Please check your layer.conf files and other BBPATH " \
+ "settings to remove the current working directory " \
+ "references.\n" \
+ "Parsed BBPATH is" + str(bbpaths));
+
+ oes_bb_conf = d.getVar( 'OES_BITBAKE_CONF')
+ if not oes_bb_conf:
+ status.addresult('You are not using the OpenEmbedded version of conf/bitbake.conf. This means your environment is misconfigured, in particular check BBPATH.\n')
+
+ # The length of TMPDIR can't be longer than 410
+ status.addresult(check_path_length(tmpdir, "TMPDIR", 410))
+
+ # Check that TMPDIR isn't located on nfs
+ status.addresult(check_not_nfs(tmpdir, "TMPDIR"))
+
+ # Check for case-insensitive file systems (such as Linux in Docker on
+ # macOS with default HFS+ file system)
+ status.addresult(check_case_sensitive(tmpdir, "TMPDIR"))
+
+def sanity_check_locale(d):
+ """
+ Currently bitbake switches locale to en_US.UTF-8 so check that this locale actually exists.
+ """
+ import locale
+ try:
+ locale.setlocale(locale.LC_ALL, "en_US.UTF-8")
+ except locale.Error:
+ raise_sanity_error("Your system needs to support the en_US.UTF-8 locale.", d)
+
+def check_sanity_everybuild(status, d):
+ import os, stat
+ # Sanity tests which test the users environment so need to run at each build (or are so cheap
+ # it makes sense to always run them.
+
+ if 0 == os.getuid():
+ raise_sanity_error("Do not use Bitbake as root.", d)
+
+ # Check the Python version, we now have a minimum of Python 3.6
+ import sys
+ if sys.hexversion < 0x030600F0:
+ status.addresult('The system requires at least Python 3.6 to run. Please update your Python interpreter.\n')
+
+ # Check the bitbake version meets minimum requirements
+ minversion = d.getVar('BB_MIN_VERSION')
+ if bb.utils.vercmp_string_op(bb.__version__, minversion, "<"):
+ status.addresult('Bitbake version %s is required and version %s was found\n' % (minversion, bb.__version__))
+
+ sanity_check_locale(d)
+
+ paths = d.getVar('PATH').split(":")
+ if "." in paths or "./" in paths or "" in paths:
+ status.addresult("PATH contains '.', './' or '' (empty element), which will break the build, please remove this.\nParsed PATH is " + str(paths) + "\n")
+
+ #Check if bitbake is present in PATH environment variable
+ bb_check = bb.utils.which(d.getVar('PATH'), 'bitbake')
+ if not bb_check:
+ bb.warn("bitbake binary is not found in PATH, did you source the script?")
+
+ # Check whether 'inherit' directive is found (used for a class to inherit)
+ # in conf file it's supposed to be uppercase INHERIT
+ inherit = d.getVar('inherit')
+ if inherit:
+ status.addresult("Please don't use inherit directive in your local.conf. The directive is supposed to be used in classes and recipes only to inherit of bbclasses. Here INHERIT should be used.\n")
+
+ # Check that the DISTRO is valid, if set
+ # need to take into account DISTRO renaming DISTRO
+ distro = d.getVar('DISTRO')
+ if distro and distro != "nodistro":
+ if not ( check_conf_exists("conf/distro/${DISTRO}.conf", d) or check_conf_exists("conf/distro/include/${DISTRO}.inc", d) ):
+ status.addresult("DISTRO '%s' not found. Please set a valid DISTRO in your local.conf\n" % d.getVar("DISTRO"))
+
+ # Check that these variables don't use tilde-expansion as we don't do that
+ for v in ("TMPDIR", "DL_DIR", "SSTATE_DIR"):
+ if d.getVar(v).startswith("~"):
+ status.addresult("%s uses ~ but Bitbake will not expand this, use an absolute path or variables." % v)
+
+ # Check that DL_DIR is set, exists and is writable. In theory, we should never even hit the check if DL_DIR isn't
+ # set, since so much relies on it being set.
+ dldir = d.getVar('DL_DIR')
+ if not dldir:
+ status.addresult("DL_DIR is not set. Your environment is misconfigured, check that DL_DIR is set, and if the directory exists, that it is writable. \n")
+ if os.path.exists(dldir) and not os.access(dldir, os.W_OK):
+ status.addresult("DL_DIR: %s exists but you do not appear to have write access to it. \n" % dldir)
+ check_symlink(dldir, d)
+
+ # Check that the MACHINE is valid, if it is set
+ machinevalid = True
+ if d.getVar('MACHINE'):
+ if not check_conf_exists("conf/machine/${MACHINE}.conf", d):
+ status.addresult('MACHINE=%s is invalid. Please set a valid MACHINE in your local.conf, environment or other configuration file.\n' % (d.getVar('MACHINE')))
+ machinevalid = False
+ else:
+ status.addresult(check_sanity_validmachine(d))
+ else:
+ status.addresult('Please set a MACHINE in your local.conf or environment\n')
+ machinevalid = False
+ if machinevalid:
+ status.addresult(check_toolchain(d))
+
+ # Check that the SDKMACHINE is valid, if it is set
+ if d.getVar('SDKMACHINE'):
+ if not check_conf_exists("conf/machine-sdk/${SDKMACHINE}.conf", d):
+ status.addresult('Specified SDKMACHINE value is not valid\n')
+ elif d.getVar('SDK_ARCH', False) == "${BUILD_ARCH}":
+ status.addresult('SDKMACHINE is set, but SDK_ARCH has not been changed as a result - SDKMACHINE may have been set too late (e.g. in the distro configuration)\n')
+
+ # If SDK_VENDOR looks like "-my-sdk" then the triples are badly formed so fail early
+ sdkvendor = d.getVar("SDK_VENDOR")
+ if not (sdkvendor.startswith("-") and sdkvendor.count("-") == 1):
+ status.addresult("SDK_VENDOR should be of the form '-foosdk' with a single dash; found '%s'\n" % sdkvendor)
+
+ check_supported_distro(d)
+
+ omask = os.umask(0o022)
+ if omask & 0o755:
+ status.addresult("Please use a umask which allows a+rx and u+rwx\n")
+ os.umask(omask)
+
+ if d.getVar('TARGET_ARCH') == "arm":
+ # This path is no longer user-readable in modern (very recent) Linux
+ try:
+ if os.path.exists("/proc/sys/vm/mmap_min_addr"):
+ f = open("/proc/sys/vm/mmap_min_addr", "r")
+ try:
+ if (int(f.read().strip()) > 65536):
+ status.addresult("/proc/sys/vm/mmap_min_addr is not <= 65536. This will cause problems with qemu so please fix the value (as root).\n\nTo fix this in later reboots, set vm.mmap_min_addr = 65536 in /etc/sysctl.conf.\n")
+ finally:
+ f.close()
+ except:
+ pass
+
+ for checkdir in ['COREBASE', 'TMPDIR']:
+ val = d.getVar(checkdir)
+ if val.find('..') != -1:
+ status.addresult("Error, you have '..' in your %s directory path. Please ensure the variable contains an absolute path as this can break some recipe builds in obtuse ways." % checkdir)
+ if val.find('+') != -1:
+ status.addresult("Error, you have an invalid character (+) in your %s directory path. Please move the installation to a directory which doesn't include any + characters." % checkdir)
+ if val.find('@') != -1:
+ status.addresult("Error, you have an invalid character (@) in your %s directory path. Please move the installation to a directory which doesn't include any @ characters." % checkdir)
+ if val.find(' ') != -1:
+ status.addresult("Error, you have a space in your %s directory path. Please move the installation to a directory which doesn't include a space since autotools doesn't support this." % checkdir)
+ if val.find('%') != -1:
+ status.addresult("Error, you have an invalid character (%) in your %s directory path which causes problems with python string formatting. Please move the installation to a directory which doesn't include any % characters." % checkdir)
+
+ # Check the format of MIRRORS, PREMIRRORS and SSTATE_MIRRORS
+ import re
+ mirror_vars = ['MIRRORS', 'PREMIRRORS', 'SSTATE_MIRRORS']
+ protocols = ['http', 'ftp', 'file', 'https', \
+ 'git', 'gitsm', 'hg', 'osc', 'p4', 'svn', \
+ 'bzr', 'cvs', 'npm', 'sftp', 'ssh', 's3', 'az', 'ftps']
+ for mirror_var in mirror_vars:
+ mirrors = (d.getVar(mirror_var) or '').replace('\\n', ' ').split()
+
+ # Split into pairs
+ if len(mirrors) % 2 != 0:
+ bb.warn('Invalid mirror variable value for %s: %s, should contain paired members.' % (mirror_var, str(mirrors)))
+ continue
+ mirrors = list(zip(*[iter(mirrors)]*2))
+
+ for mirror_entry in mirrors:
+ pattern, mirror = mirror_entry
+
+ decoded = bb.fetch2.decodeurl(pattern)
+ try:
+ pattern_scheme = re.compile(decoded[0])
+ except re.error as exc:
+ bb.warn('Invalid scheme regex (%s) in %s; %s' % (pattern, mirror_var, mirror_entry))
+ continue
+
+ if not any(pattern_scheme.match(protocol) for protocol in protocols):
+ bb.warn('Invalid protocol (%s) in %s: %s' % (decoded[0], mirror_var, mirror_entry))
+ continue
+
+ if not any(mirror.startswith(protocol + '://') for protocol in protocols):
+ bb.warn('Invalid protocol in %s: %s' % (mirror_var, mirror_entry))
+ continue
+
+ if mirror.startswith('file://'):
+ import urllib
+ check_symlink(urllib.parse.urlparse(mirror).path, d)
+ # SSTATE_MIRROR ends with a /PATH string
+ if mirror.endswith('/PATH'):
+ # remove /PATH$ from SSTATE_MIRROR to get a working
+ # base directory path
+ mirror_base = urllib.parse.urlparse(mirror[:-1*len('/PATH')]).path
+ check_symlink(mirror_base, d)
+
+ # Check sstate mirrors aren't being used with a local hash server and no remote
+ hashserv = d.getVar("BB_HASHSERVE")
+ if d.getVar("SSTATE_MIRRORS") and hashserv and hashserv.startswith("unix://") and not d.getVar("BB_HASHSERVE_UPSTREAM"):
+ bb.warn("You are using a local hash equivalence server but have configured an sstate mirror. This will likely mean no sstate will match from the mirror. You may wish to disable the hash equivalence use (BB_HASHSERVE), or use a hash equivalence server alongside the sstate mirror.")
+
+ # Check that TMPDIR hasn't changed location since the last time we were run
+ tmpdir = d.getVar('TMPDIR')
+ checkfile = os.path.join(tmpdir, "saved_tmpdir")
+ if os.path.exists(checkfile):
+ with open(checkfile, "r") as f:
+ saved_tmpdir = f.read().strip()
+ if (saved_tmpdir != tmpdir):
+ status.addresult("Error, TMPDIR has changed location. You need to either move it back to %s or delete it and rebuild\n" % saved_tmpdir)
+ else:
+ bb.utils.mkdirhier(tmpdir)
+ # Remove setuid, setgid and sticky bits from TMPDIR
+ try:
+ os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISUID)
+ os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISGID)
+ os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISVTX)
+ except OSError as exc:
+ bb.warn("Unable to chmod TMPDIR: %s" % exc)
+ with open(checkfile, "w") as f:
+ f.write(tmpdir)
+
+ # If /bin/sh is a symlink, check that it points to dash or bash
+ if os.path.islink('/bin/sh'):
+ real_sh = os.path.realpath('/bin/sh')
+ # Due to update-alternatives, the shell name may take various
+ # forms, such as /bin/dash, bin/bash, /bin/bash.bash ...
+ if '/dash' not in real_sh and '/bash' not in real_sh:
+ status.addresult("Error, /bin/sh links to %s, must be dash or bash\n" % real_sh)
+
+def check_sanity(sanity_data):
+ class SanityStatus(object):
+ def __init__(self):
+ self.messages = ""
+ self.network_error = False
+
+ def addresult(self, message):
+ if message:
+ self.messages = self.messages + message
+
+ status = SanityStatus()
+
+ tmpdir = sanity_data.getVar('TMPDIR')
+ sstate_dir = sanity_data.getVar('SSTATE_DIR')
+
+ check_symlink(sstate_dir, sanity_data)
+
+ # Check saved sanity info
+ last_sanity_version = 0
+ last_tmpdir = ""
+ last_sstate_dir = ""
+ last_nativelsbstr = ""
+ sanityverfile = sanity_data.expand("${TOPDIR}/cache/sanity_info")
+ if os.path.exists(sanityverfile):
+ with open(sanityverfile, 'r') as f:
+ for line in f:
+ if line.startswith('SANITY_VERSION'):
+ last_sanity_version = int(line.split()[1])
+ if line.startswith('TMPDIR'):
+ last_tmpdir = line.split()[1]
+ if line.startswith('SSTATE_DIR'):
+ last_sstate_dir = line.split()[1]
+ if line.startswith('NATIVELSBSTRING'):
+ last_nativelsbstr = line.split()[1]
+
+ check_sanity_everybuild(status, sanity_data)
+
+ sanity_version = int(sanity_data.getVar('SANITY_VERSION') or 1)
+ network_error = False
+ # NATIVELSBSTRING var may have been overridden with "universal", so
+ # get actual host distribution id and version
+ nativelsbstr = lsb_distro_identifier(sanity_data)
+ if last_sanity_version < sanity_version or last_nativelsbstr != nativelsbstr:
+ check_sanity_version_change(status, sanity_data)
+ status.addresult(check_sanity_sstate_dir_change(sstate_dir, sanity_data))
+ else:
+ if last_sstate_dir != sstate_dir:
+ status.addresult(check_sanity_sstate_dir_change(sstate_dir, sanity_data))
+
+ if os.path.exists(os.path.dirname(sanityverfile)) and not status.messages:
+ with open(sanityverfile, 'w') as f:
+ f.write("SANITY_VERSION %s\n" % sanity_version)
+ f.write("TMPDIR %s\n" % tmpdir)
+ f.write("SSTATE_DIR %s\n" % sstate_dir)
+ f.write("NATIVELSBSTRING %s\n" % nativelsbstr)
+
+ sanity_handle_abichanges(status, sanity_data)
+
+ if status.messages != "":
+ raise_sanity_error(sanity_data.expand(status.messages), sanity_data, status.network_error)
+
+# Create a copy of the datastore and finalise it to ensure appends and
+# overrides are set - the datastore has yet to be finalised at ConfigParsed
+def copy_data(e):
+ sanity_data = bb.data.createCopy(e.data)
+ sanity_data.finalize()
+ return sanity_data
+
+addhandler config_reparse_eventhandler
+config_reparse_eventhandler[eventmask] = "bb.event.ConfigParsed"
+python config_reparse_eventhandler() {
+ sanity_check_conffiles(e.data)
+}
+
+addhandler check_sanity_eventhandler
+check_sanity_eventhandler[eventmask] = "bb.event.SanityCheck bb.event.NetworkTest"
+python check_sanity_eventhandler() {
+ if bb.event.getName(e) == "SanityCheck":
+ sanity_data = copy_data(e)
+ check_sanity(sanity_data)
+ if e.generateevents:
+ sanity_data.setVar("SANITY_USE_EVENTS", "1")
+ bb.event.fire(bb.event.SanityCheckPassed(), e.data)
+ elif bb.event.getName(e) == "NetworkTest":
+ sanity_data = copy_data(e)
+ if e.generateevents:
+ sanity_data.setVar("SANITY_USE_EVENTS", "1")
+ bb.event.fire(bb.event.NetworkTestFailed() if check_connectivity(sanity_data) else bb.event.NetworkTestPassed(), e.data)
+
+ return
+}
diff --git a/poky/meta/classes-global/sstate.bbclass b/poky/meta/classes-global/sstate.bbclass
new file mode 100644
index 0000000..cd77c58
--- /dev/null
+++ b/poky/meta/classes-global/sstate.bbclass
@@ -0,0 +1,1364 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+SSTATE_VERSION = "10"
+
+SSTATE_ZSTD_CLEVEL ??= "8"
+
+SSTATE_MANIFESTS ?= "${TMPDIR}/sstate-control"
+SSTATE_MANFILEPREFIX = "${SSTATE_MANIFESTS}/manifest-${SSTATE_MANMACH}-${PN}"
+
+def generate_sstatefn(spec, hash, taskname, siginfo, d):
+ if taskname is None:
+ return ""
+ extension = ".tar.zst"
+ # 8 chars reserved for siginfo
+ limit = 254 - 8
+ if siginfo:
+ limit = 254
+ extension = ".tar.zst.siginfo"
+ if not hash:
+ hash = "INVALID"
+ fn = spec + hash + "_" + taskname + extension
+ # If the filename is too long, attempt to reduce it
+ if len(fn) > limit:
+ components = spec.split(":")
+ # Fields 0,5,6 are mandatory, 1 is most useful, 2,3,4 are just for information
+ # 7 is for the separators
+ avail = (limit - len(hash + "_" + taskname + extension) - len(components[0]) - len(components[1]) - len(components[5]) - len(components[6]) - 7) // 3
+ components[2] = components[2][:avail]
+ components[3] = components[3][:avail]
+ components[4] = components[4][:avail]
+ spec = ":".join(components)
+ fn = spec + hash + "_" + taskname + extension
+ if len(fn) > limit:
+ bb.fatal("Unable to reduce sstate name to less than 255 chararacters")
+ return hash[:2] + "/" + hash[2:4] + "/" + fn
+
+SSTATE_PKGARCH = "${PACKAGE_ARCH}"
+SSTATE_PKGSPEC = "sstate:${PN}:${PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}:${PV}:${PR}:${SSTATE_PKGARCH}:${SSTATE_VERSION}:"
+SSTATE_SWSPEC = "sstate:${PN}::${PV}:${PR}::${SSTATE_VERSION}:"
+SSTATE_PKGNAME = "${SSTATE_EXTRAPATH}${@generate_sstatefn(d.getVar('SSTATE_PKGSPEC'), d.getVar('BB_UNIHASH'), d.getVar('SSTATE_CURRTASK'), False, d)}"
+SSTATE_PKG = "${SSTATE_DIR}/${SSTATE_PKGNAME}"
+SSTATE_EXTRAPATH = ""
+SSTATE_EXTRAPATHWILDCARD = ""
+SSTATE_PATHSPEC = "${SSTATE_DIR}/${SSTATE_EXTRAPATHWILDCARD}*/*/${SSTATE_PKGSPEC}*_${SSTATE_PATH_CURRTASK}.tar.zst*"
+
+# explicitly make PV to depend on evaluated value of PV variable
+PV[vardepvalue] = "${PV}"
+
+# We don't want the sstate to depend on things like the distro string
+# of the system, we let the sstate paths take care of this.
+SSTATE_EXTRAPATH[vardepvalue] = ""
+SSTATE_EXTRAPATHWILDCARD[vardepvalue] = ""
+
+# For multilib rpm the allarch packagegroup files can overwrite (in theory they're identical)
+SSTATE_ALLOW_OVERLAP_FILES = "${DEPLOY_DIR}/licenses/"
+# Avoid docbook/sgml catalog warnings for now
+SSTATE_ALLOW_OVERLAP_FILES += "${STAGING_ETCDIR_NATIVE}/sgml ${STAGING_DATADIR_NATIVE}/sgml"
+# sdk-provides-dummy-nativesdk and nativesdk-buildtools-perl-dummy overlap for different SDKMACHINE
+SSTATE_ALLOW_OVERLAP_FILES += "${DEPLOY_DIR_RPM}/sdk_provides_dummy_nativesdk/ ${DEPLOY_DIR_IPK}/sdk-provides-dummy-nativesdk/"
+SSTATE_ALLOW_OVERLAP_FILES += "${DEPLOY_DIR_RPM}/buildtools_dummy_nativesdk/ ${DEPLOY_DIR_IPK}/buildtools-dummy-nativesdk/"
+# target-sdk-provides-dummy overlaps that allarch is disabled when multilib is used
+SSTATE_ALLOW_OVERLAP_FILES += "${COMPONENTS_DIR}/sdk-provides-dummy-target/ ${DEPLOY_DIR_RPM}/sdk_provides_dummy_target/ ${DEPLOY_DIR_IPK}/sdk-provides-dummy-target/"
+# Archive the sources for many architectures in one deploy folder
+SSTATE_ALLOW_OVERLAP_FILES += "${DEPLOY_DIR_SRC}"
+# ovmf/grub-efi/systemd-boot/intel-microcode multilib recipes can generate identical overlapping files
+SSTATE_ALLOW_OVERLAP_FILES += "${DEPLOY_DIR_IMAGE}/ovmf"
+SSTATE_ALLOW_OVERLAP_FILES += "${DEPLOY_DIR_IMAGE}/grub-efi"
+SSTATE_ALLOW_OVERLAP_FILES += "${DEPLOY_DIR_IMAGE}/systemd-boot"
+SSTATE_ALLOW_OVERLAP_FILES += "${DEPLOY_DIR_IMAGE}/microcode"
+
+SSTATE_SCAN_FILES ?= "*.la *-config *_config postinst-*"
+SSTATE_SCAN_CMD ??= 'find ${SSTATE_BUILDDIR} \( -name "${@"\" -o -name \"".join(d.getVar("SSTATE_SCAN_FILES").split())}" \) -type f'
+SSTATE_SCAN_CMD_NATIVE ??= 'grep -Irl -e ${RECIPE_SYSROOT} -e ${RECIPE_SYSROOT_NATIVE} -e ${HOSTTOOLS_DIR} ${SSTATE_BUILDDIR}'
+SSTATE_HASHEQUIV_FILEMAP ?= " \
+ populate_sysroot:*/postinst-useradd-*:${TMPDIR} \
+ populate_sysroot:*/postinst-useradd-*:${COREBASE} \
+ populate_sysroot:*/postinst-useradd-*:regex-\s(PATH|PSEUDO_IGNORE_PATHS|HOME|LOGNAME|OMP_NUM_THREADS|USER)=.*\s \
+ populate_sysroot:*/crossscripts/*:${TMPDIR} \
+ populate_sysroot:*/crossscripts/*:${COREBASE} \
+ "
+
+BB_HASHFILENAME = "False ${SSTATE_PKGSPEC} ${SSTATE_SWSPEC}"
+
+SSTATE_ARCHS = " \
+ ${BUILD_ARCH} \
+ ${BUILD_ARCH}_${ORIGNATIVELSBSTRING} \
+ ${BUILD_ARCH}_${SDK_ARCH}_${SDK_OS} \
+ ${SDK_ARCH}_${SDK_OS} \
+ ${SDK_ARCH}_${PACKAGE_ARCH} \
+ allarch \
+ ${PACKAGE_ARCH} \
+ ${PACKAGE_EXTRA_ARCHS} \
+ ${MACHINE_ARCH}"
+SSTATE_ARCHS[vardepsexclude] = "ORIGNATIVELSBSTRING"
+
+SSTATE_MANMACH ?= "${SSTATE_PKGARCH}"
+
+SSTATECREATEFUNCS += "sstate_hardcode_path"
+SSTATECREATEFUNCS[vardeps] = "SSTATE_SCAN_FILES"
+SSTATEPOSTCREATEFUNCS = ""
+SSTATEPREINSTFUNCS = ""
+SSTATEPOSTUNPACKFUNCS = "sstate_hardcode_path_unpack"
+SSTATEPOSTINSTFUNCS = ""
+EXTRA_STAGING_FIXMES ?= "HOSTTOOLS_DIR"
+
+# Check whether sstate exists for tasks that support sstate and are in the
+# locked signatures file.
+SIGGEN_LOCKEDSIGS_SSTATE_EXISTS_CHECK ?= 'error'
+
+# Check whether the task's computed hash matches the task's hash in the
+# locked signatures file.
+SIGGEN_LOCKEDSIGS_TASKSIG_CHECK ?= "error"
+
+# The GnuPG key ID and passphrase to use to sign sstate archives (or unset to
+# not sign)
+SSTATE_SIG_KEY ?= ""
+SSTATE_SIG_PASSPHRASE ?= ""
+# Whether to verify the GnUPG signatures when extracting sstate archives
+SSTATE_VERIFY_SIG ?= "0"
+# List of signatures to consider valid.
+SSTATE_VALID_SIGS ??= ""
+SSTATE_VALID_SIGS[vardepvalue] = ""
+
+SSTATE_HASHEQUIV_METHOD ?= "oe.sstatesig.OEOuthashBasic"
+SSTATE_HASHEQUIV_METHOD[doc] = "The fully-qualified function used to calculate \
+ the output hash for a task, which in turn is used to determine equivalency. \
+ "
+
+SSTATE_HASHEQUIV_REPORT_TASKDATA ?= "0"
+SSTATE_HASHEQUIV_REPORT_TASKDATA[doc] = "Report additional useful data to the \
+ hash equivalency server, such as PN, PV, taskname, etc. This information \
+ is very useful for developers looking at task data, but may leak sensitive \
+ data if the equivalence server is public. \
+ "
+
+python () {
+ if bb.data.inherits_class('native', d):
+ d.setVar('SSTATE_PKGARCH', d.getVar('BUILD_ARCH', False))
+ elif bb.data.inherits_class('crosssdk', d):
+ d.setVar('SSTATE_PKGARCH', d.expand("${BUILD_ARCH}_${SDK_ARCH}_${SDK_OS}"))
+ elif bb.data.inherits_class('cross', d):
+ d.setVar('SSTATE_PKGARCH', d.expand("${BUILD_ARCH}"))
+ elif bb.data.inherits_class('nativesdk', d):
+ d.setVar('SSTATE_PKGARCH', d.expand("${SDK_ARCH}_${SDK_OS}"))
+ elif bb.data.inherits_class('cross-canadian', d):
+ d.setVar('SSTATE_PKGARCH', d.expand("${SDK_ARCH}_${PACKAGE_ARCH}"))
+ elif bb.data.inherits_class('allarch', d) and d.getVar("PACKAGE_ARCH") == "all":
+ d.setVar('SSTATE_PKGARCH', "allarch")
+ else:
+ d.setVar('SSTATE_MANMACH', d.expand("${PACKAGE_ARCH}"))
+
+ if bb.data.inherits_class('native', d) or bb.data.inherits_class('crosssdk', d) or bb.data.inherits_class('cross', d):
+ d.setVar('SSTATE_EXTRAPATH', "${NATIVELSBSTRING}/")
+ d.setVar('BB_HASHFILENAME', "True ${SSTATE_PKGSPEC} ${SSTATE_SWSPEC}")
+ d.setVar('SSTATE_EXTRAPATHWILDCARD', "${NATIVELSBSTRING}/")
+
+ unique_tasks = sorted(set((d.getVar('SSTATETASKS') or "").split()))
+ d.setVar('SSTATETASKS', " ".join(unique_tasks))
+ for task in unique_tasks:
+ d.prependVarFlag(task, 'prefuncs', "sstate_task_prefunc ")
+ d.appendVarFlag(task, 'postfuncs', " sstate_task_postfunc")
+ d.setVarFlag(task, 'network', '1')
+ d.setVarFlag(task + "_setscene", 'network', '1')
+}
+
+def sstate_init(task, d):
+ ss = {}
+ ss['task'] = task
+ ss['dirs'] = []
+ ss['plaindirs'] = []
+ ss['lockfiles'] = []
+ ss['lockfiles-shared'] = []
+ return ss
+
+def sstate_state_fromvars(d, task = None):
+ if task is None:
+ task = d.getVar('BB_CURRENTTASK')
+ if not task:
+ bb.fatal("sstate code running without task context?!")
+ task = task.replace("_setscene", "")
+
+ if task.startswith("do_"):
+ task = task[3:]
+ inputs = (d.getVarFlag("do_" + task, 'sstate-inputdirs') or "").split()
+ outputs = (d.getVarFlag("do_" + task, 'sstate-outputdirs') or "").split()
+ plaindirs = (d.getVarFlag("do_" + task, 'sstate-plaindirs') or "").split()
+ lockfiles = (d.getVarFlag("do_" + task, 'sstate-lockfile') or "").split()
+ lockfilesshared = (d.getVarFlag("do_" + task, 'sstate-lockfile-shared') or "").split()
+ interceptfuncs = (d.getVarFlag("do_" + task, 'sstate-interceptfuncs') or "").split()
+ fixmedir = d.getVarFlag("do_" + task, 'sstate-fixmedir') or ""
+ if not task or len(inputs) != len(outputs):
+ bb.fatal("sstate variables not setup correctly?!")
+
+ if task == "populate_lic":
+ d.setVar("SSTATE_PKGSPEC", "${SSTATE_SWSPEC}")
+ d.setVar("SSTATE_EXTRAPATH", "")
+ d.setVar('SSTATE_EXTRAPATHWILDCARD', "")
+
+ ss = sstate_init(task, d)
+ for i in range(len(inputs)):
+ sstate_add(ss, inputs[i], outputs[i], d)
+ ss['lockfiles'] = lockfiles
+ ss['lockfiles-shared'] = lockfilesshared
+ ss['plaindirs'] = plaindirs
+ ss['interceptfuncs'] = interceptfuncs
+ ss['fixmedir'] = fixmedir
+ return ss
+
+def sstate_add(ss, source, dest, d):
+ if not source.endswith("/"):
+ source = source + "/"
+ if not dest.endswith("/"):
+ dest = dest + "/"
+ source = os.path.normpath(source)
+ dest = os.path.normpath(dest)
+ srcbase = os.path.basename(source)
+ ss['dirs'].append([srcbase, source, dest])
+ return ss
+
+def sstate_install(ss, d):
+ import oe.path
+ import oe.sstatesig
+ import subprocess
+
+ sharedfiles = []
+ shareddirs = []
+ bb.utils.mkdirhier(d.expand("${SSTATE_MANIFESTS}"))
+
+ sstateinst = d.expand("${WORKDIR}/sstate-install-%s/" % ss['task'])
+
+ manifest, d2 = oe.sstatesig.sstate_get_manifest_filename(ss['task'], d)
+
+ if os.access(manifest, os.R_OK):
+ bb.fatal("Package already staged (%s)?!" % manifest)
+
+ d.setVar("SSTATE_INST_POSTRM", manifest + ".postrm")
+
+ locks = []
+ for lock in ss['lockfiles-shared']:
+ locks.append(bb.utils.lockfile(lock, True))
+ for lock in ss['lockfiles']:
+ locks.append(bb.utils.lockfile(lock))
+
+ for state in ss['dirs']:
+ bb.debug(2, "Staging files from %s to %s" % (state[1], state[2]))
+ for walkroot, dirs, files in os.walk(state[1]):
+ for file in files:
+ srcpath = os.path.join(walkroot, file)
+ dstpath = srcpath.replace(state[1], state[2])
+ #bb.debug(2, "Staging %s to %s" % (srcpath, dstpath))
+ sharedfiles.append(dstpath)
+ for dir in dirs:
+ srcdir = os.path.join(walkroot, dir)
+ dstdir = srcdir.replace(state[1], state[2])
+ #bb.debug(2, "Staging %s to %s" % (srcdir, dstdir))
+ if os.path.islink(srcdir):
+ sharedfiles.append(dstdir)
+ continue
+ if not dstdir.endswith("/"):
+ dstdir = dstdir + "/"
+ shareddirs.append(dstdir)
+
+ # Check the file list for conflicts against files which already exist
+ overlap_allowed = (d.getVar("SSTATE_ALLOW_OVERLAP_FILES") or "").split()
+ match = []
+ for f in sharedfiles:
+ if os.path.exists(f) and not os.path.islink(f):
+ f = os.path.normpath(f)
+ realmatch = True
+ for w in overlap_allowed:
+ w = os.path.normpath(w)
+ if f.startswith(w):
+ realmatch = False
+ break
+ if realmatch:
+ match.append(f)
+ sstate_search_cmd = "grep -rlF '%s' %s --exclude=master.list | sed -e 's:^.*/::'" % (f, d.expand("${SSTATE_MANIFESTS}"))
+ search_output = subprocess.Popen(sstate_search_cmd, shell=True, stdout=subprocess.PIPE).communicate()[0]
+ if search_output:
+ match.append(" (matched in %s)" % search_output.decode('utf-8').rstrip())
+ else:
+ match.append(" (not matched to any task)")
+ if match:
+ bb.error("The recipe %s is trying to install files into a shared " \
+ "area when those files already exist. Those files and their manifest " \
+ "location are:\n %s\nPlease verify which recipe should provide the " \
+ "above files.\n\nThe build has stopped, as continuing in this scenario WILL " \
+ "break things - if not now, possibly in the future (we've seen builds fail " \
+ "several months later). If the system knew how to recover from this " \
+ "automatically it would, however there are several different scenarios " \
+ "which can result in this and we don't know which one this is. It may be " \
+ "you have switched providers of something like virtual/kernel (e.g. from " \
+ "linux-yocto to linux-yocto-dev), in that case you need to execute the " \
+ "clean task for both recipes and it will resolve this error. It may be " \
+ "you changed DISTRO_FEATURES from systemd to udev or vice versa. Cleaning " \
+ "those recipes should again resolve this error, however switching " \
+ "DISTRO_FEATURES on an existing build directory is not supported - you " \
+ "should really clean out tmp and rebuild (reusing sstate should be safe). " \
+ "It could be the overlapping files detected are harmless in which case " \
+ "adding them to SSTATE_ALLOW_OVERLAP_FILES may be the correct solution. It could " \
+ "also be your build is including two different conflicting versions of " \
+ "things (e.g. bluez 4 and bluez 5 and the correct solution for that would " \
+ "be to resolve the conflict. If in doubt, please ask on the mailing list, " \
+ "sharing the error and filelist above." % \
+ (d.getVar('PN'), "\n ".join(match)))
+ bb.fatal("If the above message is too much, the simpler version is you're advised to wipe out tmp and rebuild (reusing sstate is fine). That will likely fix things in most (but not all) cases.")
+
+ if ss['fixmedir'] and os.path.exists(ss['fixmedir'] + "/fixmepath.cmd"):
+ sharedfiles.append(ss['fixmedir'] + "/fixmepath.cmd")
+ sharedfiles.append(ss['fixmedir'] + "/fixmepath")
+
+ # Write out the manifest
+ f = open(manifest, "w")
+ for file in sharedfiles:
+ f.write(file + "\n")
+
+ # We want to ensure that directories appear at the end of the manifest
+ # so that when we test to see if they should be deleted any contents
+ # added by the task will have been removed first.
+ dirs = sorted(shareddirs, key=len)
+ # Must remove children first, which will have a longer path than the parent
+ for di in reversed(dirs):
+ f.write(di + "\n")
+ f.close()
+
+ # Append to the list of manifests for this PACKAGE_ARCH
+
+ i = d2.expand("${SSTATE_MANIFESTS}/index-${SSTATE_MANMACH}")
+ l = bb.utils.lockfile(i + ".lock")
+ filedata = d.getVar("STAMP") + " " + d2.getVar("SSTATE_MANFILEPREFIX") + " " + d.getVar("WORKDIR") + "\n"
+ manifests = []
+ if os.path.exists(i):
+ with open(i, "r") as f:
+ manifests = f.readlines()
+ # We append new entries, we don't remove older entries which may have the same
+ # manifest name but different versions from stamp/workdir. See below.
+ if filedata not in manifests:
+ with open(i, "a+") as f:
+ f.write(filedata)
+ bb.utils.unlockfile(l)
+
+ # Run the actual file install
+ for state in ss['dirs']:
+ if os.path.exists(state[1]):
+ oe.path.copyhardlinktree(state[1], state[2])
+
+ for postinst in (d.getVar('SSTATEPOSTINSTFUNCS') or '').split():
+ # All hooks should run in the SSTATE_INSTDIR
+ bb.build.exec_func(postinst, d, (sstateinst,))
+
+ for lock in locks:
+ bb.utils.unlockfile(lock)
+
+sstate_install[vardepsexclude] += "SSTATE_ALLOW_OVERLAP_FILES STATE_MANMACH SSTATE_MANFILEPREFIX"
+sstate_install[vardeps] += "${SSTATEPOSTINSTFUNCS}"
+
+def sstate_installpkg(ss, d):
+ from oe.gpg_sign import get_signer
+
+ sstateinst = d.expand("${WORKDIR}/sstate-install-%s/" % ss['task'])
+ d.setVar("SSTATE_CURRTASK", ss['task'])
+ sstatefetch = d.getVar('SSTATE_PKGNAME')
+ sstatepkg = d.getVar('SSTATE_PKG')
+
+ if not os.path.exists(sstatepkg):
+ pstaging_fetch(sstatefetch, d)
+
+ if not os.path.isfile(sstatepkg):
+ bb.note("Sstate package %s does not exist" % sstatepkg)
+ return False
+
+ sstate_clean(ss, d)
+
+ d.setVar('SSTATE_INSTDIR', sstateinst)
+
+ if bb.utils.to_boolean(d.getVar("SSTATE_VERIFY_SIG"), False):
+ if not os.path.isfile(sstatepkg + '.sig'):
+ bb.warn("No signature file for sstate package %s, skipping acceleration..." % sstatepkg)
+ return False
+ signer = get_signer(d, 'local')
+ if not signer.verify(sstatepkg + '.sig', d.getVar("SSTATE_VALID_SIGS")):
+ bb.warn("Cannot verify signature on sstate package %s, skipping acceleration..." % sstatepkg)
+ return False
+
+ # Empty sstateinst directory, ensure its clean
+ if os.path.exists(sstateinst):
+ oe.path.remove(sstateinst)
+ bb.utils.mkdirhier(sstateinst)
+
+ sstateinst = d.getVar("SSTATE_INSTDIR")
+ d.setVar('SSTATE_FIXMEDIR', ss['fixmedir'])
+
+ for f in (d.getVar('SSTATEPREINSTFUNCS') or '').split() + ['sstate_unpack_package']:
+ # All hooks should run in the SSTATE_INSTDIR
+ bb.build.exec_func(f, d, (sstateinst,))
+
+ return sstate_installpkgdir(ss, d)
+
+def sstate_installpkgdir(ss, d):
+ import oe.path
+ import subprocess
+
+ sstateinst = d.getVar("SSTATE_INSTDIR")
+ d.setVar('SSTATE_FIXMEDIR', ss['fixmedir'])
+
+ for f in (d.getVar('SSTATEPOSTUNPACKFUNCS') or '').split():
+ # All hooks should run in the SSTATE_INSTDIR
+ bb.build.exec_func(f, d, (sstateinst,))
+
+ def prepdir(dir):
+ # remove dir if it exists, ensure any parent directories do exist
+ if os.path.exists(dir):
+ oe.path.remove(dir)
+ bb.utils.mkdirhier(dir)
+ oe.path.remove(dir)
+
+ for state in ss['dirs']:
+ prepdir(state[1])
+ bb.utils.rename(sstateinst + state[0], state[1])
+ sstate_install(ss, d)
+
+ for plain in ss['plaindirs']:
+ workdir = d.getVar('WORKDIR')
+ sharedworkdir = os.path.join(d.getVar('TMPDIR'), "work-shared")
+ src = sstateinst + "/" + plain.replace(workdir, '')
+ if sharedworkdir in plain:
+ src = sstateinst + "/" + plain.replace(sharedworkdir, '')
+ dest = plain
+ bb.utils.mkdirhier(src)
+ prepdir(dest)
+ bb.utils.rename(src, dest)
+
+ return True
+
+python sstate_hardcode_path_unpack () {
+ # Fixup hardcoded paths
+ #
+ # Note: The logic below must match the reverse logic in
+ # sstate_hardcode_path(d)
+ import subprocess
+
+ sstateinst = d.getVar('SSTATE_INSTDIR')
+ sstatefixmedir = d.getVar('SSTATE_FIXMEDIR')
+ fixmefn = sstateinst + "fixmepath"
+ if os.path.isfile(fixmefn):
+ staging_target = d.getVar('RECIPE_SYSROOT')
+ staging_host = d.getVar('RECIPE_SYSROOT_NATIVE')
+
+ if bb.data.inherits_class('native', d) or bb.data.inherits_class('cross-canadian', d):
+ sstate_sed_cmd = "sed -i -e 's:FIXMESTAGINGDIRHOST:%s:g'" % (staging_host)
+ elif bb.data.inherits_class('cross', d) or bb.data.inherits_class('crosssdk', d):
+ sstate_sed_cmd = "sed -i -e 's:FIXMESTAGINGDIRTARGET:%s:g; s:FIXMESTAGINGDIRHOST:%s:g'" % (staging_target, staging_host)
+ else:
+ sstate_sed_cmd = "sed -i -e 's:FIXMESTAGINGDIRTARGET:%s:g'" % (staging_target)
+
+ extra_staging_fixmes = d.getVar('EXTRA_STAGING_FIXMES') or ''
+ for fixmevar in extra_staging_fixmes.split():
+ fixme_path = d.getVar(fixmevar)
+ sstate_sed_cmd += " -e 's:FIXME_%s:%s:g'" % (fixmevar, fixme_path)
+
+ # Add sstateinst to each filename in fixmepath, use xargs to efficiently call sed
+ sstate_hardcode_cmd = "sed -e 's:^:%s:g' %s | xargs %s" % (sstateinst, fixmefn, sstate_sed_cmd)
+
+ # Defer do_populate_sysroot relocation command
+ if sstatefixmedir:
+ bb.utils.mkdirhier(sstatefixmedir)
+ with open(sstatefixmedir + "/fixmepath.cmd", "w") as f:
+ sstate_hardcode_cmd = sstate_hardcode_cmd.replace(fixmefn, sstatefixmedir + "/fixmepath")
+ sstate_hardcode_cmd = sstate_hardcode_cmd.replace(sstateinst, "FIXMEFINALSSTATEINST")
+ sstate_hardcode_cmd = sstate_hardcode_cmd.replace(staging_host, "FIXMEFINALSSTATEHOST")
+ sstate_hardcode_cmd = sstate_hardcode_cmd.replace(staging_target, "FIXMEFINALSSTATETARGET")
+ f.write(sstate_hardcode_cmd)
+ bb.utils.copyfile(fixmefn, sstatefixmedir + "/fixmepath")
+ return
+
+ bb.note("Replacing fixme paths in sstate package: %s" % (sstate_hardcode_cmd))
+ subprocess.check_call(sstate_hardcode_cmd, shell=True)
+
+ # Need to remove this or we'd copy it into the target directory and may
+ # conflict with another writer
+ os.remove(fixmefn)
+}
+
+def sstate_clean_cachefile(ss, d):
+ import oe.path
+
+ if d.getVarFlag('do_%s' % ss['task'], 'task'):
+ d.setVar("SSTATE_PATH_CURRTASK", ss['task'])
+ sstatepkgfile = d.getVar('SSTATE_PATHSPEC')
+ bb.note("Removing %s" % sstatepkgfile)
+ oe.path.remove(sstatepkgfile)
+
+def sstate_clean_cachefiles(d):
+ for task in (d.getVar('SSTATETASKS') or "").split():
+ ld = d.createCopy()
+ ss = sstate_state_fromvars(ld, task)
+ sstate_clean_cachefile(ss, ld)
+
+def sstate_clean_manifest(manifest, d, canrace=False, prefix=None):
+ import oe.path
+
+ mfile = open(manifest)
+ entries = mfile.readlines()
+ mfile.close()
+
+ for entry in entries:
+ entry = entry.strip()
+ if prefix and not entry.startswith("/"):
+ entry = prefix + "/" + entry
+ bb.debug(2, "Removing manifest: %s" % entry)
+ # We can race against another package populating directories as we're removing them
+ # so we ignore errors here.
+ try:
+ if entry.endswith("/"):
+ if os.path.islink(entry[:-1]):
+ os.remove(entry[:-1])
+ elif os.path.exists(entry) and len(os.listdir(entry)) == 0 and not canrace:
+ # Removing directories whilst builds are in progress exposes a race. Only
+ # do it in contexts where it is safe to do so.
+ os.rmdir(entry[:-1])
+ else:
+ os.remove(entry)
+ except OSError:
+ pass
+
+ postrm = manifest + ".postrm"
+ if os.path.exists(manifest + ".postrm"):
+ import subprocess
+ os.chmod(postrm, 0o755)
+ subprocess.check_call(postrm, shell=True)
+ oe.path.remove(postrm)
+
+ oe.path.remove(manifest)
+
+def sstate_clean(ss, d):
+ import oe.path
+ import glob
+
+ d2 = d.createCopy()
+ stamp_clean = d.getVar("STAMPCLEAN")
+ extrainf = d.getVarFlag("do_" + ss['task'], 'stamp-extra-info')
+ if extrainf:
+ d2.setVar("SSTATE_MANMACH", extrainf)
+ wildcard_stfile = "%s.do_%s*.%s" % (stamp_clean, ss['task'], extrainf)
+ else:
+ wildcard_stfile = "%s.do_%s*" % (stamp_clean, ss['task'])
+
+ manifest = d2.expand("${SSTATE_MANFILEPREFIX}.%s" % ss['task'])
+
+ if os.path.exists(manifest):
+ locks = []
+ for lock in ss['lockfiles-shared']:
+ locks.append(bb.utils.lockfile(lock))
+ for lock in ss['lockfiles']:
+ locks.append(bb.utils.lockfile(lock))
+
+ sstate_clean_manifest(manifest, d, canrace=True)
+
+ for lock in locks:
+ bb.utils.unlockfile(lock)
+
+ # Remove the current and previous stamps, but keep the sigdata.
+ #
+ # The glob() matches do_task* which may match multiple tasks, for
+ # example: do_package and do_package_write_ipk, so we need to
+ # exactly match *.do_task.* and *.do_task_setscene.*
+ rm_stamp = '.do_%s.' % ss['task']
+ rm_setscene = '.do_%s_setscene.' % ss['task']
+ # For BB_SIGNATURE_HANDLER = "noop"
+ rm_nohash = ".do_%s" % ss['task']
+ for stfile in glob.glob(wildcard_stfile):
+ # Keep the sigdata
+ if ".sigdata." in stfile or ".sigbasedata." in stfile:
+ continue
+ # Preserve taint files in the stamps directory
+ if stfile.endswith('.taint'):
+ continue
+ if rm_stamp in stfile or rm_setscene in stfile or \
+ stfile.endswith(rm_nohash):
+ oe.path.remove(stfile)
+
+sstate_clean[vardepsexclude] = "SSTATE_MANFILEPREFIX"
+
+CLEANFUNCS += "sstate_cleanall"
+
+python sstate_cleanall() {
+ bb.note("Removing shared state for package %s" % d.getVar('PN'))
+
+ manifest_dir = d.getVar('SSTATE_MANIFESTS')
+ if not os.path.exists(manifest_dir):
+ return
+
+ tasks = d.getVar('SSTATETASKS').split()
+ for name in tasks:
+ ld = d.createCopy()
+ shared_state = sstate_state_fromvars(ld, name)
+ sstate_clean(shared_state, ld)
+}
+
+python sstate_hardcode_path () {
+ import subprocess, platform
+
+ # Need to remove hardcoded paths and fix these when we install the
+ # staging packages.
+ #
+ # Note: the logic in this function needs to match the reverse logic
+ # in sstate_installpkg(ss, d)
+
+ staging_target = d.getVar('RECIPE_SYSROOT')
+ staging_host = d.getVar('RECIPE_SYSROOT_NATIVE')
+ sstate_builddir = d.getVar('SSTATE_BUILDDIR')
+
+ sstate_sed_cmd = "sed -i -e 's:%s:FIXMESTAGINGDIRHOST:g'" % staging_host
+ if bb.data.inherits_class('native', d) or bb.data.inherits_class('cross-canadian', d):
+ sstate_grep_cmd = "grep -l -e '%s'" % (staging_host)
+ elif bb.data.inherits_class('cross', d) or bb.data.inherits_class('crosssdk', d):
+ sstate_grep_cmd = "grep -l -e '%s' -e '%s'" % (staging_target, staging_host)
+ sstate_sed_cmd += " -e 's:%s:FIXMESTAGINGDIRTARGET:g'" % staging_target
+ else:
+ sstate_grep_cmd = "grep -l -e '%s' -e '%s'" % (staging_target, staging_host)
+ sstate_sed_cmd += " -e 's:%s:FIXMESTAGINGDIRTARGET:g'" % staging_target
+
+ extra_staging_fixmes = d.getVar('EXTRA_STAGING_FIXMES') or ''
+ for fixmevar in extra_staging_fixmes.split():
+ fixme_path = d.getVar(fixmevar)
+ sstate_sed_cmd += " -e 's:%s:FIXME_%s:g'" % (fixme_path, fixmevar)
+ sstate_grep_cmd += " -e '%s'" % (fixme_path)
+
+ fixmefn = sstate_builddir + "fixmepath"
+
+ sstate_scan_cmd = d.getVar('SSTATE_SCAN_CMD')
+ sstate_filelist_cmd = "tee %s" % (fixmefn)
+
+ # fixmepath file needs relative paths, drop sstate_builddir prefix
+ sstate_filelist_relative_cmd = "sed -i -e 's:^%s::g' %s" % (sstate_builddir, fixmefn)
+
+ xargs_no_empty_run_cmd = '--no-run-if-empty'
+ if platform.system() == 'Darwin':
+ xargs_no_empty_run_cmd = ''
+
+ # Limit the fixpaths and sed operations based on the initial grep search
+ # This has the side effect of making sure the vfs cache is hot
+ sstate_hardcode_cmd = "%s | xargs %s | %s | xargs %s %s" % (sstate_scan_cmd, sstate_grep_cmd, sstate_filelist_cmd, xargs_no_empty_run_cmd, sstate_sed_cmd)
+
+ bb.note("Removing hardcoded paths from sstate package: '%s'" % (sstate_hardcode_cmd))
+ subprocess.check_output(sstate_hardcode_cmd, shell=True, cwd=sstate_builddir)
+
+ # If the fixmefn is empty, remove it..
+ if os.stat(fixmefn).st_size == 0:
+ os.remove(fixmefn)
+ else:
+ bb.note("Replacing absolute paths in fixmepath file: '%s'" % (sstate_filelist_relative_cmd))
+ subprocess.check_output(sstate_filelist_relative_cmd, shell=True)
+}
+
+def sstate_package(ss, d):
+ import oe.path
+ import time
+
+ tmpdir = d.getVar('TMPDIR')
+
+ fixtime = False
+ if ss['task'] == "package":
+ fixtime = True
+
+ def fixtimestamp(root, path):
+ f = os.path.join(root, path)
+ if os.lstat(f).st_mtime > sde:
+ os.utime(f, (sde, sde), follow_symlinks=False)
+
+ sstatebuild = d.expand("${WORKDIR}/sstate-build-%s/" % ss['task'])
+ sde = int(d.getVar("SOURCE_DATE_EPOCH") or time.time())
+ d.setVar("SSTATE_CURRTASK", ss['task'])
+ bb.utils.remove(sstatebuild, recurse=True)
+ bb.utils.mkdirhier(sstatebuild)
+ for state in ss['dirs']:
+ if not os.path.exists(state[1]):
+ continue
+ srcbase = state[0].rstrip("/").rsplit('/', 1)[0]
+ # Find and error for absolute symlinks. We could attempt to relocate but its not
+ # clear where the symlink is relative to in this context. We could add that markup
+ # to sstate tasks but there aren't many of these so better just avoid them entirely.
+ for walkroot, dirs, files in os.walk(state[1]):
+ for file in files + dirs:
+ if fixtime:
+ fixtimestamp(walkroot, file)
+ srcpath = os.path.join(walkroot, file)
+ if not os.path.islink(srcpath):
+ continue
+ link = os.readlink(srcpath)
+ if not os.path.isabs(link):
+ continue
+ if not link.startswith(tmpdir):
+ continue
+ bb.error("sstate found an absolute path symlink %s pointing at %s. Please replace this with a relative link." % (srcpath, link))
+ bb.debug(2, "Preparing tree %s for packaging at %s" % (state[1], sstatebuild + state[0]))
+ bb.utils.rename(state[1], sstatebuild + state[0])
+
+ workdir = d.getVar('WORKDIR')
+ sharedworkdir = os.path.join(d.getVar('TMPDIR'), "work-shared")
+ for plain in ss['plaindirs']:
+ pdir = plain.replace(workdir, sstatebuild)
+ if sharedworkdir in plain:
+ pdir = plain.replace(sharedworkdir, sstatebuild)
+ bb.utils.mkdirhier(plain)
+ bb.utils.mkdirhier(pdir)
+ bb.utils.rename(plain, pdir)
+ if fixtime:
+ fixtimestamp(pdir, "")
+ for walkroot, dirs, files in os.walk(pdir):
+ for file in files + dirs:
+ fixtimestamp(walkroot, file)
+
+ d.setVar('SSTATE_BUILDDIR', sstatebuild)
+ d.setVar('SSTATE_INSTDIR', sstatebuild)
+
+ if d.getVar('SSTATE_SKIP_CREATION') == '1':
+ return
+
+ sstate_create_package = ['sstate_report_unihash', 'sstate_create_package']
+ if d.getVar('SSTATE_SIG_KEY'):
+ sstate_create_package.append('sstate_sign_package')
+
+ for f in (d.getVar('SSTATECREATEFUNCS') or '').split() + \
+ sstate_create_package + \
+ (d.getVar('SSTATEPOSTCREATEFUNCS') or '').split():
+ # All hooks should run in SSTATE_BUILDDIR.
+ bb.build.exec_func(f, d, (sstatebuild,))
+
+ # SSTATE_PKG may have been changed by sstate_report_unihash
+ siginfo = d.getVar('SSTATE_PKG') + ".siginfo"
+ if not os.path.exists(siginfo):
+ bb.siggen.dump_this_task(siginfo, d)
+ else:
+ try:
+ os.utime(siginfo, None)
+ except PermissionError:
+ pass
+ except OSError as e:
+ # Handle read-only file systems gracefully
+ import errno
+ if e.errno != errno.EROFS:
+ raise e
+
+ return
+
+sstate_package[vardepsexclude] += "SSTATE_SIG_KEY"
+
+def pstaging_fetch(sstatefetch, d):
+ import bb.fetch2
+
+ # Only try and fetch if the user has configured a mirror
+ mirrors = d.getVar('SSTATE_MIRRORS')
+ if not mirrors:
+ return
+
+ # Copy the data object and override DL_DIR and SRC_URI
+ localdata = bb.data.createCopy(d)
+
+ dldir = localdata.expand("${SSTATE_DIR}")
+ bb.utils.mkdirhier(dldir)
+
+ localdata.delVar('MIRRORS')
+ localdata.setVar('FILESPATH', dldir)
+ localdata.setVar('DL_DIR', dldir)
+ localdata.setVar('PREMIRRORS', mirrors)
+ localdata.setVar('SRCPV', d.getVar('SRCPV'))
+
+ # if BB_NO_NETWORK is set but we also have SSTATE_MIRROR_ALLOW_NETWORK,
+ # we'll want to allow network access for the current set of fetches.
+ if bb.utils.to_boolean(localdata.getVar('BB_NO_NETWORK')) and \
+ bb.utils.to_boolean(localdata.getVar('SSTATE_MIRROR_ALLOW_NETWORK')):
+ localdata.delVar('BB_NO_NETWORK')
+
+ # Try a fetch from the sstate mirror, if it fails just return and
+ # we will build the package
+ uris = ['file://{0};downloadfilename={0}'.format(sstatefetch),
+ 'file://{0}.siginfo;downloadfilename={0}.siginfo'.format(sstatefetch)]
+ if bb.utils.to_boolean(d.getVar("SSTATE_VERIFY_SIG"), False):
+ uris += ['file://{0}.sig;downloadfilename={0}.sig'.format(sstatefetch)]
+
+ for srcuri in uris:
+ localdata.setVar('SRC_URI', srcuri)
+ try:
+ fetcher = bb.fetch2.Fetch([srcuri], localdata, cache=False)
+ fetcher.checkstatus()
+ fetcher.download()
+
+ except bb.fetch2.BBFetchException:
+ pass
+
+pstaging_fetch[vardepsexclude] += "SRCPV"
+
+
+def sstate_setscene(d):
+ shared_state = sstate_state_fromvars(d)
+ accelerate = sstate_installpkg(shared_state, d)
+ if not accelerate:
+ msg = "No sstate archive obtainable, will run full task instead."
+ bb.warn(msg)
+ raise bb.BBHandledException(msg)
+
+python sstate_task_prefunc () {
+ shared_state = sstate_state_fromvars(d)
+ sstate_clean(shared_state, d)
+}
+sstate_task_prefunc[dirs] = "${WORKDIR}"
+
+python sstate_task_postfunc () {
+ shared_state = sstate_state_fromvars(d)
+
+ for intercept in shared_state['interceptfuncs']:
+ bb.build.exec_func(intercept, d, (d.getVar("WORKDIR"),))
+
+ omask = os.umask(0o002)
+ if omask != 0o002:
+ bb.note("Using umask 0o002 (not %0o) for sstate packaging" % omask)
+ sstate_package(shared_state, d)
+ os.umask(omask)
+
+ sstateinst = d.getVar("SSTATE_INSTDIR")
+ d.setVar('SSTATE_FIXMEDIR', shared_state['fixmedir'])
+
+ sstate_installpkgdir(shared_state, d)
+
+ bb.utils.remove(d.getVar("SSTATE_BUILDDIR"), recurse=True)
+}
+sstate_task_postfunc[dirs] = "${WORKDIR}"
+
+
+#
+# Shell function to generate a sstate package from a directory
+# set as SSTATE_BUILDDIR. Will be run from within SSTATE_BUILDDIR.
+#
+sstate_create_package () {
+ # Exit early if it already exists
+ if [ -e ${SSTATE_PKG} ]; then
+ touch ${SSTATE_PKG} 2>/dev/null || true
+ return
+ fi
+
+ mkdir --mode=0775 -p `dirname ${SSTATE_PKG}`
+ TFILE=`mktemp ${SSTATE_PKG}.XXXXXXXX`
+
+ OPT="-cS"
+ ZSTD="zstd -${SSTATE_ZSTD_CLEVEL} -T${ZSTD_THREADS}"
+ # Use pzstd if available
+ if [ -x "$(command -v pzstd)" ]; then
+ ZSTD="pzstd -${SSTATE_ZSTD_CLEVEL} -p ${ZSTD_THREADS}"
+ fi
+
+ # Need to handle empty directories
+ if [ "$(ls -A)" ]; then
+ set +e
+ tar -I "$ZSTD" $OPT -f $TFILE *
+ ret=$?
+ if [ $ret -ne 0 ] && [ $ret -ne 1 ]; then
+ exit 1
+ fi
+ set -e
+ else
+ tar -I "$ZSTD" $OPT --file=$TFILE --files-from=/dev/null
+ fi
+ chmod 0664 $TFILE
+ # Skip if it was already created by some other process
+ if [ -h ${SSTATE_PKG} ] && [ ! -e ${SSTATE_PKG} ]; then
+ # There is a symbolic link, but it links to nothing.
+ # Forcefully replace it with the new file.
+ ln -f $TFILE ${SSTATE_PKG} || true
+ elif [ ! -e ${SSTATE_PKG} ]; then
+ # Move into place using ln to attempt an atomic op.
+ # Abort if it already exists
+ ln $TFILE ${SSTATE_PKG} || true
+ else
+ touch ${SSTATE_PKG} 2>/dev/null || true
+ fi
+ rm $TFILE
+}
+
+python sstate_sign_package () {
+ from oe.gpg_sign import get_signer
+
+
+ signer = get_signer(d, 'local')
+ sstate_pkg = d.getVar('SSTATE_PKG')
+ if os.path.exists(sstate_pkg + '.sig'):
+ os.unlink(sstate_pkg + '.sig')
+ signer.detach_sign(sstate_pkg, d.getVar('SSTATE_SIG_KEY', False), None,
+ d.getVar('SSTATE_SIG_PASSPHRASE'), armor=False)
+}
+
+python sstate_report_unihash() {
+ report_unihash = getattr(bb.parse.siggen, 'report_unihash', None)
+
+ if report_unihash:
+ ss = sstate_state_fromvars(d)
+ report_unihash(os.getcwd(), ss['task'], d)
+}
+
+#
+# Shell function to decompress and prepare a package for installation
+# Will be run from within SSTATE_INSTDIR.
+#
+sstate_unpack_package () {
+ ZSTD="zstd -T${ZSTD_THREADS}"
+ # Use pzstd if available
+ if [ -x "$(command -v pzstd)" ]; then
+ ZSTD="pzstd -p ${ZSTD_THREADS}"
+ fi
+
+ tar -I "$ZSTD" -xvpf ${SSTATE_PKG}
+ # update .siginfo atime on local/NFS mirror if it is a symbolic link
+ [ ! -h ${SSTATE_PKG}.siginfo ] || [ ! -e ${SSTATE_PKG}.siginfo ] || touch -a ${SSTATE_PKG}.siginfo 2>/dev/null || true
+ # update each symbolic link instead of any referenced file
+ touch --no-dereference ${SSTATE_PKG} 2>/dev/null || true
+ [ ! -e ${SSTATE_PKG}.sig ] || touch --no-dereference ${SSTATE_PKG}.sig 2>/dev/null || true
+ [ ! -e ${SSTATE_PKG}.siginfo ] || touch --no-dereference ${SSTATE_PKG}.siginfo 2>/dev/null || true
+}
+
+BB_HASHCHECK_FUNCTION = "sstate_checkhashes"
+
+def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, summary=True, **kwargs):
+ found = set()
+ missed = set()
+
+ def gethash(task):
+ return sq_data['unihash'][task]
+
+ def getpathcomponents(task, d):
+ # Magic data from BB_HASHFILENAME
+ splithashfn = sq_data['hashfn'][task].split(" ")
+ spec = splithashfn[1]
+ if splithashfn[0] == "True":
+ extrapath = d.getVar("NATIVELSBSTRING") + "/"
+ else:
+ extrapath = ""
+
+ tname = bb.runqueue.taskname_from_tid(task)[3:]
+
+ if tname in ["fetch", "unpack", "patch", "populate_lic", "preconfigure"] and splithashfn[2]:
+ spec = splithashfn[2]
+ extrapath = ""
+
+ return spec, extrapath, tname
+
+ def getsstatefile(tid, siginfo, d):
+ spec, extrapath, tname = getpathcomponents(tid, d)
+ return extrapath + generate_sstatefn(spec, gethash(tid), tname, siginfo, d)
+
+ for tid in sq_data['hash']:
+
+ sstatefile = d.expand("${SSTATE_DIR}/" + getsstatefile(tid, siginfo, d))
+
+ if os.path.exists(sstatefile):
+ found.add(tid)
+ bb.debug(2, "SState: Found valid sstate file %s" % sstatefile)
+ else:
+ missed.add(tid)
+ bb.debug(2, "SState: Looked for but didn't find file %s" % sstatefile)
+
+ foundLocal = len(found)
+ mirrors = d.getVar("SSTATE_MIRRORS")
+ if mirrors:
+ # Copy the data object and override DL_DIR and SRC_URI
+ localdata = bb.data.createCopy(d)
+
+ dldir = localdata.expand("${SSTATE_DIR}")
+ localdata.delVar('MIRRORS')
+ localdata.setVar('FILESPATH', dldir)
+ localdata.setVar('DL_DIR', dldir)
+ localdata.setVar('PREMIRRORS', mirrors)
+
+ bb.debug(2, "SState using premirror of: %s" % mirrors)
+
+ # if BB_NO_NETWORK is set but we also have SSTATE_MIRROR_ALLOW_NETWORK,
+ # we'll want to allow network access for the current set of fetches.
+ if bb.utils.to_boolean(localdata.getVar('BB_NO_NETWORK')) and \
+ bb.utils.to_boolean(localdata.getVar('SSTATE_MIRROR_ALLOW_NETWORK')):
+ localdata.delVar('BB_NO_NETWORK')
+
+ from bb.fetch2 import FetchConnectionCache
+ def checkstatus_init():
+ while not connection_cache_pool.full():
+ connection_cache_pool.put(FetchConnectionCache())
+
+ def checkstatus_end():
+ while not connection_cache_pool.empty():
+ connection_cache = connection_cache_pool.get()
+ connection_cache.close_connections()
+
+ def checkstatus(arg):
+ (tid, sstatefile) = arg
+
+ connection_cache = connection_cache_pool.get()
+ localdata2 = bb.data.createCopy(localdata)
+ srcuri = "file://" + sstatefile
+ localdata2.setVar('SRC_URI', srcuri)
+ bb.debug(2, "SState: Attempting to fetch %s" % srcuri)
+
+ import traceback
+
+ try:
+ fetcher = bb.fetch2.Fetch(srcuri.split(), localdata2,
+ connection_cache=connection_cache)
+ fetcher.checkstatus()
+ bb.debug(2, "SState: Successful fetch test for %s" % srcuri)
+ found.add(tid)
+ missed.remove(tid)
+ except bb.fetch2.FetchError as e:
+ bb.debug(2, "SState: Unsuccessful fetch test for %s (%s)\n%s" % (srcuri, repr(e), traceback.format_exc()))
+ except Exception as e:
+ bb.error("SState: cannot test %s: %s\n%s" % (srcuri, repr(e), traceback.format_exc()))
+
+ connection_cache_pool.put(connection_cache)
+
+ if progress:
+ bb.event.fire(bb.event.ProcessProgress(msg, len(tasklist) - thread_worker.tasks.qsize()), d)
+
+ tasklist = []
+ for tid in missed:
+ sstatefile = d.expand(getsstatefile(tid, siginfo, d))
+ tasklist.append((tid, sstatefile))
+
+ if tasklist:
+ nproc = min(int(d.getVar("BB_NUMBER_THREADS")), len(tasklist))
+
+ progress = len(tasklist) >= 100
+ if progress:
+ msg = "Checking sstate mirror object availability"
+ bb.event.fire(bb.event.ProcessStarted(msg, len(tasklist)), d)
+
+ # Have to setup the fetcher environment here rather than in each thread as it would race
+ fetcherenv = bb.fetch2.get_fetcher_environment(d)
+ with bb.utils.environment(**fetcherenv):
+ bb.event.enable_threadlock()
+ import concurrent.futures
+ from queue import Queue
+ connection_cache_pool = Queue(nproc)
+ checkstatus_init()
+ with concurrent.futures.ThreadPoolExecutor(max_workers=nproc) as executor:
+ executor.map(checkstatus, tasklist.copy())
+ checkstatus_end()
+ bb.event.disable_threadlock()
+
+ if progress:
+ bb.event.fire(bb.event.ProcessFinished(msg), d)
+
+ inheritlist = d.getVar("INHERIT")
+ if "toaster" in inheritlist:
+ evdata = {'missed': [], 'found': []};
+ for tid in missed:
+ sstatefile = d.expand(getsstatefile(tid, False, d))
+ evdata['missed'].append((bb.runqueue.fn_from_tid(tid), bb.runqueue.taskname_from_tid(tid), gethash(tid), sstatefile ) )
+ for tid in found:
+ sstatefile = d.expand(getsstatefile(tid, False, d))
+ evdata['found'].append((bb.runqueue.fn_from_tid(tid), bb.runqueue.taskname_from_tid(tid), gethash(tid), sstatefile ) )
+ bb.event.fire(bb.event.MetadataEvent("MissedSstate", evdata), d)
+
+ if summary:
+ # Print some summary statistics about the current task completion and how much sstate
+ # reuse there was. Avoid divide by zero errors.
+ total = len(sq_data['hash'])
+ complete = 0
+ if currentcount:
+ complete = (len(found) + currentcount) / (total + currentcount) * 100
+ match = 0
+ if total:
+ match = len(found) / total * 100
+ bb.plain("Sstate summary: Wanted %d Local %d Mirrors %d Missed %d Current %d (%d%% match, %d%% complete)" %
+ (total, foundLocal, len(found)-foundLocal, len(missed), currentcount, match, complete))
+
+ if hasattr(bb.parse.siggen, "checkhashes"):
+ bb.parse.siggen.checkhashes(sq_data, missed, found, d)
+
+ return found
+setscene_depvalid[vardepsexclude] = "SSTATE_EXCLUDEDEPS_SYSROOT"
+
+BB_SETSCENE_DEPVALID = "setscene_depvalid"
+
+def setscene_depvalid(task, taskdependees, notneeded, d, log=None):
+ # taskdependees is a dict of tasks which depend on task, each being a 3 item list of [PN, TASKNAME, FILENAME]
+ # task is included in taskdependees too
+ # Return - False - We need this dependency
+ # - True - We can skip this dependency
+ import re
+
+ def logit(msg, log):
+ if log is not None:
+ log.append(msg)
+ else:
+ bb.debug(2, msg)
+
+ logit("Considering setscene task: %s" % (str(taskdependees[task])), log)
+
+ directtasks = ["do_populate_lic", "do_deploy_source_date_epoch", "do_shared_workdir", "do_stash_locale", "do_gcc_stash_builddir", "do_create_spdx"]
+
+ def isNativeCross(x):
+ return x.endswith("-native") or "-cross-" in x or "-crosssdk" in x or x.endswith("-cross")
+
+ # We only need to trigger deploy_source_date_epoch through direct dependencies
+ if taskdependees[task][1] in directtasks:
+ return True
+
+ # We only need to trigger packagedata through direct dependencies
+ # but need to preserve packagedata on packagedata links
+ if taskdependees[task][1] == "do_packagedata":
+ for dep in taskdependees:
+ if taskdependees[dep][1] == "do_packagedata":
+ return False
+ return True
+
+ for dep in taskdependees:
+ logit(" considering dependency: %s" % (str(taskdependees[dep])), log)
+ if task == dep:
+ continue
+ if dep in notneeded:
+ continue
+ # do_package_write_* and do_package doesn't need do_package
+ if taskdependees[task][1] == "do_package" and taskdependees[dep][1] in ['do_package', 'do_package_write_deb', 'do_package_write_ipk', 'do_package_write_rpm', 'do_packagedata', 'do_package_qa']:
+ continue
+ # do_package_write_* need do_populate_sysroot as they're mainly postinstall dependencies
+ if taskdependees[task][1] == "do_populate_sysroot" and taskdependees[dep][1] in ['do_package_write_deb', 'do_package_write_ipk', 'do_package_write_rpm']:
+ return False
+ # do_package/packagedata/package_qa/deploy don't need do_populate_sysroot
+ if taskdependees[task][1] == "do_populate_sysroot" and taskdependees[dep][1] in ['do_package', 'do_packagedata', 'do_package_qa', 'do_deploy']:
+ continue
+ # Native/Cross packages don't exist and are noexec anyway
+ if isNativeCross(taskdependees[dep][0]) and taskdependees[dep][1] in ['do_package_write_deb', 'do_package_write_ipk', 'do_package_write_rpm', 'do_packagedata', 'do_package', 'do_package_qa']:
+ continue
+
+ # This is due to the [depends] in useradd.bbclass complicating matters
+ # The logic *is* reversed here due to the way hard setscene dependencies are injected
+ if (taskdependees[task][1] == 'do_package' or taskdependees[task][1] == 'do_populate_sysroot') and taskdependees[dep][0].endswith(('shadow-native', 'shadow-sysroot', 'base-passwd', 'pseudo-native')) and taskdependees[dep][1] == 'do_populate_sysroot':
+ continue
+
+ # Consider sysroot depending on sysroot tasks
+ if taskdependees[task][1] == 'do_populate_sysroot' and taskdependees[dep][1] == 'do_populate_sysroot':
+ # Allow excluding certain recursive dependencies. If a recipe needs it should add a
+ # specific dependency itself, rather than relying on one of its dependees to pull
+ # them in.
+ # See also http://lists.openembedded.org/pipermail/openembedded-core/2018-January/146324.html
+ not_needed = False
+ excludedeps = d.getVar('_SSTATE_EXCLUDEDEPS_SYSROOT')
+ if excludedeps is None:
+ # Cache the regular expressions for speed
+ excludedeps = []
+ for excl in (d.getVar('SSTATE_EXCLUDEDEPS_SYSROOT') or "").split():
+ excludedeps.append((re.compile(excl.split('->', 1)[0]), re.compile(excl.split('->', 1)[1])))
+ d.setVar('_SSTATE_EXCLUDEDEPS_SYSROOT', excludedeps)
+ for excl in excludedeps:
+ if excl[0].match(taskdependees[dep][0]):
+ if excl[1].match(taskdependees[task][0]):
+ not_needed = True
+ break
+ if not_needed:
+ continue
+ # For meta-extsdk-toolchain we want all sysroot dependencies
+ if taskdependees[dep][0] == 'meta-extsdk-toolchain':
+ return False
+ # Native/Cross populate_sysroot need their dependencies
+ if isNativeCross(taskdependees[task][0]) and isNativeCross(taskdependees[dep][0]):
+ return False
+ # Target populate_sysroot depended on by cross tools need to be installed
+ if isNativeCross(taskdependees[dep][0]):
+ return False
+ # Native/cross tools depended upon by target sysroot are not needed
+ # Add an exception for shadow-native as required by useradd.bbclass
+ if isNativeCross(taskdependees[task][0]) and taskdependees[task][0] != 'shadow-native':
+ continue
+ # Target populate_sysroot need their dependencies
+ return False
+
+ if taskdependees[dep][1] in directtasks:
+ continue
+
+ # Safe fallthrough default
+ logit(" Default setscene dependency fall through due to dependency: %s" % (str(taskdependees[dep])), log)
+ return False
+ return True
+
+addhandler sstate_eventhandler
+sstate_eventhandler[eventmask] = "bb.build.TaskSucceeded"
+python sstate_eventhandler() {
+ d = e.data
+ writtensstate = d.getVar('SSTATE_CURRTASK')
+ if not writtensstate:
+ taskname = d.getVar("BB_RUNTASK")[3:]
+ spec = d.getVar('SSTATE_PKGSPEC')
+ swspec = d.getVar('SSTATE_SWSPEC')
+ if taskname in ["fetch", "unpack", "patch", "populate_lic", "preconfigure"] and swspec:
+ d.setVar("SSTATE_PKGSPEC", "${SSTATE_SWSPEC}")
+ d.setVar("SSTATE_EXTRAPATH", "")
+ d.setVar("SSTATE_CURRTASK", taskname)
+ siginfo = d.getVar('SSTATE_PKG') + ".siginfo"
+ if not os.path.exists(siginfo):
+ bb.siggen.dump_this_task(siginfo, d)
+ else:
+ try:
+ os.utime(siginfo, None)
+ except PermissionError:
+ pass
+ except OSError as e:
+ # Handle read-only file systems gracefully
+ import errno
+ if e.errno != errno.EROFS:
+ raise e
+
+}
+
+SSTATE_PRUNE_OBSOLETEWORKDIR ?= "1"
+
+#
+# Event handler which removes manifests and stamps file for recipes which are no
+# longer 'reachable' in a build where they once were. 'Reachable' refers to
+# whether a recipe is parsed so recipes in a layer which was removed would no
+# longer be reachable. Switching between systemd and sysvinit where recipes
+# became skipped would be another example.
+#
+# Also optionally removes the workdir of those tasks/recipes
+#
+addhandler sstate_eventhandler_reachablestamps
+sstate_eventhandler_reachablestamps[eventmask] = "bb.event.ReachableStamps"
+python sstate_eventhandler_reachablestamps() {
+ import glob
+ d = e.data
+ stamps = e.stamps.values()
+ removeworkdir = (d.getVar("SSTATE_PRUNE_OBSOLETEWORKDIR", False) == "1")
+ preservestampfile = d.expand('${SSTATE_MANIFESTS}/preserve-stamps')
+ preservestamps = []
+ if os.path.exists(preservestampfile):
+ with open(preservestampfile, 'r') as f:
+ preservestamps = f.readlines()
+ seen = []
+
+ # The machine index contains all the stamps this machine has ever seen in this build directory.
+ # We should only remove things which this machine once accessed but no longer does.
+ machineindex = set()
+ bb.utils.mkdirhier(d.expand("${SSTATE_MANIFESTS}"))
+ mi = d.expand("${SSTATE_MANIFESTS}/index-machine-${MACHINE}")
+ if os.path.exists(mi):
+ with open(mi, "r") as f:
+ machineindex = set(line.strip() for line in f.readlines())
+
+ for a in sorted(list(set(d.getVar("SSTATE_ARCHS").split()))):
+ toremove = []
+ i = d.expand("${SSTATE_MANIFESTS}/index-" + a)
+ if not os.path.exists(i):
+ continue
+ manseen = set()
+ ignore = []
+ with open(i, "r") as f:
+ lines = f.readlines()
+ for l in reversed(lines):
+ try:
+ (stamp, manifest, workdir) = l.split()
+ # The index may have multiple entries for the same manifest as the code above only appends
+ # new entries and there may be an entry with matching manifest but differing version in stamp/workdir.
+ # The last entry in the list is the valid one, any earlier entries with matching manifests
+ # should be ignored.
+ if manifest in manseen:
+ ignore.append(l)
+ continue
+ manseen.add(manifest)
+ if stamp not in stamps and stamp not in preservestamps and stamp in machineindex:
+ toremove.append(l)
+ if stamp not in seen:
+ bb.debug(2, "Stamp %s is not reachable, removing related manifests" % stamp)
+ seen.append(stamp)
+ except ValueError:
+ bb.fatal("Invalid line '%s' in sstate manifest '%s'" % (l, i))
+
+ if toremove:
+ msg = "Removing %d recipes from the %s sysroot" % (len(toremove), a)
+ bb.event.fire(bb.event.ProcessStarted(msg, len(toremove)), d)
+
+ removed = 0
+ for r in toremove:
+ (stamp, manifest, workdir) = r.split()
+ for m in glob.glob(manifest + ".*"):
+ if m.endswith(".postrm"):
+ continue
+ sstate_clean_manifest(m, d)
+ bb.utils.remove(stamp + "*")
+ if removeworkdir:
+ bb.utils.remove(workdir, recurse = True)
+ lines.remove(r)
+ removed = removed + 1
+ bb.event.fire(bb.event.ProcessProgress(msg, removed), d)
+
+ bb.event.fire(bb.event.ProcessFinished(msg), d)
+
+ with open(i, "w") as f:
+ for l in lines:
+ if l in ignore:
+ continue
+ f.write(l)
+ machineindex |= set(stamps)
+ with open(mi, "w") as f:
+ for l in machineindex:
+ f.write(l + "\n")
+
+ if preservestamps:
+ os.remove(preservestampfile)
+}
+
+
+#
+# Bitbake can generate an event showing which setscene tasks are 'stale',
+# i.e. which ones will be rerun. These are ones where a stamp file is present but
+# it is stable (e.g. taskhash doesn't match). With that list we can go through
+# the manifests for matching tasks and "uninstall" those manifests now. We do
+# this now rather than mid build since the distribution of files between sstate
+# objects may have changed, new tasks may run first and if those new tasks overlap
+# with the stale tasks, we'd see overlapping files messages and failures. Thankfully
+# removing these files is fast.
+#
+addhandler sstate_eventhandler_stalesstate
+sstate_eventhandler_stalesstate[eventmask] = "bb.event.StaleSetSceneTasks"
+python sstate_eventhandler_stalesstate() {
+ d = e.data
+ tasks = e.tasks
+
+ bb.utils.mkdirhier(d.expand("${SSTATE_MANIFESTS}"))
+
+ for a in list(set(d.getVar("SSTATE_ARCHS").split())):
+ toremove = []
+ i = d.expand("${SSTATE_MANIFESTS}/index-" + a)
+ if not os.path.exists(i):
+ continue
+ with open(i, "r") as f:
+ lines = f.readlines()
+ for l in lines:
+ try:
+ (stamp, manifest, workdir) = l.split()
+ for tid in tasks:
+ for s in tasks[tid]:
+ if s.startswith(stamp):
+ taskname = bb.runqueue.taskname_from_tid(tid)[3:]
+ manname = manifest + "." + taskname
+ if os.path.exists(manname):
+ bb.debug(2, "Sstate for %s is stale, removing related manifest %s" % (tid, manname))
+ toremove.append((manname, tid, tasks[tid]))
+ break
+ except ValueError:
+ bb.fatal("Invalid line '%s' in sstate manifest '%s'" % (l, i))
+
+ if toremove:
+ msg = "Removing %d stale sstate objects for arch %s" % (len(toremove), a)
+ bb.event.fire(bb.event.ProcessStarted(msg, len(toremove)), d)
+
+ removed = 0
+ for (manname, tid, stamps) in toremove:
+ sstate_clean_manifest(manname, d)
+ for stamp in stamps:
+ bb.utils.remove(stamp)
+ removed = removed + 1
+ bb.event.fire(bb.event.ProcessProgress(msg, removed), d)
+
+ bb.event.fire(bb.event.ProcessFinished(msg), d)
+}
diff --git a/poky/meta/classes-global/staging.bbclass b/poky/meta/classes-global/staging.bbclass
new file mode 100644
index 0000000..5a1f43d
--- /dev/null
+++ b/poky/meta/classes-global/staging.bbclass
@@ -0,0 +1,690 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+# These directories will be staged in the sysroot
+SYSROOT_DIRS = " \
+ ${includedir} \
+ ${libdir} \
+ ${base_libdir} \
+ ${nonarch_base_libdir} \
+ ${datadir} \
+ /sysroot-only \
+"
+
+# These directories are also staged in the sysroot when they contain files that
+# are usable on the build system
+SYSROOT_DIRS_NATIVE = " \
+ ${bindir} \
+ ${sbindir} \
+ ${base_bindir} \
+ ${base_sbindir} \
+ ${libexecdir} \
+ ${sysconfdir} \
+ ${localstatedir} \
+"
+SYSROOT_DIRS:append:class-native = " ${SYSROOT_DIRS_NATIVE}"
+SYSROOT_DIRS:append:class-cross = " ${SYSROOT_DIRS_NATIVE}"
+SYSROOT_DIRS:append:class-crosssdk = " ${SYSROOT_DIRS_NATIVE}"
+
+# These directories will not be staged in the sysroot
+SYSROOT_DIRS_IGNORE = " \
+ ${mandir} \
+ ${docdir} \
+ ${infodir} \
+ ${datadir}/X11/locale \
+ ${datadir}/applications \
+ ${datadir}/bash-completion \
+ ${datadir}/fonts \
+ ${datadir}/gtk-doc/html \
+ ${datadir}/installed-tests \
+ ${datadir}/locale \
+ ${datadir}/pixmaps \
+ ${datadir}/terminfo \
+ ${libdir}/${BPN}/ptest \
+"
+
+sysroot_stage_dir() {
+ src="$1"
+ dest="$2"
+ # if the src doesn't exist don't do anything
+ if [ ! -d "$src" ]; then
+ return
+ fi
+
+ mkdir -p "$dest"
+ rdest=$(realpath --relative-to="$src" "$dest")
+ (
+ cd $src
+ find . -print0 | cpio --null -pdlu $rdest
+ )
+}
+
+sysroot_stage_dirs() {
+ from="$1"
+ to="$2"
+
+ for dir in ${SYSROOT_DIRS}; do
+ sysroot_stage_dir "$from$dir" "$to$dir"
+ done
+
+ # Remove directories we do not care about
+ for dir in ${SYSROOT_DIRS_IGNORE}; do
+ rm -rf "$to$dir"
+ done
+}
+
+sysroot_stage_all() {
+ sysroot_stage_dirs ${D} ${SYSROOT_DESTDIR}
+}
+
+python sysroot_strip () {
+ inhibit_sysroot = d.getVar('INHIBIT_SYSROOT_STRIP')
+ if inhibit_sysroot and oe.types.boolean(inhibit_sysroot):
+ return
+
+ dstdir = d.getVar('SYSROOT_DESTDIR')
+ pn = d.getVar('PN')
+ libdir = d.getVar("libdir")
+ base_libdir = d.getVar("base_libdir")
+ qa_already_stripped = 'already-stripped' in (d.getVar('INSANE_SKIP:' + pn) or "").split()
+ strip_cmd = d.getVar("STRIP")
+
+ oe.package.strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, d,
+ qa_already_stripped=qa_already_stripped)
+}
+
+do_populate_sysroot[dirs] = "${SYSROOT_DESTDIR}"
+
+addtask populate_sysroot after do_install
+
+SYSROOT_PREPROCESS_FUNCS ?= ""
+SYSROOT_DESTDIR = "${WORKDIR}/sysroot-destdir"
+
+python do_populate_sysroot () {
+ # SYSROOT 'version' 2
+ bb.build.exec_func("sysroot_stage_all", d)
+ bb.build.exec_func("sysroot_strip", d)
+ for f in (d.getVar('SYSROOT_PREPROCESS_FUNCS') or '').split():
+ bb.build.exec_func(f, d)
+ pn = d.getVar("PN")
+ multiprov = d.getVar("BB_MULTI_PROVIDER_ALLOWED").split()
+ provdir = d.expand("${SYSROOT_DESTDIR}${base_prefix}/sysroot-providers/")
+ bb.utils.mkdirhier(provdir)
+ for p in d.getVar("PROVIDES").split():
+ if p in multiprov:
+ continue
+ p = p.replace("/", "_")
+ with open(provdir + p, "w") as f:
+ f.write(pn)
+}
+
+do_populate_sysroot[vardeps] += "${SYSROOT_PREPROCESS_FUNCS}"
+do_populate_sysroot[vardepsexclude] += "BB_MULTI_PROVIDER_ALLOWED"
+
+POPULATESYSROOTDEPS = ""
+POPULATESYSROOTDEPS:class-target = "virtual/${MLPREFIX}${HOST_PREFIX}binutils:do_populate_sysroot"
+POPULATESYSROOTDEPS:class-nativesdk = "virtual/${HOST_PREFIX}binutils-crosssdk:do_populate_sysroot"
+do_populate_sysroot[depends] += "${POPULATESYSROOTDEPS}"
+
+SSTATETASKS += "do_populate_sysroot"
+do_populate_sysroot[cleandirs] = "${SYSROOT_DESTDIR}"
+do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}"
+do_populate_sysroot[sstate-outputdirs] = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/${PN}"
+do_populate_sysroot[sstate-fixmedir] = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/${PN}"
+
+python do_populate_sysroot_setscene () {
+ sstate_setscene(d)
+}
+addtask do_populate_sysroot_setscene
+
+def staging_copyfile(c, target, dest, postinsts, seendirs):
+ import errno
+
+ destdir = os.path.dirname(dest)
+ if destdir not in seendirs:
+ bb.utils.mkdirhier(destdir)
+ seendirs.add(destdir)
+ if "/usr/bin/postinst-" in c:
+ postinsts.append(dest)
+ if os.path.islink(c):
+ linkto = os.readlink(c)
+ if os.path.lexists(dest):
+ if not os.path.islink(dest):
+ raise OSError(errno.EEXIST, "Link %s already exists as a file" % dest, dest)
+ if os.readlink(dest) == linkto:
+ return dest
+ raise OSError(errno.EEXIST, "Link %s already exists to a different location? (%s vs %s)" % (dest, os.readlink(dest), linkto), dest)
+ os.symlink(linkto, dest)
+ #bb.warn(c)
+ else:
+ try:
+ os.link(c, dest)
+ except OSError as err:
+ if err.errno == errno.EXDEV:
+ bb.utils.copyfile(c, dest)
+ else:
+ raise
+ return dest
+
+def staging_copydir(c, target, dest, seendirs):
+ if dest not in seendirs:
+ bb.utils.mkdirhier(dest)
+ seendirs.add(dest)
+
+def staging_processfixme(fixme, target, recipesysroot, recipesysrootnative, d):
+ import subprocess
+
+ if not fixme:
+ return
+ cmd = "sed -e 's:^[^/]*/:%s/:g' %s | xargs sed -i -e 's:FIXMESTAGINGDIRTARGET:%s:g; s:FIXMESTAGINGDIRHOST:%s:g'" % (target, " ".join(fixme), recipesysroot, recipesysrootnative)
+ for fixmevar in ['PSEUDO_SYSROOT', 'HOSTTOOLS_DIR', 'PKGDATA_DIR', 'PSEUDO_LOCALSTATEDIR', 'LOGFIFO']:
+ fixme_path = d.getVar(fixmevar)
+ cmd += " -e 's:FIXME_%s:%s:g'" % (fixmevar, fixme_path)
+ bb.debug(2, cmd)
+ subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
+
+
+def staging_populate_sysroot_dir(targetsysroot, nativesysroot, native, d):
+ import glob
+ import subprocess
+ import errno
+
+ fixme = []
+ postinsts = []
+ seendirs = set()
+ stagingdir = d.getVar("STAGING_DIR")
+ if native:
+ pkgarchs = ['${BUILD_ARCH}', '${BUILD_ARCH}_*']
+ targetdir = nativesysroot
+ else:
+ pkgarchs = ['${MACHINE_ARCH}']
+ pkgarchs = pkgarchs + list(reversed(d.getVar("PACKAGE_EXTRA_ARCHS").split()))
+ pkgarchs.append('allarch')
+ targetdir = targetsysroot
+
+ bb.utils.mkdirhier(targetdir)
+ for pkgarch in pkgarchs:
+ for manifest in glob.glob(d.expand("${SSTATE_MANIFESTS}/manifest-%s-*.populate_sysroot" % pkgarch)):
+ if manifest.endswith("-initial.populate_sysroot"):
+ # skip libgcc-initial due to file overlap
+ continue
+ if not native and (manifest.endswith("-native.populate_sysroot") or "nativesdk-" in manifest):
+ continue
+ if native and not (manifest.endswith("-native.populate_sysroot") or manifest.endswith("-cross.populate_sysroot") or "-cross-" in manifest):
+ continue
+ tmanifest = targetdir + "/" + os.path.basename(manifest)
+ if os.path.exists(tmanifest):
+ continue
+ try:
+ os.link(manifest, tmanifest)
+ except OSError as err:
+ if err.errno == errno.EXDEV:
+ bb.utils.copyfile(manifest, tmanifest)
+ else:
+ raise
+ with open(manifest, "r") as f:
+ for l in f:
+ l = l.strip()
+ if l.endswith("/fixmepath"):
+ fixme.append(l)
+ continue
+ if l.endswith("/fixmepath.cmd"):
+ continue
+ dest = l.replace(stagingdir, "")
+ dest = targetdir + "/" + "/".join(dest.split("/")[3:])
+ if l.endswith("/"):
+ staging_copydir(l, targetdir, dest, seendirs)
+ continue
+ try:
+ staging_copyfile(l, targetdir, dest, postinsts, seendirs)
+ except FileExistsError:
+ continue
+
+ staging_processfixme(fixme, targetdir, targetsysroot, nativesysroot, d)
+ for p in postinsts:
+ subprocess.check_output(p, shell=True, stderr=subprocess.STDOUT)
+
+#
+# Manifests here are complicated. The main sysroot area has the unpacked sstate
+# which us unrelocated and tracked by the main sstate manifests. Each recipe
+# specific sysroot has manifests for each dependency that is installed there.
+# The task hash is used to tell whether the data needs to be reinstalled. We
+# use a symlink to point to the currently installed hash. There is also a
+# "complete" stamp file which is used to mark if installation completed. If
+# something fails (e.g. a postinst), this won't get written and we would
+# remove and reinstall the dependency. This also means partially installed
+# dependencies should get cleaned up correctly.
+#
+
+python extend_recipe_sysroot() {
+ import copy
+ import subprocess
+ import errno
+ import collections
+ import glob
+
+ taskdepdata = d.getVar("BB_TASKDEPDATA", False)
+ mytaskname = d.getVar("BB_RUNTASK")
+ if mytaskname.endswith("_setscene"):
+ mytaskname = mytaskname.replace("_setscene", "")
+ workdir = d.getVar("WORKDIR")
+ #bb.warn(str(taskdepdata))
+ pn = d.getVar("PN")
+ stagingdir = d.getVar("STAGING_DIR")
+ sharedmanifests = d.getVar("COMPONENTS_DIR") + "/manifests"
+ recipesysroot = d.getVar("RECIPE_SYSROOT")
+ recipesysrootnative = d.getVar("RECIPE_SYSROOT_NATIVE")
+
+ # Detect bitbake -b usage
+ nodeps = d.getVar("BB_LIMITEDDEPS") or False
+ if nodeps:
+ lock = bb.utils.lockfile(recipesysroot + "/sysroot.lock")
+ staging_populate_sysroot_dir(recipesysroot, recipesysrootnative, True, d)
+ staging_populate_sysroot_dir(recipesysroot, recipesysrootnative, False, d)
+ bb.utils.unlockfile(lock)
+ return
+
+ start = None
+ configuredeps = []
+ owntaskdeps = []
+ for dep in taskdepdata:
+ data = taskdepdata[dep]
+ if data[1] == mytaskname and data[0] == pn:
+ start = dep
+ elif data[0] == pn:
+ owntaskdeps.append(data[1])
+ if start is None:
+ bb.fatal("Couldn't find ourself in BB_TASKDEPDATA?")
+
+ # We need to figure out which sysroot files we need to expose to this task.
+ # This needs to match what would get restored from sstate, which is controlled
+ # ultimately by calls from bitbake to setscene_depvalid().
+ # That function expects a setscene dependency tree. We build a dependency tree
+ # condensed to inter-sstate task dependencies, similar to that used by setscene
+ # tasks. We can then call into setscene_depvalid() and decide
+ # which dependencies we can "see" and should expose in the recipe specific sysroot.
+ setscenedeps = copy.deepcopy(taskdepdata)
+
+ start = set([start])
+
+ sstatetasks = d.getVar("SSTATETASKS").split()
+ # Add recipe specific tasks referenced by setscene_depvalid()
+ sstatetasks.append("do_stash_locale")
+ sstatetasks.append("do_deploy")
+
+ def print_dep_tree(deptree):
+ data = ""
+ for dep in deptree:
+ deps = " " + "\n ".join(deptree[dep][3]) + "\n"
+ data = data + "%s:\n %s\n %s\n%s %s\n %s\n" % (deptree[dep][0], deptree[dep][1], deptree[dep][2], deps, deptree[dep][4], deptree[dep][5])
+ return data
+
+ #bb.note("Full dep tree is:\n%s" % print_dep_tree(taskdepdata))
+
+ #bb.note(" start2 is %s" % str(start))
+
+ # If start is an sstate task (like do_package) we need to add in its direct dependencies
+ # else the code below won't recurse into them.
+ for dep in set(start):
+ for dep2 in setscenedeps[dep][3]:
+ start.add(dep2)
+ start.remove(dep)
+
+ #bb.note(" start3 is %s" % str(start))
+
+ # Create collapsed do_populate_sysroot -> do_populate_sysroot tree
+ for dep in taskdepdata:
+ data = setscenedeps[dep]
+ if data[1] not in sstatetasks:
+ for dep2 in setscenedeps:
+ data2 = setscenedeps[dep2]
+ if dep in data2[3]:
+ data2[3].update(setscenedeps[dep][3])
+ data2[3].remove(dep)
+ if dep in start:
+ start.update(setscenedeps[dep][3])
+ start.remove(dep)
+ del setscenedeps[dep]
+
+ # Remove circular references
+ for dep in setscenedeps:
+ if dep in setscenedeps[dep][3]:
+ setscenedeps[dep][3].remove(dep)
+
+ #bb.note("Computed dep tree is:\n%s" % print_dep_tree(setscenedeps))
+ #bb.note(" start is %s" % str(start))
+
+ # Direct dependencies should be present and can be depended upon
+ for dep in sorted(set(start)):
+ if setscenedeps[dep][1] == "do_populate_sysroot":
+ if dep not in configuredeps:
+ configuredeps.append(dep)
+ bb.note("Direct dependencies are %s" % str(configuredeps))
+ #bb.note(" or %s" % str(start))
+
+ msgbuf = []
+ # Call into setscene_depvalid for each sub-dependency and only copy sysroot files
+ # for ones that would be restored from sstate.
+ done = list(start)
+ next = list(start)
+ while next:
+ new = []
+ for dep in next:
+ data = setscenedeps[dep]
+ for datadep in data[3]:
+ if datadep in done:
+ continue
+ taskdeps = {}
+ taskdeps[dep] = setscenedeps[dep][:2]
+ taskdeps[datadep] = setscenedeps[datadep][:2]
+ retval = setscene_depvalid(datadep, taskdeps, [], d, msgbuf)
+ if retval:
+ msgbuf.append("Skipping setscene dependency %s for installation into the sysroot" % datadep)
+ continue
+ done.append(datadep)
+ new.append(datadep)
+ if datadep not in configuredeps and setscenedeps[datadep][1] == "do_populate_sysroot":
+ configuredeps.append(datadep)
+ msgbuf.append("Adding dependency on %s" % setscenedeps[datadep][0])
+ else:
+ msgbuf.append("Following dependency on %s" % setscenedeps[datadep][0])
+ next = new
+
+ # This logging is too verbose for day to day use sadly
+ #bb.debug(2, "\n".join(msgbuf))
+
+ depdir = recipesysrootnative + "/installeddeps"
+ bb.utils.mkdirhier(depdir)
+ bb.utils.mkdirhier(sharedmanifests)
+
+ lock = bb.utils.lockfile(recipesysroot + "/sysroot.lock")
+
+ fixme = {}
+ seendirs = set()
+ postinsts = []
+ multilibs = {}
+ manifests = {}
+ # All files that we're going to be installing, to find conflicts.
+ fileset = {}
+
+ invalidate_tasks = set()
+ for f in os.listdir(depdir):
+ removed = []
+ if not f.endswith(".complete"):
+ continue
+ f = depdir + "/" + f
+ if os.path.islink(f) and not os.path.exists(f):
+ bb.note("%s no longer exists, removing from sysroot" % f)
+ lnk = os.readlink(f.replace(".complete", ""))
+ sstate_clean_manifest(depdir + "/" + lnk, d, canrace=True, prefix=workdir)
+ os.unlink(f)
+ os.unlink(f.replace(".complete", ""))
+ removed.append(os.path.basename(f.replace(".complete", "")))
+
+ # If we've removed files from the sysroot above, the task that installed them may still
+ # have a stamp file present for the task. This is probably invalid right now but may become
+ # valid again if the user were to change configuration back for example. Since we've removed
+ # the files a task might need, remove the stamp file too to force it to rerun.
+ # YOCTO #14790
+ if removed:
+ for i in glob.glob(depdir + "/index.*"):
+ if i.endswith("." + mytaskname):
+ continue
+ with open(i, "r") as f:
+ for l in f:
+ if l.startswith("TaskDeps:"):
+ continue
+ l = l.strip()
+ if l in removed:
+ invalidate_tasks.add(i.rsplit(".", 1)[1])
+ break
+ for t in invalidate_tasks:
+ bb.note("Invalidating stamps for task %s" % t)
+ bb.build.clean_stamp(t, d)
+
+ installed = []
+ for dep in configuredeps:
+ c = setscenedeps[dep][0]
+ if mytaskname in ["do_sdk_depends", "do_populate_sdk_ext"] and c.endswith("-initial"):
+ bb.note("Skipping initial setscene dependency %s for installation into the sysroot" % c)
+ continue
+ installed.append(c)
+
+ # We want to remove anything which this task previously installed but is no longer a dependency
+ taskindex = depdir + "/" + "index." + mytaskname
+ if os.path.exists(taskindex):
+ potential = []
+ with open(taskindex, "r") as f:
+ for l in f:
+ l = l.strip()
+ if l not in installed:
+ fl = depdir + "/" + l
+ if not os.path.exists(fl):
+ # Was likely already uninstalled
+ continue
+ potential.append(l)
+ # We need to ensure no other task needs this dependency. We hold the sysroot
+ # lock so we ca search the indexes to check
+ if potential:
+ for i in glob.glob(depdir + "/index.*"):
+ if i.endswith("." + mytaskname):
+ continue
+ with open(i, "r") as f:
+ for l in f:
+ if l.startswith("TaskDeps:"):
+ prevtasks = l.split()[1:]
+ if mytaskname in prevtasks:
+ # We're a dependency of this task so we can clear items out the sysroot
+ break
+ l = l.strip()
+ if l in potential:
+ potential.remove(l)
+ for l in potential:
+ fl = depdir + "/" + l
+ bb.note("Task %s no longer depends on %s, removing from sysroot" % (mytaskname, l))
+ lnk = os.readlink(fl)
+ sstate_clean_manifest(depdir + "/" + lnk, d, canrace=True, prefix=workdir)
+ os.unlink(fl)
+ os.unlink(fl + ".complete")
+
+ msg_exists = []
+ msg_adding = []
+
+ # Handle all removals first since files may move between recipes
+ for dep in configuredeps:
+ c = setscenedeps[dep][0]
+ if c not in installed:
+ continue
+ taskhash = setscenedeps[dep][5]
+ taskmanifest = depdir + "/" + c + "." + taskhash
+
+ if os.path.exists(depdir + "/" + c):
+ lnk = os.readlink(depdir + "/" + c)
+ if lnk == c + "." + taskhash and os.path.exists(depdir + "/" + c + ".complete"):
+ continue
+ else:
+ bb.note("%s exists in sysroot, but is stale (%s vs. %s), removing." % (c, lnk, c + "." + taskhash))
+ sstate_clean_manifest(depdir + "/" + lnk, d, canrace=True, prefix=workdir)
+ os.unlink(depdir + "/" + c)
+ if os.path.lexists(depdir + "/" + c + ".complete"):
+ os.unlink(depdir + "/" + c + ".complete")
+ elif os.path.lexists(depdir + "/" + c):
+ os.unlink(depdir + "/" + c)
+
+ binfiles = {}
+ # Now handle installs
+ for dep in configuredeps:
+ c = setscenedeps[dep][0]
+ if c not in installed:
+ continue
+ taskhash = setscenedeps[dep][5]
+ taskmanifest = depdir + "/" + c + "." + taskhash
+
+ if os.path.exists(depdir + "/" + c):
+ lnk = os.readlink(depdir + "/" + c)
+ if lnk == c + "." + taskhash and os.path.exists(depdir + "/" + c + ".complete"):
+ msg_exists.append(c)
+ continue
+
+ msg_adding.append(c)
+
+ os.symlink(c + "." + taskhash, depdir + "/" + c)
+
+ manifest, d2 = oe.sstatesig.find_sstate_manifest(c, setscenedeps[dep][2], "populate_sysroot", d, multilibs)
+ if d2 is not d:
+ # If we don't do this, the recipe sysroot will be placed in the wrong WORKDIR for multilibs
+ # We need a consistent WORKDIR for the image
+ d2.setVar("WORKDIR", d.getVar("WORKDIR"))
+ destsysroot = d2.getVar("RECIPE_SYSROOT")
+ # We put allarch recipes into the default sysroot
+ if manifest and "allarch" in manifest:
+ destsysroot = d.getVar("RECIPE_SYSROOT")
+
+ native = False
+ if c.endswith("-native") or "-cross-" in c or "-crosssdk" in c:
+ native = True
+
+ if manifest:
+ newmanifest = collections.OrderedDict()
+ targetdir = destsysroot
+ if native:
+ targetdir = recipesysrootnative
+ if targetdir not in fixme:
+ fixme[targetdir] = []
+ fm = fixme[targetdir]
+
+ with open(manifest, "r") as f:
+ manifests[dep] = manifest
+ for l in f:
+ l = l.strip()
+ if l.endswith("/fixmepath"):
+ fm.append(l)
+ continue
+ if l.endswith("/fixmepath.cmd"):
+ continue
+ dest = l.replace(stagingdir, "")
+ dest = "/" + "/".join(dest.split("/")[3:])
+ newmanifest[l] = targetdir + dest
+
+ # Check if files have already been installed by another
+ # recipe and abort if they have, explaining what recipes are
+ # conflicting.
+ hashname = targetdir + dest
+ if not hashname.endswith("/"):
+ if hashname in fileset:
+ bb.fatal("The file %s is installed by both %s and %s, aborting" % (dest, c, fileset[hashname]))
+ else:
+ fileset[hashname] = c
+
+ # Having multiple identical manifests in each sysroot eats diskspace so
+ # create a shared pool of them and hardlink if we can.
+ # We create the manifest in advance so that if something fails during installation,
+ # or the build is interrupted, subsequent exeuction can cleanup.
+ sharedm = sharedmanifests + "/" + os.path.basename(taskmanifest)
+ if not os.path.exists(sharedm):
+ smlock = bb.utils.lockfile(sharedm + ".lock")
+ # Can race here. You'd think it just means we may not end up with all copies hardlinked to each other
+ # but python can lose file handles so we need to do this under a lock.
+ if not os.path.exists(sharedm):
+ with open(sharedm, 'w') as m:
+ for l in newmanifest:
+ dest = newmanifest[l]
+ m.write(dest.replace(workdir + "/", "") + "\n")
+ bb.utils.unlockfile(smlock)
+ try:
+ os.link(sharedm, taskmanifest)
+ except OSError as err:
+ if err.errno == errno.EXDEV:
+ bb.utils.copyfile(sharedm, taskmanifest)
+ else:
+ raise
+ # Finally actually install the files
+ for l in newmanifest:
+ dest = newmanifest[l]
+ if l.endswith("/"):
+ staging_copydir(l, targetdir, dest, seendirs)
+ continue
+ if "/bin/" in l or "/sbin/" in l:
+ # defer /*bin/* files until last in case they need libs
+ binfiles[l] = (targetdir, dest)
+ else:
+ staging_copyfile(l, targetdir, dest, postinsts, seendirs)
+
+ # Handle deferred binfiles
+ for l in binfiles:
+ (targetdir, dest) = binfiles[l]
+ staging_copyfile(l, targetdir, dest, postinsts, seendirs)
+
+ bb.note("Installed into sysroot: %s" % str(msg_adding))
+ bb.note("Skipping as already exists in sysroot: %s" % str(msg_exists))
+
+ for f in fixme:
+ staging_processfixme(fixme[f], f, recipesysroot, recipesysrootnative, d)
+
+ for p in postinsts:
+ subprocess.check_output(p, shell=True, stderr=subprocess.STDOUT)
+
+ for dep in manifests:
+ c = setscenedeps[dep][0]
+ os.symlink(manifests[dep], depdir + "/" + c + ".complete")
+
+ with open(taskindex, "w") as f:
+ f.write("TaskDeps: " + " ".join(owntaskdeps) + "\n")
+ for l in sorted(installed):
+ f.write(l + "\n")
+
+ bb.utils.unlockfile(lock)
+}
+extend_recipe_sysroot[vardepsexclude] += "MACHINE_ARCH PACKAGE_EXTRA_ARCHS SDK_ARCH BUILD_ARCH SDK_OS BB_TASKDEPDATA"
+
+do_prepare_recipe_sysroot[deptask] = "do_populate_sysroot"
+python do_prepare_recipe_sysroot () {
+ bb.build.exec_func("extend_recipe_sysroot", d)
+}
+addtask do_prepare_recipe_sysroot before do_configure after do_fetch
+
+python staging_taskhandler() {
+ bbtasks = e.tasklist
+ for task in bbtasks:
+ deps = d.getVarFlag(task, "depends")
+ if task == "do_configure" or (deps and "populate_sysroot" in deps):
+ d.prependVarFlag(task, "prefuncs", "extend_recipe_sysroot ")
+}
+staging_taskhandler[eventmask] = "bb.event.RecipeTaskPreProcess"
+addhandler staging_taskhandler
+
+
+#
+# Target build output, stored in do_populate_sysroot or do_package can depend
+# not only upon direct dependencies but also indirect ones. A good example is
+# linux-libc-headers. The toolchain depends on this but most target recipes do
+# not. There are some headers which are not used by the toolchain build and do
+# not change the toolchain task output, hence the task hashes can change without
+# changing the sysroot output of that recipe yet they can influence others.
+#
+# A specific example is rtc.h which can change rtcwake.c in util-linux but is not
+# used in the glibc or gcc build. To account for this, we need to account for the
+# populate_sysroot hashes in the task output hashes.
+#
+python target_add_sysroot_deps () {
+ current_task = "do_" + d.getVar("BB_CURRENTTASK")
+ if current_task not in ["do_populate_sysroot", "do_package"]:
+ return
+
+ pn = d.getVar("PN")
+ if pn.endswith("-native"):
+ return
+
+ taskdepdata = d.getVar("BB_TASKDEPDATA", False)
+ deps = {}
+ for dep in taskdepdata.values():
+ if dep[1] == "do_populate_sysroot" and not dep[0].endswith(("-native", "-initial")) and "-cross-" not in dep[0] and dep[0] != pn:
+ deps[dep[0]] = dep[6]
+
+ d.setVar("HASHEQUIV_EXTRA_SIGDATA", "\n".join("%s: %s" % (k, deps[k]) for k in sorted(deps.keys())))
+}
+SSTATECREATEFUNCS += "target_add_sysroot_deps"
+
diff --git a/poky/meta/classes-global/uninative.bbclass b/poky/meta/classes-global/uninative.bbclass
new file mode 100644
index 0000000..42c5f8f
--- /dev/null
+++ b/poky/meta/classes-global/uninative.bbclass
@@ -0,0 +1,179 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+UNINATIVE_LOADER ?= "${UNINATIVE_STAGING_DIR}-uninative/${BUILD_ARCH}-linux/lib/${@bb.utils.contains('BUILD_ARCH', 'x86_64', 'ld-linux-x86-64.so.2', '', d)}${@bb.utils.contains('BUILD_ARCH', 'i686', 'ld-linux.so.2', '', d)}${@bb.utils.contains('BUILD_ARCH', 'aarch64', 'ld-linux-aarch64.so.1', '', d)}${@bb.utils.contains('BUILD_ARCH', 'ppc64le', 'ld64.so.2', '', d)}"
+UNINATIVE_STAGING_DIR ?= "${STAGING_DIR}"
+
+UNINATIVE_URL ?= "unset"
+UNINATIVE_TARBALL ?= "${BUILD_ARCH}-nativesdk-libc-${UNINATIVE_VERSION}.tar.xz"
+# Example checksums
+#UNINATIVE_CHECKSUM[aarch64] = "dead"
+#UNINATIVE_CHECKSUM[i686] = "dead"
+#UNINATIVE_CHECKSUM[x86_64] = "dead"
+UNINATIVE_DLDIR ?= "${DL_DIR}/uninative/"
+
+# Enabling uninative will change the following variables so they need to go the parsing ignored variables list to prevent multiple recipe parsing
+BB_HASHCONFIG_IGNORE_VARS += "NATIVELSBSTRING SSTATEPOSTUNPACKFUNCS BUILD_LDFLAGS"
+
+addhandler uninative_event_fetchloader
+uninative_event_fetchloader[eventmask] = "bb.event.BuildStarted"
+
+addhandler uninative_event_enable
+uninative_event_enable[eventmask] = "bb.event.ConfigParsed"
+
+python uninative_event_fetchloader() {
+ """
+ This event fires on the parent and will try to fetch the tarball if the
+ loader isn't already present.
+ """
+
+ chksum = d.getVarFlag("UNINATIVE_CHECKSUM", d.getVar("BUILD_ARCH"))
+ if not chksum:
+ bb.fatal("Uninative selected but not configured correctly, please set UNINATIVE_CHECKSUM[%s]" % d.getVar("BUILD_ARCH"))
+
+ loader = d.getVar("UNINATIVE_LOADER")
+ loaderchksum = loader + ".chksum"
+ if os.path.exists(loader) and os.path.exists(loaderchksum):
+ with open(loaderchksum, "r") as f:
+ readchksum = f.read().strip()
+ if readchksum == chksum:
+ return
+
+ import subprocess
+ try:
+ # Save and restore cwd as Fetch.download() does a chdir()
+ olddir = os.getcwd()
+
+ tarball = d.getVar("UNINATIVE_TARBALL")
+ tarballdir = os.path.join(d.getVar("UNINATIVE_DLDIR"), chksum)
+ tarballpath = os.path.join(tarballdir, tarball)
+
+ if not os.path.exists(tarballpath + ".done"):
+ bb.utils.mkdirhier(tarballdir)
+ if d.getVar("UNINATIVE_URL") == "unset":
+ bb.fatal("Uninative selected but not configured, please set UNINATIVE_URL")
+
+ localdata = bb.data.createCopy(d)
+ localdata.setVar('FILESPATH', "")
+ localdata.setVar('DL_DIR', tarballdir)
+ # Our games with path manipulation of DL_DIR mean standard PREMIRRORS don't work
+ # and we can't easily put 'chksum' into the url path from a url parameter with
+ # the current fetcher url handling
+ premirrors = bb.fetch2.mirror_from_string(localdata.getVar("PREMIRRORS"))
+ for line in premirrors:
+ try:
+ (find, replace) = line
+ except ValueError:
+ continue
+ if find.startswith("http"):
+ localdata.appendVar("PREMIRRORS", " ${UNINATIVE_URL}${UNINATIVE_TARBALL} %s/uninative/%s/${UNINATIVE_TARBALL}" % (replace, chksum))
+
+ srcuri = d.expand("${UNINATIVE_URL}${UNINATIVE_TARBALL};sha256sum=%s" % chksum)
+ bb.note("Fetching uninative binary shim %s (will check PREMIRRORS first)" % srcuri)
+
+ fetcher = bb.fetch2.Fetch([srcuri], localdata, cache=False)
+ fetcher.download()
+ localpath = fetcher.localpath(srcuri)
+ if localpath != tarballpath and os.path.exists(localpath) and not os.path.exists(tarballpath):
+ # Follow the symlink behavior from the bitbake fetch2.
+ # This will cover the case where an existing symlink is broken
+ # as well as if there are two processes trying to create it
+ # at the same time.
+ if os.path.islink(tarballpath):
+ # Broken symbolic link
+ os.unlink(tarballpath)
+
+ # Deal with two processes trying to make symlink at once
+ try:
+ os.symlink(localpath, tarballpath)
+ except FileExistsError:
+ pass
+
+ # ldd output is "ldd (Ubuntu GLIBC 2.23-0ubuntu10) 2.23", extract last option from first line
+ glibcver = subprocess.check_output(["ldd", "--version"]).decode('utf-8').split('\n')[0].split()[-1]
+ if bb.utils.vercmp_string(d.getVar("UNINATIVE_MAXGLIBCVERSION"), glibcver) < 0:
+ raise RuntimeError("Your host glibc version (%s) is newer than that in uninative (%s). Disabling uninative so that sstate is not corrupted." % (glibcver, d.getVar("UNINATIVE_MAXGLIBCVERSION")))
+
+ cmd = d.expand("\
+mkdir -p ${UNINATIVE_STAGING_DIR}-uninative; \
+cd ${UNINATIVE_STAGING_DIR}-uninative; \
+tar -xJf ${UNINATIVE_DLDIR}/%s/${UNINATIVE_TARBALL}; \
+${UNINATIVE_STAGING_DIR}-uninative/relocate_sdk.py \
+ ${UNINATIVE_STAGING_DIR}-uninative/${BUILD_ARCH}-linux \
+ ${UNINATIVE_LOADER} \
+ ${UNINATIVE_LOADER} \
+ ${UNINATIVE_STAGING_DIR}-uninative/${BUILD_ARCH}-linux/${bindir_native}/patchelf-uninative \
+ ${UNINATIVE_STAGING_DIR}-uninative/${BUILD_ARCH}-linux${base_libdir_native}/libc*.so*" % chksum)
+ subprocess.check_output(cmd, shell=True)
+
+ with open(loaderchksum, "w") as f:
+ f.write(chksum)
+
+ enable_uninative(d)
+
+ except RuntimeError as e:
+ bb.warn(str(e))
+ except bb.fetch2.BBFetchException as exc:
+ bb.warn("Disabling uninative as unable to fetch uninative tarball: %s" % str(exc))
+ bb.warn("To build your own uninative loader, please bitbake uninative-tarball and set UNINATIVE_TARBALL appropriately.")
+ except subprocess.CalledProcessError as exc:
+ bb.warn("Disabling uninative as unable to install uninative tarball: %s" % str(exc))
+ bb.warn("To build your own uninative loader, please bitbake uninative-tarball and set UNINATIVE_TARBALL appropriately.")
+ finally:
+ os.chdir(olddir)
+}
+
+python uninative_event_enable() {
+ """
+ This event handler is called in the workers and is responsible for setting
+ up uninative if a loader is found.
+ """
+ enable_uninative(d)
+}
+
+def enable_uninative(d):
+ loader = d.getVar("UNINATIVE_LOADER")
+ if os.path.exists(loader):
+ bb.debug(2, "Enabling uninative")
+ d.setVar("NATIVELSBSTRING", "universal%s" % oe.utils.host_gcc_version(d))
+ d.appendVar("SSTATEPOSTUNPACKFUNCS", " uninative_changeinterp")
+ d.appendVarFlag("SSTATEPOSTUNPACKFUNCS", "vardepvalueexclude", "| uninative_changeinterp")
+ d.appendVar("BUILD_LDFLAGS", " -Wl,--allow-shlib-undefined -Wl,--dynamic-linker=${UNINATIVE_LOADER}")
+ d.appendVarFlag("BUILD_LDFLAGS", "vardepvalueexclude", "| -Wl,--allow-shlib-undefined -Wl,--dynamic-linker=${UNINATIVE_LOADER}")
+ d.appendVarFlag("BUILD_LDFLAGS", "vardepsexclude", "UNINATIVE_LOADER")
+ d.prependVar("PATH", "${STAGING_DIR}-uninative/${BUILD_ARCH}-linux${bindir_native}:")
+
+python uninative_changeinterp () {
+ import subprocess
+ import stat
+ import oe.qa
+
+ if not (bb.data.inherits_class('native', d) or bb.data.inherits_class('crosssdk', d) or bb.data.inherits_class('cross', d)):
+ return
+
+ sstateinst = d.getVar('SSTATE_INSTDIR')
+ for walkroot, dirs, files in os.walk(sstateinst):
+ for file in files:
+ if file.endswith(".so") or ".so." in file:
+ continue
+ f = os.path.join(walkroot, file)
+ if os.path.islink(f):
+ continue
+ s = os.stat(f)
+ if not ((s[stat.ST_MODE] & stat.S_IXUSR) or (s[stat.ST_MODE] & stat.S_IXGRP) or (s[stat.ST_MODE] & stat.S_IXOTH)):
+ continue
+ elf = oe.qa.ELFFile(f)
+ try:
+ elf.open()
+ except oe.qa.NotELFFileError:
+ continue
+ if not elf.isDynamic():
+ continue
+
+ os.chmod(f, s[stat.ST_MODE] | stat.S_IWUSR)
+ subprocess.check_output(("patchelf-uninative", "--set-interpreter", d.getVar("UNINATIVE_LOADER"), f), stderr=subprocess.STDOUT)
+ os.chmod(f, s[stat.ST_MODE])
+}
diff --git a/poky/meta/classes-global/utility-tasks.bbclass b/poky/meta/classes-global/utility-tasks.bbclass
new file mode 100644
index 0000000..ae2da33
--- /dev/null
+++ b/poky/meta/classes-global/utility-tasks.bbclass
@@ -0,0 +1,60 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+addtask listtasks
+do_listtasks[nostamp] = "1"
+python do_listtasks() {
+ taskdescs = {}
+ maxlen = 0
+ for e in d.keys():
+ if d.getVarFlag(e, 'task'):
+ maxlen = max(maxlen, len(e))
+ if e.endswith('_setscene'):
+ desc = "%s (setscene version)" % (d.getVarFlag(e[:-9], 'doc') or '')
+ else:
+ desc = d.getVarFlag(e, 'doc') or ''
+ taskdescs[e] = desc
+
+ tasks = sorted(taskdescs.keys())
+ for taskname in tasks:
+ bb.plain("%s %s" % (taskname.ljust(maxlen), taskdescs[taskname]))
+}
+
+CLEANFUNCS ?= ""
+
+T:task-clean = "${LOG_DIR}/cleanlogs/${PN}"
+addtask clean
+do_clean[nostamp] = "1"
+python do_clean() {
+ """clear the build and temp directories"""
+ dir = d.expand("${WORKDIR}")
+ bb.note("Removing " + dir)
+ oe.path.remove(dir)
+
+ dir = "%s.*" % d.getVar('STAMP')
+ bb.note("Removing " + dir)
+ oe.path.remove(dir)
+
+ for f in (d.getVar('CLEANFUNCS') or '').split():
+ bb.build.exec_func(f, d)
+}
+
+addtask checkuri
+do_checkuri[nostamp] = "1"
+do_checkuri[network] = "1"
+python do_checkuri() {
+ src_uri = (d.getVar('SRC_URI') or "").split()
+ if len(src_uri) == 0:
+ return
+
+ try:
+ fetcher = bb.fetch2.Fetch(src_uri, d)
+ fetcher.checkstatus()
+ except bb.fetch2.BBFetchException as e:
+ bb.fatal(str(e))
+}
+
+
diff --git a/poky/meta/classes-global/utils.bbclass b/poky/meta/classes-global/utils.bbclass
new file mode 100644
index 0000000..8d797ff
--- /dev/null
+++ b/poky/meta/classes-global/utils.bbclass
@@ -0,0 +1,369 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+oe_soinstall() {
+ # Purpose: Install shared library file and
+ # create the necessary links
+ # Example: oe_soinstall libfoo.so.1.2.3 ${D}${libdir}
+ libname=`basename $1`
+ case "$libname" in
+ *.so)
+ bbfatal "oe_soinstall: Shared library must haved versioned filename (e.g. libfoo.so.1.2.3)"
+ ;;
+ esac
+ install -m 755 $1 $2/$libname
+ sonamelink=`${HOST_PREFIX}readelf -d $1 |grep 'Library soname:' |sed -e 's/.*\[\(.*\)\].*/\1/'`
+ if [ -z $sonamelink ]; then
+ bbfatal "oe_soinstall: $libname is missing ELF tag 'SONAME'."
+ fi
+ solink=`echo $libname | sed -e 's/\.so\..*/.so/'`
+ ln -sf $libname $2/$sonamelink
+ ln -sf $libname $2/$solink
+}
+
+oe_libinstall() {
+ # Purpose: Install a library, in all its forms
+ # Example
+ #
+ # oe_libinstall libltdl ${STAGING_LIBDIR}/
+ # oe_libinstall -C src/libblah libblah ${D}/${libdir}/
+ dir=""
+ libtool=""
+ silent=""
+ require_static=""
+ require_shared=""
+ while [ "$#" -gt 0 ]; do
+ case "$1" in
+ -C)
+ shift
+ dir="$1"
+ ;;
+ -s)
+ silent=1
+ ;;
+ -a)
+ require_static=1
+ ;;
+ -so)
+ require_shared=1
+ ;;
+ -*)
+ bbfatal "oe_libinstall: unknown option: $1"
+ ;;
+ *)
+ break;
+ ;;
+ esac
+ shift
+ done
+
+ libname="$1"
+ shift
+ destpath="$1"
+ if [ -z "$destpath" ]; then
+ bbfatal "oe_libinstall: no destination path specified"
+ fi
+
+ __runcmd () {
+ if [ -z "$silent" ]; then
+ echo >&2 "oe_libinstall: $*"
+ fi
+ $*
+ }
+
+ if [ -z "$dir" ]; then
+ dir=`pwd`
+ fi
+
+ dotlai=$libname.lai
+
+ # Sanity check that the libname.lai is unique
+ number_of_files=`(cd $dir; find . -name "$dotlai") | wc -l`
+ if [ $number_of_files -gt 1 ]; then
+ bbfatal "oe_libinstall: $dotlai is not unique in $dir"
+ fi
+
+
+ dir=$dir`(cd $dir;find . -name "$dotlai") | sed "s/^\.//;s/\/$dotlai\$//;q"`
+ olddir=`pwd`
+ __runcmd cd $dir
+
+ lafile=$libname.la
+
+ # If such file doesn't exist, try to cut version suffix
+ if [ ! -f "$lafile" ]; then
+ libname1=`echo "$libname" | sed 's/-[0-9.]*$//'`
+ lafile1=$libname.la
+ if [ -f "$lafile1" ]; then
+ libname=$libname1
+ lafile=$lafile1
+ fi
+ fi
+
+ if [ -f "$lafile" ]; then
+ # libtool archive
+ eval `cat $lafile|grep "^library_names="`
+ libtool=1
+ else
+ library_names="$libname.so* $libname.dll.a $libname.*.dylib"
+ fi
+
+ __runcmd install -d $destpath/
+ dota=$libname.a
+ if [ -f "$dota" -o -n "$require_static" ]; then
+ rm -f $destpath/$dota
+ __runcmd install -m 0644 $dota $destpath/
+ fi
+ if [ -f "$dotlai" -a -n "$libtool" ]; then
+ rm -f $destpath/$libname.la
+ __runcmd install -m 0644 $dotlai $destpath/$libname.la
+ fi
+
+ for name in $library_names; do
+ files=`eval echo $name`
+ for f in $files; do
+ if [ ! -e "$f" ]; then
+ if [ -n "$libtool" ]; then
+ bbfatal "oe_libinstall: $dir/$f not found."
+ fi
+ elif [ -L "$f" ]; then
+ __runcmd cp -P "$f" $destpath/
+ elif [ ! -L "$f" ]; then
+ libfile="$f"
+ rm -f $destpath/$libfile
+ __runcmd install -m 0755 $libfile $destpath/
+ fi
+ done
+ done
+
+ if [ -z "$libfile" ]; then
+ if [ -n "$require_shared" ]; then
+ bbfatal "oe_libinstall: unable to locate shared library"
+ fi
+ elif [ -z "$libtool" ]; then
+ # special case hack for non-libtool .so.#.#.# links
+ baselibfile=`basename "$libfile"`
+ if (echo $baselibfile | grep -qE '^lib.*\.so\.[0-9.]*$'); then
+ sonamelink=`${HOST_PREFIX}readelf -d $libfile |grep 'Library soname:' |sed -e 's/.*\[\(.*\)\].*/\1/'`
+ solink=`echo $baselibfile | sed -e 's/\.so\..*/.so/'`
+ if [ -n "$sonamelink" -a x"$baselibfile" != x"$sonamelink" ]; then
+ __runcmd ln -sf $baselibfile $destpath/$sonamelink
+ fi
+ __runcmd ln -sf $baselibfile $destpath/$solink
+ fi
+ fi
+
+ __runcmd cd "$olddir"
+}
+
+create_cmdline_wrapper () {
+ # Create a wrapper script where commandline options are needed
+ #
+ # These are useful to work around relocation issues, by passing extra options
+ # to a program
+ #
+ # Usage: create_cmdline_wrapper FILENAME <extra-options>
+
+ cmd=$1
+ shift
+
+ echo "Generating wrapper script for $cmd"
+
+ mv $cmd $cmd.real
+ cmdname=`basename $cmd`
+ dirname=`dirname $cmd`
+ cmdoptions=$@
+ if [ "${base_prefix}" != "" ]; then
+ relpath=`python3 -c "import os; print(os.path.relpath('${D}${base_prefix}', '$dirname'))"`
+ cmdoptions=`echo $@ | sed -e "s:${base_prefix}:\\$realdir/$relpath:g"`
+ fi
+ cat <<END >$cmd
+#!/bin/bash
+realpath=\`readlink -fn \$0\`
+realdir=\`dirname \$realpath\`
+exec -a \$realdir/$cmdname \$realdir/$cmdname.real $cmdoptions "\$@"
+END
+ chmod +x $cmd
+}
+
+create_cmdline_shebang_wrapper () {
+ # Create a wrapper script where commandline options are needed
+ #
+ # These are useful to work around shebang relocation issues, where shebangs are too
+ # long or have arguments in them, thus preventing them from using the /usr/bin/env
+ # shebang
+ #
+ # Usage: create_cmdline_wrapper FILENAME <extra-options>
+
+ cmd=$1
+ shift
+
+ echo "Generating wrapper script for $cmd"
+
+ # Strip #! and get remaining interpreter + arg
+ argument="$(sed -ne 's/^#! *//p;q' $cmd)"
+ # strip the shebang from the real script as we do not want it to be usable anyway
+ tail -n +2 $cmd > $cmd.real
+ chown --reference=$cmd $cmd.real
+ chmod --reference=$cmd $cmd.real
+ rm -f $cmd
+ cmdname=$(basename $cmd)
+ dirname=$(dirname $cmd)
+ cmdoptions=$@
+ if [ "${base_prefix}" != "" ]; then
+ relpath=`python3 -c "import os; print(os.path.relpath('${D}${base_prefix}', '$dirname'))"`
+ cmdoptions=`echo $@ | sed -e "s:${base_prefix}:\\$realdir/$relpath:g"`
+ fi
+ cat <<END >$cmd
+#!/usr/bin/env bash
+realpath=\`readlink -fn \$0\`
+realdir=\`dirname \$realpath\`
+exec -a \$realdir/$cmdname $argument \$realdir/$cmdname.real $cmdoptions "\$@"
+END
+ chmod +x $cmd
+}
+
+create_wrapper () {
+ # Create a wrapper script where extra environment variables are needed
+ #
+ # These are useful to work around relocation issues, by setting environment
+ # variables which point to paths in the filesystem.
+ #
+ # Usage: create_wrapper FILENAME [[VAR=VALUE]..]
+
+ cmd=$1
+ shift
+
+ echo "Generating wrapper script for $cmd"
+
+ mv $cmd $cmd.real
+ cmdname=`basename $cmd`
+ dirname=`dirname $cmd`
+ exportstring=$@
+ if [ "${base_prefix}" != "" ]; then
+ relpath=`python3 -c "import os; print(os.path.relpath('${D}${base_prefix}', '$dirname'))"`
+ exportstring=`echo $@ | sed -e "s:${base_prefix}:\\$realdir/$relpath:g"`
+ fi
+ cat <<END >$cmd
+#!/bin/bash
+realpath=\`readlink -fn \$0\`
+realdir=\`dirname \$realpath\`
+export $exportstring
+exec -a "\$0" \$realdir/$cmdname.real "\$@"
+END
+ chmod +x $cmd
+}
+
+# Copy files/directories from $1 to $2 but using hardlinks
+# (preserve symlinks)
+hardlinkdir () {
+ from=$1
+ to=$2
+ (cd $from; find . -print0 | cpio --null -pdlu $to)
+}
+
+
+def check_app_exists(app, d):
+ app = d.expand(app).split()[0].strip()
+ path = d.getVar('PATH')
+ return bool(bb.utils.which(path, app))
+
+def explode_deps(s):
+ return bb.utils.explode_deps(s)
+
+def base_set_filespath(path, d):
+ filespath = []
+ extrapaths = (d.getVar("FILESEXTRAPATHS") or "")
+ # Remove default flag which was used for checking
+ extrapaths = extrapaths.replace("__default:", "")
+ # Don't prepend empty strings to the path list
+ if extrapaths != "":
+ path = extrapaths.split(":") + path
+ # The ":" ensures we have an 'empty' override
+ overrides = (":" + (d.getVar("FILESOVERRIDES") or "")).split(":")
+ overrides.reverse()
+ for o in overrides:
+ for p in path:
+ if p != "":
+ filespath.append(os.path.join(p, o))
+ return ":".join(filespath)
+
+def extend_variants(d, var, extend, delim=':'):
+ """Return a string of all bb class extend variants for the given extend"""
+ variants = []
+ whole = d.getVar(var) or ""
+ for ext in whole.split():
+ eext = ext.split(delim)
+ if len(eext) > 1 and eext[0] == extend:
+ variants.append(eext[1])
+ return " ".join(variants)
+
+def multilib_pkg_extend(d, pkg):
+ variants = (d.getVar("MULTILIB_VARIANTS") or "").split()
+ if not variants:
+ return pkg
+ pkgs = pkg
+ for v in variants:
+ pkgs = pkgs + " " + v + "-" + pkg
+ return pkgs
+
+def get_multilib_datastore(variant, d):
+ return oe.utils.get_multilib_datastore(variant, d)
+
+def all_multilib_tune_values(d, var, unique = True, need_split = True, delim = ' '):
+ """Return a string of all ${var} in all multilib tune configuration"""
+ values = []
+ variants = (d.getVar("MULTILIB_VARIANTS") or "").split() + ['']
+ for item in variants:
+ localdata = get_multilib_datastore(item, d)
+ # We need WORKDIR to be consistent with the original datastore
+ localdata.setVar("WORKDIR", d.getVar("WORKDIR"))
+ value = localdata.getVar(var) or ""
+ if value != "":
+ if need_split:
+ for item in value.split(delim):
+ values.append(item)
+ else:
+ values.append(value)
+ if unique:
+ #we do this to keep order as much as possible
+ ret = []
+ for value in values:
+ if not value in ret:
+ ret.append(value)
+ else:
+ ret = values
+ return " ".join(ret)
+
+def all_multilib_tune_list(vars, d):
+ """
+ Return a list of ${VAR} for each variable VAR in vars from each
+ multilib tune configuration.
+ Is safe to be called from a multilib recipe/context as it can
+ figure out the original tune and remove the multilib overrides.
+ """
+ values = {}
+ for v in vars:
+ values[v] = []
+ values['ml'] = ['']
+
+ variants = (d.getVar("MULTILIB_VARIANTS") or "").split() + ['']
+ for item in variants:
+ localdata = get_multilib_datastore(item, d)
+ values[v].append(localdata.getVar(v))
+ values['ml'].append(item)
+ return values
+all_multilib_tune_list[vardepsexclude] = "OVERRIDES"
+
+# If the user hasn't set up their name/email, set some defaults
+check_git_config() {
+ if ! git config user.email > /dev/null ; then
+ git config --local user.email "${PATCH_GIT_USER_EMAIL}"
+ fi
+ if ! git config user.name > /dev/null ; then
+ git config --local user.name "${PATCH_GIT_USER_NAME}"
+ fi
+}