poky: subtree update:c67f57c09e..c6bc20857c
Adrian Freihofer (2):
oe-publish-sdk: fix layers init via ssh
oe-publish-sdk: add --keep-orig option
Alexander Kanavin (68):
meta-selftest: correct the virgl test for 5.8 kernels
bison: upgrade 3.6.4 -> 3.7.1
util-linux: upgrade 2.35.2 -> 2.36
python3-numpy: upgrade 1.19.0 -> 1.19.1
python3-setuptools: upgrade 49.3.1 -> 49.6.0
rsync: upgrade 3.2.2 -> 3.2.3
util-linux: merge .inc into .bb
acpica: upgrade 20200528 -> 20200717
asciidoc: upgrade 9.0.1 -> 9.0.2
cryptodev: upgrade 1.10 -> 1.11
diffoscope: upgrade 153 -> 156
epiphany: upgrade 3.36.3 -> 3.36.4
font-alias: upgrade 1.0.3 -> 1.0.4
gtk+3: upgrade 3.24.21 -> 3.24.22
libcheck: upgrade 0.15.0 -> 0.15.2
libinput: upgrade 1.16.0 -> 1.16.1
libpipeline: upgrade 1.5.2 -> 1.5.3
libx11: upgrade 1.6.9 -> 1.6.11
linux-firmware: upgrade 20200619 -> 20200721
man-pages: upgrade 5.07 -> 5.08
mc: upgrade 4.8.24 -> 4.8.25
mesa: upgrade 20.1.4 -> 20.1.5
piglit: upgrade to latest revision
re2c: upgrade 2.0 -> 2.0.2
sysstat: upgrade 12.2.2 -> 12.4.0
vala: upgrade 0.48.7 -> 0.48.9
bootchart2: update 0.14.8 -> 0.14.9
harfbuzz: convert to meson, enable gobject introspection
pango: update 1.44.7 -> 1.46.0
boost: update 1.73.0 -> 1.74.0
xev: update 1.2.3 -> 1.2.4
wpebackend-fdo: update 1.6.1 -> 1.7.1
gpgme: update 1.13.1 -> 1.14.0
libpsl: update 0.21.0 -> 0.21.1.
gettext: update 0.20.2 -> 0.21
cmake: update 3.17.3 -> 3.18.1
linux-firmware: update 20200721 -> 20200817
meson: update 0.55.0 -> 0.55.1
systemd-boot: bump version to 246.2
json-glib: inherit upstream-version-is-even
packagegroup-core-device-devel: remove
oeqa/x32lib: rework to use readelf from the host
oeqa/multilib: rework to use readelf from the host
oeqa/multilib: un-skip the connman test
poky.conf: do not install packagegroup-core-device-devel into qemu images
glib-2.0: update 2.64.4 -> 2.64.5
cmake: upgrade 3.18.1 -> 3.18.2
libxcrypt: upgrade 4.4.16 -> 4.4.17
debianutils: upgrade 4.11 -> 4.11.1
enchant2: upgrade 2.2.8 -> 2.2.9
harfbuzz: upgrade 2.7.1 -> 2.7.2
libmpc: upgrade 1.1.0 -> 1.2.0
librepo: upgrade 1.12.0 -> 1.12.1
libuv: upgrade 1.38.1 -> 1.39.0
msmtp: upgrade 1.8.11 -> 1.8.12
ninja: upgrade 1.10.0 -> 1.10.1
p11-kit: upgrade 0.23.20 -> 0.23.21
pango: upgrade 1.46.0 -> 1.46.1
re2c: upgrade 2.0.2 -> 2.0.3
resolvconf: upgrade 1.82 -> 1.83
stress-ng: upgrade 0.11.18 -> 0.11.19
gnu-config: update to latest revision
nasm: update 2.15.03 -> 2.15.05
libva-utils: fix upstream version check
gnupg: update 2.2.21 -> 2.2.22
libx11: update 1.6.11 -> 1.6.12
mesa: update 20.1.5 -> 20.1.6
xserver-xorg: update 1.20.8 -> 1.20.9
Andrey Zhizhikin (1):
insane: check for missing update-alternatives inherit
Anibal Limon (1):
recipes-kernel: linux-firmware add qcom-venus-{5.2,5.4} packages
Aníbal Limón (1):
recipes-graphics/xorg-xserver: Add patch to fix segfault when probe
Armin Kuster (2):
bind: update to 9.11.22 ESV
core-image-sato: qemumips use 512 mem
Bruce Ashfield (30):
linux-yocto/5.4: update to v5.4.59
linux-yocto/5.8: update to v5.8.2
yocto-bsp: update to v5.4.56
yocto-bsp: update to v5.4.58
qemu: bump default reference kernel to v5.8
linux-yocto/5.8: fix perf and virtio_scsi warnings
linux-yocto-rt/5.8: fix lttng-modules build
linux-yocto/5.8: selftests/bpf: Prevent runqslower from racing on building bpftool
linux-yocto/5.8: disable CONFIG_NFS_DISABLE_UDP_SUPPORT
poky: set preferred version for linux-yocto to be v5.8
poky-tiny: set preferred version to 5.8
poky: add preferred version for linux-yocto-rt
linux-yocto/5.8: update to v5.8.3
linux-yocto/5.4: update to v5.4.60
kernel: config cleanups for 5.8+
linux-yocto/5.4: update to v5.4.61
linux-yocto/5.8: update to v5.8.4
linux-yocto/5.8: disable IKHEADERS in default builds
kernel-yocto: allow promotion of configuration warnings to errors
kernel-yocto: checksum all modifications to available kernel fragments directories
lttng-modules/devupstream: bump to latest 2.12 commits
linux-yocto-dev: bump to v5.9+
linux-yocto/5.8: update to v5.8.5
kernel-devsrc: account for HOSTCC and HOSTCXX
linux-yocto/config: netfilter: Enable nat for ipv4 and ipv6
linux-yocto/5.8: update to v5.8.8
linux-yocto/5.4: update to v5.4.64
linux-yocto/config: configuration warning cleanup
linux-yocto/5.8: update to v5.8.9
linux-yocto/5.4: update to v5.4.65
Changhyeok Bae (2):
iw: upgrade 5.4 -> 5.8
iputils: upgrade s20190709 -> s20200821
Chris Laplante (12):
bitbake: compat.py: remove file since it no longer actually implements anything
bitbake: COW: formatting
bitbake: COW: migrate test suite into tests/cow
cve-update-db-native: add progress handler
cve-check/cve-update-db-native: use lockfile to fix usage under multiconfig
cve-update-db-native: use context manager for cve_f
cve-check: avoid FileNotFoundError if no do_cve_check task has run
bitbake: utils: process_profilelog: use context manager
bitbake: utils: fix UnboundLocalError when _print_exception raises
cve-update-db-native: be less magical about checking whether the cve-check class is enabled
cve-update-db-native: move -journal checking into do_fetch
cve-update-db-native: remove unused variable
Christophe GUIBOUT (1):
initramfs-framework: support kernel cmdline with double quotes
Denys Dmytriyenko (2):
weston: upgrade 8.0.0 -> 9.0.0
cryptodev: bump 1 commit past 1.11 to fix 5.9-rc1+
Diego Sueiro (2):
license_image.bbclass: Create symlink to the image license manifest dir
license_image.bbclass: Fix symlink to the image license manifest dir creation
Douglas Royds (1):
tcmode-default: Drop gcc-cross-initial, gcc-crosssdk-initial references
Frazer Clews (1):
bitbake: lib: fix most undefined code picked up by pylint
Geoff Parker (1):
systemd-serialgetty: Replace sed quoting using ' with " to allow var expansion
Jacob Kroon (1):
gcc10: Don't default back to -fcommon
Jean-Francois Dagenais (1):
bitbake: siggen: clean_basepath: remove recipe full path when virtual:xyz present
Jens Rehsack (1):
lttng-modules: backport patches from 2.12.x to fix 5.4.64+ and 5.8.9+ builds
Joe Slater (1):
pseudo: fix renaming to self
Jon Mason (4):
cortex-m0plus.inc: change file permissions
tune-cortexa55.inc: clean-up ARMv8.2a uses
tune-cortexa57-cortexa53.inc: add CRC and set march
tune-cortexa*: Cleanups
Joshua Watt (8):
wic: Add 512 Byte alignment to --offset
oeqa: runtime_tests: Extra GPG debugging
oeqa: sdk: Capture stderr output
oeqa: reproducible: Fix test not producing diffs
diffoscope: upgrade 156 -> 158
bitbake: bitbake: Add parsing torture test
bitbake: cooker: Block SIGINT in worker processes
sphinx: dev-manual: Clarify that virtual providers do not apply to runtime dependencies
Kai Kang (1):
dhcpcd: 9.1.4 -> 9.2.0
Kevin Hao (1):
meta-yocto-bsp: Bump to the v5.8 kernel
Khairul Rohaizzat Jamaluddin (1):
wic/bootimg-efi: IMAGE_EFI_BOOT_FILES variable added to separate bootimg-efi and bootimg-partition
Khem Raj (24):
gcc-cross-canadian: Install gcc/g++ wrappers for musl
uninative: Upgrade to 2.9
packagegroup-core-tools-profile: Disable lttng-modules for riscv64
lttng-modules: Disable on riscv64
kexec-tools: Fix build with -fno-common on ppc
lttng-tools: Do not build for riscv64
util-linux: Allow update alternatives for additional apps
lttng-tools: lttng-ust works on riscv64
json-glib: Backport a build fix with clang
rpcbind: Use update-alternatives for rpcinfo
go: Upgrade to 1.15 major release
weston-init: Redefine weston service and add socket activation option
musl: Upgrade to latest master
libucontext: Recognise riscv32 architecture
linuxloader.bbclass: Define riscv32 ldso for musl
populate_sdk_ext: Do not assume local.conf will always exist
weston: plane_add_prop() calls break musl atomic modesetting
weston-init: Enable RDP screen share
weston-init: Do not use fbdev backend
weston-init: Select drm/fbdev backends for qemu machines
oeqa/weston: Fix tests to run with systemd
core-image-weston: Bump qemu memory to 512M
go: Update to 1.15.2 minor release
bind: Inherit update-alternatives
Mark Hatle (6):
package_tar.bbclass: Sync to the other package_* classes
kernel.bbclass: Remove do_install[prefunc] no longer needed
buildhistory.bbclass: Rework to use read_subpackage_metadata
kernel.bbclass: Move away from calling package_get_auto_pr
package.bbclass: hash equivalency and pr service
bitbake: process.py: Handle SystemExit exception to eliminate backtrace
Mark Morton (1):
sphinx: test-manual code block, link, and format update
Martin Jansa (7):
devtool: expand SRC_URI when guessing recipe update mode
image-artifact-names: introduce new bbclass and move some variables into it
kernel.bbclass: use bash variables like imageType, base_name without {}
kernel.bbclass: eliminate (initramfs_)symlink_name variables
kernel.bbclass: use camelCase notation for bash variables in do_deploy
*-initramfs: don't use .rootfs IMAGE_NAME_SUFFIX
bitbake.conf: use ${TCMODE}-${TCLIBC} directory for CACHE
Matt Madison (1):
image.bbclass: fix REPRODUCIBLE_TIMESTAMP_ROOTFS reference
Michael Gloff (2):
sysvinit rc: Use PSPLASH_FIFO_DIR for progress fifo
sysvinit: Remove ${B} assignment
Michael Tretter (1):
devtool: deploy-target: Fix size calculation for hard links
Ming Liu (2):
systemd: split systemd specific udev rules into its own package
libubootenv: inherit uboot-config
Mingli Yu (3):
qemu: always define unknown_lock_type
qemu: override DEBUG_BUILD
bison: remove the parallel build patch
Naveen Saini (1):
lib/oe/recipeutils.py: add support for BBFILES_DYNAMIC
Nicolas Dechesne (73):
linux-libc-headers: kernel headers are installed in STAGING_KERNEL_BUILDDIR
bitbake: sphinx: add initial build infrastructure
bitbake: sphinx: initial sphinx support
bitbake: sphinx: bitbake-user-manual: use builtin sphinx glossary
bitbake: sphinx: switch to readthedocs theme
bitbake: sphinx: override theme CSS
bitbake: sphinx: fixup for links
bitbake: sphinx: fix links inside notes
bitbake: sphinx: fixes all remaining warnings
bitbake: sphinx: Makefile.sphinx: add clean and publish targets
bitbake: sphinx: tweak html output a bit
bitbake: sphinx: add SPDX headers
bitbake: sphinx: index: move the boilerplate at the end of the page
bitbake: sphinx: conf: enable extlinks extension
bitbake: sphinx: add releases page
bitbake: sphinx: bitbake-user-manual: insert additional blank line after title
bitbake: sphinx: last manual round of fixes/improvements
bitbake: sphinx: update style for important, caution and warnings
bitbake: sphinx: remove leading '/'
bitbake: sphinx: theme_override: properly set font for verbatim text
bitbake: bitbake-user-manual: fix bad links
sphinx: add initial build infrastructure
sphinx: initial sphinx support
sphinx: ref-variables: use builtin sphinx glossary
sphinx: overview-manual: add figures
sphinx: switch to readthedocs theme
sphinx: Add SPDX license headers
sphinx: add CSS theme override
sphinx: bsp-guide: add figures
sphinx: add Yocto project logo
sphinx: conf: update copyright
sphinx: conf: add substitutions/global variables
sphinx: add boilerplate file
sphinx: add boilerplate to manuals
sphinx: ref-manual: add revision history table
sphinx: add a general index
sphinx: conf.py: enable sphinx.ext.autosectionlabel
sphinx: ref-manual: use builtin glossary for the Terms section
sphinx: fix internal links
sphinx: ref-manual: fix typo
sphinx: fix custom term links
sphinx: manual updates for some links
sphinx: dev-manual add figures
sphinx: kernel-dev: add figures
sphinx: profile-manual: add figures
sphinx: fix up bold text for informalexample container
sphinx: ref-manual: add figures
sphinx: sdk-manual: add figures
sphinx: test-manual: add figures
sphinx: toaster-manual: add figures
sphinx: add links for Yocto project website
sphinx: fix links when the link text should be displayed
sphinx: add links to terms in the BitBake glossary
sphinx: add links to section in the Bitbake manual
sphinx: setup extlink for docs.yoctoproject.org
sphinx: enable intersphinx extension
sphinx: insert blank below between title and toc
sphinx: fix up terms related to kernel-fitimage
sphinx: conf: a few rendering tweaks
sphinx: makefile: add publish target
sphinx: conf: include CSS/JS files, the proper way
sphinx: convert 'what I wish I'd known'
sphinx: convert 'transitioning to a custom environment'
sphinx: ref-manual: fix heading for oe-init-build-env
sphinx: brief-yoctoprojectqs: fix up all remaining rendering issues
sphinx: Makefile.sphinx improvements
sphinx: convert bsp-guide
sphinx: remove leading '/'
sphinx: update style for important, caution and warnings
sphinx: profile-manual: convert profile-manual
sphinx: theme_override: properly set font for verbatim text
sphinx: theme_override: add tying-it-together admonition
sphinx: conf: exclude adt-manual/*.rst
Oleksandr Kravchuk (1):
ell: update to 0.33
Ovidiu Panait (1):
libxml2: Fix CVE-2020-24977
Peter A. Bigot (2):
bluez5: fix builds that require ell support
timezone: include leap second data in tzdata-core
Peter Bergin (1):
systemd: avoid failing if no udev rules provided
Pierre-Jean Texier (2):
libubootenv: upgrade 0.3 -> 0.3.1
diffoscope: upgrade 158 -> 160
Quentin Schulz (16):
sphinx: brief-yoctoprojectqs: remove redundant welcome
sphinx: brief-yoctoprojectqs: fix ambiguous note for cyclone5 example
sphinx: brief-yoctoprojectqs: add missing boilerplate
sphinx: overview-manual: add link to AUH how-to section
sphinx: overview-manual: fix bitbake basic explanation
sphinx: brief-yoctoprojectqs: add note on branch consistency between layers
sphinx: what-i-wish-id-known: update "don't be fooled by doc search results"
sphinx: overview-manual: remove highlight in bold section
sphinx: replace special quotes with single and double quotes
sphinx: fix incorrect indentations
sphinx: brief-yoctoprojectqs: put other distros note after Ubuntu-specific packages
sphinx: fix a few typos or missing/too many words
sphinx: "highlight" some variables, tasks or files
sphinx: fix or add missing links and remove mention of Eclipse workflow
ref-manual: examples: hello-autotools: upgrade to 2.10
ref-manual: examples: libxpm: add relative path to .inc
Rahul Kumar (1):
systemd-serialgetty: Fix sed expression quoting
Rasmus Villemoes (1):
kernel.bbclass: run do_symlink_kernsrc before do_patch
Richard Purdie (74):
nativesdk-sdk-provides-dummy: Add /bin/sh
bitbake: fetch2/wget: Remove buffering parameter
bitbake: cooker: Ensure parse_quit thread is closed down
bitbake: cooker: Explictly shut down the sync thread
bitbake: fetch2: Drop cups.org from wget status checks
bitbake: build/msg: Cleanup verbose option handling
bitbake: cooker/cookerdata/main: Improve loglevel handling
bitbake: cookerdata: Ensure UI options are updated to the server
bitbake: cooker/cookerdata: Ensure UI event log is updated from commandline
bitbake: cooker: Defer configuration init to after UI connection
bitbake: server/process: Move the socket code to server process only
bitbake: main/server/process: Drop configuration object passing
bitbake: cooker: Ensure BB_ORIGENV is updated by changes to configuration.env
bitbake: server/process: Log extra threads at exit
bitbake: server/process: Add bitbake-server and exec() a new server process
bitbake: runqueue: Don't use sys.argv
bitbake: cooker: Ensure cooker's enviroment is updated on updateConfig
connman-gnome/matchbox-desktop: Remove file:// globbing
selftest/recipetool: Drop globbing SRC_URI test, no longer supported
local.conf.sample: Document memory resident bitbake
bitbake: fetch2: Drop globbing supprt in file:// SRC_URIs
bitbake: server/process: Use sys.executable for bitbake-server
bitbake: process: Avoid bb.utils.timeout
bitbake: utils: Drop broken timeout function
bitbake: server/process: Fix typo in code causing tracebacks
oeqa/selftest: Apply patch to fix cpio build with -fno-common
runqemu: Show an error for conflicting graphics options
lttng: Move platform logic to dedicated inc file
patchelf: upgrade 0.11 -> 0.12
build-appliance/packagegroup-core-base-utils: Replace dhcp-client/dhcp-server with dhcpcd/kea
selftest/prservice: Improve test failure message
iputils: Adapt ${PN}-tftpd package dependency to PACKAGECONFIG
bitbake: process/knotty: Improve early exception handling
bitbake: cooker/cookerdata: Use BBHandledException, not sys.exit()
bitbake: cookerdata: Fix exception raise statements
bitbake: process: Avoid printing binary strings for leftover processes
bitbake: server/process: Ensure logging is flushed
bitbake: server/process: Don't show tracebacks if the lockfile is removed
bitbake: cooker: Ensure parser replacement calls parser final_cleanup
bitbake: cooker: Assign a name to the sync thread to aid debugging
bitbake: server/process: Ensure we don't keep looping if some other server is started
bitbake: server/process: Prefix the log data with pid/time information
bitbake: server/process: Note when commands complete in logs
bitbake: cooker: Ensure parser is cleaned up
runqemu: Add a hook to allow it to renice
bitbake: cooker: Avoid parser deadlocks
bitbake: cooker: Ensure parser worker signal handlers are default
selftest/signing: Ensure build path relocation is safe
oeqa/concurrencytest: Improve builddir path manipulations
bitbake: cooker/command: Fix disconnection handling
bitbake: tinfoil: Ensure sockets don't leak even when exceptions occur
bitbake: tests/fetch: Move away from problematic freedesktop.org urls
bitbake: sphinx: Enhance the sphinx experience/nagivation with:
bitbake: sphinx: theme_override: Use bold for emphasis text
Revert "qemu: always define unknown_lock_type"
Revert "core-image-sato: qemumips use 512 mem"
sphinx: Organize top level docs
sphinx: releases.rst: Add index/links to docs for previous releases
sphinx: boilerplate.rst: Drop versions notes as we have better navigation now
sphinx: boilerplate.rst: Sphinx puts the copyright elsewhere
sphinx: history: Move revision history to its own section
sphinx: manuals: Move boilerplate after toctree
sphinx: Add support for multiple docs version
sphinx: index.rst: Fix links
sphinx: ref-system-requirements: Improve formatting of the notes sections, merging them
sphinx: ref-manual links fixes and many other cleanups to import
sphinx: dev-manual: Various URL, code block and other fixes to imported data
sphinx: sdk-manual: Various URL, code block and other fixes to imported data
sphinx: kernel-dev: Various URL, code block and other fixes to imported data
sphinx: theme_override: Use bold for emphasis text
sphinx: ref-tasks: Add populate_sdk_ext task definition
sphinx: ref-manual/migration: Split each release into its own file
sphinx: overview-manual: Various URL, code block and other fixes to imported data
build-appliance-image: Update to master head revision
Robert Yang (3):
bitbake: cooker.py: Save prioritized BBFILES to BBFILES_PRIORITIZED
bitbake: utils.py: get_file_layer(): Exit the loop when file is matched
bitbake: utils.py: get_file_layer(): Improve performance
Ross Burton (25):
package.bbclass: explode the RPROVIDES so we don't think the versions are provides
elfutils: silence a new QA warning
insane: improve gnu-hash-style warning
gdk-pixbuf: add tests PACKAGECONFIG
debianutils: change SRC_URI to use snapshot.debian.org
insane: only load real files as ELF
autoconf: consolidate SRC_URI
autoconf: consolidate DEPENDS
kea: no need to depend on kea-native
kea: don't use PACKAGECONFIG inappropriately
kea: bump to 1.7.10
help2man: rewrite recipe
local.conf.sample.extended: remove help2man reference
curl: add vendors to CVE_PRODUCT to exclude false positives
harfbuzz: update patch status
harfbuzz: fix a build race around hb-version.h
cmake: whitelist CVE-2016-10642
ncurses: remove config.cache
qemu: fix CVE-2020-14364
cve-update-db-native: remove unused import
cve-update-db-native: add more logging when fetching
cve-update-db-native: use fetch task
alsa-plugins: improve .la removal
sato-screenshot: improve .la removal
buildhistory-diff: use BUILDDIR to know where buildhistory is
Saul Wold (1):
gnupg: uprev 2.2.22 -> 2.2.23
Stacy Gaikovaia (2):
bison: uprev from 3.7.1 to 3.7.2
valgrind: fix memcheck vgtests remove fullpath-after flags
Steve Sakoman (1):
xinput-calibrator: change SRC_URI to branch with libinput support
Sumit Garg (1):
insane: fix gnu-hash-style check
TeohJayShen (1):
oeqa/runtime: add test for matchbox-terminal
Tim Orling (1):
sphinx: toaster-manual: fix vars, links, code blocks
Vijai Kumar K (2):
image_types_wic: Add ASSUME_PROVIDED to WICVARS
wic: misc: Add /bin to the list of searchpaths
Yanfei Xu (1):
kernel-yocto: only replace leading -I in include paths
Yi Zhao (1):
glib-networking: add ptest
Zhixiong Chi (1):
gnutls: CVE-2020-24659
akuster (8):
log4cplus: move meta-oe pkg to core
kea: Move from meta-networking
maintainers.inc: Add me as kea & log4plus maintainer.
dhcpcd: Move from meta-network as OE-Core needs a client
maintainers.inc: Add me as dhcpcd maintainer
dhcp: remove from core
bind: Add 9.16.x
bind: 9.11 remove
hongxu (1):
sysstat: fix installed-vs-shipped QA Issue in systemd
zangrc (4):
libcap:upgrade 2.42 -> 2.43
libcap-ng:upgrade 0.7.10 -> 0.7.11
libgpg-error:upgrade 1.38 -> 1.39
at-spi2-core:upgrade 2.36.0 -> 2.36.1
Signed-off-by: Andrew Geissler <geissonator@yahoo.com>
Change-Id: I5542f5eea751a2641342e945725fd687cd74bebe
diff --git a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
new file mode 100644
index 0000000..f62ddff
--- /dev/null
+++ b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
@@ -0,0 +1,652 @@
+.. SPDX-License-Identifier: CC-BY-2.5
+
+=====================
+File Download Support
+=====================
+
+|
+
+BitBake's fetch module is a standalone piece of library code that deals
+with the intricacies of downloading source code and files from remote
+systems. Fetching source code is one of the cornerstones of building
+software. As such, this module forms an important part of BitBake.
+
+The current fetch module is called "fetch2" and refers to the fact that
+it is the second major version of the API. The original version is
+obsolete and has been removed from the codebase. Thus, in all cases,
+"fetch" refers to "fetch2" in this manual.
+
+The Download (Fetch)
+====================
+
+BitBake takes several steps when fetching source code or files. The
+fetcher codebase deals with two distinct processes in order: obtaining
+the files from somewhere (cached or otherwise) and then unpacking those
+files into a specific location and perhaps in a specific way. Getting
+and unpacking the files is often optionally followed by patching.
+Patching, however, is not covered by this module.
+
+The code to execute the first part of this process, a fetch, looks
+something like the following: ::
+
+ src_uri = (d.getVar('SRC_URI') or "").split()
+ fetcher = bb.fetch2.Fetch(src_uri, d)
+ fetcher.download()
+
+This code sets up an instance of the fetch class. The instance uses a
+space-separated list of URLs from the :term:`SRC_URI`
+variable and then calls the ``download`` method to download the files.
+
+The instantiation of the fetch class is usually followed by: ::
+
+ rootdir = l.getVar('WORKDIR')
+ fetcher.unpack(rootdir)
+
+This code unpacks the downloaded files to the specified by ``WORKDIR``.
+
+.. note::
+
+ For convenience, the naming in these examples matches the variables
+ used by OpenEmbedded. If you want to see the above code in action,
+ examine the OpenEmbedded class file ``base.bbclass``
+ .
+
+The ``SRC_URI`` and ``WORKDIR`` variables are not hardcoded into the
+fetcher, since those fetcher methods can be (and are) called with
+different variable names. In OpenEmbedded for example, the shared state
+(sstate) code uses the fetch module to fetch the sstate files.
+
+When the ``download()`` method is called, BitBake tries to resolve the
+URLs by looking for source files in a specific search order:
+
+- *Pre-mirror Sites:* BitBake first uses pre-mirrors to try and find
+ source files. These locations are defined using the
+ :term:`PREMIRRORS` variable.
+
+- *Source URI:* If pre-mirrors fail, BitBake uses the original URL (e.g
+ from ``SRC_URI``).
+
+- *Mirror Sites:* If fetch failures occur, BitBake next uses mirror
+ locations as defined by the :term:`MIRRORS` variable.
+
+For each URL passed to the fetcher, the fetcher calls the submodule that
+handles that particular URL type. This behavior can be the source of
+some confusion when you are providing URLs for the ``SRC_URI`` variable.
+Consider the following two URLs: ::
+
+ http://git.yoctoproject.org/git/poky;protocol=git
+ git://git.yoctoproject.org/git/poky;protocol=http
+
+In the former case, the URL is passed to the ``wget`` fetcher, which does not
+understand "git". Therefore, the latter case is the correct form since the Git
+fetcher does know how to use HTTP as a transport.
+
+Here are some examples that show commonly used mirror definitions: ::
+
+ PREMIRRORS ?= "\
+ bzr://.*/.\* http://somemirror.org/sources/ \\n \
+ cvs://.*/.\* http://somemirror.org/sources/ \\n \
+ git://.*/.\* http://somemirror.org/sources/ \\n \
+ hg://.*/.\* http://somemirror.org/sources/ \\n \
+ osc://.*/.\* http://somemirror.org/sources/ \\n \
+ p4://.*/.\* http://somemirror.org/sources/ \\n \
+ svn://.*/.\* http://somemirror.org/sources/ \\n"
+
+ MIRRORS =+ "\
+ ftp://.*/.\* http://somemirror.org/sources/ \\n \
+ http://.*/.\* http://somemirror.org/sources/ \\n \
+ https://.*/.\* http://somemirror.org/sources/ \\n"
+
+It is useful to note that BitBake
+supports cross-URLs. It is possible to mirror a Git repository on an
+HTTP server as a tarball. This is what the ``git://`` mapping in the
+previous example does.
+
+Since network accesses are slow, BitBake maintains a cache of files
+downloaded from the network. Any source files that are not local (i.e.
+downloaded from the Internet) are placed into the download directory,
+which is specified by the :term:`DL_DIR` variable.
+
+File integrity is of key importance for reproducing builds. For
+non-local archive downloads, the fetcher code can verify SHA-256 and MD5
+checksums to ensure the archives have been downloaded correctly. You can
+specify these checksums by using the ``SRC_URI`` variable with the
+appropriate varflags as follows: ::
+
+ SRC_URI[md5sum] = "value"
+ SRC_URI[sha256sum] = "value"
+
+You can also specify the checksums as
+parameters on the ``SRC_URI`` as shown below: ::
+
+ SRC_URI = "http://example.com/foobar.tar.bz2;md5sum=4a8e0f237e961fd7785d19d07fdb994d"
+
+If multiple URIs exist, you can specify the checksums either directly as
+in the previous example, or you can name the URLs. The following syntax
+shows how you name the URIs: ::
+
+ SRC_URI = "http://example.com/foobar.tar.bz2;name=foo"
+ SRC_URI[foo.md5sum] = 4a8e0f237e961fd7785d19d07fdb994d
+
+After a file has been downloaded and
+has had its checksum checked, a ".done" stamp is placed in ``DL_DIR``.
+BitBake uses this stamp during subsequent builds to avoid downloading or
+comparing a checksum for the file again.
+
+.. note::
+
+ It is assumed that local storage is safe from data corruption. If
+ this were not the case, there would be bigger issues to worry about.
+
+If :term:`BB_STRICT_CHECKSUM` is set, any
+download without a checksum triggers an error message. The
+:term:`BB_NO_NETWORK` variable can be used to
+make any attempted network access a fatal error, which is useful for
+checking that mirrors are complete as well as other things.
+
+.. _bb-the-unpack:
+
+The Unpack
+==========
+
+The unpack process usually immediately follows the download. For all
+URLs except Git URLs, BitBake uses the common ``unpack`` method.
+
+A number of parameters exist that you can specify within the URL to
+govern the behavior of the unpack stage:
+
+- *unpack:* Controls whether the URL components are unpacked. If set to
+ "1", which is the default, the components are unpacked. If set to
+ "0", the unpack stage leaves the file alone. This parameter is useful
+ when you want an archive to be copied in and not be unpacked.
+
+- *dos:* Applies to ``.zip`` and ``.jar`` files and specifies whether
+ to use DOS line ending conversion on text files.
+
+- *basepath:* Instructs the unpack stage to strip the specified
+ directories from the source path when unpacking.
+
+- *subdir:* Unpacks the specific URL to the specified subdirectory
+ within the root directory.
+
+The unpack call automatically decompresses and extracts files with ".Z",
+".z", ".gz", ".xz", ".zip", ".jar", ".ipk", ".rpm". ".srpm", ".deb" and
+".bz2" extensions as well as various combinations of tarball extensions.
+
+As mentioned, the Git fetcher has its own unpack method that is
+optimized to work with Git trees. Basically, this method works by
+cloning the tree into the final directory. The process is completed
+using references so that there is only one central copy of the Git
+metadata needed.
+
+.. _bb-fetchers:
+
+Fetchers
+========
+
+As mentioned earlier, the URL prefix determines which fetcher submodule
+BitBake uses. Each submodule can support different URL parameters, which
+are described in the following sections.
+
+.. _local-file-fetcher:
+
+Local file fetcher (``file://``)
+--------------------------------
+
+This submodule handles URLs that begin with ``file://``. The filename
+you specify within the URL can be either an absolute or relative path to
+a file. If the filename is relative, the contents of the
+:term:`FILESPATH` variable is used in the same way
+``PATH`` is used to find executables. If the file cannot be found, it is
+assumed that it is available in :term:`DL_DIR` by the
+time the ``download()`` method is called.
+
+If you specify a directory, the entire directory is unpacked.
+
+Here are a couple of example URLs, the first relative and the second
+absolute: ::
+
+ SRC_URI = "file://relativefile.patch"
+ SRC_URI = "file:///Users/ich/very_important_software"
+
+.. _http-ftp-fetcher:
+
+HTTP/FTP wget fetcher (``http://``, ``ftp://``, ``https://``)
+-------------------------------------------------------------
+
+This fetcher obtains files from web and FTP servers. Internally, the
+fetcher uses the wget utility.
+
+The executable and parameters used are specified by the
+``FETCHCMD_wget`` variable, which defaults to sensible values. The
+fetcher supports a parameter "downloadfilename" that allows the name of
+the downloaded file to be specified. Specifying the name of the
+downloaded file is useful for avoiding collisions in
+:term:`DL_DIR` when dealing with multiple files that
+have the same name.
+
+Some example URLs are as follows: ::
+
+ SRC_URI = "http://oe.handhelds.org/not_there.aac"
+ SRC_URI = "ftp://oe.handhelds.org/not_there_as_well.aac"
+ SRC_URI = "ftp://you@oe.handhelds.org/home/you/secret.plan"
+
+.. note::
+
+ Because URL parameters are delimited by semi-colons, this can
+ introduce ambiguity when parsing URLs that also contain semi-colons,
+ for example:
+ ::
+
+ SRC_URI = "http://abc123.org/git/?p=gcc/gcc.git;a=snapshot;h=a5dd47"
+
+
+ Such URLs should should be modified by replacing semi-colons with '&'
+ characters:
+ ::
+
+ SRC_URI = "http://abc123.org/git/?p=gcc/gcc.git&a=snapshot&h=a5dd47"
+
+
+ In most cases this should work. Treating semi-colons and '&' in
+ queries identically is recommended by the World Wide Web Consortium
+ (W3C). Note that due to the nature of the URL, you may have to
+ specify the name of the downloaded file as well:
+ ::
+
+ SRC_URI = "http://abc123.org/git/?p=gcc/gcc.git&a=snapshot&h=a5dd47;downloadfilename=myfile.bz2"
+
+
+.. _cvs-fetcher:
+
+CVS fetcher (``(cvs://``)
+-------------------------
+
+This submodule handles checking out files from the CVS version control
+system. You can configure it using a number of different variables:
+
+- :term:`FETCHCMD_cvs <FETCHCMD>`: The name of the executable to use when running
+ the ``cvs`` command. This name is usually "cvs".
+
+- :term:`SRCDATE`: The date to use when fetching the CVS source code. A
+ special value of "now" causes the checkout to be updated on every
+ build.
+
+- :term:`CVSDIR`: Specifies where a temporary
+ checkout is saved. The location is often ``DL_DIR/cvs``.
+
+- CVS_PROXY_HOST: The name to use as a "proxy=" parameter to the
+ ``cvs`` command.
+
+- CVS_PROXY_PORT: The port number to use as a "proxyport="
+ parameter to the ``cvs`` command.
+
+As well as the standard username and password URL syntax, you can also
+configure the fetcher with various URL parameters:
+
+The supported parameters are as follows:
+
+- *"method":* The protocol over which to communicate with the CVS
+ server. By default, this protocol is "pserver". If "method" is set to
+ "ext", BitBake examines the "rsh" parameter and sets ``CVS_RSH``. You
+ can use "dir" for local directories.
+
+- *"module":* Specifies the module to check out. You must supply this
+ parameter.
+
+- *"tag":* Describes which CVS TAG should be used for the checkout. By
+ default, the TAG is empty.
+
+- *"date":* Specifies a date. If no "date" is specified, the
+ :term:`SRCDATE` of the configuration is used to
+ checkout a specific date. The special value of "now" causes the
+ checkout to be updated on every build.
+
+- *"localdir":* Used to rename the module. Effectively, you are
+ renaming the output directory to which the module is unpacked. You
+ are forcing the module into a special directory relative to
+ :term:`CVSDIR`.
+
+- *"rsh":* Used in conjunction with the "method" parameter.
+
+- *"scmdata":* Causes the CVS metadata to be maintained in the tarball
+ the fetcher creates when set to "keep". The tarball is expanded into
+ the work directory. By default, the CVS metadata is removed.
+
+- *"fullpath":* Controls whether the resulting checkout is at the
+ module level, which is the default, or is at deeper paths.
+
+- *"norecurse":* Causes the fetcher to only checkout the specified
+ directory with no recurse into any subdirectories.
+
+- *"port":* The port to which the CVS server connects.
+
+Some example URLs are as follows: ::
+
+ SRC_URI = "cvs://CVSROOT;module=mymodule;tag=some-version;method=ext"
+ SRC_URI = "cvs://CVSROOT;module=mymodule;date=20060126;localdir=usethat"
+
+.. _svn-fetcher:
+
+Subversion (SVN) Fetcher (``svn://``)
+-------------------------------------
+
+This fetcher submodule fetches code from the Subversion source control
+system. The executable used is specified by ``FETCHCMD_svn``, which
+defaults to "svn". The fetcher's temporary working directory is set by
+:term:`SVNDIR`, which is usually ``DL_DIR/svn``.
+
+The supported parameters are as follows:
+
+- *"module":* The name of the svn module to checkout. You must provide
+ this parameter. You can think of this parameter as the top-level
+ directory of the repository data you want.
+
+- *"path_spec":* A specific directory in which to checkout the
+ specified svn module.
+
+- *"protocol":* The protocol to use, which defaults to "svn". If
+ "protocol" is set to "svn+ssh", the "ssh" parameter is also used.
+
+- *"rev":* The revision of the source code to checkout.
+
+- *"scmdata":* Causes the “.svn” directories to be available during
+ compile-time when set to "keep". By default, these directories are
+ removed.
+
+- *"ssh":* An optional parameter used when "protocol" is set to
+ "svn+ssh". You can use this parameter to specify the ssh program used
+ by svn.
+
+- *"transportuser":* When required, sets the username for the
+ transport. By default, this parameter is empty. The transport
+ username is different than the username used in the main URL, which
+ is passed to the subversion command.
+
+Following are three examples using svn: ::
+
+ SRC_URI = "svn://myrepos/proj1;module=vip;protocol=http;rev=667"
+ SRC_URI = "svn://myrepos/proj1;module=opie;protocol=svn+ssh"
+ SRC_URI = "svn://myrepos/proj1;module=trunk;protocol=http;path_spec=${MY_DIR}/proj1"
+
+.. _git-fetcher:
+
+Git Fetcher (``git://``)
+------------------------
+
+This fetcher submodule fetches code from the Git source control system.
+The fetcher works by creating a bare clone of the remote into
+:term:`GITDIR`, which is usually ``DL_DIR/git2``. This
+bare clone is then cloned into the work directory during the unpack
+stage when a specific tree is checked out. This is done using alternates
+and by reference to minimize the amount of duplicate data on the disk
+and make the unpack process fast. The executable used can be set with
+``FETCHCMD_git``.
+
+This fetcher supports the following parameters:
+
+- *"protocol":* The protocol used to fetch the files. The default is
+ "git" when a hostname is set. If a hostname is not set, the Git
+ protocol is "file". You can also use "http", "https", "ssh" and
+ "rsync".
+
+- *"nocheckout":* Tells the fetcher to not checkout source code when
+ unpacking when set to "1". Set this option for the URL where there is
+ a custom routine to checkout code. The default is "0".
+
+- *"rebaseable":* Indicates that the upstream Git repository can be
+ rebased. You should set this parameter to "1" if revisions can become
+ detached from branches. In this case, the source mirror tarball is
+ done per revision, which has a loss of efficiency. Rebasing the
+ upstream Git repository could cause the current revision to disappear
+ from the upstream repository. This option reminds the fetcher to
+ preserve the local cache carefully for future use. The default value
+ for this parameter is "0".
+
+- *"nobranch":* Tells the fetcher to not check the SHA validation for
+ the branch when set to "1". The default is "0". Set this option for
+ the recipe that refers to the commit that is valid for a tag instead
+ of the branch.
+
+- *"bareclone":* Tells the fetcher to clone a bare clone into the
+ destination directory without checking out a working tree. Only the
+ raw Git metadata is provided. This parameter implies the "nocheckout"
+ parameter as well.
+
+- *"branch":* The branch(es) of the Git tree to clone. If unset, this
+ is assumed to be "master". The number of branch parameters much match
+ the number of name parameters.
+
+- *"rev":* The revision to use for the checkout. The default is
+ "master".
+
+- *"tag":* Specifies a tag to use for the checkout. To correctly
+ resolve tags, BitBake must access the network. For that reason, tags
+ are often not used. As far as Git is concerned, the "tag" parameter
+ behaves effectively the same as the "rev" parameter.
+
+- *"subpath":* Limits the checkout to a specific subpath of the tree.
+ By default, the whole tree is checked out.
+
+- *"destsuffix":* The name of the path in which to place the checkout.
+ By default, the path is ``git/``.
+
+- *"usehead":* Enables local ``git://`` URLs to use the current branch
+ HEAD as the revision for use with ``AUTOREV``. The "usehead"
+ parameter implies no branch and only works when the transfer protocol
+ is ``file://``.
+
+Here are some example URLs: ::
+
+ SRC_URI = "git://git.oe.handhelds.org/git/vip.git;tag=version-1"
+ SRC_URI = "git://git.oe.handhelds.org/git/vip.git;protocol=http"
+
+.. _gitsm-fetcher:
+
+Git Submodule Fetcher (``gitsm://``)
+------------------------------------
+
+This fetcher submodule inherits from the :ref:`Git
+fetcher<bitbake-user-manual/bitbake-user-manual-fetching:git fetcher
+(\`\`git://\`\`)>` and extends that fetcher's behavior by fetching a
+repository's submodules. :term:`SRC_URI` is passed to the Git fetcher as
+described in the :ref:`bitbake-user-manual/bitbake-user-manual-fetching:git
+fetcher (\`\`git://\`\`)` section.
+
+.. note::
+
+ You must clean a recipe when switching between '``git://``' and
+ '``gitsm://``' URLs.
+
+ The Git Submodules fetcher is not a complete fetcher implementation.
+ The fetcher has known issues where it does not use the normal source
+ mirroring infrastructure properly. Further, the submodule sources it
+ fetches are not visible to the licensing and source archiving
+ infrastructures.
+
+.. _clearcase-fetcher:
+
+ClearCase Fetcher (``ccrc://``)
+-------------------------------
+
+This fetcher submodule fetches code from a
+`ClearCase <http://en.wikipedia.org/wiki/Rational_ClearCase>`__
+repository.
+
+To use this fetcher, make sure your recipe has proper
+:term:`SRC_URI`, :term:`SRCREV`, and
+:term:`PV` settings. Here is an example: ::
+
+ SRC_URI = "ccrc://cc.example.org/ccrc;vob=/example_vob;module=/example_module"
+ SRCREV = "EXAMPLE_CLEARCASE_TAG"
+ PV = "${@d.getVar("SRCREV", False).replace("/", "+")}"
+
+The fetcher uses the ``rcleartool`` or
+``cleartool`` remote client, depending on which one is available.
+
+Following are options for the ``SRC_URI`` statement:
+
+- *vob*: The name, which must include the prepending "/" character,
+ of the ClearCase VOB. This option is required.
+
+- *module*: The module, which must include the prepending "/"
+ character, in the selected VOB.
+
+ .. note::
+
+ The module and vob options are combined to create the load rule in the
+ view config spec. As an example, consider the vob and module values from
+ the SRC_URI statement at the start of this section. Combining those values
+ results in the following: ::
+
+ load /example_vob/example_module
+
+- *proto*: The protocol, which can be either ``http`` or ``https``.
+
+By default, the fetcher creates a configuration specification. If you
+want this specification written to an area other than the default, use
+the ``CCASE_CUSTOM_CONFIG_SPEC`` variable in your recipe to define where
+the specification is written.
+
+.. note::
+
+ the SRCREV loses its functionality if you specify this variable. However,
+ SRCREV is still used to label the archive after a fetch even though it does
+ not define what is fetched.
+
+Here are a couple of other behaviors worth mentioning:
+
+- When using ``cleartool``, the login of ``cleartool`` is handled by
+ the system. The login require no special steps.
+
+- In order to use ``rcleartool`` with authenticated users, an
+ "rcleartool login" is necessary before using the fetcher.
+
+.. _perforce-fetcher:
+
+Perforce Fetcher (``p4://``)
+----------------------------
+
+This fetcher submodule fetches code from the
+`Perforce <https://www.perforce.com/>`__ source control system. The
+executable used is specified by ``FETCHCMD_p4``, which defaults to "p4".
+The fetcher's temporary working directory is set by
+:term:`P4DIR`, which defaults to "DL_DIR/p4".
+The fetcher does not make use of a perforce client, instead it
+relies on ``p4 files`` to retrieve a list of
+files and ``p4 print`` to transfer the content
+of those files locally.
+
+To use this fetcher, make sure your recipe has proper
+:term:`SRC_URI`, :term:`SRCREV`, and
+:term:`PV` values. The p4 executable is able to use the
+config file defined by your system's ``P4CONFIG`` environment variable
+in order to define the Perforce server URL and port, username, and
+password if you do not wish to keep those values in a recipe itself. If
+you choose not to use ``P4CONFIG``, or to explicitly set variables that
+``P4CONFIG`` can contain, you can specify the ``P4PORT`` value, which is
+the server's URL and port number, and you can specify a username and
+password directly in your recipe within ``SRC_URI``.
+
+Here is an example that relies on ``P4CONFIG`` to specify the server URL
+and port, username, and password, and fetches the Head Revision: ::
+
+ SRC_URI = "p4://example-depot/main/source/..."
+ SRCREV = "${AUTOREV}"
+ PV = "p4-${SRCPV}"
+ S = "${WORKDIR}/p4"
+
+Here is an example that specifies the server URL and port, username, and
+password, and fetches a Revision based on a Label: ::
+
+ P4PORT = "tcp:p4server.example.net:1666"
+ SRC_URI = "p4://user:passwd@example-depot/main/source/..."
+ SRCREV = "release-1.0"
+ PV = "p4-${SRCPV}"
+ S = "${WORKDIR}/p4"
+
+.. note::
+
+ You should always set S to "${WORKDIR}/p4" in your recipe.
+
+By default, the fetcher strips the depot location from the local file paths. In
+the above example, the content of ``example-depot/main/source/`` will be placed
+in ``${WORKDIR}/p4``. For situations where preserving parts of the remote depot
+paths locally is desirable, the fetcher supports two parameters:
+
+- *"module":*
+ The top-level depot location or directory to fetch. The value of this
+ parameter can also point to a single file within the depot, in which case
+ the local file path will include the module path.
+- *"remotepath":*
+ When used with the value "``keep``", the fetcher will mirror the full depot
+ paths locally for the specified location, even in combination with the
+ ``module`` parameter.
+
+Here is an example use of the the ``module`` parameter: ::
+
+ SRC_URI = "p4://user:passwd@example-depot/main;module=source/..."
+
+In this case, the content of the top-level directory ``source/`` will be fetched
+to ``${P4DIR}``, including the directory itself. The top-level directory will
+be accesible at ``${P4DIR}/source/``.
+
+Here is an example use of the the ``remotepath`` parameter: ::
+
+ SRC_URI = "p4://user:passwd@example-depot/main;module=source/...;remotepath=keep"
+
+In this case, the content of the top-level directory ``source/`` will be fetched
+to ``${P4DIR}``, but the complete depot paths will be mirrored locally. The
+top-level directory will be accessible at
+``${P4DIR}/example-depot/main/source/``.
+
+.. _repo-fetcher:
+
+Repo Fetcher (``repo://``)
+--------------------------
+
+This fetcher submodule fetches code from ``google-repo`` source control
+system. The fetcher works by initiating and syncing sources of the
+repository into :term:`REPODIR`, which is usually
+``${DL_DIR}/repo``.
+
+This fetcher supports the following parameters:
+
+- *"protocol":* Protocol to fetch the repository manifest (default:
+ git).
+
+- *"branch":* Branch or tag of repository to get (default: master).
+
+- *"manifest":* Name of the manifest file (default: ``default.xml``).
+
+Here are some example URLs: ::
+
+ SRC_URI = "repo://REPOROOT;protocol=git;branch=some_branch;manifest=my_manifest.xml"
+ SRC_URI = "repo://REPOROOT;protocol=file;branch=some_branch;manifest=my_manifest.xml"
+
+Other Fetchers
+--------------
+
+Fetch submodules also exist for the following:
+
+- Bazaar (``bzr://``)
+
+- Mercurial (``hg://``)
+
+- npm (``npm://``)
+
+- OSC (``osc://``)
+
+- Secure FTP (``sftp://``)
+
+- Secure Shell (``ssh://``)
+
+- Trees using Git Annex (``gitannex://``)
+
+No documentation currently exists for these lesser used fetcher
+submodules. However, you might find the code helpful and readable.
+
+Auto Revisions
+==============
+
+We need to document ``AUTOREV`` and ``SRCREV_FORMAT`` here.