master: subtree updates oct 12 2023
poky: e444d2bed0..8d0ba08aa6:
Alassane Yattara (1):
bitbake: toaster: Monitoring - implement Django logging system
Alexander Kanavin (1):
nghttp2: update 1.56.0 -> 1.57.0
Alexis Lothoré (2):
oeqa/utils/gitarchive: fix tag pattern searching
oeqa/utils/gitarchive: ensure tag matches regex before getting its fields
Andreas Cord-Landwehr (1):
wayland: consider pkgconfig sysroot for pkgdatadir
André Draszik (1):
wic: fix a typo in help output
Archana Polampalli (1):
gstreamer: upgrade 1.22.5 -> 1.22.6
Bruce Ashfield (1):
linux-yocto/6.5: integrate fixes for sanity issues
Chris Laplante (1):
recipetool/create_buildsys_python: use importlib instead of imp
Daniel McGregor (2):
meson: upgrade 1.2.1 -> 1.2.2
libtirpc: conditionally enable gssapi
Daniel Semkowicz (3):
uboot-extlinux-config.bbclass: Remove repeated space character
uboot-extlinux-config.bbclass: Uppercase "menu title" entry
uboot-extlinux-config.bbclass: Add menu title configuration
Fabio Estevam (1):
u-boot: Upgrade to 2023.10
Fahad Arslan (1):
linux-firmware: create separate package for cirrus and cnm firmwares
Jermain Horsman (1):
scripts/oe-setup-layers: Update how to determine if directory is git repo
Jose Quaresma (4):
curl: 8.3.0 -> 8.4.0
go: update 1.20.7 -> 1.20.8
go: update 1.20.8 -> 1.20.9
go: update 1.20.9 -> 1.20.10
Joshua Watt (6):
bitbake: hashserv: Add remove API
bitbake: bitbake-hashclient: Add remove subcommand
bitbake: hashserv: Extend get_outhash API to optionally include unihash
bitbake: hashserv: Add API to clean unused entries
bitbake: bitbake-hashclient: Add clean-unused subcommand
overview: Add note about non-reproducibility side effects
Julien Stephan (4):
bitbake.conf: include bblock.conf
sstatesig: add a new info level for SIGGEN_LOCKEDSIGS_TASKSIG_CHECK
scripts/bblock: add a script to lock/unlock recipes
oeqa/selftest/bblock: add self test for bblock tool
Khem Raj (1):
python3-docutils: Rename utilities to their canonical names
Lee Chee Yang (1):
migration-guides: add release notes for 4.0.13
Marcus Flyckt (1):
devtool/upgrade: check all git config locations
Markus Volk (2):
mesa: Upgrade 23.1.8 -> 23.2.1
gtk: Upgrade 4.12.1 -> 4.12.3
Marlon Rodriguez Garcia (1):
bitbake: toaster: update selenium version and code syntax
Martijn de Gouw (1):
busybox: Set PATH in syslog initscript
Michael Opdenacker (1):
ref-manual: releases.svg: Scarthgap is now version 5.0
Mikko Rapeli (2):
qemurunner.py: detect login prompt without without utf-8 conversion
openssh: drop sudo from ptest dependencies
Peter Kjellerstedt (4):
externalsrc.bbclass: Support specifying patterns in CONFIGURE_FILES
autotools.bbclass: Add *.m4 to CONFIGURE_FILES
packages.bbclass: Correct the check for conflicts with renamed packages
cmake.bbclass: Add *.cmake to CONFIGURE_FILES
Quentin Schulz (1):
uboot-extlinux-config.bbclass: fix missed override syntax migration
Randy MacLeod (1):
strace: skip so_peerpidfd test
Rasmus Villemoes (1):
openssh: update sshd_check_keys script to make use of 'sshd -G'
Richard Purdie (14):
wic: Add console parameters to qemux86 canned-wks
qemurunner: Skip conversion from/to binary data for logfile
qemurunner: Use backslashreplace with utf8 to make invalid characters clear
qemurunner: Log the second serial console as well as the first
qemurunner: Show both the login console log and all logging upon failure
oeqa/concurrencytest: Remove invalid buffering option
bitbake: selftest/fetch: Ensure top level directory timestamp doesn't break test
cryptodev: Update to latest git for 6.5 kernel fixes
qemux86/qemuarm: Drop kernel version overrides
oeqa/qemurunner: Add newlines serial workaround
runqemu/qemurunner: Use nodelay with tcp serial connections
oeqa/qemurunner: Add extra logging when console doesn't appear
poky/poky-tiny: Switch to the 6.5 kernel
oeqa/qemurunner: Ensure we retry after BrokenPipeError
Ross Burton (12):
python3-numpy: remove obsolete reproducible workaround
libx11: upgrade to 1.8.7
libxpm: upgrade to 3.5.17
qemuboot: reduce default size of software I/O translation buffer
sysvinit-initab: rewrite loop to generate inittab
ttyrun: add new recipe
sysvinit-inittab: use ttyrun to run getty only if the terminal exists
busybox: use ttyrun to run getty only if the terminal exists
oeqa/selftest: don't skip test_read_only_image on qemuarm64
meta/conf/machine: remove SERIAL_CONSOLES_CHECK
busybox-inittab: fix console handling
oeqa/runtime/_qemutiny: rewrite test to be functional
Sundeep KOKKONDA (1):
rust: reproducibility issue fix
Trevor Gamblin (1):
dev-manual: fix testimage usage instructions
Yogita Urade (1):
qemu: fix CVE-2023-42467
meta-openembedded: ea42cec2ec..62039a2c33:
Ahmad Fatoum (1):
signing.bbclass: don't export OPENSSL environment variables globally
Beniamin Sandu (1):
libnet: upgrade version v1.2 -> v1.3
Benjamin Bara (1):
libvpx: upgrade 1.13.0 -> 1.13.1
Chen Qi (1):
libblockdev: fix QA error in case of multilib
Christophe Vu-Brugier (3):
libnvme: upgrade 1.5 -> 1.6
nvme-cli: upgrade 2.5 -> 2.6
libnvme: apply patch already upstream to fix build with musl
Clément Péron (1):
Revert "protobuf: stage protoc binary to sysroot"
Daniel Klauer (1):
graphviz: Fix build to not use $prefix as search dir
Denys Zagorui (1):
libbpf: add arm, powerpc and mips64 to COMPATIBLE_HOST
Fabien Thomas (8):
meta-filesystems/layer.conf : Add meta-networking dependency
Add static-passwd and static-group files
Add static-passwd and static-group files
Add static-passwd and static-group files
Add static-passwd and static-group files
Add static-passwd and static-group files
Add static-passwd and static-group files
Add static-passwd and static-group files
Gianfranco Costamagna (6):
dlt-daemon: Make it work without systemd
dlt-daemon: Enable experimental coredumphandler feature
dlt-daemon: update patch 544.patch
dlt-daemon: do not disable dlt-system build when systemd is set to off
dlt-daemon: Add an additional fix for non-systemd builds
cpprestsdk: fix typo in comment, tag is actually 2.0.18
Jeffrey Pautler (1):
bolt: change product name used for CVE checking
Joe Slater (1):
nginx: add configure option
Johannes Kauffmann (1):
open62541: add Backport status and link to patch
Jörg Sommer (1):
collectd: Use https in SRC_URI, add HOMEPAGE
Khem Raj (16):
python3-pyroute2: Add missing dependency on sqlite3 for ptests
python3-pylint: Upgrade to 3.0.0
python3-lz4: use python3-unittest-automake-output
minicoredumber: Fix ptest reporting
images: Inherit from core-image-base
images: Delete layer specific base images
images: Rename <layer>-image to <layer>-image-all
images: Rename ptest images to rhyme with oe-core ptest images
ptest-image: Switch to using core-image-minimal
stressapptest: Upgrade to 1.0.11 release
klibc: Upgrade to 2.0.13 release
libnvme: Fix test builds on musl
kernel-selftest: Build bpf tests again
ptest-packagelists-meta-oe: Add kernel-selftest to x86/x86-64 images
kernel-selftest: Copy the .config from kernel build
kernel-selftest: Use clang options when clang is available
Markus Volk (3):
pugixml: Update 1.13 -> 1.14
pipewire: Upgrade 0.3.80 -> 0.3.81
gnome-control-center: Fix polkit gettext issue
Martin Jansa (1):
opencv: Fix build with protobuf v22 and dnn enabled
Mickael RAMILISON (1):
python3-rapidjson: add ptest
Pawel Langowski (1):
recipes-connectivity: Add tayga recipe
Philip-Dylan Gleonec (1):
cukinia: Fix license field
Thomas Roos (1):
python3-boto3, python3-botocore: remove recipes
Tom Hochstein (1):
libcamera: Avoid build break in signature recalculation
meta-arm: 95789365f7..e914891eee:
Jon Mason (1):
arm-bsp/u-boot: add recipe for 2023.07.02
meta-security: aca6d4a9e7..3f7d40b0fc:
Rasmus Villemoes (3):
fail2ban: add systemd support
fail2ban: change sqlite3 dependency to python3-sqlite3
fail2ban: add useful recommendations
Change-Id: I93672642f4e0392adc6223fdc4e073910b817bc9
Signed-off-by: Andrew Geissler <geissonator@yahoo.com>
diff --git a/poky/bitbake/bin/bitbake-hashclient b/poky/bitbake/bin/bitbake-hashclient
index 494f175..3f265e8 100755
--- a/poky/bitbake/bin/bitbake-hashclient
+++ b/poky/bitbake/bin/bitbake-hashclient
@@ -113,6 +113,19 @@
with lock:
pbar.update()
+ def handle_remove(args, client):
+ where = {k: v for k, v in args.where}
+ if where:
+ result = client.remove(where)
+ print("Removed %d row(s)" % (result["count"]))
+ else:
+ print("No query specified")
+
+ def handle_clean_unused(args, client):
+ result = client.clean_unused(args.max_age)
+ print("Removed %d rows" % (result["count"]))
+ return 0
+
parser = argparse.ArgumentParser(description='Hash Equivalence Client')
parser.add_argument('--address', default=DEFAULT_ADDRESS, help='Server address (default "%(default)s")')
parser.add_argument('--log', default='WARNING', help='Set logging level')
@@ -137,6 +150,15 @@
help='Include string in outhash')
stress_parser.set_defaults(func=handle_stress)
+ remove_parser = subparsers.add_parser('remove', help="Remove hash entries")
+ remove_parser.add_argument("--where", "-w", metavar="KEY VALUE", nargs=2, action="append", default=[],
+ help="Remove entries from table where KEY == VALUE")
+ remove_parser.set_defaults(func=handle_remove)
+
+ clean_unused_parser = subparsers.add_parser('clean-unused', help="Remove unused database entries")
+ clean_unused_parser.add_argument("max_age", metavar="SECONDS", type=int, help="Remove unused entries older than SECONDS old")
+ clean_unused_parser.set_defaults(func=handle_clean_unused)
+
args = parser.parse_args()
logger = logging.getLogger('hashserv')
diff --git a/poky/bitbake/lib/bb/tests/fetch.py b/poky/bitbake/lib/bb/tests/fetch.py
index c5d15e9..eeb7a31 100644
--- a/poky/bitbake/lib/bb/tests/fetch.py
+++ b/poky/bitbake/lib/bb/tests/fetch.py
@@ -684,11 +684,13 @@
archive = tarfile.open(os.path.join(self.dldir, self.recipe_tarball))
self.assertNotEqual(len(archive.members), 0)
for member in archive.members:
- self.assertEqual(member.uname, 'oe')
- self.assertEqual(member.uid, 0)
- self.assertEqual(member.gname, 'oe')
- self.assertEqual(member.gid, 0)
- self.assertEqual(member.mtime, mtime)
+ if member.name == ".":
+ continue
+ self.assertEqual(member.uname, 'oe', "user name for %s differs" % member.name)
+ self.assertEqual(member.uid, 0, "uid for %s differs" % member.name)
+ self.assertEqual(member.gname, 'oe', "group name for %s differs" % member.name)
+ self.assertEqual(member.gid, 0, "gid for %s differs" % member.name)
+ self.assertEqual(member.mtime, mtime, "mtime for %s differs" % member.name)
class FetcherLocalTest(FetcherTest):
diff --git a/poky/bitbake/lib/hashserv/client.py b/poky/bitbake/lib/hashserv/client.py
index b2aa102..f676d26 100644
--- a/poky/bitbake/lib/hashserv/client.py
+++ b/poky/bitbake/lib/hashserv/client.py
@@ -83,10 +83,10 @@
{"get": {"taskhash": taskhash, "method": method, "all": all_properties}}
)
- async def get_outhash(self, method, outhash, taskhash):
+ async def get_outhash(self, method, outhash, taskhash, with_unihash=True):
await self._set_mode(self.MODE_NORMAL)
return await self.send_message(
- {"get-outhash": {"outhash": outhash, "taskhash": taskhash, "method": method}}
+ {"get-outhash": {"outhash": outhash, "taskhash": taskhash, "method": method, "with_unihash": with_unihash}}
)
async def get_stats(self):
@@ -101,6 +101,14 @@
await self._set_mode(self.MODE_NORMAL)
return (await self.send_message({"backfill-wait": None}))["tasks"]
+ async def remove(self, where):
+ await self._set_mode(self.MODE_NORMAL)
+ return await self.send_message({"remove": {"where": where}})
+
+ async def clean_unused(self, max_age):
+ await self._set_mode(self.MODE_NORMAL)
+ return await self.send_message({"clean-unused": {"max_age_seconds": max_age}})
+
class Client(bb.asyncrpc.Client):
def __init__(self):
@@ -115,6 +123,8 @@
"get_stats",
"reset_stats",
"backfill_wait",
+ "remove",
+ "clean_unused",
)
def _get_async_client(self):
diff --git a/poky/bitbake/lib/hashserv/server.py b/poky/bitbake/lib/hashserv/server.py
index d40a2ab..45bf476 100644
--- a/poky/bitbake/lib/hashserv/server.py
+++ b/poky/bitbake/lib/hashserv/server.py
@@ -4,7 +4,7 @@
#
from contextlib import closing, contextmanager
-from datetime import datetime
+from datetime import datetime, timedelta
import enum
import asyncio
import logging
@@ -186,6 +186,8 @@
'report-equiv': self.handle_equivreport,
'reset-stats': self.handle_reset_stats,
'backfill-wait': self.handle_backfill_wait,
+ 'remove': self.handle_remove,
+ 'clean-unused': self.handle_clean_unused,
})
def validate_proto_version(self):
@@ -269,27 +271,42 @@
method = request['method']
outhash = request['outhash']
taskhash = request['taskhash']
+ with_unihash = request.get("with_unihash", True)
with closing(self.db.cursor()) as cursor:
- d = await self.get_outhash(cursor, method, outhash, taskhash)
+ d = await self.get_outhash(cursor, method, outhash, taskhash, with_unihash)
self.write_message(d)
- async def get_outhash(self, cursor, method, outhash, taskhash):
+ async def get_outhash(self, cursor, method, outhash, taskhash, with_unihash=True):
d = None
- cursor.execute(
- '''
- SELECT *, unihashes_v2.unihash AS unihash FROM outhashes_v2
- INNER JOIN unihashes_v2 ON unihashes_v2.method=outhashes_v2.method AND unihashes_v2.taskhash=outhashes_v2.taskhash
- WHERE outhashes_v2.method=:method AND outhashes_v2.outhash=:outhash
- ORDER BY outhashes_v2.created ASC
- LIMIT 1
- ''',
- {
- 'method': method,
- 'outhash': outhash,
- }
- )
+ if with_unihash:
+ cursor.execute(
+ '''
+ SELECT *, unihashes_v2.unihash AS unihash FROM outhashes_v2
+ INNER JOIN unihashes_v2 ON unihashes_v2.method=outhashes_v2.method AND unihashes_v2.taskhash=outhashes_v2.taskhash
+ WHERE outhashes_v2.method=:method AND outhashes_v2.outhash=:outhash
+ ORDER BY outhashes_v2.created ASC
+ LIMIT 1
+ ''',
+ {
+ 'method': method,
+ 'outhash': outhash,
+ }
+ )
+ else:
+ cursor.execute(
+ """
+ SELECT * FROM outhashes_v2
+ WHERE outhashes_v2.method=:method AND outhashes_v2.outhash=:outhash
+ ORDER BY outhashes_v2.created ASC
+ LIMIT 1
+ """,
+ {
+ 'method': method,
+ 'outhash': outhash,
+ }
+ )
row = cursor.fetchone()
if row is not None:
@@ -499,6 +516,50 @@
await self.backfill_queue.join()
self.write_message(d)
+ async def handle_remove(self, request):
+ condition = request["where"]
+ if not isinstance(condition, dict):
+ raise TypeError("Bad condition type %s" % type(condition))
+
+ def do_remove(columns, table_name, cursor):
+ nonlocal condition
+ where = {}
+ for c in columns:
+ if c in condition and condition[c] is not None:
+ where[c] = condition[c]
+
+ if where:
+ query = ('DELETE FROM %s WHERE ' % table_name) + ' AND '.join("%s=:%s" % (k, k) for k in where.keys())
+ cursor.execute(query, where)
+ return cursor.rowcount
+
+ return 0
+
+ count = 0
+ with closing(self.db.cursor()) as cursor:
+ count += do_remove(OUTHASH_TABLE_COLUMNS, "outhashes_v2", cursor)
+ count += do_remove(UNIHASH_TABLE_COLUMNS, "unihashes_v2", cursor)
+ self.db.commit()
+
+ self.write_message({"count": count})
+
+ async def handle_clean_unused(self, request):
+ max_age = request["max_age_seconds"]
+ with closing(self.db.cursor()) as cursor:
+ cursor.execute(
+ """
+ DELETE FROM outhashes_v2 WHERE created<:oldest AND NOT EXISTS (
+ SELECT unihashes_v2.id FROM unihashes_v2 WHERE unihashes_v2.method=outhashes_v2.method AND unihashes_v2.taskhash=outhashes_v2.taskhash LIMIT 1
+ )
+ """,
+ {
+ "oldest": datetime.now() - timedelta(seconds=-max_age)
+ }
+ )
+ count = cursor.rowcount
+
+ self.write_message({"count": count})
+
def query_equivalent(self, cursor, method, taskhash):
# This is part of the inner loop and must be as fast as possible
cursor.execute(
diff --git a/poky/bitbake/lib/hashserv/tests.py b/poky/bitbake/lib/hashserv/tests.py
index f6b85ae..f343c58 100644
--- a/poky/bitbake/lib/hashserv/tests.py
+++ b/poky/bitbake/lib/hashserv/tests.py
@@ -84,6 +84,7 @@
result = self.client.report_unihash(taskhash, self.METHOD, outhash, unihash)
self.assertEqual(result['unihash'], unihash, 'Server returned bad unihash')
+ return taskhash, outhash, unihash
def test_create_equivalent(self):
# Tests that a second reported task with the same outhash will be
@@ -125,6 +126,57 @@
self.assertClientGetHash(self.client, taskhash, unihash)
+ def test_remove_taskhash(self):
+ taskhash, outhash, unihash = self.test_create_hash()
+ result = self.client.remove({"taskhash": taskhash})
+ self.assertGreater(result["count"], 0)
+ self.assertClientGetHash(self.client, taskhash, None)
+
+ result_outhash = self.client.get_outhash(self.METHOD, outhash, taskhash)
+ self.assertIsNone(result_outhash)
+
+ def test_remove_unihash(self):
+ taskhash, outhash, unihash = self.test_create_hash()
+ result = self.client.remove({"unihash": unihash})
+ self.assertGreater(result["count"], 0)
+ self.assertClientGetHash(self.client, taskhash, None)
+
+ def test_remove_outhash(self):
+ taskhash, outhash, unihash = self.test_create_hash()
+ result = self.client.remove({"outhash": outhash})
+ self.assertGreater(result["count"], 0)
+
+ result_outhash = self.client.get_outhash(self.METHOD, outhash, taskhash)
+ self.assertIsNone(result_outhash)
+
+ def test_remove_method(self):
+ taskhash, outhash, unihash = self.test_create_hash()
+ result = self.client.remove({"method": self.METHOD})
+ self.assertGreater(result["count"], 0)
+ self.assertClientGetHash(self.client, taskhash, None)
+
+ result_outhash = self.client.get_outhash(self.METHOD, outhash, taskhash)
+ self.assertIsNone(result_outhash)
+
+ def test_clean_unused(self):
+ taskhash, outhash, unihash = self.test_create_hash()
+
+ # Clean the database, which should not remove anything because all hashes an in-use
+ result = self.client.clean_unused(0)
+ self.assertEqual(result["count"], 0)
+ self.assertClientGetHash(self.client, taskhash, unihash)
+
+ # Remove the unihash. The row in the outhash table should still be present
+ self.client.remove({"unihash": unihash})
+ result_outhash = self.client.get_outhash(self.METHOD, outhash, taskhash, False)
+ self.assertIsNotNone(result_outhash)
+
+ # Now clean with no minimum age which will remove the outhash
+ result = self.client.clean_unused(0)
+ self.assertEqual(result["count"], 1)
+ result_outhash = self.client.get_outhash(self.METHOD, outhash, taskhash, False)
+ self.assertIsNone(result_outhash)
+
def test_huge_message(self):
# Simple test that hashes can be created
taskhash = 'c665584ee6817aa99edfc77a44dd853828279370'
diff --git a/poky/bitbake/lib/toaster/bldcollector/views.py b/poky/bitbake/lib/toaster/bldcollector/views.py
index 04cd8b3..bdf38ae 100644
--- a/poky/bitbake/lib/toaster/bldcollector/views.py
+++ b/poky/bitbake/lib/toaster/bldcollector/views.py
@@ -14,8 +14,11 @@
import toastermain
from django.views.decorators.csrf import csrf_exempt
+from toastermain.logs import log_view_mixin
+
@csrf_exempt
+@log_view_mixin
def eventfile(request):
""" Receives a file by POST, and runs toaster-eventreply on this file """
if request.method != "POST":
diff --git a/poky/bitbake/lib/toaster/logs/.gitignore b/poky/bitbake/lib/toaster/logs/.gitignore
new file mode 100644
index 0000000..e5ebf25
--- /dev/null
+++ b/poky/bitbake/lib/toaster/logs/.gitignore
@@ -0,0 +1 @@
+*.log*
diff --git a/poky/bitbake/lib/toaster/orm/models.py b/poky/bitbake/lib/toaster/orm/models.py
index f9fcf9e..0d503a5 100644
--- a/poky/bitbake/lib/toaster/orm/models.py
+++ b/poky/bitbake/lib/toaster/orm/models.py
@@ -1733,7 +1733,7 @@
packages_conf += "\""
base_recipe_path = self.get_base_recipe_file()
- if base_recipe_path:
+ if base_recipe_path and os.path.isfile(base_recipe_path):
base_recipe = open(base_recipe_path, 'r').read()
else:
# Pass back None to trigger error message to user
diff --git a/poky/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py b/poky/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py
index 644d45f..9a4e27a 100644
--- a/poky/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py
+++ b/poky/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py
@@ -21,6 +21,7 @@
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
+from selenium.webdriver.common.by import By
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.common.exceptions import NoSuchElementException, \
StaleElementReferenceException, TimeoutException
@@ -32,9 +33,7 @@
browser = env_browser
if browser == 'chrome':
- return webdriver.Chrome(
- service_args=["--verbose", "--log-path=selenium.log"]
- )
+ return webdriver.Chrome()
elif browser == 'firefox':
return webdriver.Firefox()
elif browser == 'marionette':
@@ -153,11 +152,11 @@
def find(self, selector):
""" Find single element by CSS selector """
- return self.driver.find_element_by_css_selector(selector)
+ return self.driver.find_element(By.CSS_SELECTOR, selector)
def find_all(self, selector):
""" Find all elements matching CSS selector """
- return self.driver.find_elements_by_css_selector(selector)
+ return self.driver.find_elements(By.CSS_SELECTOR, selector)
def element_exists(self, selector):
"""
diff --git a/poky/bitbake/lib/toaster/tests/browser/test_all_builds_page.py b/poky/bitbake/lib/toaster/tests/browser/test_all_builds_page.py
index 8423d3d..d4312bb 100644
--- a/poky/bitbake/lib/toaster/tests/browser/test_all_builds_page.py
+++ b/poky/bitbake/lib/toaster/tests/browser/test_all_builds_page.py
@@ -7,7 +7,7 @@
# SPDX-License-Identifier: GPL-2.0-only
#
-import re
+import re, time
from django.urls import reverse
from django.utils import timezone
@@ -15,6 +15,8 @@
from orm.models import BitbakeVersion, Release, Project, Build, Target
+from selenium.webdriver.common.by import By
+
class TestAllBuildsPage(SeleniumTestCase):
""" Tests for all builds page /builds/ """
@@ -91,7 +93,7 @@
found_row = None
for row in rows:
- outcome_links = row.find_elements_by_css_selector(selector)
+ outcome_links = row.find_elements(By.CSS_SELECTOR, selector)
if len(outcome_links) == 1:
found_row = row
break
@@ -131,17 +133,19 @@
url = reverse('all-builds')
self.get(url)
+ # should see a rebuild button for non-command-line builds
+ selector = 'div[data-latest-build-result="%s"] .rebuild-btn' % build1.id
+ time.sleep(2)
+ run_again_button = self.find_all(selector)
+ self.assertEqual(len(run_again_button), 1,
+ 'should see a rebuild button for non-cli builds')
+
# shouldn't see a rebuild button for command-line builds
selector = 'div[data-latest-build-result="%s"] .rebuild-btn' % default_build.id
run_again_button = self.find_all(selector)
self.assertEqual(len(run_again_button), 0,
'should not see a rebuild button for cli builds')
- # should see a rebuild button for non-command-line builds
- selector = 'div[data-latest-build-result="%s"] .rebuild-btn' % build1.id
- run_again_button = self.find_all(selector)
- self.assertEqual(len(run_again_button), 1,
- 'should see a rebuild button for non-cli builds')
def test_tooltips_on_project_name(self):
"""
@@ -198,24 +202,24 @@
# test recent builds area for successful build
element = self._get_build_time_element(build1)
- links = element.find_elements_by_css_selector('a')
+ links = element.find_elements(By.CSS_SELECTOR, 'a')
msg = 'should be a link on the build time for a successful recent build'
self.assertEquals(len(links), 1, msg)
# test recent builds area for failed build
element = self._get_build_time_element(build2)
- links = element.find_elements_by_css_selector('a')
+ links = element.find_elements(By.CSS_SELECTOR, 'a')
msg = 'should not be a link on the build time for a failed recent build'
self.assertEquals(len(links), 0, msg)
# test the time column for successful build
build1_row = self._get_row_for_build(build1)
- links = build1_row.find_elements_by_css_selector('td.time a')
+ links = build1_row.find_elements(By.CSS_SELECTOR, 'td.time a')
msg = 'should be a link on the build time for a successful build'
self.assertEquals(len(links), 1, msg)
# test the time column for failed build
build2_row = self._get_row_for_build(build2)
- links = build2_row.find_elements_by_css_selector('td.time a')
+ links = build2_row.find_elements(By.CSS_SELECTOR, 'td.time a')
msg = 'should not be a link on the build time for a failed build'
self.assertEquals(len(links), 0, msg)
diff --git a/poky/bitbake/lib/toaster/tests/browser/test_all_projects_page.py b/poky/bitbake/lib/toaster/tests/browser/test_all_projects_page.py
index 15b0340..3389d32 100644
--- a/poky/bitbake/lib/toaster/tests/browser/test_all_projects_page.py
+++ b/poky/bitbake/lib/toaster/tests/browser/test_all_projects_page.py
@@ -16,6 +16,8 @@
from orm.models import BitbakeVersion, Release, Project, Build
from orm.models import ProjectVariable
+from selenium.webdriver.common.by import By
+
class TestAllProjectsPage(SeleniumTestCase):
""" Browser tests for projects page /projects/ """
@@ -117,7 +119,7 @@
# check the release text for the default project
selector = 'span[data-project-field="release"] span.text-muted'
- element = default_project_row.find_element_by_css_selector(selector)
+ element = default_project_row.find_element(By.CSS_SELECTOR, selector)
text = element.text.strip()
self.assertEqual(text, 'Not applicable',
'release should be "not applicable" for default project')
@@ -127,7 +129,7 @@
# check the link in the release cell for the other project
selector = 'span[data-project-field="release"]'
- element = other_project_row.find_element_by_css_selector(selector)
+ element = other_project_row.find_element(By.CSS_SELECTOR, selector)
text = element.text.strip()
self.assertEqual(text, self.release.name,
'release name should be shown for non-default project')
@@ -152,7 +154,7 @@
# check the machine cell for the default project
selector = 'span[data-project-field="machine"] span.text-muted'
- element = default_project_row.find_element_by_css_selector(selector)
+ element = default_project_row.find_element(By.CSS_SELECTOR, selector)
text = element.text.strip()
self.assertEqual(text, 'Not applicable',
'machine should be not applicable for default project')
@@ -162,7 +164,7 @@
# check the link in the machine cell for the other project
selector = 'span[data-project-field="machine"]'
- element = other_project_row.find_element_by_css_selector(selector)
+ element = other_project_row.find_element(By.CSS_SELECTOR, selector)
text = element.text.strip()
self.assertEqual(text, self.MACHINE_NAME,
'machine name should be shown for non-default project')
@@ -187,7 +189,7 @@
# check the link on the name field
selector = 'span[data-project-field="name"] a'
- element = default_project_row.find_element_by_css_selector(selector)
+ element = default_project_row.find_element(By.CSS_SELECTOR, selector)
link_url = element.get_attribute('href').strip()
expected_url = reverse('projectbuilds', args=(self.default_project.id,))
msg = 'link on default project name should point to builds but was %s' % link_url
@@ -198,7 +200,7 @@
# check the link for the other project
selector = 'span[data-project-field="name"] a'
- element = other_project_row.find_element_by_css_selector(selector)
+ element = other_project_row.find_element(By.CSS_SELECTOR, selector)
link_url = element.get_attribute('href').strip()
expected_url = reverse('project', args=(self.project.id,))
msg = 'link on project name should point to configuration but was %s' % link_url
diff --git a/poky/bitbake/lib/toaster/tests/browser/test_builddashboard_page.py b/poky/bitbake/lib/toaster/tests/browser/test_builddashboard_page.py
index efcd89b..1afa4a4 100644
--- a/poky/bitbake/lib/toaster/tests/browser/test_builddashboard_page.py
+++ b/poky/bitbake/lib/toaster/tests/browser/test_builddashboard_page.py
@@ -15,6 +15,8 @@
from orm.models import Project, Release, BitbakeVersion, Build, LogMessage
from orm.models import Layer, Layer_Version, Recipe, CustomImageRecipe, Variable
+from selenium.webdriver.common.by import By
+
class TestBuildDashboardPage(SeleniumTestCase):
""" Tests for the build dashboard /build/X """
@@ -183,7 +185,7 @@
found = False
for element in message_elements:
- log_message_text = element.find_element_by_tag_name('pre').text.strip()
+ log_message_text = element.find_element(By.TAG_NAME, 'pre').text.strip()
text_matches = (log_message_text == expected_text)
log_message_pk = element.get_attribute('data-log-message-id')
@@ -213,7 +215,7 @@
the WebElement modal match the list of text values in expected
"""
# labels containing the radio buttons we're testing for
- labels = modal.find_elements_by_css_selector(".radio")
+ labels = modal.find_elements(By.CSS_SELECTOR,".radio")
labels_text = [lab.text for lab in labels]
self.assertEqual(len(labels_text), len(expected))
@@ -248,7 +250,7 @@
selector = '[data-role="edit-custom-image-trigger"]'
self.click(selector)
- modal = self.driver.find_element_by_id('edit-custom-image-modal')
+ modal = self.driver.find_element(By.ID, 'edit-custom-image-modal')
self.wait_until_visible("#edit-custom-image-modal")
# recipes we expect to see in the edit custom image modal
@@ -270,7 +272,7 @@
selector = '[data-role="new-custom-image-trigger"]'
self.click(selector)
- modal = self.driver.find_element_by_id('new-custom-image-modal')
+ modal = self.driver.find_element(By.ID,'new-custom-image-modal')
self.wait_until_visible("#new-custom-image-modal")
# recipes we expect to see in the new custom image modal
diff --git a/poky/bitbake/lib/toaster/tests/browser/test_most_recent_builds_states.py b/poky/bitbake/lib/toaster/tests/browser/test_most_recent_builds_states.py
index 7844aaa..a34a092 100644
--- a/poky/bitbake/lib/toaster/tests/browser/test_most_recent_builds_states.py
+++ b/poky/bitbake/lib/toaster/tests/browser/test_most_recent_builds_states.py
@@ -6,7 +6,7 @@
#
# Copyright (C) 2013-2016 Intel Corporation
#
-
+import time
from django.urls import reverse
from django.utils import timezone
from tests.browser.selenium_helpers import SeleniumTestCase
@@ -14,6 +14,8 @@
from orm.models import Project, Build, Task, Recipe, Layer, Layer_Version
from bldcontrol.models import BuildRequest
+from selenium.webdriver.common.by import By
+
class TestMostRecentBuildsStates(SeleniumTestCase):
""" Test states update correctly in most recent builds area """
@@ -62,7 +64,7 @@
element = self.wait_until_visible(selector)
bar_selector = '#recipes-parsed-percentage-bar-%s' % build.id
- bar_element = element.find_element_by_css_selector(bar_selector)
+ bar_element = element.find_element(By.CSS_SELECTOR, bar_selector)
self.assertEqual(bar_element.value_of_css_property('width'), '0px',
'recipe parse progress should be at 0')
@@ -73,7 +75,7 @@
self.get(url)
element = self.wait_until_visible(selector)
- bar_element = element.find_element_by_css_selector(bar_selector)
+ bar_element = element.find_element(By.CSS_SELECTOR, bar_selector)
recipe_bar_updated = lambda driver: \
bar_element.get_attribute('style') == 'width: 50%;'
msg = 'recipe parse progress bar should update to 50%'
@@ -107,7 +109,7 @@
element = self.wait_until_visible(selector)
bar_selector = '#build-pc-done-bar-%s' % build.id
- bar_element = element.find_element_by_css_selector(bar_selector)
+ bar_element = element.find_element(By.CSS_SELECTOR, bar_selector)
task_bar_updated = lambda driver: \
bar_element.get_attribute('style') == 'width: 50%;'
@@ -121,7 +123,7 @@
self.get(url)
element = self.wait_until_visible(selector)
- bar_element = element.find_element_by_css_selector(bar_selector)
+ bar_element = element.find_element(By.CSS_SELECTOR, bar_selector)
task_bar_updated = lambda driver: \
bar_element.get_attribute('style') == 'width: 100%;'
msg = 'tasks progress bar should update to 100%'
diff --git a/poky/bitbake/lib/toaster/tests/browser/test_new_custom_image_page.py b/poky/bitbake/lib/toaster/tests/browser/test_new_custom_image_page.py
index 9906ae4..6361f40 100644
--- a/poky/bitbake/lib/toaster/tests/browser/test_new_custom_image_page.py
+++ b/poky/bitbake/lib/toaster/tests/browser/test_new_custom_image_page.py
@@ -6,6 +6,7 @@
#
# SPDX-License-Identifier: GPL-2.0-only
#
+from bldcontrol.models import BuildEnvironment
from django.urls import reverse
from tests.browser.selenium_helpers import SeleniumTestCase
@@ -18,6 +19,9 @@
CUSTOM_IMAGE_NAME = 'roopa-doopa'
def setUp(self):
+ BuildEnvironment.objects.get_or_create(
+ betype=BuildEnvironment.TYPE_LOCAL,
+ )
release = Release.objects.create(
name='baz',
bitbake_version=BitbakeVersion.objects.create(name='v1')
diff --git a/poky/bitbake/lib/toaster/tests/browser/test_new_project_page.py b/poky/bitbake/lib/toaster/tests/browser/test_new_project_page.py
index e20a1f6..f4b2708 100644
--- a/poky/bitbake/lib/toaster/tests/browser/test_new_project_page.py
+++ b/poky/bitbake/lib/toaster/tests/browser/test_new_project_page.py
@@ -6,11 +6,13 @@
#
# SPDX-License-Identifier: GPL-2.0-only
#
+import time
from django.urls import reverse
from tests.browser.selenium_helpers import SeleniumTestCase
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import InvalidElementStateException
+from selenium.webdriver.common.by import By
from orm.models import Project, Release, BitbakeVersion
@@ -47,13 +49,14 @@
url = reverse('newproject')
self.get(url)
-
self.enter_text('#new-project-name', project_name)
select = Select(self.find('#projectversion'))
select.select_by_value(str(self.release.pk))
+ time.sleep(1)
self.click("#create-project-button")
+ time.sleep(2)
# We should get redirected to the new project's page with the
# notification at the top
@@ -84,6 +87,12 @@
select = Select(self.find('#projectversion'))
select.select_by_value(str(self.release.pk))
+ radio = self.driver.find_element(By.ID, 'type-new')
+ radio.click()
+
+ self.click("#create-project-button")
+ time.sleep(2)
+
element = self.wait_until_visible('#hint-error-project-name')
self.assertTrue(("Project names must be unique" in element.text),
@@ -96,6 +105,7 @@
except InvalidElementStateException:
pass
+ time.sleep(2)
self.assertTrue(
(Project.objects.filter(name=project_name).count() == 1),
"New project not found in database")
diff --git a/poky/bitbake/lib/toaster/tests/browser/test_project_config_page.py b/poky/bitbake/lib/toaster/tests/browser/test_project_config_page.py
index 944bcb2..7b21460 100644
--- a/poky/bitbake/lib/toaster/tests/browser/test_project_config_page.py
+++ b/poky/bitbake/lib/toaster/tests/browser/test_project_config_page.py
@@ -11,6 +11,7 @@
from tests.browser.selenium_helpers import SeleniumTestCase
from orm.models import BitbakeVersion, Release, Project, ProjectVariable
+from selenium.webdriver.common.by import By
class TestProjectConfigsPage(SeleniumTestCase):
""" Test data at /project/X/builds is displayed correctly """
@@ -66,7 +67,7 @@
self.enter_text('#new-imagefs_types', imagefs_type)
- checkboxes = self.driver.find_elements_by_xpath("//input[@class='fs-checkbox-fstypes']")
+ checkboxes = self.driver.find_elements(By.XPATH, "//input[@class='fs-checkbox-fstypes']")
for checkbox in checkboxes:
if checkbox.get_attribute("value") == "btrfs":
@@ -95,7 +96,7 @@
for checkbox in checkboxes:
if checkbox.get_attribute("value") == "cpio":
checkbox.click()
- element = self.driver.find_element_by_id('new-imagefs_types')
+ element = self.driver.find_element(By.ID, 'new-imagefs_types')
self.wait_until_visible('#new-imagefs_types')
@@ -129,7 +130,7 @@
self.assertTrue((self.INVALID_PATH_START_TEXT in element.text), msg)
# downloads dir path has a space
- self.driver.find_element_by_id('new-dl_dir').clear()
+ self.driver.find_element(By.ID, 'new-dl_dir').clear()
self.enter_text('#new-dl_dir', '/foo/bar a')
element = self.wait_until_visible('#hintError-dl_dir')
@@ -137,7 +138,7 @@
self.assertTrue((self.INVALID_PATH_CHAR_TEXT in element.text), msg)
# downloads dir path starts with ${...} but has a space
- self.driver.find_element_by_id('new-dl_dir').clear()
+ self.driver.find_element(By.ID,'new-dl_dir').clear()
self.enter_text('#new-dl_dir', '${TOPDIR}/down foo')
element = self.wait_until_visible('#hintError-dl_dir')
@@ -145,18 +146,18 @@
self.assertTrue((self.INVALID_PATH_CHAR_TEXT in element.text), msg)
# downloads dir path starts with /
- self.driver.find_element_by_id('new-dl_dir').clear()
+ self.driver.find_element(By.ID,'new-dl_dir').clear()
self.enter_text('#new-dl_dir', '/bar/foo')
- hidden_element = self.driver.find_element_by_id('hintError-dl_dir')
+ hidden_element = self.driver.find_element(By.ID,'hintError-dl_dir')
self.assertEqual(hidden_element.is_displayed(), False,
'downloads directory path valid but treated as invalid')
# downloads dir path starts with ${...}
- self.driver.find_element_by_id('new-dl_dir').clear()
+ self.driver.find_element(By.ID,'new-dl_dir').clear()
self.enter_text('#new-dl_dir', '${TOPDIR}/down')
- hidden_element = self.driver.find_element_by_id('hintError-dl_dir')
+ hidden_element = self.driver.find_element(By.ID,'hintError-dl_dir')
self.assertEqual(hidden_element.is_displayed(), False,
'downloads directory path valid but treated as invalid')
@@ -184,7 +185,7 @@
self.assertTrue((self.INVALID_PATH_START_TEXT in element.text), msg)
# path has a space
- self.driver.find_element_by_id('new-sstate_dir').clear()
+ self.driver.find_element(By.ID, 'new-sstate_dir').clear()
self.enter_text('#new-sstate_dir', '/foo/bar a')
element = self.wait_until_visible('#hintError-sstate_dir')
@@ -192,7 +193,7 @@
self.assertTrue((self.INVALID_PATH_CHAR_TEXT in element.text), msg)
# path starts with ${...} but has a space
- self.driver.find_element_by_id('new-sstate_dir').clear()
+ self.driver.find_element(By.ID,'new-sstate_dir').clear()
self.enter_text('#new-sstate_dir', '${TOPDIR}/down foo')
element = self.wait_until_visible('#hintError-sstate_dir')
@@ -200,18 +201,18 @@
self.assertTrue((self.INVALID_PATH_CHAR_TEXT in element.text), msg)
# path starts with /
- self.driver.find_element_by_id('new-sstate_dir').clear()
+ self.driver.find_element(By.ID,'new-sstate_dir').clear()
self.enter_text('#new-sstate_dir', '/bar/foo')
- hidden_element = self.driver.find_element_by_id('hintError-sstate_dir')
+ hidden_element = self.driver.find_element(By.ID, 'hintError-sstate_dir')
self.assertEqual(hidden_element.is_displayed(), False,
'sstate directory path valid but treated as invalid')
# paths starts with ${...}
- self.driver.find_element_by_id('new-sstate_dir').clear()
+ self.driver.find_element(By.ID, 'new-sstate_dir').clear()
self.enter_text('#new-sstate_dir', '${TOPDIR}/down')
- hidden_element = self.driver.find_element_by_id('hintError-sstate_dir')
+ hidden_element = self.driver.find_element(By.ID, 'hintError-sstate_dir')
self.assertEqual(hidden_element.is_displayed(), False,
'sstate directory path valid but treated as invalid')
diff --git a/poky/bitbake/lib/toaster/tests/browser/test_toastertable_ui.py b/poky/bitbake/lib/toaster/tests/browser/test_toastertable_ui.py
index e82d5ec..e00c30a 100644
--- a/poky/bitbake/lib/toaster/tests/browser/test_toastertable_ui.py
+++ b/poky/bitbake/lib/toaster/tests/browser/test_toastertable_ui.py
@@ -13,6 +13,7 @@
from django.utils import timezone
from tests.browser.selenium_helpers import SeleniumTestCase
from orm.models import BitbakeVersion, Release, Project, Build
+from selenium.webdriver.common.by import By
class TestToasterTableUI(SeleniumTestCase):
"""
@@ -33,7 +34,7 @@
table: WebElement for a ToasterTable
"""
selector = 'thead a.sorted'
- heading = table.find_element_by_css_selector(selector)
+ heading = table.find_element(By.CSS_SELECTOR, selector)
return heading.get_attribute('innerHTML').strip()
def _get_datetime_from_cell(self, row, selector):
@@ -45,7 +46,7 @@
selector: CSS selector to use to find the cell containing the date time
string
"""
- cell = row.find_element_by_css_selector(selector)
+ cell = row.find_element(By.CSS_SELECTOR, selector)
cell_text = cell.get_attribute('innerHTML').strip()
return datetime.strptime(cell_text, '%d/%m/%y %H:%M')
@@ -105,7 +106,7 @@
self.click('#checkbox-started_on')
# sort by started_on column
- links = table.find_elements_by_css_selector('th.started_on a')
+ links = table.find_elements(By.CSS_SELECTOR, 'th.started_on a')
for link in links:
if link.get_attribute('innerHTML').strip() == 'Started on':
link.click()
diff --git a/poky/bitbake/lib/toaster/tests/builds/test_core_image_min.py b/poky/bitbake/lib/toaster/tests/builds/test_core_image_min.py
index 44b6cbe..9cdaa15 100644
--- a/poky/bitbake/lib/toaster/tests/builds/test_core_image_min.py
+++ b/poky/bitbake/lib/toaster/tests/builds/test_core_image_min.py
@@ -26,6 +26,7 @@
def setUp(self):
self.completed_build = self.build("core-image-minimal")
+ self.built = self.target_already_built("core-image-minimal")
# Check if build name is unique - tc_id=795
def test_Build_Unique_Name(self):
diff --git a/poky/bitbake/lib/toaster/tests/functional/functional_helpers.py b/poky/bitbake/lib/toaster/tests/functional/functional_helpers.py
index 5c4ea71..c3191f6 100644
--- a/poky/bitbake/lib/toaster/tests/functional/functional_helpers.py
+++ b/poky/bitbake/lib/toaster/tests/functional/functional_helpers.py
@@ -16,6 +16,9 @@
from tests.browser.selenium_helpers_base import SeleniumTestCaseBase
from tests.builds.buildtest import load_build_environment
+from bldcontrol.models import BuildEnvironment
+from selenium.webdriver.common.by import By
+from selenium.common.exceptions import NoSuchElementException
logger = logging.getLogger("toaster")
@@ -30,6 +33,8 @@
raise RuntimeError("Please initialise django with the tests settings: " \
"DJANGO_SETTINGS_MODULE='toastermain.settings_test'")
+ if BuildEnvironment.objects.count() == 0:
+ BuildEnvironment.objects.create(betype=BuildEnvironment.TYPE_LOCAL)
load_build_environment()
# start toaster
@@ -74,8 +79,8 @@
"""
try:
table_element = self.get_table_element(table_id)
- element = table_element.find_element_by_link_text(link_text)
- except self.NoSuchElementException:
+ element = table_element.find_element(By.LINK_TEXT, link_text)
+ except NoSuchElementException:
print('no element found')
raise
return element
@@ -85,8 +90,8 @@
#return whole-table element
element_xpath = "//*[@id='" + table_id + "']"
try:
- element = self.driver.find_element_by_xpath(element_xpath)
- except self.NoSuchElementException:
+ element = self.driver.find_element(By.XPATH, element_xpath)
+ except NoSuchElementException:
raise
return element
row = coordinate[0]
@@ -95,8 +100,8 @@
#return whole-row element
element_xpath = "//*[@id='" + table_id + "']/tbody/tr[" + str(row) + "]"
try:
- element = self.driver.find_element_by_xpath(element_xpath)
- except self.NoSuchElementException:
+ element = self.driver.find_element(By.XPATH, element_xpath)
+ except NoSuchElementException:
return False
return element
#now we are looking for an element with specified X and Y
@@ -104,7 +109,7 @@
element_xpath = "//*[@id='" + table_id + "']/tbody/tr[" + str(row) + "]/td[" + str(column) + "]"
try:
- element = self.driver.find_element_by_xpath(element_xpath)
- except self.NoSuchElementException:
+ element = self.driver.find_element(By.XPATH, element_xpath)
+ except NoSuchElementException:
return False
return element
diff --git a/poky/bitbake/lib/toaster/tests/functional/test_functional_basic.py b/poky/bitbake/lib/toaster/tests/functional/test_functional_basic.py
index 5683e38..067ad99 100644
--- a/poky/bitbake/lib/toaster/tests/functional/test_functional_basic.py
+++ b/poky/bitbake/lib/toaster/tests/functional/test_functional_basic.py
@@ -10,6 +10,7 @@
import re
from tests.functional.functional_helpers import SeleniumFunctionalTestCase
from orm.models import Project
+from selenium.webdriver.common.by import By
class FuntionalTestBasic(SeleniumFunctionalTestCase):
@@ -17,10 +18,10 @@
def test_create_slenium_project(self):
project_name = 'selenium-project'
self.get('')
- self.driver.find_element_by_link_text("To start building, create your first Toaster project").click()
- self.driver.find_element_by_id("new-project-name").send_keys(project_name)
- self.driver.find_element_by_id('projectversion').click()
- self.driver.find_element_by_id("create-project-button").click()
+ self.driver.find_element(By.LINK_TEXT, "To start building, create your first Toaster project").click()
+ self.driver.find_element(By.ID, "new-project-name").send_keys(project_name)
+ self.driver.find_element(By.ID, 'projectversion').click()
+ self.driver.find_element(By.ID, "create-project-button").click()
element = self.wait_until_visible('#project-created-notification')
self.assertTrue(self.element_exists('#project-created-notification'),'Project creation notification not shown')
self.assertTrue(project_name in element.text,
@@ -35,77 +36,77 @@
self.find_element_by_link_text_in_table('projectstable', 'selenium-project').click()
self.assertTrue(self.element_exists('#config-nav'),'Configuration Tab does not exist')
project_URL=self.get_URL()
- self.driver.find_element_by_xpath('//a[@href="'+project_URL+'"]').click()
+ self.driver.find_element(By.XPATH, '//a[@href="'+project_URL+'"]').click()
try:
- self.driver.find_element_by_xpath("//*[@id='config-nav']/ul/li/a[@href="+'"'+project_URL+'customimages/"'+"]").click()
- self.assertTrue(re.search("Custom images",self.driver.find_element_by_xpath("//div[@class='col-md-10']").text),'Custom images information is not loading properly')
+ self.driver.find_element(By.XPATH, "//*[@id='config-nav']/ul/li/a[@href="+'"'+project_URL+'customimages/"'+"]").click()
+ self.assertTrue(re.search("Custom images",self.driver.find_element(By.XPATH, "//div[@class='col-md-10']").text),'Custom images information is not loading properly')
except:
self.fail(msg='No Custom images tab available')
try:
- self.driver.find_element_by_xpath("//*[@id='config-nav']/ul/li/a[@href="+'"'+project_URL+'images/"'+"]").click()
- self.assertTrue(re.search("Compatible image recipes",self.driver.find_element_by_xpath("//div[@class='col-md-10']").text),'The Compatible image recipes information is not loading properly')
+ self.driver.find_element(By.XPATH, "//*[@id='config-nav']/ul/li/a[@href="+'"'+project_URL+'images/"'+"]").click()
+ self.assertTrue(re.search("Compatible image recipes",self.driver.find_element(By.XPATH, "//div[@class='col-md-10']").text),'The Compatible image recipes information is not loading properly')
except:
self.fail(msg='No Compatible image tab available')
try:
- self.driver.find_element_by_xpath("//*[@id='config-nav']/ul/li/a[@href="+'"'+project_URL+'softwarerecipes/"'+"]").click()
- self.assertTrue(re.search("Compatible software recipes",self.driver.find_element_by_xpath("//div[@class='col-md-10']").text),'The Compatible software recipe information is not loading properly')
+ self.driver.find_element(By.XPATH, "//*[@id='config-nav']/ul/li/a[@href="+'"'+project_URL+'softwarerecipes/"'+"]").click()
+ self.assertTrue(re.search("Compatible software recipes",self.driver.find_element(By.XPATH, "//div[@class='col-md-10']").text),'The Compatible software recipe information is not loading properly')
except:
self.fail(msg='No Compatible software recipe tab available')
try:
- self.driver.find_element_by_xpath("//*[@id='config-nav']/ul/li/a[@href="+'"'+project_URL+'machines/"'+"]").click()
- self.assertTrue(re.search("Compatible machines",self.driver.find_element_by_xpath("//div[@class='col-md-10']").text),'The Compatible machine information is not loading properly')
+ self.driver.find_element(By.XPATH, "//*[@id='config-nav']/ul/li/a[@href="+'"'+project_URL+'machines/"'+"]").click()
+ self.assertTrue(re.search("Compatible machines",self.driver.find_element(By.XPATH, "//div[@class='col-md-10']").text),'The Compatible machine information is not loading properly')
except:
self.fail(msg='No Compatible machines tab available')
try:
- self.driver.find_element_by_xpath("//*[@id='config-nav']/ul/li/a[@href="+'"'+project_URL+'layers/"'+"]").click()
- self.assertTrue(re.search("Compatible layers",self.driver.find_element_by_xpath("//div[@class='col-md-10']").text),'The Compatible layer information is not loading properly')
+ self.driver.find_element(By.XPATH, "//*[@id='config-nav']/ul/li/a[@href="+'"'+project_URL+'layers/"'+"]").click()
+ self.assertTrue(re.search("Compatible layers",self.driver.find_element(By.XPATH, "//div[@class='col-md-10']").text),'The Compatible layer information is not loading properly')
except:
self.fail(msg='No Compatible layers tab available')
try:
- self.driver.find_element_by_xpath("//*[@id='config-nav']/ul/li/a[@href="+'"'+project_URL+'configuration"'+"]").click()
- self.assertTrue(re.search("Bitbake variables",self.driver.find_element_by_xpath("//div[@class='col-md-10']").text),'The Bitbake variables information is not loading properly')
+ self.driver.find_element(By.XPATH, "//*[@id='config-nav']/ul/li/a[@href="+'"'+project_URL+'configuration"'+"]").click()
+ self.assertTrue(re.search("Bitbake variables",self.driver.find_element(By.XPATH, "//div[@class='col-md-10']").text),'The Bitbake variables information is not loading properly')
except:
self.fail(msg='No Bitbake variables tab available')
# testcase (1516)
def test_review_configuration_information(self):
self.get('')
- self.driver.find_element_by_xpath("//div[@id='global-nav']/ul/li/a[@href="+'"'+'/toastergui/projects/'+'"'+"]").click()
+ self.driver.find_element(By.XPATH, "//div[@id='global-nav']/ul/li/a[@href="+'"'+'/toastergui/projects/'+'"'+"]").click()
self.wait_until_visible('#projectstable')
self.find_element_by_link_text_in_table('projectstable', 'selenium-project').click()
project_URL=self.get_URL()
try:
self.assertTrue(self.element_exists('#machine-section'),'Machine section for the project configuration page does not exist')
- self.assertTrue(re.search("qemux86",self.driver.find_element_by_xpath("//span[@id='project-machine-name']").text),'The machine type is not assigned')
- self.driver.find_element_by_xpath("//span[@id='change-machine-toggle']").click()
+ self.assertTrue(re.search("qemux86",self.driver.find_element(By.XPATH, "//span[@id='project-machine-name']").text),'The machine type is not assigned')
+ self.driver.find_element(By.XPATH, "//span[@id='change-machine-toggle']").click()
self.wait_until_visible('#select-machine-form')
self.wait_until_visible('#cancel-machine-change')
- self.driver.find_element_by_xpath("//form[@id='select-machine-form']/a[@id='cancel-machine-change']").click()
+ self.driver.find_element(By.XPATH, "//form[@id='select-machine-form']/a[@id='cancel-machine-change']").click()
except:
self.fail(msg='The machine information is wrong in the configuration page')
try:
- self.driver.find_element_by_id('no-most-built')
+ self.driver.find_element(By.ID, 'no-most-built')
except:
self.fail(msg='No Most built information in project detail page')
try:
- self.assertTrue(re.search("Yocto Project master",self.driver.find_element_by_xpath("//span[@id='project-release-title']").text),'The project release is not defined')
+ self.assertTrue(re.search("Yocto Project master",self.driver.find_element(By.XPATH, "//span[@id='project-release-title']").text),'The project release is not defined')
except:
self.fail(msg='No project release title information in project detail page')
try:
- self.driver.find_element_by_xpath("//div[@id='layer-container']")
- self.assertTrue(re.search("3",self.driver.find_element_by_id("project-layers-count").text),'There should be 3 layers listed in the layer count')
- layer_list = self.driver.find_element_by_id("layers-in-project-list")
- layers = layer_list.find_elements_by_tag_name("li")
+ self.driver.find_element(By.XPATH, "//div[@id='layer-container']")
+ self.assertTrue(re.search("3",self.driver.find_element(By.ID, "project-layers-count").text),'There should be 3 layers listed in the layer count')
+ layer_list = self.driver.find_element(By.ID, "layers-in-project-list")
+ layers = layer_list.find_elements(By.TAG_NAME, "li")
for layer in layers:
if re.match ("openembedded-core",layer.text):
print ("openembedded-core layer is a default layer in the project configuration")
@@ -121,60 +122,61 @@
# testcase (1517)
def test_verify_machine_information(self):
self.get('')
- self.driver.find_element_by_xpath("//div[@id='global-nav']/ul/li/a[@href="+'"'+'/toastergui/projects/'+'"'+"]").click()
+ self.driver.find_element(By.XPATH, "//div[@id='global-nav']/ul/li/a[@href="+'"'+'/toastergui/projects/'+'"'+"]").click()
self.wait_until_visible('#projectstable')
self.find_element_by_link_text_in_table('projectstable', 'selenium-project').click()
try:
self.assertTrue(self.element_exists('#machine-section'),'Machine section for the project configuration page does not exist')
- self.assertTrue(re.search("qemux86",self.driver.find_element_by_id("project-machine-name").text),'The machine type is not assigned')
- self.driver.find_element_by_id("change-machine-toggle").click()
+ self.assertTrue(re.search("qemux86",self.driver.find_element(By.ID, "project-machine-name").text),'The machine type is not assigned')
+ self.driver.find_element(By.ID, "change-machine-toggle").click()
self.wait_until_visible('#select-machine-form')
self.wait_until_visible('#cancel-machine-change')
- self.driver.find_element_by_id("cancel-machine-change").click()
+ self.driver.find_element(By.ID, "cancel-machine-change").click()
except:
self.fail(msg='The machine information is wrong in the configuration page')
# testcase (1518)
def test_verify_most_built_recipes_information(self):
self.get('')
- self.driver.find_element_by_xpath("//div[@id='global-nav']/ul/li/a[@href="+'"'+'/toastergui/projects/'+'"'+"]").click()
+ self.driver.find_element(By.XPATH, "//div[@id='global-nav']/ul/li/a[@href="+'"'+'/toastergui/projects/'+'"'+"]").click()
+
self.wait_until_visible('#projectstable')
self.find_element_by_link_text_in_table('projectstable', 'selenium-project').click()
project_URL=self.get_URL()
try:
- self.assertTrue(re.search("You haven't built any recipes yet",self.driver.find_element_by_id("no-most-built").text),'Default message of no builds is not present')
- self.driver.find_element_by_xpath("//div[@id='no-most-built']/p/a[@href="+'"'+project_URL+'images/"'+"]").click()
- self.assertTrue(re.search("Compatible image recipes",self.driver.find_element_by_xpath("//div[@class='col-md-10']").text),'The Choose a recipe to build link is not working properly')
+ self.assertTrue(re.search("You haven't built any recipes yet",self.driver.find_element(By.ID, "no-most-built").text),'Default message of no builds is not present')
+ self.driver.find_element(By.XPATH, "//div[@id='no-most-built']/p/a[@href="+'"'+project_URL+'images/"'+"]").click()
+ self.assertTrue(re.search("Compatible image recipes",self.driver.find_element(By.XPATH, "//div[@class='col-md-10']").text),'The Choose a recipe to build link is not working properly')
except:
self.fail(msg='No Most built information in project detail page')
# testcase (1519)
def test_verify_project_release_information(self):
self.get('')
- self.driver.find_element_by_xpath("//div[@id='global-nav']/ul/li/a[@href="+'"'+'/toastergui/projects/'+'"'+"]").click()
+ self.driver.find_element(By.XPATH, "//div[@id='global-nav']/ul/li/a[@href="+'"'+'/toastergui/projects/'+'"'+"]").click()
self.wait_until_visible('#projectstable')
self.find_element_by_link_text_in_table('projectstable', 'selenium-project').click()
try:
- self.assertTrue(re.search("Yocto Project master",self.driver.find_element_by_id("project-release-title").text),'The project release is not defined')
+ self.assertTrue(re.search("Yocto Project master",self.driver.find_element(By.ID, "project-release-title").text),'The project release is not defined')
except:
self.fail(msg='No project release title information in project detail page')
# testcase (1520)
def test_verify_layer_information(self):
self.get('')
- self.driver.find_element_by_xpath("//div[@id='global-nav']/ul/li/a[@href="+'"'+'/toastergui/projects/'+'"'+"]").click()
+ self.driver.find_element(By.XPATH, "//div[@id='global-nav']/ul/li/a[@href="+'"'+'/toastergui/projects/'+'"'+"]").click()
self.wait_until_visible('#projectstable')
self.find_element_by_link_text_in_table('projectstable', 'selenium-project').click()
project_URL=self.get_URL()
try:
- self.driver.find_element_by_xpath("//div[@id='layer-container']")
- self.assertTrue(re.search("3",self.driver.find_element_by_id("project-layers-count").text),'There should be 3 layers listed in the layer count')
- layer_list = self.driver.find_element_by_id("layers-in-project-list")
- layers = layer_list.find_elements_by_tag_name("li")
+ self.driver.find_element(By.XPATH, "//div[@id='layer-container']")
+ self.assertTrue(re.search("3",self.driver.find_element(By.ID, "project-layers-count").text),'There should be 3 layers listed in the layer count')
+ layer_list = self.driver.find_element(By.ID, "layers-in-project-list")
+ layers = layer_list.find_element(By.TAG_NAME, "li")
for layer in layers:
if re.match ("openembedded-core",layer.text):
@@ -186,43 +188,43 @@
else:
self.fail(msg='default layers are missing from the project configuration')
- self.driver.find_element_by_xpath("//input[@id='layer-add-input']")
- self.driver.find_element_by_xpath("//button[@id='add-layer-btn']")
- self.driver.find_element_by_xpath("//div[@id='layer-container']/form[@class='form-inline']/p/a[@id='view-compatible-layers']")
- self.driver.find_element_by_xpath("//div[@id='layer-container']/form[@class='form-inline']/p/a[@href="+'"'+project_URL+'importlayer"'+"]")
+ self.driver.find_element(By.XPATH, "//input[@id='layer-add-input']")
+ self.driver.find_element(By.XPATH, "//button[@id='add-layer-btn']")
+ self.driver.find_element(By.XPATH, "//div[@id='layer-container']/form[@class='form-inline']/p/a[@id='view-compatible-layers']")
+ self.driver.find_element(By.XPATH, "//div[@id='layer-container']/form[@class='form-inline']/p/a[@href="+'"'+project_URL+'importlayer"'+"]")
except:
self.fail(msg='No Layer information in project detail page')
# testcase (1521)
def test_verify_project_detail_links(self):
self.get('')
- self.driver.find_element_by_xpath("//div[@id='global-nav']/ul/li/a[@href="+'"'+'/toastergui/projects/'+'"'+"]").click()
+ self.driver.find_element(By.XPATH, "//div[@id='global-nav']/ul/li/a[@href="+'"'+'/toastergui/projects/'+'"'+"]").click()
self.wait_until_visible('#projectstable')
self.find_element_by_link_text_in_table('projectstable', 'selenium-project').click()
project_URL=self.get_URL()
- self.driver.find_element_by_xpath("//div[@id='project-topbar']/ul[@class='nav nav-tabs']/li[@id='topbar-configuration-tab']/a[@href="+'"'+project_URL+'"'+"]").click()
- self.assertTrue(re.search("Configuration",self.driver.find_element_by_xpath("//div[@id='project-topbar']/ul[@class='nav nav-tabs']/li[@id='topbar-configuration-tab']/a[@href="+'"'+project_URL+'"'+"]").text), 'Configuration tab in project topbar is misspelled')
+ self.driver.find_element(By.XPATH, "//div[@id='project-topbar']/ul[@class='nav nav-tabs']/li[@id='topbar-configuration-tab']/a[@href="+'"'+project_URL+'"'+"]").click()
+ self.assertTrue(re.search("Configuration",self.driver.find_element(By.XPATH, "//div[@id='project-topbar']/ul[@class='nav nav-tabs']/li[@id='topbar-configuration-tab']/a[@href="+'"'+project_URL+'"'+"]").text), 'Configuration tab in project topbar is misspelled')
try:
- self.driver.find_element_by_xpath("//div[@id='project-topbar']/ul[@class='nav nav-tabs']/li/a[@href="+'"'+project_URL+'builds/"'+"]").click()
- self.assertTrue(re.search("Builds",self.driver.find_element_by_xpath("//div[@id='project-topbar']/ul[@class='nav nav-tabs']/li/a[@href="+'"'+project_URL+'builds/"'+"]").text), 'Builds tab in project topbar is misspelled')
- self.driver.find_element_by_xpath("//div[@id='empty-state-projectbuildstable']")
+ self.driver.find_element(By.XPATH, "//div[@id='project-topbar']/ul[@class='nav nav-tabs']/li/a[@href="+'"'+project_URL+'builds/"'+"]").click()
+ self.assertTrue(re.search("Builds",self.driver.find_element(By.XPATH, "//div[@id='project-topbar']/ul[@class='nav nav-tabs']/li/a[@href="+'"'+project_URL+'builds/"'+"]").text), 'Builds tab in project topbar is misspelled')
+ self.driver.find_element(By.XPATH, "//div[@id='empty-state-projectbuildstable']")
except:
self.fail(msg='Builds tab information is not present')
try:
- self.driver.find_element_by_xpath("//div[@id='project-topbar']/ul[@class='nav nav-tabs']/li/a[@href="+'"'+project_URL+'importlayer"'+"]").click()
- self.assertTrue(re.search("Import layer",self.driver.find_element_by_xpath("//div[@id='project-topbar']/ul[@class='nav nav-tabs']/li/a[@href="+'"'+project_URL+'importlayer"'+"]").text), 'Import layer tab in project topbar is misspelled')
- self.driver.find_element_by_xpath("//fieldset[@id='repo-select']")
- self.driver.find_element_by_xpath("//fieldset[@id='git-repo']")
+ self.driver.find_element(By.XPATH, "//div[@id='project-topbar']/ul[@class='nav nav-tabs']/li/a[@href="+'"'+project_URL+'importlayer"'+"]").click()
+ self.assertTrue(re.search("Import layer",self.driver.find_element(By.XPATH, "//div[@id='project-topbar']/ul[@class='nav nav-tabs']/li/a[@href="+'"'+project_URL+'importlayer"'+"]").text), 'Import layer tab in project topbar is misspelled')
+ self.driver.find_element(By.XPATH, "//fieldset[@id='repo-select']")
+ self.driver.find_element(By.XPATH, "//fieldset[@id='git-repo']")
except:
self.fail(msg='Import layer tab not loading properly')
try:
- self.driver.find_element_by_xpath("//div[@id='project-topbar']/ul[@class='nav nav-tabs']/li/a[@href="+'"'+project_URL+'newcustomimage/"'+"]").click()
- self.assertTrue(re.search("New custom image",self.driver.find_element_by_xpath("//div[@id='project-topbar']/ul[@class='nav nav-tabs']/li/a[@href="+'"'+project_URL+'newcustomimage/"'+"]").text), 'New custom image tab in project topbar is misspelled')
- self.assertTrue(re.search("Select the image recipe you want to customise",self.driver.find_element_by_xpath("//div[@class='col-md-12']/h2").text),'The new custom image tab is not loading correctly')
+ self.driver.find_element(By.XPATH, "//div[@id='project-topbar']/ul[@class='nav nav-tabs']/li/a[@href="+'"'+project_URL+'newcustomimage/"'+"]").click()
+ self.assertTrue(re.search("New custom image",self.driver.find_element(By.XPATH, "//div[@id='project-topbar']/ul[@class='nav nav-tabs']/li/a[@href="+'"'+project_URL+'newcustomimage/"'+"]").text), 'New custom image tab in project topbar is misspelled')
+ self.assertTrue(re.search("Select the image recipe you want to customise",self.driver.find_element(By.XPATH, "//div[@class='col-md-12']/h2").text),'The new custom image tab is not loading correctly')
except:
self.fail(msg='New custom image tab not loading properly')
diff --git a/poky/bitbake/lib/toaster/tests/toaster-tests-requirements.txt b/poky/bitbake/lib/toaster/tests/toaster-tests-requirements.txt
index 4f9fcc4..f30ac07 100644
--- a/poky/bitbake/lib/toaster/tests/toaster-tests-requirements.txt
+++ b/poky/bitbake/lib/toaster/tests/toaster-tests-requirements.txt
@@ -1 +1 @@
-selenium==2.49.2
+selenium>=4.13.0
diff --git a/poky/bitbake/lib/toaster/tests/views/test_views.py b/poky/bitbake/lib/toaster/tests/views/test_views.py
index 735d596..f962e76 100644
--- a/poky/bitbake/lib/toaster/tests/views/test_views.py
+++ b/poky/bitbake/lib/toaster/tests/views/test_views.py
@@ -19,6 +19,7 @@
from orm.models import CustomImageRecipe
from orm.models import CustomImagePackage
+from bldcontrol.models import BuildEnvironment
import inspect
import toastergui
@@ -45,6 +46,9 @@
self.cust_package = CustomImagePackage.objects.first()
self.package = Package.objects.first()
self.lver = Layer_Version.objects.first()
+ if BuildEnvironment.objects.count() == 0:
+ BuildEnvironment.objects.create(betype=BuildEnvironment.TYPE_LOCAL)
+
def test_get_base_call_returns_html(self):
"""Basic test for all-projects view"""
diff --git a/poky/bitbake/lib/toaster/toastergui/views.py b/poky/bitbake/lib/toaster/toastergui/views.py
index 552ff16..cc8517b 100644
--- a/poky/bitbake/lib/toaster/toastergui/views.py
+++ b/poky/bitbake/lib/toaster/toastergui/views.py
@@ -34,6 +34,8 @@
import logging
+from toastermain.logs import log_view_mixin
+
logger = logging.getLogger("toaster")
# Project creation and managed build enable
@@ -56,6 +58,7 @@
return guessed_type
# single point to add global values into the context before rendering
+@log_view_mixin
def toaster_render(request, page, context):
context['project_enable'] = project_enable
context['project_specific'] = is_project_specific
@@ -665,6 +668,7 @@
return response
from django.http import HttpResponse
+@log_view_mixin
def xhr_dirinfo(request, build_id, target_id):
top = request.GET.get('start', '/')
return HttpResponse(_get_dir_entries(build_id, target_id, top), content_type = "application/json")
@@ -1612,6 +1616,7 @@
from django.views.decorators.csrf import csrf_exempt
@csrf_exempt
+ @log_view_mixin
def xhr_testreleasechange(request, pid):
def response(data):
return HttpResponse(jsonfilter(data),
@@ -1648,6 +1653,7 @@
except Exception as e:
return response({"error": str(e) })
+ @log_view_mixin
def xhr_configvaredit(request, pid):
try:
prj = Project.objects.get(id = pid)
@@ -1726,6 +1732,7 @@
return HttpResponse(json.dumps({"error":str(e) + "\n" + traceback.format_exc()}), content_type = "application/json")
+ @log_view_mixin
def customrecipe_download(request, pid, recipe_id):
recipe = get_object_or_404(CustomImageRecipe, pk=recipe_id)
diff --git a/poky/bitbake/lib/toaster/toastergui/widgets.py b/poky/bitbake/lib/toaster/toastergui/widgets.py
index 5369691..b32abf4 100644
--- a/poky/bitbake/lib/toaster/toastergui/widgets.py
+++ b/poky/bitbake/lib/toaster/toastergui/widgets.py
@@ -32,6 +32,7 @@
import os
from toastergui.tablefilter import TableFilterMap
+from toastermain.logs import log_view_mixin
try:
from urllib import unquote_plus
@@ -84,6 +85,7 @@
return context
+ @log_view_mixin
def get(self, request, *args, **kwargs):
if request.GET.get('format', None) == 'json':
@@ -305,6 +307,7 @@
self.setup_columns(**kwargs)
+ self.apply_orderby('pk')
if search:
self.apply_search(search)
if filters:
@@ -414,6 +417,7 @@
def __init__(self, *args, **kwargs):
super(ToasterTypeAhead, self).__init__()
+ @log_view_mixin
def get(self, request, *args, **kwargs):
def response(data):
return HttpResponse(json.dumps(data,
@@ -469,6 +473,7 @@
return False
+ @log_view_mixin
def get(self, request, *args, **kwargs):
"""
Returns a list of builds in JSON format.
diff --git a/poky/bitbake/lib/toaster/toastermain/logs.py b/poky/bitbake/lib/toaster/toastermain/logs.py
new file mode 100644
index 0000000..f995398
--- /dev/null
+++ b/poky/bitbake/lib/toaster/toastermain/logs.py
@@ -0,0 +1,153 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+import logging
+import json
+from pathlib import Path
+from django.http import HttpRequest
+
+BASE_DIR = Path(__file__).resolve(strict=True).parent.parent
+
+
+def log_api_request(request, response, view, logger_name='api'):
+ """Helper function for LogAPIMixin"""
+
+ repjson = {
+ 'view': view,
+ 'path': request.path,
+ 'method': request.method,
+ 'status': response.status_code
+ }
+
+ logger = logging.getLogger(logger_name)
+ logger.info(
+ json.dumps(repjson, indent=4, separators=(", ", " : "))
+ )
+
+
+def log_view_mixin(view):
+ def log_view_request(*args, **kwargs):
+ # get request from args else kwargs
+ request = None
+ if len(args) > 0:
+ for req in args:
+ if isinstance(req, HttpRequest):
+ request = req
+ break
+ elif request is None:
+ request = kwargs.get('request')
+
+ response = view(*args, **kwargs)
+ log_api_request(
+ request, response, request.resolver_match.view_name, 'toaster')
+ return response
+ return log_view_request
+
+
+
+class LogAPIMixin:
+ """Logs API requests
+
+ tested with:
+ - APIView
+ - ModelViewSet
+ - ReadOnlyModelViewSet
+ - GenericAPIView
+
+ Note: you can set `view_name` attribute in View to override get_view_name()
+ """
+
+ def get_view_name(self):
+ if hasattr(self, 'view_name'):
+ return self.view_name
+ return super().get_view_name()
+
+ def finalize_response(self, request, response, *args, **kwargs):
+ log_api_request(request, response, self.get_view_name())
+ return super().finalize_response(request, response, *args, **kwargs)
+
+
+LOGGING_SETTINGS = {
+ 'version': 1,
+ 'disable_existing_loggers': False,
+ 'filters': {
+ 'require_debug_false': {
+ '()': 'django.utils.log.RequireDebugFalse'
+ }
+ },
+ 'formatters': {
+ 'datetime': {
+ 'format': '%(asctime)s %(levelname)s %(message)s'
+ },
+ 'verbose': {
+ 'format': '{levelname} {asctime} {module} {name}.{funcName} {process:d} {thread:d} {message}',
+ 'datefmt': "%d/%b/%Y %H:%M:%S",
+ 'style': '{',
+ },
+ 'api': {
+ 'format': '\n{levelname} {asctime} {name}.{funcName}:\n{message}',
+ 'style': '{'
+ }
+ },
+ 'handlers': {
+ 'mail_admins': {
+ 'level': 'ERROR',
+ 'filters': ['require_debug_false'],
+ 'class': 'django.utils.log.AdminEmailHandler'
+ },
+ 'console': {
+ 'level': 'DEBUG',
+ 'class': 'logging.StreamHandler',
+ 'formatter': 'datetime',
+ },
+ 'file_django': {
+ 'level': 'INFO',
+ 'class': 'logging.handlers.TimedRotatingFileHandler',
+ 'filename': BASE_DIR / 'logs/django.log',
+ 'when': 'D', # interval type
+ 'interval': 1, # defaults to 1
+ 'backupCount': 10, # how many files to keep
+ 'formatter': 'verbose',
+ },
+ 'file_api': {
+ 'level': 'INFO',
+ 'class': 'logging.handlers.TimedRotatingFileHandler',
+ 'filename': BASE_DIR / 'logs/api.log',
+ 'when': 'D',
+ 'interval': 1,
+ 'backupCount': 10,
+ 'formatter': 'verbose',
+ },
+ 'file_toaster': {
+ 'level': 'INFO',
+ 'class': 'logging.handlers.TimedRotatingFileHandler',
+ 'filename': BASE_DIR / 'logs/toaster.log',
+ 'when': 'D',
+ 'interval': 1,
+ 'backupCount': 10,
+ 'formatter': 'verbose',
+ },
+ },
+ 'loggers': {
+ 'django.request': {
+ 'handlers': ['file_django', 'console'],
+ 'level': 'WARN',
+ 'propagate': True,
+ },
+ 'django': {
+ 'handlers': ['file_django', 'console'],
+ 'level': 'WARNING',
+ 'propogate': True,
+ },
+ 'toaster': {
+ 'handlers': ['file_toaster'],
+ 'level': 'INFO',
+ 'propagate': False,
+ },
+ 'api': {
+ 'handlers': ['file_api'],
+ 'level': 'INFO',
+ 'propagate': False,
+ }
+ }
+}
diff --git a/poky/bitbake/lib/toaster/toastermain/settings.py b/poky/bitbake/lib/toaster/toastermain/settings.py
index 609c85d..b083cf5 100644
--- a/poky/bitbake/lib/toaster/toastermain/settings.py
+++ b/poky/bitbake/lib/toaster/toastermain/settings.py
@@ -9,6 +9,8 @@
# Django settings for Toaster project.
import os
+from pathlib import Path
+from toastermain.logs import LOGGING_SETTINGS
DEBUG = True
@@ -186,7 +188,13 @@
'django.template.loaders.app_directories.Loader',
#'django.template.loaders.eggs.Loader',
],
- 'string_if_invalid': InvalidString("%s"),
+ # https://docs.djangoproject.com/en/4.2/ref/templates/api/#how-invalid-variables-are-handled
+ # Generally, string_if_invalid should only be enabled in order to debug
+ # a specific template problem, then cleared once debugging is complete.
+ # If you assign a value other than '' to string_if_invalid,
+ # you will experience rendering problems with these templates and sites.
+ # 'string_if_invalid': InvalidString("%s"),
+ 'string_if_invalid': "",
'debug': DEBUG,
},
},
@@ -242,6 +250,9 @@
'django.contrib.humanize',
'bldcollector',
'toastermain',
+
+ # 3rd-lib
+ "log_viewer",
)
@@ -302,43 +313,22 @@
# the site admins on every HTTP 500 error when DEBUG=False.
# See http://docs.djangoproject.com/en/dev/topics/logging for
# more details on how to customize your logging configuration.
-LOGGING = {
- 'version': 1,
- 'disable_existing_loggers': False,
- 'filters': {
- 'require_debug_false': {
- '()': 'django.utils.log.RequireDebugFalse'
- }
- },
- 'formatters': {
- 'datetime': {
- 'format': '%(asctime)s %(levelname)s %(message)s'
- }
- },
- 'handlers': {
- 'mail_admins': {
- 'level': 'ERROR',
- 'filters': ['require_debug_false'],
- 'class': 'django.utils.log.AdminEmailHandler'
- },
- 'console': {
- 'level': 'DEBUG',
- 'class': 'logging.StreamHandler',
- 'formatter': 'datetime',
- }
- },
- 'loggers': {
- 'toaster' : {
- 'handlers': ['console'],
- 'level': 'DEBUG',
- },
- 'django.request': {
- 'handlers': ['console'],
- 'level': 'WARN',
- 'propagate': True,
- },
- }
-}
+LOGGING = LOGGING_SETTINGS
+
+# Build paths inside the project like this: BASE_DIR / 'subdir'.
+BASE_DIR = Path(__file__).resolve(strict=True).parent.parent
+
+# LOG VIEWER
+# https://pypi.org/project/django-log-viewer/
+LOG_VIEWER_FILES_PATTERN = '*.log*'
+LOG_VIEWER_FILES_DIR = os.path.join(BASE_DIR, 'logs')
+LOG_VIEWER_PAGE_LENGTH = 25 # total log lines per-page
+LOG_VIEWER_MAX_READ_LINES = 100000 # total log lines will be read
+LOG_VIEWER_PATTERNS = ['INFO', 'DEBUG', 'WARNING', 'ERROR', 'CRITICAL']
+
+# Optionally you can set the next variables in order to customize the admin:
+LOG_VIEWER_FILE_LIST_TITLE = "Logs list"
+
if DEBUG and SQL_DEBUG:
LOGGING['loggers']['django.db.backends'] = {
diff --git a/poky/bitbake/lib/toaster/toastermain/urls.py b/poky/bitbake/lib/toaster/toastermain/urls.py
index 0360302..3be46fc 100644
--- a/poky/bitbake/lib/toaster/toastermain/urls.py
+++ b/poky/bitbake/lib/toaster/toastermain/urls.py
@@ -28,6 +28,8 @@
# url(r'^admin/doc/', include('django.contrib.admindocs.urls')),
+ url(r'^logs/', include('log_viewer.urls')),
+
# This is here to maintain backward compatibility and will be deprecated
# in the future.
url(r'^orm/eventfile$', bldcollector.views.eventfile),