Add expectedJsonChecker tool
Tool to cross check an expected set of JSON entries against an input set
of JSON entries. Optionally, the applying of filters to entries within
the expected set using logical operations is allowed against another set
of entries within a JSON file.
Traversing/cross-checking JSON dictionary objects are supported, lists
are not supported yet.
Change-Id: I9537fd8d94316b90e301a8f671ed9d1c3ab2dc0e
Signed-off-by: Matthew Barth <msbarth@us.ibm.com>
diff --git a/msbarth/LICENSE b/msbarth/LICENSE
new file mode 100644
index 0000000..5f62e17
--- /dev/null
+++ b/msbarth/LICENSE
@@ -0,0 +1,14 @@
+Copyright 2019 IBM Corporation
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
diff --git a/msbarth/README.md b/msbarth/README.md
new file mode 100644
index 0000000..a9c0e80
--- /dev/null
+++ b/msbarth/README.md
@@ -0,0 +1,331 @@
+# Expected JSON Checker tool
+
+A tool that cross checks an expected set of JSON entries, by a given index,
+with a given input set of JSON entries. In addition, there's an ability to
+filter expected entries by using logical operations against entries within
+another file contain JSON entries. This filtering functionality on cross
+checking expected entries with a set of input is optional
+
+Expected entries that only want to be found within the input JSON should use
+a value of `{}`. This denotes the value for the given property should be
+treated as a `don't care` and ignored during the cross-check.
+
+## Intention:
+
+The intention for this tool's creation was to provide an ability to cross check
+entries within the BMC's enumerated sensor JSON output against an expected set
+of entries for a specific machine. In addition, the expected set of entries are
+different based on specific entries within inventory. So given a dump of a
+machine's enumerated sensor data and inventory data to separate JSON files,
+an expected set of entries that should be contained within the sensor data
+is created for this machine. These JSON files are then fed into this tool to
+determine if all the expected set of entries are found within the sensor data.
+The machine's name was used as the index into the expected JSON to allow the
+same expected JSON file to be used across multiple machines instead of having
+separate expected JSON files per machine(since what's expected will likely be
+different across different machines).
+
+## (OPTIONAL) Filtering:
+
+Filters can be used for whether or not a set of entries within the expected
+JSON file should be included when cross checking with the input set of JSON
+entries. This is useful in allowing a single set of expected entries to be
+used and add/remove entries based on some other JSON file's contents.
+
+### Supported logical operations:
+
+- \$and : Performs an AND operation on an array with at least two
+ expressions and returns the document that meets all the
+ expressions. i.e.) {"$and": [{"age": 5}, {"name": "Joe"}]}
+- \$or : Performs an OR operation on an array with at least two
+ expressions and returns the documents that meet at least one of
+ the expressions. i.e.) {"$or": [{"age": 4}, {"name": "Joe"}]}
+- \$nor : Performs a NOR operation on an array with at least two
+ expressions and returns the documents that do not meet any of
+ the expressions. i.e.) {"$nor": [{"age": 3}, {"name": "Moe"}]}
+- \$not : Performs a NOT operation on the specified expression and
+ returns the documents that do not meet the expression.
+ i.e.) {"$not": {"age": 4}}
+
+## Example Usage:
+
+Expected JSON set of entries for a `witherspoon` index(expected.json):
+```
+{
+ "witherspoon": {
+ "/xyz/openbmc_project/sensors/fan_tach/fan0_0": {
+ "Functional": true,
+ "Target": {},
+ "Value": {}
+ },
+ "/xyz/openbmc_project/sensors/fan_tach/fan0_1": {
+ "Functional": true,
+ "Value": {}
+ },
+ "$op": {
+ "$and": [{
+ "/xyz/openbmc_project/inventory/system/chassis":{"WaterCooled": false}
+ }],
+ "$input": [{
+ "/xyz/openbmc_project/sensors/fan_tach/fan1_0": {
+ "Functional": true,
+ "Target": {},
+ "Value": {}
+ },
+ "/xyz/openbmc_project/sensors/fan_tach/fan1_1": {
+ "Functional": true,
+ "Value": {}
+ }
+ }]
+ }
+ }
+}
+```
+
+Input JSON set of entries(input.json):
+```
+{
+ "data": {
+ "/xyz/openbmc_project/sensors/fan_tach/fan0_0": {
+ "CriticalAlarmHigh": false,
+ "CriticalAlarmLow": true,
+ "CriticalHigh": 12076,
+ "CriticalLow": 2974,
+ "Functional": true,
+ "MaxValue": 0,
+ "MinValue": 0,
+ "Scale": 0,
+ "Target": 10500,
+ "Unit": "xyz.openbmc_project.Sensor.Value.Unit.RPMS",
+ "Value": 0
+ },
+ "/xyz/openbmc_project/sensors/fan_tach/fan0_0/chassis": {
+ "endpoints": [
+ "/xyz/openbmc_project/inventory/system/chassis"
+ ]
+ },
+ "/xyz/openbmc_project/sensors/fan_tach/fan0_0/inventory": {
+ "endpoints": [
+ "/xyz/openbmc_project/inventory/system/chassis/motherboard/fan0"
+ ]
+ },
+ "/xyz/openbmc_project/sensors/fan_tach/fan0_1": {
+ "CriticalAlarmHigh": false,
+ "CriticalAlarmLow": false,
+ "CriticalHigh": 12076,
+ "CriticalLow": 2974,
+ "Functional": true,
+ "MaxValue": 0,
+ "MinValue": 0,
+ "Scale": 0,
+ "Unit": "xyz.openbmc_project.Sensor.Value.Unit.RPMS",
+ "Value": 3393
+ },
+ "/xyz/openbmc_project/sensors/fan_tach/fan0_1/chassis": {
+ "endpoints": [
+ "/xyz/openbmc_project/inventory/system/chassis"
+ ]
+ },
+ "/xyz/openbmc_project/sensors/fan_tach/fan0_1/inventory": {
+ "endpoints": [
+ "/xyz/openbmc_project/inventory/system/chassis/motherboard/fan0"
+ ]
+ },
+ "/xyz/openbmc_project/sensors/fan_tach/fan1_0": {
+ "CriticalAlarmHigh": false,
+ "CriticalAlarmLow": true,
+ "CriticalHigh": 12076,
+ "CriticalLow": 2974,
+ "Functional": true,
+ "MaxValue": 0,
+ "MinValue": 0,
+ "Scale": 0,
+ "Target": 10500,
+ "Unit": "xyz.openbmc_project.Sensor.Value.Unit.RPMS",
+ "Value": 0
+ },
+ "/xyz/openbmc_project/sensors/fan_tach/fan1_0/chassis": {
+ "endpoints": [
+ "/xyz/openbmc_project/inventory/system/chassis"
+ ]
+ },
+ "/xyz/openbmc_project/sensors/fan_tach/fan1_0/inventory": {
+ "endpoints": [
+ "/xyz/openbmc_project/inventory/system/chassis/motherboard/fan1"
+ ]
+ },
+ "/xyz/openbmc_project/sensors/fan_tach/fan1_1": {
+ "CriticalAlarmHigh": false,
+ "CriticalAlarmLow": false,
+ "CriticalHigh": 12076,
+ "CriticalLow": 2974,
+ "Functional": true,
+ "MaxValue": 0,
+ "MinValue": 0,
+ "Scale": 0,
+ "Unit": "xyz.openbmc_project.Sensor.Value.Unit.RPMS",
+ "Value": 3409
+ },
+ "/xyz/openbmc_project/sensors/fan_tach/fan1_1/chassis": {
+ "endpoints": [
+ "/xyz/openbmc_project/inventory/system/chassis"
+ ]
+ },
+ "/xyz/openbmc_project/sensors/fan_tach/fan1_1/inventory": {
+ "endpoints": [
+ "/xyz/openbmc_project/inventory/system/chassis/motherboard/fan1"
+ ]
+ },
+ "/xyz/openbmc_project/sensors/power/ps0_input_power": {
+ "CriticalAlarmHigh": false,
+ "CriticalAlarmLow": true,
+ "CriticalHigh": 2500000000,
+ "CriticalLow": 0,
+ "Functional": true,
+ "MaxValue": 0,
+ "MinValue": 0,
+ "Scale": -6,
+ "Unit": "xyz.openbmc_project.Sensor.Value.Unit.Watts",
+ "Value": 0,
+ "WarningAlarmHigh": false,
+ "WarningAlarmLow": true,
+ "WarningHigh": 2350000000,
+ "WarningLow": 0
+ },
+ "/xyz/openbmc_project/sensors/power/ps0_input_power/chassis": {
+ "endpoints": [
+ "/xyz/openbmc_project/inventory/system/chassis"
+ ]
+ },
+ "/xyz/openbmc_project/sensors/power/ps0_input_power/inventory": {
+ "endpoints": [
+ "/xyz/openbmc_project/inventory/system/chassis/motherboard/powersupply0"
+ ]
+ },
+ "/xyz/openbmc_project/sensors/power/ps1_input_power": {
+ "CriticalAlarmHigh": false,
+ "CriticalAlarmLow": false,
+ "CriticalHigh": 2500000000,
+ "CriticalLow": 0,
+ "Functional": true,
+ "MaxValue": 0,
+ "MinValue": 0,
+ "Scale": -6,
+ "Unit": "xyz.openbmc_project.Sensor.Value.Unit.Watts",
+ "Value": 18000000,
+ "WarningAlarmHigh": false,
+ "WarningAlarmLow": false,
+ "WarningHigh": 2350000000,
+ "WarningLow": 0
+ },
+ "/xyz/openbmc_project/sensors/power/ps1_input_power/chassis": {
+ "endpoints": [
+ "/xyz/openbmc_project/inventory/system/chassis"
+ ]
+ },
+ "/xyz/openbmc_project/sensors/power/ps1_input_power/inventory": {
+ "endpoints": [
+ "/xyz/openbmc_project/inventory/system/chassis/motherboard/powersupply1"
+ ]
+ },
+ "/xyz/openbmc_project/sensors/temperature/ambient": {
+ "CriticalAlarmHigh": false,
+ "CriticalAlarmLow": false,
+ "CriticalHigh": 35000,
+ "CriticalLow": 0,
+ "Functional": true,
+ "MaxValue": 0,
+ "MinValue": 0,
+ "Scale": -3,
+ "Unit": "xyz.openbmc_project.Sensor.Value.Unit.DegreesC",
+ "Value": 22420,
+ "WarningAlarmHigh": false,
+ "WarningAlarmLow": false,
+ "WarningHigh": 25000,
+ "WarningLow": 0
+ }
+ },
+ "message": "200 OK",
+ "status": "ok"
+}
+```
+
+Filter JSON set of entries(filter.json):
+```
+{
+ "data": {
+ "/xyz/openbmc_project/inventory/system": {
+ "AssetTag": "",
+ "BuildDate": "",
+ "Cached": false,
+ "FieldReplaceable": false,
+ "Manufacturer": "",
+ "Model": "8335-GTA ",
+ "PartNumber": "",
+ "Present": true,
+ "PrettyName": "",
+ "SerialNumber": "1234567 "
+ },
+ "/xyz/openbmc_project/inventory/system/chassis": {
+ "AirCooled": true,
+ "Type": "RackMount",
+ "WaterCooled": false
+ },
+ "/xyz/openbmc_project/inventory/system/chassis/activation": {
+ "endpoints": [
+ "/xyz/openbmc_project/software/224cd310"
+ ]
+ },
+ "/xyz/openbmc_project/inventory/system/chassis/motherboard/cpu0": {
+ "BuildDate": "1996-01-01 - 00:00:00",
+ "Cached": false,
+ "FieldReplaceable": true,
+ "Functional": true,
+ "Manufacturer": "IBM",
+ "Model": "",
+ "PartNumber": "02CY211",
+ "Present": true,
+ "PrettyName": "PROCESSOR MODULE",
+ "SerialNumber": "YA1934302447",
+ "Version": "22"
+ },
+ "/xyz/openbmc_project/inventory/system/chassis/motherboard/cpu0/core0": {
+ "Associations": [
+ [
+ "sensors",
+ "inventory",
+ "/xyz/openbmc_project/sensors/temperature/p0_core0_temp"
+ ]
+ ],
+ "Functional": true,
+ "Present": true,
+ "PrettyName": ""
+ },
+ "/xyz/openbmc_project/inventory/system/chassis/motherboard/cpu0/core1": {
+ "Associations": [
+ [
+ "sensors",
+ "inventory",
+ "/xyz/openbmc_project/sensors/temperature/p0_core1_temp"
+ ]
+ ],
+ "Functional": true,
+ "Present": true,
+ "PrettyName": ""
+ }
+ },
+ "message": "200 OK",
+ "status": "ok"
+}
+```
+
+Invoke the tool(with everything expected found):
+```
+> expectedJsonChecker.py witherspoon expected.json input.json -f filter.json
+```
+Invoke the tool(with modified fan1_0 `Functional` property to `False`):
+```
+> expectedJsonChecker.py witherspoon expected.json input.json -f filter.json
+NOT FOUND:
+/xyz/openbmc_project/sensors/fan_tach/fan1_0: {u'Functional': False}
+```
diff --git a/msbarth/expectedJsonChecker.py b/msbarth/expectedJsonChecker.py
new file mode 100755
index 0000000..d232a4f
--- /dev/null
+++ b/msbarth/expectedJsonChecker.py
@@ -0,0 +1,200 @@
+#!/usr/bin/python
+
+"""
+This script reads in JSON files containing a set of expected entries to be
+found within a given input JSON file. An optional "filtering" JSON
+file may also be provided that contains a set of data that will be used to
+filter configured expected entries from being checked in the input JSON.
+"""
+
+import os
+import sys
+import json
+from argparse import ArgumentParser
+
+
+def findEntry(entry, jsonObject):
+
+ if isinstance(jsonObject, dict):
+ for key in jsonObject:
+ if key == entry:
+ return jsonObject[key]
+ else:
+ found = findEntry(entry, jsonObject[key])
+ if found:
+ return found
+
+
+def buildDict(entry, jsonObject, resultDict):
+
+ key = list(entry)[0]
+ jsonObject = findEntry(key, jsonObject)
+ if jsonObject is None:
+ return {}
+ entry = entry[key]
+ if isinstance(entry, dict):
+ resultDict[key] = buildDict(entry, jsonObject, resultDict)
+ else:
+ return {key: jsonObject}
+
+
+def doAnd(andList, filters):
+
+ allTrue = True
+ for entry in andList:
+ # $and entries must be atleast a single layer dict
+ value = dict()
+ buildDict(entry, filters, value)
+ if value != entry:
+ allTrue = False
+
+ return allTrue
+
+
+def doOr(orList, filters):
+
+ anyTrue = False
+ for entry in orList:
+ # $or entries must be atleast a single layer dict
+ value = dict()
+ buildDict(entry, filters, value)
+ if value == entry:
+ anyTrue = True
+ break
+
+ return anyTrue
+
+
+def doNor(norList, filters):
+
+ allFalse = True
+ for entry in norList:
+ # $nor entries must be atleast a single layer dict
+ value = dict()
+ buildDict(entry, filters, value)
+ if value == entry:
+ allFalse = False
+
+ return allFalse
+
+
+def doNot(notDict, filters):
+
+ # $not entry must be atleast a single layer dict
+ value = dict()
+ buildDict(notDict, filters, value)
+ if value == notDict:
+ return False
+
+ return True
+
+
+def applyFilters(expected, filters):
+
+ switch = {
+ # $and - Performs an AND operation on an array with at least two
+ # expressions and returns the document that meets all the
+ # expressions. i.e.) {"$and": [{"age": 5}, {"name": "Joe"}]}
+ "$and": doAnd,
+ # $or - Performs an OR operation on an array with at least two
+ # expressions and returns the documents that meet at least one of
+ # the expressions. i.e.) {"$or": [{"age": 4}, {"name": "Joe"}]}
+ "$or": doOr,
+ # $nor - Performs a NOR operation on an array with at least two
+ # expressions and returns the documents that do not meet any of
+ # the expressions. i.e.) {"$nor": [{"age": 3}, {"name": "Moe"}]}
+ "$nor": doNor,
+ # $not - Performs a NOT operation on the specified expression and
+ # returns the documents that do not meet the expression.
+ # i.e.) {"$not": {"age": 4}}
+ "$not": doNot
+ }
+
+ isExpected = {}
+ for entry in expected:
+ expectedList = list()
+ if entry == "$op":
+ addInput = True
+ for op in expected[entry]:
+ if op != "$input":
+ func = switch.get(op)
+ if not func(expected[entry][op], filters):
+ addInput = False
+ if addInput:
+ expectedList = expected[entry]["$input"]
+ else:
+ expectedList = [dict({entry: expected[entry]})]
+
+ for i in expectedList:
+ for key in i:
+ isExpected[key] = i[key]
+
+ return isExpected
+
+
+def findExpected(expected, input):
+
+ result = {}
+ for key in expected:
+ jsonObject = findEntry(key, input)
+ if isinstance(expected[key], dict) and expected[key] and jsonObject:
+ notExpected = findExpected(expected[key], jsonObject)
+ if notExpected:
+ result[key] = notExpected
+ else:
+ # If expected value is not "dont care" and
+ # does not equal what's expected
+ if str(expected[key]) != "{}" and expected[key] != jsonObject:
+ if jsonObject is None:
+ result[key] = None
+ else:
+ result[key] = expected[key]
+ return result
+
+
+if __name__ == '__main__':
+ parser = ArgumentParser(
+ description="Expected JSON cross-checker. Similar to a JSON schema \
+ validator, however this cross-checks a set of expected \
+ property states against the contents of a JSON input \
+ file with the ability to apply an optional set of \
+ filters against what's expected based on the property \
+ states within the provided filter JSON.")
+
+ parser.add_argument('index',
+ help='Index name into a set of entries within the \
+ expected JSON file')
+ parser.add_argument('expected_json',
+ help='JSON input file containing the expected set of \
+ entries, by index name, to be contained within \
+ the JSON input file')
+ parser.add_argument('input_json',
+ help='JSON input file containing the JSON data to be \
+ cross-checked against what is expected')
+ parser.add_argument('-f', '--filters', dest='filter_json',
+ help='JSON file containing path:property:value \
+ associations to optional filters configured \
+ within the expected set of JSON entries')
+
+ args = parser.parse_args()
+
+ with open(args.expected_json, 'r') as expected_json:
+ expected = json.load(expected_json) or {}
+ with open(args.input_json, 'r') as input_json:
+ input = json.load(input_json) or {}
+
+ filters = {}
+ if args.filter_json:
+ with open(args.filter_json, 'r') as filters_json:
+ filters = json.load(filters_json) or {}
+
+ if args.index in expected and expected[args.index] is not None:
+ expected = applyFilters(expected[args.index], filters)
+ result = findExpected(expected, input)
+ if result:
+ print("NOT FOUND:")
+ for key in result:
+ print(key + ": " + str(result[key]))
+ else:
+ print("Error: " + args.index + " not found in " + args.expected_json)
+ sys.exit(1)