| *** Settings *** |
| Documentation Error logging utility keywords. |
| |
| Resource rest_client.robot |
| Variables ../data/variables.py |
| Variables ../data/pel_variables.py |
| |
| *** Variables *** |
| |
| |
| # Define variables for use by callers of 'Get Error Logs'. |
| ${low_severity_errlog_regex} \\.(Informational|Notice|Debug)$ |
| &{low_severity_errlog_filter} Severity=${low_severity_errlog_regex} |
| &{low_severity_errlog_filter_args} filter_dict=${low_severity_errlog_filter} regex=${True} invert=${True} |
| # The following is equivalent to &{low_severity_errlog_filter_args} but the name may be more intuitive for |
| # users. Example usage: |
| # ${err_logs}= Get Error Logs &{filter_low_severity_errlogs} |
| &{filter_low_severity_errlogs} &{low_severity_errlog_filter_args} |
| |
| *** Keywords *** |
| |
| Get Logging Entry List |
| [Documentation] Get logging entry and return the object list. |
| |
| ${entry_list}= Create List |
| ${resp}= OpenBMC Get Request ${BMC_LOGGING_ENTRY}list quiet=${1} |
| Return From Keyword If ${resp.status_code} == ${HTTP_NOT_FOUND} |
| ${jsondata}= To JSON ${resp.content} |
| |
| FOR ${entry} IN @{jsondata["data"]} |
| Continue For Loop If '${entry.rsplit('/', 1)[1]}' == 'callout' |
| Append To List ${entry_list} ${entry} |
| END |
| |
| # Logging entries list. |
| # ['/xyz/openbmc_project/logging/entry/14', |
| # '/xyz/openbmc_project/logging/entry/15'] |
| [Return] ${entry_list} |
| |
| |
| Logging Entry Should Exist |
| [Documentation] Find the matching message id and return the entry id. |
| [Arguments] ${message_id} |
| |
| # Description of argument(s): |
| # message_id Logging message string. |
| # Example: "xyz.openbmc_project.Common.Error.InternalFailure" |
| |
| @{elog_entries}= Get Logging Entry List |
| |
| FOR ${entry} IN @{elog_entries} |
| ${resp}= Read Properties ${entry} |
| ${status}= Run Keyword And Return Status |
| ... Should Be Equal As Strings ${message_id} ${resp["Message"]} |
| Return From Keyword If ${status} == ${TRUE} ${entry} |
| END |
| |
| Fail No ${message_id} logging entry found. |
| |
| |
| Get Error Logs |
| [Documentation] Return the BMC error logs as a dictionary. |
| [Arguments] ${quiet}=1 &{filter_struct_args} |
| |
| # Example of call using pre-defined filter args (defined above). |
| |
| # ${err_logs}= Get Error Logs &{filter_low_severity_errlogs} |
| |
| # In this example, all error logs with "Severity" fields that are neither Informational, Debug nor |
| # Notice will be returned. |
| |
| # Description of argument(s): |
| # quiet Indicates whether this keyword should run without any output to the |
| # console, 0 = verbose, 1 = quiet. |
| # filter_struct_args filter_struct args (e.g. filter_dict, regex, etc.) to be passed directly |
| # to the Filter Struct keyword. See its prolog for details. |
| |
| # The length of the returned dictionary indicates how many logs there are. |
| |
| # Use 'Print Error Logs' to print. Example: |
| |
| # Print Error Logs ${error_logs} Message. |
| |
| ${status} ${error_logs}= Run Keyword And Ignore Error Read Properties |
| ... /xyz/openbmc_project/logging/entry/enumerate timeout=30 quiet=${quiet} |
| Return From Keyword If '${status}' == 'FAIL' &{EMPTY} |
| ${num_filter_struct_args}= Get Length ${filter_struct_args} |
| Return From Keyword If '${num_filter_struct_args}' == '${0}' ${error_logs} |
| ${filtered_error_logs}= Filter Struct ${error_logs} &{filter_struct_args} |
| [Return] ${filtered_error_logs} |
| |
| |
| Get IPMI SEL Setting |
| [Documentation] Returns status for given IPMI SEL setting. |
| [Arguments] ${setting} |
| # Description of argument(s): |
| # setting SEL setting which needs to be read(e.g. "Last Add Time"). |
| |
| ${resp}= Run IPMI Standard Command sel info |
| |
| ${setting_line}= Get Lines Containing String ${resp} ${setting} |
| ... case-insensitive |
| ${setting_status}= Fetch From Right ${setting_line} :${SPACE} |
| |
| [Return] ${setting_status} |
| |
| |
| Verify Watchdog Errorlog Content |
| [Documentation] Verify watchdog errorlog content. |
| # Example: |
| # "/xyz/openbmc_project/logging/entry/1": |
| # { |
| # "AdditionalData": [], |
| # "Id": 1, |
| # "Message": "org.open_power.Host.Boot.Error.WatchdogTimedOut", |
| # "Resolved": 0, |
| # "Severity": "xyz.openbmc_project.Logging.Entry.Level.Error", |
| # "Timestamp": 1492715244828, |
| # "Associations": [] |
| # }, |
| |
| ${elog_entry}= Get URL List ${BMC_LOGGING_ENTRY} |
| ${elog}= Read Properties ${elog_entry[0]} |
| Should Be Equal As Strings |
| ... ${elog["Message"]} org.open_power.Host.Boot.Error.WatchdogTimedOut |
| ... msg=Watchdog timeout error log was not found. |
| Should Be Equal As Strings |
| ... ${elog["Severity"]} xyz.openbmc_project.Logging.Entry.Level.Error |
| ... msg=Watchdog timeout severity unexpected value. |
| |
| |
| Logging Test Binary Exist |
| [Documentation] Verify existence of prerequisite logging-test. |
| Open Connection And Log In |
| ${out} ${stderr}= Execute Command |
| ... which /tmp/tarball/bin/logging-test return_stderr=True |
| Should Be Empty ${stderr} msg=Logging Test stderr is non-empty. |
| Should Contain ${out} logging-test |
| ... msg=Logging test returned unexpected result. |
| |
| Clear Existing Error Logs |
| [Documentation] If error log isn't empty, reboot the BMC to clear the log. |
| ${resp}= OpenBMC Get Request ${BMC_LOGGING_ENTRY}${1} |
| Return From Keyword If ${resp.status_code} == ${HTTP_NOT_FOUND} |
| Initiate BMC Reboot |
| Wait Until Keyword Succeeds 10 min 10 sec |
| ... Is BMC Ready |
| ${resp}= OpenBMC Get Request ${BMC_LOGGING_ENTRY}${1} |
| Should Be Equal As Strings ${resp.status_code} ${HTTP_NOT_FOUND} |
| ... msg=Could not clear BMC error logs. |
| |
| |
| Create Test PEL Log |
| [Documentation] Generate test PEL log. |
| [Arguments] ${pel_type}=Internal Failure |
| |
| # Description of argument(s): |
| # pel_type The PEL type (e.g. Internal Failure, FRU Callout, Procedural Callout). |
| |
| # Test PEL log entry example: |
| # { |
| # "0x5000002D": { |
| # "SRC": "BD8D1002", |
| # "Message": "An application had an internal failure", |
| # "PLID": "0x5000002D", |
| # "CreatorID": "BMC", |
| # "Subsystem": "BMC Firmware", |
| # "Commit Time": "02/25/2020 04:47:09", |
| # "Sev": "Unrecoverable Error", |
| # "CompID": "0x1000" |
| # } |
| # } |
| |
| Run Keyword If '${pel_type}' == 'Internal Failure' |
| ... BMC Execute Command ${CMD_INTERNAL_FAILURE} |
| ... ELSE IF '${pel_type}' == 'FRU Callout' |
| ... BMC Execute Command ${CMD_FRU_CALLOUT} |
| ... ELSE IF '${pel_type}' == 'Procedure And Symbolic FRU Callout' |
| ... BMC Execute Command ${CMD_PROCEDURAL_SYMBOLIC_FRU_CALLOUT} |
| |
| |
| Create Test Error Log |
| [Documentation] Generate test error log. |
| # Test error log entry example: |
| # "/xyz/openbmc_project/logging/entry/1": { |
| # "AdditionalData": [ |
| # "STRING=FOO" |
| # ], |
| # "Id": 1, |
| # "Message": "example.xyz.openbmc_project.Example.Elog.AutoTestSimple", |
| # "Severity": "xyz.openbmc_project.Logging.Entry.Level.Error", |
| # "Timestamp": 1487743963328, |
| # "Associations": [] |
| # } |
| BMC Execute Command /tmp/tarball/bin/logging-test -c AutoTestSimple |
| |
| Count Error Entries |
| [Documentation] Count Error entries. |
| ${resp}= OpenBMC Get Request ${BMC_LOGGING_ENTRY} |
| Should Be Equal As Strings ${resp.status_code} ${HTTP_OK} |
| ... msg=Failed to get error logs. |
| ${jsondata}= To JSON ${resp.content} |
| ${count}= Get Length ${jsondata["data"]} |
| [Return] ${count} |
| |
| Verify Test Error Log |
| [Documentation] Verify test error log entries. |
| ${elog_entry}= Get URL List ${BMC_LOGGING_ENTRY} |
| ${entry_id}= Read Attribute ${elog_entry[0]} Message |
| Should Be Equal ${entry_id} |
| ... example.xyz.openbmc_project.Example.Elog.AutoTestSimple |
| ... msg=Error log not from AutoTestSimple. |
| ${entry_id}= Read Attribute ${elog_entry[0]} Severity |
| Should Be Equal ${entry_id} |
| ... xyz.openbmc_project.Logging.Entry.Level.Error |
| ... msg=Error log severity mismatch. |
| |
| Delete Error Logs And Verify |
| [Documentation] Delete all error logs and verify. |
| Delete All Error Logs |
| ${resp}= OpenBMC Get Request ${BMC_LOGGING_ENTRY}list quiet=${1} |
| Should Be Equal As Strings ${resp.status_code} ${HTTP_NOT_FOUND} |
| ... msg=Error logs not deleted as expected. |
| |
| |
| Install Tarball |
| [Documentation] Install tarball on BMC. |
| Should Not Be Empty ${DEBUG_TARBALL_PATH} |
| ... msg=Debug tarball path value is required. |
| BMC Execute Command rm -rf /tmp/tarball |
| Install Debug Tarball On BMC ${DEBUG_TARBALL_PATH} |
| |
| |
| Get Event Logs |
| [Documentation] Get all available EventLog entries. |
| |
| #{ |
| # "@odata.context": "/redfish/v1/$metadata#LogEntryCollection.LogEntryCollection", |
| # "@odata.id": "/redfish/v1/Systems/system/LogServices/EventLog/Entries", |
| # "@odata.type": "#LogEntryCollection.LogEntryCollection", |
| # "Description": "Collection of System Event Log Entries", |
| # "Members": [ |
| # { |
| # "@odata.context": "/redfish/v1/$metadata#LogEntry.LogEntry", |
| # "@odata.id": "/redfish/v1/Systems/system/LogServices/EventLog/Entries/1", |
| # "@odata.type": "#LogEntry.v1_4_0.LogEntry", |
| # "Created": "2019-05-29T13:19:27+00:00", |
| # "EntryType": "Event", |
| # "Id": "1", |
| # "Message": "org.open_power.Host.Error.Event", |
| # "Name": "System DBus Event Log Entry", |
| # "Severity": "Critical" |
| # } |
| # ], |
| # "Members@odata.count": 1, |
| # "Name": "System Event Log Entries" |
| #} |
| |
| ${members}= Redfish.Get Attribute ${EVENT_LOG_URI}Entries Members |
| [Return] ${members} |
| |
| |
| Get Event Logs Not Ok |
| [Documentation] Get all event logs where the 'Severity' is not 'OK'. |
| |
| ${members}= Get Event Logs |
| ${severe_logs}= Evaluate [elog for elog in $members if elog['Severity'] != 'OK'] |
| [Return] ${severe_logs} |
| |
| |
| Get Number Of Event Logs |
| [Documentation] Return the number of EventLog members. |
| |
| ${members}= Get Event Logs |
| ${num_members}= Get Length ${members} |
| [Return] ${num_members} |
| |
| |
| Redfish Purge Event Log |
| [Documentation] Do Redfish EventLog purge. |
| |
| ${target_action}= redfish_utils.Get Target Actions |
| ... /redfish/v1/Systems/system/LogServices/EventLog/ LogService.ClearLog |
| Redfish.Post ${target_action} body={'target': '${target_action}'} |
| ... valid_status_codes=[${HTTP_OK}, ${HTTP_NO_CONTENT}] |
| |