blob: 0a0bfbbfdda000467f06d613586e31e81c30150b [file] [log] [blame]
Michael Tritz935c18e2017-03-08 11:45:41 -06001*** Settings ***
2Documentation Test Error logging.
3
4Resource ../lib/connection_client.robot
5Resource ../lib/openbmc_ffdc.robot
6Resource ../lib/utils.robot
7Resource ../lib/state_manager.robot
Rahul Maheshwari92972e72017-06-22 00:59:59 -05008Resource ../lib/ipmi_client.robot
Michael Tritz935c18e2017-03-08 11:45:41 -06009
10Suite Setup Run Keywords Verify logging-test AND
George Keishing184a89a2017-06-27 07:12:16 -050011... Delete Error Logs And Verify
Michael Tritz935c18e2017-03-08 11:45:41 -060012Test Setup Open Connection And Log In
George Keishing845d9dc2017-06-16 14:49:29 -050013Test Teardown Post Test Case Execution
George Keishing184a89a2017-06-27 07:12:16 -050014Suite Teardown Delete Error Logs And Verify
Michael Tritz935c18e2017-03-08 11:45:41 -060015
16*** Test Cases ***
17
18Create Test Error And Verify
19 [Documentation] Create error logs and verify via REST.
20 [Tags] Create_Test_Error_And_Verify
21
22 Create Test Error Log
23 Verify Test Error Log
24
25
26Test Error Persistency On Restart
George Keishing184a89a2017-06-27 07:12:16 -050027 [Documentation] Restart logging service and verify error logs.
Michael Tritz935c18e2017-03-08 11:45:41 -060028 [Tags] Test_Error_Persistency_On_Restart
29
30 Create Test Error Log
31 Verify Test Error Log
32 Execute Command On BMC
33 ... systemctl restart xyz.openbmc_project.Logging.service
34 Sleep 10s reason=Wait for logging service to restart properly.
George Keishing184a89a2017-06-27 07:12:16 -050035 Verify Test Error Log
Michael Tritz935c18e2017-03-08 11:45:41 -060036
37
38Test Error Persistency On Reboot
George Keishing184a89a2017-06-27 07:12:16 -050039 [Documentation] Reboot BMC and verify error logs.
Michael Tritz935c18e2017-03-08 11:45:41 -060040 [Tags] Test_Error_Persistency_On_Reboot
41
42 Create Test Error Log
43 Verify Test Error Log
44 Initiate BMC Reboot
45 Wait Until Keyword Succeeds 10 min 10 sec
46 ... Is BMC Ready
George Keishing184a89a2017-06-27 07:12:16 -050047 Verify Test Error Log
Michael Tritz935c18e2017-03-08 11:45:41 -060048
49
George Keishing04bc4812017-03-30 03:33:31 -050050Create Test Error And Verify Resolved Field
51 [Documentation] Create error log and verify "Resolved"
52 ... field is 0.
53 [Tags] Create_Test_Error_And_Verify_Resolved_Field
54
55 # Example Error log:
56 # "/xyz/openbmc_project/logging/entry/1": {
57 # "AdditionalData": [
58 # "STRING=FOO"
59 # ],
60 # "Id": 1,
61 # "Message": "example.xyz.openbmc_project.Example.Elog.AutoTestSimple",
62 # "Resolved": 0,
63 # "Severity": "xyz.openbmc_project.Logging.Entry.Level.Error",
64 # "Timestamp": 1490817164983,
65 # "associations": []
66 # },
67
68 # It's work in progress, but it's mnfg need. To mark an error as
69 # resolved, without deleting the error, mfg will set this bool
70 # property.
71 # In this test context we are making sure "Resolved" field is "0"
72 # by default.
73
George Keishing6eb10b82017-06-06 10:02:09 -050074 Delete Error Logs
George Keishing04bc4812017-03-30 03:33:31 -050075 Create Test Error Log
George Keishing184a89a2017-06-27 07:12:16 -050076 ${elog_entry}= Get URL List ${BMC_LOGGING_ENTRY}
77 ${resolved}= Read Attribute ${elog_entry[0]} Resolved
George Keishing04bc4812017-03-30 03:33:31 -050078 Should Be True ${resolved} == 0
79
80
81Create Test Errors And Verify Time Stamp
82 [Documentation] Create error logs and verify time stamp.
83 [Tags] Create_Test_Error_And_Verify_Time_Stamp
84
85 # Example Error logs:
86 # "/xyz/openbmc_project/logging/entry/1": {
87 # "AdditionalData": [
88 # "STRING=FOO"
89 # ],
90 # "Id": 1,
91 # "Message": "example.xyz.openbmc_project.Example.Elog.AutoTestSimple",
92 # "Resolved": 0,
93 # "Severity": "xyz.openbmc_project.Logging.Entry.Level.Error",
94 # "Timestamp": 1490818990051, <--- Time stamp
95 # "associations": []
96 # },
97 # "/xyz/openbmc_project/logging/entry/2": {
98 # "AdditionalData": [
99 # "STRING=FOO"
100 # ],
101 # "Id": 2,
102 # "Message": "example.xyz.openbmc_project.Example.Elog.AutoTestSimple",
103 # "Resolved": 0,
104 # "Severity": "xyz.openbmc_project.Logging.Entry.Level.Error",
105 # "Timestamp": 1490818992116, <---- Time stamp
106 # "associations": []
107 # },
108
George Keishing6eb10b82017-06-06 10:02:09 -0500109 Delete Error Logs
George Keishing04bc4812017-03-30 03:33:31 -0500110 Create Test Error Log
111 Create Test Error Log
112 # The error log generated is associated with the epoc time and unique
113 # for every error and in increasing time stamp.
George Keishing184a89a2017-06-27 07:12:16 -0500114 ${elog_entry}= Get URL List ${BMC_LOGGING_ENTRY}
115 ${time_stamp1}= Read Attribute ${elog_entry[0]} Timestamp
116 ${time_stamp2}= Read Attribute ${elog_entry[1]} Timestamp
George Keishing6eb10b82017-06-06 10:02:09 -0500117 Should Be True ${time_stamp2} > ${time_stamp1}
George Keishing04bc4812017-03-30 03:33:31 -0500118
Sweta Potthuri9ce8eeb2017-05-08 04:20:11 -0500119Create Test Error Log And Delete
120 [Documentation] Create an error log and delete it.
121 [Tags] Create_Test_Error_Log_And_Delete
122
123 Delete Error Logs And Verify
124 Create Test Error Log
125 Delete Error Logs And Verify
126
127Create Multiple Test Error Logs And Delete All
128 [Documentation] Create multiple error logs and delete all.
129 [Tags] Create_Multiple_Test_Error_Logs_And_Delete_All
130
131 Delete Error Logs And Verify
132 Create Test Error Log
133 Create Test Error Log
134 Create Test Error Log
135 Delete Error Logs And Verify
136
137Create Two Test Error Logs And Delete One
138 [Documentation] Create two error logs and delete the first entry.
139 [Tags] Create_Two_Test_Error_Logs_And_Delete_One
140
141 Delete Error Logs And Verify
142 Create Test Error Log
George Keishing184a89a2017-06-27 07:12:16 -0500143 ${elog_entry}= Get URL List ${BMC_LOGGING_ENTRY}
Sweta Potthuri9ce8eeb2017-05-08 04:20:11 -0500144 Create Test Error Log
George Keishing184a89a2017-06-27 07:12:16 -0500145 Delete Error log Entry ${elog_entry[0]}
146 ${resp}= OpenBMC Get Request ${elog_entry[0]}
Sweta Potthuri9ce8eeb2017-05-08 04:20:11 -0500147 Should Be Equal As Strings ${resp.status_code} ${HTTP_NOT_FOUND}
George Keishing04bc4812017-03-30 03:33:31 -0500148
Rahul Maheshwari92972e72017-06-22 00:59:59 -0500149
150Verify IPMI SEL Version
151 [Documentation] Verify IPMI SEL's version info.
152 [Tags] Verify_IPMI_SEL_Version
153
154 ${version_info}= Get IPMI SEL Setting Version
155 ${setting_status}= Fetch From Left ${version_info} (
156 ${setting_status}= Evaluate $setting_status.replace(' ','')
157
158 Should Be True ${setting_status} >= 1.5
Rahul Maheshwarib44b6ec2017-07-05 00:23:03 -0500159 Should Contain ${version_info} v2 compliant case_insensitive=True
Rahul Maheshwari92972e72017-06-22 00:59:59 -0500160
161
George Keishing845d9dc2017-06-16 14:49:29 -0500162Verify Watchdog Timedout Error
163 [Documentation] Trigger watchdog timed out and verify errorlog generated.
164 [Tags] Verify_Watchdog_Timedout_Error
165
166 # Clear errors if there are any.
167 Delete Error Logs
168
169 Initiate Host Boot
170
171 # Check if the watchdog interface is created.
172 Wait Until Keyword Succeeds 3 min 10 sec
173 ... Read Properties /xyz/openbmc_project/watchdog/host0
174
175 Trigger Host Watchdog Error
176
177 Verify Watchdog Errorlog Content
178
179
Rahul Maheshwarib44b6ec2017-07-05 00:23:03 -0500180Verify IPMI SEL Delete
181 [Documentation] Verify IPMI SEL delete operation.
182 [Tags] Verify_IPMI_SEL_Delete
183
184 Delete Error Logs And Verify
185 Create Test Error Log
186
187 ${sel_list}= Run IPMI Standard Command sel list
188 # Example of SEL List:
189 # 4 | 04/21/2017 | 10:51:16 | System Event #0x01 | Undetermined system hardware failure | Asserted
190
191 ${sel_entry}= Fetch from Left ${sel_list} |
192 ${sel_entry}= Evaluate $sel_entry.replace(' ','')
193 ${sel_entry}= Convert To Integer 0x${sel_entry}
194
195 ${sel_delete}= Run IPMI Standard Command sel delete ${sel_entry}
196 Should Be Equal As Strings ${sel_delete} Deleted entry ${sel_entry}
197 ... case_insensitive=True
198
199 ${sel_list}= Run IPMI Standard Command sel list
200 Should Be Equal As Strings ${sel_list} SEL has no entries
201 ... case_insensitive=True
202
203
204Verify Empty SEL
205 [Documentation] Verify empty SEL list.
206 [Tags] Verify_Empty_SEL
207
208 Delete Error Logs And Verify
209
210 ${resp}= Run IPMI Standard Command sel list
211 Should Contain ${resp} SEL has no entries case_insensitive=True
212
213
214Delete Non Existing SEL Entry
215 [Documentation] Delete non existing SEL entry.
216 [Tags] Delete_Non_Existing_SEL_Entry
217
218 Delete Error Logs And Verify
219 ${sel_delete}= Run Keyword And Expect Error *
220 ... Run IPMI Standard Command sel delete 100
221 Should Contain ${sel_delete} Unable to delete entry
222 ... case_insensitive=True
223
224
225Delete Invalid SEL Entry
226 [Documentation] Delete invalid SEL entry.
227 [Tags] Delete_Invalid_SEL_Entry
228
229 ${sel_delete}= Run Keyword And Expect Error *
230 ... Run IPMI Standard Command sel delete abc
231 Should Contain ${sel_delete} Given SEL ID 'abc' is invalid
232 ... case_insensitive=True
233
234
235
Michael Tritz935c18e2017-03-08 11:45:41 -0600236*** Keywords ***
237
Rahul Maheshwari92972e72017-06-22 00:59:59 -0500238Get IPMI SEL Setting
239 [Documentation] Returns status for given IPMI SEL setting.
240 [Arguments] ${setting}
241 # Description of argument(s):
242 # setting SEL setting which needs to be read(e.g. "Last Add Time").
243
244 ${resp}= Run IPMI Standard Command sel info
245
246 ${setting_line}= Get Lines Containing String ${resp} ${setting}
247 ... case-insensitive
Rahul Maheshwarib44b6ec2017-07-05 00:23:03 -0500248 ${setting_status}= Fetch From Right ${setting_line} :${SPACE}
Rahul Maheshwari92972e72017-06-22 00:59:59 -0500249
250 [Return] ${setting_status}
251
252
George Keishing845d9dc2017-06-16 14:49:29 -0500253Verify Watchdog Errorlog Content
254 [Documentation] Verify watchdog errorlog content.
255 # Example:
256 # "/xyz/openbmc_project/logging/entry/1":
257 # {
258 # "AdditionalData": [],
259 # "Id": 1,
260 # "Message": "org.open_power.Host.Error.WatchdogTimedOut",
261 # "Resolved": 0,
262 # "Severity": "xyz.openbmc_project.Logging.Entry.Level.Informational",
263 # "Timestamp": 1492715244828,
264 # "associations": []
265 # },
266
George Keishing184a89a2017-06-27 07:12:16 -0500267 ${elog_entry}= Get URL List ${BMC_LOGGING_ENTRY}
268 ${elog}= Read Properties ${elog_entry[0]}
George Keishing845d9dc2017-06-16 14:49:29 -0500269 Should Be Equal As Strings
270 ... ${elog["Message"]} org.open_power.Host.Error.WatchdogTimedOut
271 Should Not Be Equal As Strings
272 ... ${elog["Severity"]} xyz.openbmc_project.Logging.Entry.Level.Informational
273
274
Michael Tritz935c18e2017-03-08 11:45:41 -0600275Verify logging-test
276 [Documentation] Verify existence of prerequisite logging-test.
277
278 Open Connection And Log In
279 ${out} ${stderr}= Execute Command which logging-test return_stderr=True
280 Should Be Empty ${stderr}
281 Should Contain ${out} logging-test
282
283Clear Existing Error Logs
284 [Documentation] If error log isn't empty, reboot the BMC to clear the log.
285
286 ${resp}= OpenBMC Get Request ${BMC_LOGGING_ENTRY}${1}
287 Return From Keyword If ${resp.status_code} == ${HTTP_NOT_FOUND}
288 Initiate BMC Reboot
289 Wait Until Keyword Succeeds 10 min 10 sec
290 ... Is BMC Ready
291 ${resp}= OpenBMC Get Request ${BMC_LOGGING_ENTRY}${1}
292 Should Be Equal As Strings ${resp.status_code} ${HTTP_NOT_FOUND}
293
294Create Test Error Log
295 [Documentation] Generate test error log.
296
297 # Test error log entry example:
298 # "/xyz/openbmc_project/logging/entry/1": {
299 # "AdditionalData": [
300 # "STRING=FOO"
301 # ],
302 # "Id": 1,
303 # "Message": "example.xyz.openbmc_project.Example.Elog.AutoTestSimple",
304 # "Severity": "xyz.openbmc_project.Logging.Entry.Level.Error",
305 # "Timestamp": 1487743963328,
306 # "associations": []
307 # }
308
309 Execute Command On BMC logging-test -c AutoTestSimple
310
311Verify Test Error Log
312 [Documentation] Verify test error log entries.
George Keishing184a89a2017-06-27 07:12:16 -0500313 ${elog_entry}= Get URL List ${BMC_LOGGING_ENTRY}
314 ${entry_id}= Read Attribute ${elog_entry[0]} Message
Sweta Potthuri9ce8eeb2017-05-08 04:20:11 -0500315 Should Be Equal ${entry_id}
Michael Tritz935c18e2017-03-08 11:45:41 -0600316 ... example.xyz.openbmc_project.Example.Elog.AutoTestSimple
George Keishing184a89a2017-06-27 07:12:16 -0500317 ${entry_id}= Read Attribute ${elog_entry[0]} Severity
Sweta Potthuri9ce8eeb2017-05-08 04:20:11 -0500318 Should Be Equal ${entry_id}
Michael Tritz935c18e2017-03-08 11:45:41 -0600319 ... xyz.openbmc_project.Logging.Entry.Level.Error
George Keishing04bc4812017-03-30 03:33:31 -0500320
Sweta Potthuri9ce8eeb2017-05-08 04:20:11 -0500321Delete Error Logs And Verify
322 [Documentation] Delete all error logs and verify.
323
George Keishing6eb10b82017-06-06 10:02:09 -0500324 Delete Error Logs
Sweta Potthuri9ce8eeb2017-05-08 04:20:11 -0500325 ${resp}= OpenBMC Get Request ${BMC_LOGGING_ENTRY}/list
326 Should Be Equal As Strings ${resp.status_code} ${HTTP_NOT_FOUND}
George Keishing845d9dc2017-06-16 14:49:29 -0500327
328Post Test Case Execution
329 [Documentation] Do the post test teardown.
330 # 1. Capture FFDC on test failure.
331 # 2. Delete error logs.
332 # 3. Close all open SSH connections.
333 # 4. Clear all REST sessions.
334
335 FFDC On Test Case Fail
336 Delete Error Logs
337 Close All Connections
338 Flush REST Sessions
339