blob: 2349bb37b6108ec49d552e13ffbad9809c663cbb [file] [log] [blame]
Rahul Maheshwari2568f872020-09-23 01:03:39 -05001*** Settings ***
2
3Documentation Test BMC dump functionality of OpenBMC.
4
George Keishing88ec2802021-08-04 03:04:22 -05005Resource ../../lib/bmc_redfish_resource.robot
6Resource ../../lib/boot_utils.robot
Rahul Maheshwari2568f872020-09-23 01:03:39 -05007Resource ../../lib/dump_utils.robot
George Keishing88ec2802021-08-04 03:04:22 -05008Resource ../../lib/openbmc_ffdc.robot
manashsarma7e1f2c52023-02-03 03:18:40 -06009Variables ../../data/pel_variables.py
Rahul Maheshwari2568f872020-09-23 01:03:39 -050010
George Keishingf514ccb2021-02-02 23:55:56 -060011Suite Setup Redfish.Login
Rahul Maheshwari2568f872020-09-23 01:03:39 -050012Test Setup Redfish Delete All BMC Dumps
13Test Teardown Test Teardown Execution
14
Tim Leede2d1302021-06-18 01:24:11 +080015*** Variables ***
16
17# Total size of the dump in kilo bytes
18${BMC_DUMP_TOTAL_SIZE} ${1024}
19
20# Minimum space required for one bmc dump in kilo bytes
21${BMC_DUMP_MIN_SPACE_REQD} ${20}
manashsarmabd346b32022-07-20 06:22:58 -050022${MAX_DUMP_COUNT} ${20}
manashsarma395f8802022-11-17 03:09:02 -060023${BMC_DUMP_COLLECTOR_PATH} /var/lib/phosphor-debug-collector/dumps
Rahul Maheshwari2568f872020-09-23 01:03:39 -050024
25*** Test Cases ***
26
manashsarma4b5b1fa2022-11-16 02:36:58 -060027Verify Error Response For Already Deleted Dump Id
28 [Documentation] Delete non existing BMC dump and expect an error.
29 [Tags] Verify_Error_Response_For_Already_Deleted_Dump_Id
30
manashsarma9dcabad2023-02-02 03:55:17 -060031 Redfish Power Off stack_mode=skip
manashsarma4b5b1fa2022-11-16 02:36:58 -060032 ${dump_id}= Create User Initiated BMC Dump Via Redfish
33 Redfish Delete BMC Dump ${dump_id}
34 Run Keyword And Expect Error ValueError: * Redfish Delete BMC Dump ${dump_id}
35
36
Rahul Maheshwari2568f872020-09-23 01:03:39 -050037Verify User Initiated BMC Dump When Host Powered Off
38 [Documentation] Create user initiated BMC dump at host off state and
39 ... verify dump entry for it.
40 [Tags] Verify_User_Initiated_BMC_Dump_When_Host_Powered_Off
41
42 Redfish Power Off stack_mode=skip
Rahul Maheshwaribd38aa42021-08-04 01:24:39 -050043 ${dump_id}= Create User Initiated BMC Dump Via Redfish
Rahul Maheshwaribcefdf22020-10-16 07:51:34 -050044 ${dump_entries}= Get BMC Dump Entries
Rahul Maheshwari2568f872020-09-23 01:03:39 -050045 Length Should Be ${dump_entries} 1
46 List Should Contain Value ${dump_entries} ${dump_id}
47
Rahul Maheshwari2568f872020-09-23 01:03:39 -050048
Rahul Maheshwarie8e9d0c2020-10-30 02:39:01 -050049Verify User Initiated BMC Dump Size
manashsarma7831dc22022-03-08 01:49:36 -060050 [Documentation] Verify user initiated BMC dump size is under 20 MB.
Rahul Maheshwarie8e9d0c2020-10-30 02:39:01 -050051 [Tags] Verify_User_Initiated_BMC_Dump_Size
52
Rahul Maheshwaribd38aa42021-08-04 01:24:39 -050053 ${dump_id}= Create User Initiated BMC Dump Via Redfish
Rahul Maheshwarie8e9d0c2020-10-30 02:39:01 -050054 ${resp}= Redfish.Get Properties /redfish/v1/Managers/bmc/LogServices/Dump/Entries/${dump_id}
55
56 # Example of response from above Redfish GET request.
57 # "@odata.type": "#LogEntry.v1_7_0.LogEntry",
58 # "AdditionalDataSizeBytes": 31644,
59 # "AdditionalDataURI": "/redfish/v1/Managers/bmc/LogServices/Dump/attachment/9",
60 # "Created": "2020-10-23T06:32:53+00:00",
61 # "DiagnosticDataType": "Manager",
62 # "EntryType": "Event",
63 # "Id": "9",
64 # "Name": "BMC Dump Entry"
65
manashsarma7831dc22022-03-08 01:49:36 -060066 # Max size for dump is 20 MB = 20x1024x1024 Byte.
67 Should Be True 0 < ${resp["AdditionalDataSizeBytes"]} < 20971520
Rahul Maheshwarie8e9d0c2020-10-30 02:39:01 -050068
69
manashsarma7e1f2c52023-02-03 03:18:40 -060070Verify Internal Failure Initiated BMC Dump Size
71 [Documentation] Verify that the internal failure initiated BMC dump size is under 20 MB.
72 [Tags] Verify_Internal_Failure_Initiated_BMC_Dump_Size
73
74 Redfish Delete All BMC Dumps
75
76 # Create an internal failure error log.
77 BMC Execute Command ${CMD_INTERNAL_FAILURE}
78
79 # Wait for BMC dump to get generated after injecting internal failure.
80 Wait Until Keyword Succeeds 2 min 10 sec Is BMC Dump Available
81
82 # Verify that only one BMC dump is generated after injecting error.
83 ${dump_entries}= Get BMC Dump Entries
84 ${length}= Get length ${dump_entries}
85 Should Be Equal As Integers ${length} ${1}
86
87 # Max size for dump is 20 MB = 20x1024x1024 Byte.
88 ${resp}= Redfish.Get Properties
89 ... /redfish/v1/Managers/bmc/LogServices/Dump/Entries/${dump_entries[0]}
90 Should Be True 0 < ${resp["AdditionalDataSizeBytes"]} < 20971520
91
92
manashsarmabbc68bc2022-12-05 09:55:17 -060093Verify Multiple BMC Dump Creation
94 [Documentation] Verify that multiple BMC dumps can be created one after
95 ... another successfully.
96 [Tags] Verify_Multiple_BMC_Dump_Creation
97
98 ${dump_count}= Evaluate random.randint(5, 10) modules=random
99 FOR ${INDEX} IN 1 ${dump_count}
100 Create User Initiated BMC Dump Via Redfish
101 END
102
103
manashsarma395f8802022-11-17 03:09:02 -0600104Verify BMC Dump Default Location In BMC
105 [Documentation] Verify that BMC dump is created in its default location of BMC.
George Keishing224e4c72022-12-14 05:07:58 -0600106 [Tags] Verify_BMC_Dump_Default_Location_In_BMC
manashsarma395f8802022-11-17 03:09:02 -0600107
108 Redfish Delete All BMC Dumps
109 ${dump_id}= Create User Initiated BMC Dump Via Redfish
110 ${dump_file} ${stderr} ${rc}= BMC Execute Command
111 ... ls ${BMC_DUMP_COLLECTOR_PATH}/${dump_id}
112 Should Be True ${rc} == 0
George Keishing952bb0f2022-12-12 03:13:35 -0600113 Should Start With ${dump_file} BMCDUMP
manashsarma395f8802022-11-17 03:09:02 -0600114
115
George Keishing3d850982022-07-19 11:03:03 -0500116Verify User Initiated BMC Dump When Host Booted
117 [Documentation] Create user initiated BMC dump at host booted state and
118 ... verify dump entry for it.
119 [Tags] Verify_User_Initiated_BMC_Dump_When_Host_Booted
120
121 Redfish Power On stack_mode=skip
122 ${dump_id}= Create User Initiated BMC Dump Via Redfish
123 ${dump_entries}= Get BMC Dump Entries
124 Length Should Be ${dump_entries} 1
125 List Should Contain Value ${dump_entries} ${dump_id}
126
127
Rahul Maheshwari2568f872020-09-23 01:03:39 -0500128Verify Dump Persistency On Dump Service Restart
129 [Documentation] Create user dump, restart dump manager service and verify dump
130 ... persistency.
131 [Tags] Verify_Dump_Persistency_On_Dump_Service_Restart
132
Rahul Maheshwaribd38aa42021-08-04 01:24:39 -0500133 Create User Initiated BMC Dump Via Redfish
Rahul Maheshwari2568f872020-09-23 01:03:39 -0500134 ${dump_entries_before}= redfish_utils.get_member_list /redfish/v1/Managers/bmc/LogServices/Dump/Entries
135
136 # Restart dump service.
137 BMC Execute Command systemctl restart xyz.openbmc_project.Dump.Manager.service
138 Sleep 10s reason=Wait for BMC dump service to restart properly
139
140 ${dump_entries_after}= redfish_utils.get_member_list /redfish/v1/Managers/bmc/LogServices/Dump/Entries
141 Lists Should Be Equal ${dump_entries_before} ${dump_entries_after}
142
143
144Verify Dump Persistency On BMC Reset
145 [Documentation] Create user dump, reset BMC and verify dump persistency.
146 [Tags] Verify_Dump_Persistency_On_BMC_Reset
147
manashsarma8fa9c892022-10-03 06:20:58 -0500148 # Power off host so that dump is not offloaded to host OS.
149 Redfish Power Off stack_mode=skip
150
Rahul Maheshwaribd38aa42021-08-04 01:24:39 -0500151 Create User Initiated BMC Dump Via Redfish
Rahul Maheshwari2568f872020-09-23 01:03:39 -0500152 ${dump_entries_before}= redfish_utils.get_member_list /redfish/v1/Managers/bmc/LogServices/Dump/Entries
153
154 # Reset BMC.
George Keishing88ec2802021-08-04 03:04:22 -0500155 OBMC Reboot (off) stack_mode=skip
Rahul Maheshwari2568f872020-09-23 01:03:39 -0500156
157 ${dump_entries_after}= redfish_utils.get_member_list /redfish/v1/Managers/bmc/LogServices/Dump/Entries
158 Lists Should Be Equal ${dump_entries_before} ${dump_entries_after}
159
160
Rahul Maheshwari95cbceb2020-10-21 23:25:08 -0500161Delete User Initiated BMC Dump And Verify
162 [Documentation] Delete user initiated BMC dump and verify.
163 [Tags] Delete_User_Initiated_BMC_Dump_And_Verify
164
Rahul Maheshwaribd38aa42021-08-04 01:24:39 -0500165 ${dump_id}= Create User Initiated BMC Dump Via Redfish
Rahul Maheshwari95cbceb2020-10-21 23:25:08 -0500166 Redfish Delete BMC Dump ${dump_id}
167
168 ${dump_entries}= Get BMC Dump Entries
169 Should Be Empty ${dump_entries}
170
171
172Delete All User Initiated BMC Dumps And Verify
173 [Documentation] Delete all user initiated BMC dumps and verify.
174 [Tags] Delete_All_User_Initiated_BMC_Dumps_And_Verify
175
manashsarma70d840e2022-10-18 02:34:45 -0500176 # Power off host so that dump is not offloaded to host OS.
177 Redfish Power Off stack_mode=skip
178
Rahul Maheshwari95cbceb2020-10-21 23:25:08 -0500179 # Create some BMC dump.
Rahul Maheshwaribd38aa42021-08-04 01:24:39 -0500180 Create User Initiated BMC Dump Via Redfish
181 Create User Initiated BMC Dump Via Redfish
Rahul Maheshwari95cbceb2020-10-21 23:25:08 -0500182
183 Redfish Delete All BMC Dumps
184 ${dump_entries}= Get BMC Dump Entries
185 Should Be Empty ${dump_entries}
186
187
Rahul Maheshwari4f338ab2020-10-21 23:28:40 -0500188Create Two User Initiated BMC Dumps
189 [Documentation] Create two user initiated BMC dumps.
190 [Tags] Create_Two_User_Initiated_BMC_Dumps
191
Rahul Maheshwaribd38aa42021-08-04 01:24:39 -0500192 ${dump_id1}= Create User Initiated BMC Dump Via Redfish
193 ${dump_id2}= Create User Initiated BMC Dump Via Redfish
Rahul Maheshwari4f338ab2020-10-21 23:28:40 -0500194
195 ${dump_entries}= Get BMC Dump Entries
196 Length Should Be ${dump_entries} 2
197 Should Contain ${dump_entries} ${dump_id1}
198 Should Contain ${dump_entries} ${dump_id2}
199
200
201Create Two User Initiated BMC Dumps And Delete One
202 [Documentation] Create two dumps and delete the first.
203 [Tags] Create_Two_User_Initiated_BMC_Dumps_And_Delete_One
204
Rahul Maheshwaribd38aa42021-08-04 01:24:39 -0500205 ${dump_id1}= Create User Initiated BMC Dump Via Redfish
206 ${dump_id2}= Create User Initiated BMC Dump Via Redfish
Rahul Maheshwari4f338ab2020-10-21 23:28:40 -0500207
208 Redfish Delete BMC Dump ${dump_id1}
209
210 ${dump_entries}= Get BMC Dump Entries
211 Length Should Be ${dump_entries} 1
212 List Should Contain Value ${dump_entries} ${dump_id2}
213
214
215Create And Delete User Initiated BMC Dump Multiple Times
216 [Documentation] Create and delete user initiated BMC dump multiple times.
217 [Tags] Create_And_Delete_User_Initiated_BMC_Dump_Multiple_Times
218
219 FOR ${INDEX} IN 1 10
Rahul Maheshwaribd38aa42021-08-04 01:24:39 -0500220 ${dump_id}= Create User Initiated BMC Dump Via Redfish
Rahul Maheshwari4f338ab2020-10-21 23:28:40 -0500221 Redfish Delete BMC Dump ${dump_id}
222 END
223
224
Rahul Maheshwarid709c8d2020-11-01 23:03:51 -0600225Verify Maximum BMC Dump Creation
226 [Documentation] Create maximum BMC dump and verify error when dump runs out of space.
227 [Tags] Verify_Maximum_BMC_Dump_Creation
228 [Teardown] Redfish Delete All BMC Dumps
229
230 # Maximum allowed space for dump is 1024 KB. BMC typically hold 8-14 dumps
231 # before running out of this dump space. So trying to create dumps in 20
232 # iterations to run out of space.
manashsarmabd346b32022-07-20 06:22:58 -0500233 # User can key in the Maximum allowed space for bmc dump and how many iteration.
234 FOR ${n} IN RANGE 0 ${MAX_DUMP_COUNT}
Rahul Maheshwaribd38aa42021-08-04 01:24:39 -0500235 Create User Initiated BMC Dump Via Redfish
Rahul Maheshwarid709c8d2020-11-01 23:03:51 -0600236 ${dump_space}= Get Disk Usage For Dumps
Tim Leede2d1302021-06-18 01:24:11 +0800237 Exit For Loop If ${dump_space} >= (${BMC_DUMP_TOTAL_SIZE} - ${BMC_DUMP_MIN_SPACE_REQD})
Rahul Maheshwarid709c8d2020-11-01 23:03:51 -0600238 END
239
240 # Check error while creating dump when dump size is full.
241 ${payload}= Create Dictionary DiagnosticDataType=Manager
242 Redfish.Post /redfish/v1/Managers/bmc/LogServices/Dump/Actions/LogService.CollectDiagnosticData
243 ... body=${payload} valid_status_codes=[${HTTP_INTERNAL_SERVER_ERROR}]
244
245
manashsarma633cac92022-11-25 03:27:03 -0600246Verify BMC Core Dump When Host Powered Off
247 [Documentation] Verify BMC core dump after application crash at host powered off state.
248 [Tags] Verify_BMC_Core_Dump_When_Host_Powered_Off
249
250 Redfish Power Off stack_mode=skip
251
252 # Ensure all dumps are cleaned out.
253 Redfish Delete All BMC Dumps
254 Trigger Core Dump
255
256 # Verify that BMC dump is available.
257 Wait Until Keyword Succeeds 2 min 10 sec Is BMC Dump Available
258
259
manashsarma849189c2022-11-28 04:16:51 -0600260Verify Core Dump Size
261 [Documentation] Verify BMC core dump size is under 20 MB.
262 [Tags] Verify_Core_Dump_Size
263
264 Redfish Power Off stack_mode=skip
265
266 # Ensure all dumps are cleaned out.
267 Redfish Delete All BMC Dumps
268 Trigger Core Dump
269
270 # Verify that BMC dump is available.
271 Wait Until Keyword Succeeds 2 min 10 sec Is BMC Dump Available
272 ${dump_entries}= Get BMC Dump Entries
273 ${resp}= Redfish.Get Properties
274 ... /redfish/v1/Managers/bmc/LogServices/Dump/Entries/${dump_entries[0]}
275
276 # Max size for dump is 20 MB = 20x1024x1024 Byte.
277 Should Be True 0 < ${resp["AdditionalDataSizeBytes"]} < 20971520
278
279
manashsarmafcbfdf62022-12-07 06:36:28 -0600280Verify Error While Initiating BMC Dump During Dumping State
281 [Documentation] Verify error while initiating BMC dump during dumping state.
282 [Tags] Verify_Error_While_Initiating_BMC_Dump_During_Dumping_State
283
Nandish-Mattiaf5bc152023-01-30 00:49:59 -0600284 ${task_id}= Create User Initiated BMC Dump Via Redfish ${1}
manashsarmafcbfdf62022-12-07 06:36:28 -0600285
286 # Check error while initiating BMC dump while dump in progress.
287 ${payload}= Create Dictionary DiagnosticDataType=Manager
288 Redfish.Post
289 ... /redfish/v1/Managers/bmc/LogServices/Dump/Actions/LogService.CollectDiagnosticData
290 ... body=${payload} valid_status_codes=[${HTTP_SERVICE_UNAVAILABLE}]
291
Nandish-Mattiaf5bc152023-01-30 00:49:59 -0600292 # Wait for above initiated dump to complete. Otherwise, on going dump would impact next test.
293 Wait Until Keyword Succeeds 5 min 15 sec Check Task Completion ${task_id}
294
manashsarmafcbfdf62022-12-07 06:36:28 -0600295
manashsarmaaa845b72022-12-12 05:41:32 -0600296Verify BMC Dump Create Errors While Another BMC Dump In Progress
297 [Documentation] Verify BMC dump creation error until older BMC dump completion.
298 [Tags] Verify_BMC_Dump_Create_Errors_While_Another_BMC_Dump_In_Progress
299
300 # Initiate a BMC dump that returns without completion.
301 ${task_id}= Create User Initiated BMC Dump Via Redfish ${1}
302
303 # Now continue to initiate multiple dump request which is not expected to be accepted
304 # till earlier BMC dump task is completed. A limit is set to avoid risk of infinite loop.
305 ${payload}= Create Dictionary DiagnosticDataType=Manager
306 WHILE True limit=1000
307 ${task_dict}= Redfish.Get Properties /redfish/v1/TaskService/Tasks/${task_id}
308 IF '${task_dict['TaskState']}' == 'Completed' BREAK
309 Redfish.Post
310 ... /redfish/v1/Managers/bmc/LogServices/Dump/Actions/LogService.CollectDiagnosticData
311 ... body=${payload} valid_status_codes=[${HTTP_SERVICE_UNAVAILABLE}]
312 END
313
314 # The next BMC dump initiation request should be accepted as earlier dump is completed.
315 ${resp}= Redfish.Post
316 ... /redfish/v1/Managers/bmc/LogServices/Dump/Actions/LogService.CollectDiagnosticData
317 ... body=${payload} valid_status_codes=[${HTTP_ACCEPTED}]
318
319 # Wait for above initiated dump to complete. Otherwise, on going dump would impact next test.
320 Wait Until Keyword Succeeds 5 min 15 sec Check Task Completion ${resp.dict['Id']}
321
322
manashsarmaadabd3b2023-02-08 04:51:32 -0600323Verify Core Dump After Terminating Dump Manager Service
324 [Documentation] Verify initiate core dumps and kill Phosphor-dump-manager.
325 [Tags] Verify_Core_Dump_After_Terminating_Dump_Manager_Service
326
327 Redfish Power Off stack_mode=skip
328
329 # Remove all available dumps in BMC.
330 Redfish Delete All BMC Dumps
331
332 # Find the pid of the phosphor-dump-manage process and kill it.
333 ${cmd_buf}= Catenate kill -s SEGV $(pgrep phosphor-dump-manager)
334 ${cmd_output} ${stderr} ${rc}= BMC Execute Command ${cmd_buf}
335 Should Be Equal As Integers ${rc} ${0}
336
337 # Verify that BMC dump is available.
338 Wait Until Keyword Succeeds 2 min 10 sec Is BMC Dump Available
339
George Keishing4611b812023-02-16 09:44:33 -0600340 # Verifying that there is only one dump.
manashsarmaadabd3b2023-02-08 04:51:32 -0600341 ${dump_entries}= Get BMC Dump Entries
342 ${length}= Get length ${dump_entries}
343 Should Be Equal As Integers ${length} ${1}
344
345
Rahul Maheshwari2568f872020-09-23 01:03:39 -0500346*** Keywords ***
347
Rahul Maheshwaribcefdf22020-10-16 07:51:34 -0500348Get BMC Dump Entries
349 [Documentation] Return BMC dump ids list.
350
351 ${dump_uris}= redfish_utils.get_member_list /redfish/v1/Managers/bmc/LogServices/Dump/Entries
352 ${dump_ids}= Create List
353
354 FOR ${dump_uri} IN @{dump_uris}
355 ${dump_id}= Fetch From Right ${dump_uri} /
356 Append To List ${dump_ids} ${dump_id}
357 END
358
359 [Return] ${dump_ids}
360
361
manashsarma633cac92022-11-25 03:27:03 -0600362Is BMC Dump Available
363 [Documentation] Verify if BMC dump is available.
364
365 ${dump_entries}= Get BMC Dump Entries
366
367 # Verifying that BMC dump is available.
368 ${length}= Get length ${dump_entries}
369 Should Be True 0 < ${length}
370
371
Rahul Maheshwarid709c8d2020-11-01 23:03:51 -0600372Get Disk Usage For Dumps
373 [Documentation] Return disk usage in kilobyte for BMC dumps.
374
George Keishing952bb0f2022-12-12 03:13:35 -0600375 ${usage_output} ${stderr} ${rc}= BMC Execute Command du -s ${BMC_DUMP_COLLECTOR_PATH}
Rahul Maheshwarid709c8d2020-11-01 23:03:51 -0600376
377 # Example of output from above BMC cli command.
378 # $ du -s /var/lib/phosphor-debug-collector/dumps
379 # 516 /var/lib/phosphor-debug-collector/dumps
380
381 ${usage_output}= Fetch From Left ${usage_output} /
382 ${usage_output}= Convert To Integer ${usage_output}
383
384 [return] ${usage_output}
385
386
Rahul Maheshwari2568f872020-09-23 01:03:39 -0500387Test Teardown Execution
388 [Documentation] Do test teardown operation.
389
390 FFDC On Test Case Fail
391 Close All Connections