Rahul Maheshwari | 2568f87 | 2020-09-23 01:03:39 -0500 | [diff] [blame] | 1 | *** Settings *** |
| 2 | |
| 3 | Documentation Test BMC dump functionality of OpenBMC. |
| 4 | |
George Keishing | 88ec280 | 2021-08-04 03:04:22 -0500 | [diff] [blame] | 5 | Resource ../../lib/bmc_redfish_resource.robot |
| 6 | Resource ../../lib/boot_utils.robot |
Rahul Maheshwari | 2568f87 | 2020-09-23 01:03:39 -0500 | [diff] [blame] | 7 | Resource ../../lib/dump_utils.robot |
George Keishing | 88ec280 | 2021-08-04 03:04:22 -0500 | [diff] [blame] | 8 | Resource ../../lib/openbmc_ffdc.robot |
manashsarma | 7e1f2c5 | 2023-02-03 03:18:40 -0600 | [diff] [blame] | 9 | Variables ../../data/pel_variables.py |
Rahul Maheshwari | 2568f87 | 2020-09-23 01:03:39 -0500 | [diff] [blame] | 10 | |
George Keishing | f514ccb | 2021-02-02 23:55:56 -0600 | [diff] [blame] | 11 | Suite Setup Redfish.Login |
Rahul Maheshwari | 2568f87 | 2020-09-23 01:03:39 -0500 | [diff] [blame] | 12 | Test Setup Redfish Delete All BMC Dumps |
| 13 | Test Teardown Test Teardown Execution |
| 14 | |
Tim Lee | de2d130 | 2021-06-18 01:24:11 +0800 | [diff] [blame] | 15 | *** Variables *** |
| 16 | |
| 17 | # Total size of the dump in kilo bytes |
| 18 | ${BMC_DUMP_TOTAL_SIZE} ${1024} |
| 19 | |
| 20 | # Minimum space required for one bmc dump in kilo bytes |
| 21 | ${BMC_DUMP_MIN_SPACE_REQD} ${20} |
manashsarma | bd346b3 | 2022-07-20 06:22:58 -0500 | [diff] [blame] | 22 | ${MAX_DUMP_COUNT} ${20} |
manashsarma | 395f880 | 2022-11-17 03:09:02 -0600 | [diff] [blame] | 23 | ${BMC_DUMP_COLLECTOR_PATH} /var/lib/phosphor-debug-collector/dumps |
Rahul Maheshwari | 2568f87 | 2020-09-23 01:03:39 -0500 | [diff] [blame] | 24 | |
| 25 | *** Test Cases *** |
| 26 | |
manashsarma | 4b5b1fa | 2022-11-16 02:36:58 -0600 | [diff] [blame] | 27 | Verify Error Response For Already Deleted Dump Id |
| 28 | [Documentation] Delete non existing BMC dump and expect an error. |
| 29 | [Tags] Verify_Error_Response_For_Already_Deleted_Dump_Id |
| 30 | |
manashsarma | 9dcabad | 2023-02-02 03:55:17 -0600 | [diff] [blame] | 31 | Redfish Power Off stack_mode=skip |
manashsarma | 4b5b1fa | 2022-11-16 02:36:58 -0600 | [diff] [blame] | 32 | ${dump_id}= Create User Initiated BMC Dump Via Redfish |
manashsarma | 6cc5bcf | 2023-08-22 02:22:48 -0500 | [diff] [blame] | 33 | Wait Until Keyword Succeeds 15 sec 5 sec Redfish Delete BMC Dump ${dump_id} |
manashsarma | 4b5b1fa | 2022-11-16 02:36:58 -0600 | [diff] [blame] | 34 | Run Keyword And Expect Error ValueError: * Redfish Delete BMC Dump ${dump_id} |
| 35 | |
| 36 | |
Rahul Maheshwari | 2568f87 | 2020-09-23 01:03:39 -0500 | [diff] [blame] | 37 | Verify User Initiated BMC Dump When Host Powered Off |
| 38 | [Documentation] Create user initiated BMC dump at host off state and |
| 39 | ... verify dump entry for it. |
| 40 | [Tags] Verify_User_Initiated_BMC_Dump_When_Host_Powered_Off |
| 41 | |
| 42 | Redfish Power Off stack_mode=skip |
Rahul Maheshwari | bd38aa4 | 2021-08-04 01:24:39 -0500 | [diff] [blame] | 43 | ${dump_id}= Create User Initiated BMC Dump Via Redfish |
Rahul Maheshwari | bcefdf2 | 2020-10-16 07:51:34 -0500 | [diff] [blame] | 44 | ${dump_entries}= Get BMC Dump Entries |
Rahul Maheshwari | 2568f87 | 2020-09-23 01:03:39 -0500 | [diff] [blame] | 45 | Length Should Be ${dump_entries} 1 |
| 46 | List Should Contain Value ${dump_entries} ${dump_id} |
| 47 | |
Rahul Maheshwari | 2568f87 | 2020-09-23 01:03:39 -0500 | [diff] [blame] | 48 | |
Rahul Maheshwari | e8e9d0c | 2020-10-30 02:39:01 -0500 | [diff] [blame] | 49 | Verify User Initiated BMC Dump Size |
manashsarma | 7831dc2 | 2022-03-08 01:49:36 -0600 | [diff] [blame] | 50 | [Documentation] Verify user initiated BMC dump size is under 20 MB. |
Rahul Maheshwari | e8e9d0c | 2020-10-30 02:39:01 -0500 | [diff] [blame] | 51 | [Tags] Verify_User_Initiated_BMC_Dump_Size |
| 52 | |
manashsarma | e52174b | 2023-04-03 01:26:56 -0500 | [diff] [blame] | 53 | Redfish Power Off stack_mode=skip |
Rahul Maheshwari | bd38aa4 | 2021-08-04 01:24:39 -0500 | [diff] [blame] | 54 | ${dump_id}= Create User Initiated BMC Dump Via Redfish |
ganesanb | 4d43028 | 2023-04-27 14:33:23 +0000 | [diff] [blame] | 55 | ${resp}= Redfish.Get Properties /redfish/v1/Managers/${MANAGER_ID}/LogServices/Dump/Entries/${dump_id} |
Rahul Maheshwari | e8e9d0c | 2020-10-30 02:39:01 -0500 | [diff] [blame] | 56 | |
| 57 | # Example of response from above Redfish GET request. |
| 58 | # "@odata.type": "#LogEntry.v1_7_0.LogEntry", |
| 59 | # "AdditionalDataSizeBytes": 31644, |
ganesanb | 4d43028 | 2023-04-27 14:33:23 +0000 | [diff] [blame] | 60 | # "AdditionalDataURI": "/redfish/v1/Managers/${MANAGER_ID}/LogServices/Dump/attachment/9", |
Rahul Maheshwari | e8e9d0c | 2020-10-30 02:39:01 -0500 | [diff] [blame] | 61 | # "Created": "2020-10-23T06:32:53+00:00", |
| 62 | # "DiagnosticDataType": "Manager", |
| 63 | # "EntryType": "Event", |
| 64 | # "Id": "9", |
| 65 | # "Name": "BMC Dump Entry" |
| 66 | |
manashsarma | 7831dc2 | 2022-03-08 01:49:36 -0600 | [diff] [blame] | 67 | # Max size for dump is 20 MB = 20x1024x1024 Byte. |
| 68 | Should Be True 0 < ${resp["AdditionalDataSizeBytes"]} < 20971520 |
Rahul Maheshwari | e8e9d0c | 2020-10-30 02:39:01 -0500 | [diff] [blame] | 69 | |
| 70 | |
manashsarma | 7e1f2c5 | 2023-02-03 03:18:40 -0600 | [diff] [blame] | 71 | Verify Internal Failure Initiated BMC Dump Size |
| 72 | [Documentation] Verify that the internal failure initiated BMC dump size is under 20 MB. |
| 73 | [Tags] Verify_Internal_Failure_Initiated_BMC_Dump_Size |
| 74 | |
| 75 | Redfish Delete All BMC Dumps |
| 76 | |
| 77 | # Create an internal failure error log. |
| 78 | BMC Execute Command ${CMD_INTERNAL_FAILURE} |
| 79 | |
| 80 | # Wait for BMC dump to get generated after injecting internal failure. |
| 81 | Wait Until Keyword Succeeds 2 min 10 sec Is BMC Dump Available |
| 82 | |
| 83 | # Verify that only one BMC dump is generated after injecting error. |
| 84 | ${dump_entries}= Get BMC Dump Entries |
| 85 | ${length}= Get length ${dump_entries} |
| 86 | Should Be Equal As Integers ${length} ${1} |
| 87 | |
| 88 | # Max size for dump is 20 MB = 20x1024x1024 Byte. |
| 89 | ${resp}= Redfish.Get Properties |
ganesanb | 4d43028 | 2023-04-27 14:33:23 +0000 | [diff] [blame] | 90 | ... /redfish/v1/Managers/${MANAGER_ID}/LogServices/Dump/Entries/${dump_entries[0]} |
manashsarma | 7e1f2c5 | 2023-02-03 03:18:40 -0600 | [diff] [blame] | 91 | Should Be True 0 < ${resp["AdditionalDataSizeBytes"]} < 20971520 |
| 92 | |
| 93 | |
manashsarma | bbc68bc | 2022-12-05 09:55:17 -0600 | [diff] [blame] | 94 | Verify Multiple BMC Dump Creation |
| 95 | [Documentation] Verify that multiple BMC dumps can be created one after |
| 96 | ... another successfully. |
| 97 | [Tags] Verify_Multiple_BMC_Dump_Creation |
| 98 | |
manashsarma | e52174b | 2023-04-03 01:26:56 -0500 | [diff] [blame] | 99 | Redfish Power Off stack_mode=skip |
manashsarma | bbc68bc | 2022-12-05 09:55:17 -0600 | [diff] [blame] | 100 | ${dump_count}= Evaluate random.randint(5, 10) modules=random |
| 101 | FOR ${INDEX} IN 1 ${dump_count} |
| 102 | Create User Initiated BMC Dump Via Redfish |
| 103 | END |
| 104 | |
| 105 | |
manashsarma | 395f880 | 2022-11-17 03:09:02 -0600 | [diff] [blame] | 106 | Verify BMC Dump Default Location In BMC |
| 107 | [Documentation] Verify that BMC dump is created in its default location of BMC. |
George Keishing | 224e4c7 | 2022-12-14 05:07:58 -0600 | [diff] [blame] | 108 | [Tags] Verify_BMC_Dump_Default_Location_In_BMC |
manashsarma | 395f880 | 2022-11-17 03:09:02 -0600 | [diff] [blame] | 109 | |
manashsarma | e52174b | 2023-04-03 01:26:56 -0500 | [diff] [blame] | 110 | Redfish Power Off stack_mode=skip |
manashsarma | 395f880 | 2022-11-17 03:09:02 -0600 | [diff] [blame] | 111 | Redfish Delete All BMC Dumps |
| 112 | ${dump_id}= Create User Initiated BMC Dump Via Redfish |
| 113 | ${dump_file} ${stderr} ${rc}= BMC Execute Command |
| 114 | ... ls ${BMC_DUMP_COLLECTOR_PATH}/${dump_id} |
| 115 | Should Be True ${rc} == 0 |
manashsarma | c30e02a | 2023-02-20 06:46:15 -0600 | [diff] [blame] | 116 | Should Contain Any ${dump_file} BMCDUMP obmcdump |
manashsarma | 395f880 | 2022-11-17 03:09:02 -0600 | [diff] [blame] | 117 | |
| 118 | |
George Keishing | 3d85098 | 2022-07-19 11:03:03 -0500 | [diff] [blame] | 119 | Verify User Initiated BMC Dump When Host Booted |
| 120 | [Documentation] Create user initiated BMC dump at host booted state and |
| 121 | ... verify dump entry for it. |
| 122 | [Tags] Verify_User_Initiated_BMC_Dump_When_Host_Booted |
| 123 | |
| 124 | Redfish Power On stack_mode=skip |
| 125 | ${dump_id}= Create User Initiated BMC Dump Via Redfish |
| 126 | ${dump_entries}= Get BMC Dump Entries |
| 127 | Length Should Be ${dump_entries} 1 |
| 128 | List Should Contain Value ${dump_entries} ${dump_id} |
| 129 | |
| 130 | |
manashsarma | 4b5d6ac | 2023-02-08 00:04:03 -0600 | [diff] [blame] | 131 | Verify User Initiated BMC Dump At Host Booting |
| 132 | [Documentation] Create and verify user initiated BMC dump during Host is powwering on |
| 133 | ... or when host booting is in progress. |
| 134 | [Tags] Verify_User_Initiated_BMC_Dump_At_Host_Booting |
| 135 | |
manashsarma | 9712ac5 | 2023-03-09 04:54:41 -0600 | [diff] [blame] | 136 | Redfish Power Off stack_mode=skip |
manashsarma | 4b5d6ac | 2023-02-08 00:04:03 -0600 | [diff] [blame] | 137 | Redfish Delete All BMC Dumps |
| 138 | |
| 139 | # Initiate power on. |
| 140 | Redfish Power Operation On |
| 141 | Wait Until Keyword Succeeds 2 min 5 sec Is Boot Progress Changed |
| 142 | |
| 143 | # Create user initiated BMC dump and verify only one dump is available. |
| 144 | Create User Initiated BMC Dump Via Redfish |
| 145 | ${dump_entries}= Get BMC Dump Entries |
| 146 | Length Should Be ${dump_entries} 1 |
| 147 | |
| 148 | |
Rahul Maheshwari | 2568f87 | 2020-09-23 01:03:39 -0500 | [diff] [blame] | 149 | Verify Dump Persistency On Dump Service Restart |
| 150 | [Documentation] Create user dump, restart dump manager service and verify dump |
| 151 | ... persistency. |
| 152 | [Tags] Verify_Dump_Persistency_On_Dump_Service_Restart |
| 153 | |
manashsarma | e52174b | 2023-04-03 01:26:56 -0500 | [diff] [blame] | 154 | Redfish Power Off stack_mode=skip |
Rahul Maheshwari | bd38aa4 | 2021-08-04 01:24:39 -0500 | [diff] [blame] | 155 | Create User Initiated BMC Dump Via Redfish |
ganesanb | 4d43028 | 2023-04-27 14:33:23 +0000 | [diff] [blame] | 156 | ${dump_entries_before}= redfish_utils.get_member_list /redfish/v1/Managers/${MANAGER_ID}/LogServices/Dump/Entries |
Rahul Maheshwari | 2568f87 | 2020-09-23 01:03:39 -0500 | [diff] [blame] | 157 | |
| 158 | # Restart dump service. |
| 159 | BMC Execute Command systemctl restart xyz.openbmc_project.Dump.Manager.service |
| 160 | Sleep 10s reason=Wait for BMC dump service to restart properly |
| 161 | |
ganesanb | 4d43028 | 2023-04-27 14:33:23 +0000 | [diff] [blame] | 162 | ${dump_entries_after}= redfish_utils.get_member_list /redfish/v1/Managers/${MANAGER_ID}/LogServices/Dump/Entries |
Rahul Maheshwari | 2568f87 | 2020-09-23 01:03:39 -0500 | [diff] [blame] | 163 | Lists Should Be Equal ${dump_entries_before} ${dump_entries_after} |
| 164 | |
| 165 | |
| 166 | Verify Dump Persistency On BMC Reset |
| 167 | [Documentation] Create user dump, reset BMC and verify dump persistency. |
| 168 | [Tags] Verify_Dump_Persistency_On_BMC_Reset |
| 169 | |
manashsarma | 8fa9c89 | 2022-10-03 06:20:58 -0500 | [diff] [blame] | 170 | # Power off host so that dump is not offloaded to host OS. |
| 171 | Redfish Power Off stack_mode=skip |
| 172 | |
Rahul Maheshwari | bd38aa4 | 2021-08-04 01:24:39 -0500 | [diff] [blame] | 173 | Create User Initiated BMC Dump Via Redfish |
ganesanb | 4d43028 | 2023-04-27 14:33:23 +0000 | [diff] [blame] | 174 | ${dump_entries_before}= redfish_utils.get_member_list /redfish/v1/Managers/${MANAGER_ID}/LogServices/Dump/Entries |
Rahul Maheshwari | 2568f87 | 2020-09-23 01:03:39 -0500 | [diff] [blame] | 175 | |
| 176 | # Reset BMC. |
George Keishing | 88ec280 | 2021-08-04 03:04:22 -0500 | [diff] [blame] | 177 | OBMC Reboot (off) stack_mode=skip |
Rahul Maheshwari | 2568f87 | 2020-09-23 01:03:39 -0500 | [diff] [blame] | 178 | |
ganesanb | 4d43028 | 2023-04-27 14:33:23 +0000 | [diff] [blame] | 179 | ${dump_entries_after}= redfish_utils.get_member_list /redfish/v1/Managers/${MANAGER_ID}/LogServices/Dump/Entries |
Rahul Maheshwari | 2568f87 | 2020-09-23 01:03:39 -0500 | [diff] [blame] | 180 | Lists Should Be Equal ${dump_entries_before} ${dump_entries_after} |
| 181 | |
| 182 | |
Rahul Maheshwari | 95cbceb | 2020-10-21 23:25:08 -0500 | [diff] [blame] | 183 | Delete User Initiated BMC Dump And Verify |
| 184 | [Documentation] Delete user initiated BMC dump and verify. |
| 185 | [Tags] Delete_User_Initiated_BMC_Dump_And_Verify |
| 186 | |
manashsarma | e52174b | 2023-04-03 01:26:56 -0500 | [diff] [blame] | 187 | Redfish Power Off stack_mode=skip |
Rahul Maheshwari | bd38aa4 | 2021-08-04 01:24:39 -0500 | [diff] [blame] | 188 | ${dump_id}= Create User Initiated BMC Dump Via Redfish |
manashsarma | 6cc5bcf | 2023-08-22 02:22:48 -0500 | [diff] [blame] | 189 | Wait Until Keyword Succeeds 15 sec 5 sec Redfish Delete BMC Dump ${dump_id} |
Rahul Maheshwari | 95cbceb | 2020-10-21 23:25:08 -0500 | [diff] [blame] | 190 | |
| 191 | ${dump_entries}= Get BMC Dump Entries |
| 192 | Should Be Empty ${dump_entries} |
| 193 | |
| 194 | |
| 195 | Delete All User Initiated BMC Dumps And Verify |
| 196 | [Documentation] Delete all user initiated BMC dumps and verify. |
| 197 | [Tags] Delete_All_User_Initiated_BMC_Dumps_And_Verify |
| 198 | |
manashsarma | 70d840e | 2022-10-18 02:34:45 -0500 | [diff] [blame] | 199 | # Power off host so that dump is not offloaded to host OS. |
| 200 | Redfish Power Off stack_mode=skip |
| 201 | |
Rahul Maheshwari | 95cbceb | 2020-10-21 23:25:08 -0500 | [diff] [blame] | 202 | # Create some BMC dump. |
Rahul Maheshwari | bd38aa4 | 2021-08-04 01:24:39 -0500 | [diff] [blame] | 203 | Create User Initiated BMC Dump Via Redfish |
| 204 | Create User Initiated BMC Dump Via Redfish |
Rahul Maheshwari | 95cbceb | 2020-10-21 23:25:08 -0500 | [diff] [blame] | 205 | |
| 206 | Redfish Delete All BMC Dumps |
| 207 | ${dump_entries}= Get BMC Dump Entries |
| 208 | Should Be Empty ${dump_entries} |
| 209 | |
| 210 | |
Rahul Maheshwari | 4f338ab | 2020-10-21 23:28:40 -0500 | [diff] [blame] | 211 | Create Two User Initiated BMC Dumps |
| 212 | [Documentation] Create two user initiated BMC dumps. |
| 213 | [Tags] Create_Two_User_Initiated_BMC_Dumps |
| 214 | |
manashsarma | e52174b | 2023-04-03 01:26:56 -0500 | [diff] [blame] | 215 | Redfish Power Off stack_mode=skip |
Rahul Maheshwari | bd38aa4 | 2021-08-04 01:24:39 -0500 | [diff] [blame] | 216 | ${dump_id1}= Create User Initiated BMC Dump Via Redfish |
| 217 | ${dump_id2}= Create User Initiated BMC Dump Via Redfish |
Rahul Maheshwari | 4f338ab | 2020-10-21 23:28:40 -0500 | [diff] [blame] | 218 | |
| 219 | ${dump_entries}= Get BMC Dump Entries |
| 220 | Length Should Be ${dump_entries} 2 |
| 221 | Should Contain ${dump_entries} ${dump_id1} |
| 222 | Should Contain ${dump_entries} ${dump_id2} |
| 223 | |
| 224 | |
| 225 | Create Two User Initiated BMC Dumps And Delete One |
| 226 | [Documentation] Create two dumps and delete the first. |
| 227 | [Tags] Create_Two_User_Initiated_BMC_Dumps_And_Delete_One |
| 228 | |
manashsarma | e52174b | 2023-04-03 01:26:56 -0500 | [diff] [blame] | 229 | Redfish Power Off stack_mode=skip |
Rahul Maheshwari | bd38aa4 | 2021-08-04 01:24:39 -0500 | [diff] [blame] | 230 | ${dump_id1}= Create User Initiated BMC Dump Via Redfish |
| 231 | ${dump_id2}= Create User Initiated BMC Dump Via Redfish |
Rahul Maheshwari | 4f338ab | 2020-10-21 23:28:40 -0500 | [diff] [blame] | 232 | |
manashsarma | 6cc5bcf | 2023-08-22 02:22:48 -0500 | [diff] [blame] | 233 | Wait Until Keyword Succeeds 15 sec 5 sec Redfish Delete BMC Dump ${dump_id1} |
Rahul Maheshwari | 4f338ab | 2020-10-21 23:28:40 -0500 | [diff] [blame] | 234 | |
| 235 | ${dump_entries}= Get BMC Dump Entries |
| 236 | Length Should Be ${dump_entries} 1 |
| 237 | List Should Contain Value ${dump_entries} ${dump_id2} |
| 238 | |
| 239 | |
| 240 | Create And Delete User Initiated BMC Dump Multiple Times |
| 241 | [Documentation] Create and delete user initiated BMC dump multiple times. |
| 242 | [Tags] Create_And_Delete_User_Initiated_BMC_Dump_Multiple_Times |
| 243 | |
manashsarma | e52174b | 2023-04-03 01:26:56 -0500 | [diff] [blame] | 244 | Redfish Power Off stack_mode=skip |
Rahul Maheshwari | 4f338ab | 2020-10-21 23:28:40 -0500 | [diff] [blame] | 245 | FOR ${INDEX} IN 1 10 |
Rahul Maheshwari | bd38aa4 | 2021-08-04 01:24:39 -0500 | [diff] [blame] | 246 | ${dump_id}= Create User Initiated BMC Dump Via Redfish |
manashsarma | 6cc5bcf | 2023-08-22 02:22:48 -0500 | [diff] [blame] | 247 | Wait Until Keyword Succeeds 15 sec 5 sec Redfish Delete BMC Dump ${dump_id} |
Rahul Maheshwari | 4f338ab | 2020-10-21 23:28:40 -0500 | [diff] [blame] | 248 | END |
| 249 | |
| 250 | |
Rahul Maheshwari | d709c8d | 2020-11-01 23:03:51 -0600 | [diff] [blame] | 251 | Verify Maximum BMC Dump Creation |
| 252 | [Documentation] Create maximum BMC dump and verify error when dump runs out of space. |
| 253 | [Tags] Verify_Maximum_BMC_Dump_Creation |
| 254 | [Teardown] Redfish Delete All BMC Dumps |
| 255 | |
| 256 | # Maximum allowed space for dump is 1024 KB. BMC typically hold 8-14 dumps |
| 257 | # before running out of this dump space. So trying to create dumps in 20 |
| 258 | # iterations to run out of space. |
manashsarma | bd346b3 | 2022-07-20 06:22:58 -0500 | [diff] [blame] | 259 | # User can key in the Maximum allowed space for bmc dump and how many iteration. |
| 260 | FOR ${n} IN RANGE 0 ${MAX_DUMP_COUNT} |
Rahul Maheshwari | bd38aa4 | 2021-08-04 01:24:39 -0500 | [diff] [blame] | 261 | Create User Initiated BMC Dump Via Redfish |
Rahul Maheshwari | d709c8d | 2020-11-01 23:03:51 -0600 | [diff] [blame] | 262 | ${dump_space}= Get Disk Usage For Dumps |
Tim Lee | de2d130 | 2021-06-18 01:24:11 +0800 | [diff] [blame] | 263 | Exit For Loop If ${dump_space} >= (${BMC_DUMP_TOTAL_SIZE} - ${BMC_DUMP_MIN_SPACE_REQD}) |
Rahul Maheshwari | d709c8d | 2020-11-01 23:03:51 -0600 | [diff] [blame] | 264 | END |
| 265 | |
| 266 | # Check error while creating dump when dump size is full. |
| 267 | ${payload}= Create Dictionary DiagnosticDataType=Manager |
ganesanb | 4d43028 | 2023-04-27 14:33:23 +0000 | [diff] [blame] | 268 | Redfish.Post /redfish/v1/Managers/${MANAGER_ID}/LogServices/Dump/Actions/LogService.CollectDiagnosticData |
Rahul Maheshwari | d709c8d | 2020-11-01 23:03:51 -0600 | [diff] [blame] | 269 | ... body=${payload} valid_status_codes=[${HTTP_INTERNAL_SERVER_ERROR}] |
| 270 | |
| 271 | |
manashsarma | 633cac9 | 2022-11-25 03:27:03 -0600 | [diff] [blame] | 272 | Verify BMC Core Dump When Host Powered Off |
| 273 | [Documentation] Verify BMC core dump after application crash at host powered off state. |
| 274 | [Tags] Verify_BMC_Core_Dump_When_Host_Powered_Off |
| 275 | |
| 276 | Redfish Power Off stack_mode=skip |
| 277 | |
| 278 | # Ensure all dumps are cleaned out. |
| 279 | Redfish Delete All BMC Dumps |
| 280 | Trigger Core Dump |
| 281 | |
| 282 | # Verify that BMC dump is available. |
| 283 | Wait Until Keyword Succeeds 2 min 10 sec Is BMC Dump Available |
| 284 | |
| 285 | |
manashsarma | 849189c | 2022-11-28 04:16:51 -0600 | [diff] [blame] | 286 | Verify Core Dump Size |
| 287 | [Documentation] Verify BMC core dump size is under 20 MB. |
| 288 | [Tags] Verify_Core_Dump_Size |
| 289 | |
| 290 | Redfish Power Off stack_mode=skip |
| 291 | |
| 292 | # Ensure all dumps are cleaned out. |
| 293 | Redfish Delete All BMC Dumps |
| 294 | Trigger Core Dump |
| 295 | |
| 296 | # Verify that BMC dump is available. |
| 297 | Wait Until Keyword Succeeds 2 min 10 sec Is BMC Dump Available |
| 298 | ${dump_entries}= Get BMC Dump Entries |
| 299 | ${resp}= Redfish.Get Properties |
ganesanb | 4d43028 | 2023-04-27 14:33:23 +0000 | [diff] [blame] | 300 | ... /redfish/v1/Managers/${MANAGER_ID}/LogServices/Dump/Entries/${dump_entries[0]} |
manashsarma | 849189c | 2022-11-28 04:16:51 -0600 | [diff] [blame] | 301 | |
| 302 | # Max size for dump is 20 MB = 20x1024x1024 Byte. |
| 303 | Should Be True 0 < ${resp["AdditionalDataSizeBytes"]} < 20971520 |
| 304 | |
| 305 | |
manashsarma | fcbfdf6 | 2022-12-07 06:36:28 -0600 | [diff] [blame] | 306 | Verify Error While Initiating BMC Dump During Dumping State |
| 307 | [Documentation] Verify error while initiating BMC dump during dumping state. |
| 308 | [Tags] Verify_Error_While_Initiating_BMC_Dump_During_Dumping_State |
| 309 | |
manashsarma | e52174b | 2023-04-03 01:26:56 -0500 | [diff] [blame] | 310 | Redfish Power Off stack_mode=skip |
Nandish-Matti | af5bc15 | 2023-01-30 00:49:59 -0600 | [diff] [blame] | 311 | ${task_id}= Create User Initiated BMC Dump Via Redfish ${1} |
manashsarma | fcbfdf6 | 2022-12-07 06:36:28 -0600 | [diff] [blame] | 312 | |
| 313 | # Check error while initiating BMC dump while dump in progress. |
| 314 | ${payload}= Create Dictionary DiagnosticDataType=Manager |
| 315 | Redfish.Post |
ganesanb | 4d43028 | 2023-04-27 14:33:23 +0000 | [diff] [blame] | 316 | ... /redfish/v1/Managers/${MANAGER_ID}/LogServices/Dump/Actions/LogService.CollectDiagnosticData |
manashsarma | fcbfdf6 | 2022-12-07 06:36:28 -0600 | [diff] [blame] | 317 | ... body=${payload} valid_status_codes=[${HTTP_SERVICE_UNAVAILABLE}] |
| 318 | |
Nandish-Matti | af5bc15 | 2023-01-30 00:49:59 -0600 | [diff] [blame] | 319 | # Wait for above initiated dump to complete. Otherwise, on going dump would impact next test. |
| 320 | Wait Until Keyword Succeeds 5 min 15 sec Check Task Completion ${task_id} |
| 321 | |
manashsarma | fcbfdf6 | 2022-12-07 06:36:28 -0600 | [diff] [blame] | 322 | |
manashsarma | aa845b7 | 2022-12-12 05:41:32 -0600 | [diff] [blame] | 323 | Verify BMC Dump Create Errors While Another BMC Dump In Progress |
| 324 | [Documentation] Verify BMC dump creation error until older BMC dump completion. |
| 325 | [Tags] Verify_BMC_Dump_Create_Errors_While_Another_BMC_Dump_In_Progress |
| 326 | |
manashsarma | e52174b | 2023-04-03 01:26:56 -0500 | [diff] [blame] | 327 | Redfish Power Off stack_mode=skip |
| 328 | |
manashsarma | aa845b7 | 2022-12-12 05:41:32 -0600 | [diff] [blame] | 329 | # Initiate a BMC dump that returns without completion. |
| 330 | ${task_id}= Create User Initiated BMC Dump Via Redfish ${1} |
| 331 | |
| 332 | # Now continue to initiate multiple dump request which is not expected to be accepted |
| 333 | # till earlier BMC dump task is completed. A limit is set to avoid risk of infinite loop. |
| 334 | ${payload}= Create Dictionary DiagnosticDataType=Manager |
| 335 | WHILE True limit=1000 |
| 336 | ${task_dict}= Redfish.Get Properties /redfish/v1/TaskService/Tasks/${task_id} |
| 337 | IF '${task_dict['TaskState']}' == 'Completed' BREAK |
manashsarma | 785244c | 2023-03-15 05:56:18 -0500 | [diff] [blame] | 338 | ${resp}= Redfish.Post |
ganesanb | 4d43028 | 2023-04-27 14:33:23 +0000 | [diff] [blame] | 339 | ... /redfish/v1/Managers/${MANAGER_ID}/LogServices/Dump/Actions/LogService.CollectDiagnosticData |
manashsarma | 785244c | 2023-03-15 05:56:18 -0500 | [diff] [blame] | 340 | ... body=${payload} valid_status_codes=[${HTTP_SERVICE_UNAVAILABLE}, ${HTTP_ACCEPTED}] |
| 341 | |
| 342 | # Sample response of above POST request: |
| 343 | # 503 |
| 344 | # Strict-Transport-Security max-age=31536000; includeSubdomains; preload |
| 345 | # X-Frame-Options DENY |
| 346 | # Pragma no-cache |
| 347 | # Cache-Control no-Store,no-Cache |
| 348 | # X-XSS-Protection 1; mode=block |
| 349 | # X-Content-Type-Options nosniff |
| 350 | # Content-Security-Policy default-src 'none'; img-src 'self' data:; font-src 'self'; style-src |
| 351 | # 'self'; script-src 'self'; connect-src 'self' wss:; form-action 'none'; frame-ancestors 'none'; |
| 352 | # object-src 'none'; base-uri 'none' |
| 353 | # Content-Type application/json |
| 354 | # Date Thu, 16 Mar 2023 06:41:06 GMT |
| 355 | # Content-Length 573 |
| 356 | # { |
| 357 | # "error": { |
| 358 | # "@Message.ExtendedInfo": [ |
| 359 | # { |
| 360 | # "@odata.type": "#Message.v1_1_1.Message", |
| 361 | # "Message": "The request could not be performed because the resource is in standby.", |
| 362 | # "MessageArgs": [], |
| 363 | # "MessageId": "Base.1.8.1.ResourceInStandby", |
| 364 | # "MessageSeverity": "Critical", |
| 365 | # "Resolution": "Ensure that the resource is in the correct power state and resubmit the request." |
| 366 | # } |
| 367 | # ], |
| 368 | # "code": "Base.1.8.1.ResourceInStandby", |
| 369 | # "message": "The request could not be performed because the resource is in standby." |
| 370 | # } |
| 371 | # } |
| 372 | |
| 373 | # At this point the request HTTPS response should be HTTP_SERVICE_UNAVAILABLE. However, if the |
| 374 | # response contains the HTTPS response code HTTP_ACCEPTED, it means there is a dump initiation |
| 375 | # request which is completed. We verify this with below check. |
| 376 | ${resp}= Convert To String ${resp} |
| 377 | ${contains}= Run Keyword And Return Status Should Contain ${resp.split()[0]} ${HTTP_ACCEPTED} |
| 378 | IF ${contains} |
| 379 | ${task_dict}= Redfish.Get Properties /redfish/v1/TaskService/Tasks/${task_id} |
| 380 | Should Be True '${task_dict['TaskState']}' == 'Completed' |
| 381 | END |
manashsarma | aa845b7 | 2022-12-12 05:41:32 -0600 | [diff] [blame] | 382 | END |
| 383 | |
| 384 | # The next BMC dump initiation request should be accepted as earlier dump is completed. |
manashsarma | 0a47135 | 2023-04-06 06:06:04 -0500 | [diff] [blame] | 385 | # Wait for few seconds before initiating the dump. |
| 386 | Sleep 2s |
manashsarma | aa845b7 | 2022-12-12 05:41:32 -0600 | [diff] [blame] | 387 | ${resp}= Redfish.Post |
ganesanb | 4d43028 | 2023-04-27 14:33:23 +0000 | [diff] [blame] | 388 | ... /redfish/v1/Managers/${MANAGER_ID}/LogServices/Dump/Actions/LogService.CollectDiagnosticData |
manashsarma | aa845b7 | 2022-12-12 05:41:32 -0600 | [diff] [blame] | 389 | ... body=${payload} valid_status_codes=[${HTTP_ACCEPTED}] |
| 390 | |
| 391 | # Wait for above initiated dump to complete. Otherwise, on going dump would impact next test. |
| 392 | Wait Until Keyword Succeeds 5 min 15 sec Check Task Completion ${resp.dict['Id']} |
| 393 | |
| 394 | |
manashsarma | adabd3b | 2023-02-08 04:51:32 -0600 | [diff] [blame] | 395 | Verify Core Dump After Terminating Dump Manager Service |
| 396 | [Documentation] Verify initiate core dumps and kill Phosphor-dump-manager. |
| 397 | [Tags] Verify_Core_Dump_After_Terminating_Dump_Manager_Service |
| 398 | |
| 399 | Redfish Power Off stack_mode=skip |
| 400 | |
| 401 | # Remove all available dumps in BMC. |
| 402 | Redfish Delete All BMC Dumps |
| 403 | |
| 404 | # Find the pid of the phosphor-dump-manage process and kill it. |
| 405 | ${cmd_buf}= Catenate kill -s SEGV $(pgrep phosphor-dump-manager) |
| 406 | ${cmd_output} ${stderr} ${rc}= BMC Execute Command ${cmd_buf} |
| 407 | Should Be Equal As Integers ${rc} ${0} |
| 408 | |
| 409 | # Verify that BMC dump is available. |
| 410 | Wait Until Keyword Succeeds 2 min 10 sec Is BMC Dump Available |
| 411 | |
George Keishing | 4611b81 | 2023-02-16 09:44:33 -0600 | [diff] [blame] | 412 | # Verifying that there is only one dump. |
manashsarma | adabd3b | 2023-02-08 04:51:32 -0600 | [diff] [blame] | 413 | ${dump_entries}= Get BMC Dump Entries |
| 414 | ${length}= Get length ${dump_entries} |
| 415 | Should Be Equal As Integers ${length} ${1} |
| 416 | |
| 417 | |
manashsarma | 7bf47d2 | 2023-02-08 01:10:18 -0600 | [diff] [blame] | 418 | Verify Error Log And Dump For Internal Failure |
| 419 | [Documentation] Verify error log and dump for internal failure. |
| 420 | [Tags] Verify_Error_Log_And_Dump_For_Internal_Failure |
| 421 | |
| 422 | Redfish Purge Event Log |
| 423 | Redfish Delete All BMC Dumps |
| 424 | |
| 425 | # Create an internal failure error log. |
| 426 | BMC Execute Command ${CMD_INTERNAL_FAILURE} |
| 427 | |
| 428 | # With internal failure, an error log file is generated. Check if |
| 429 | # BMC has only one error log for this internal failure. |
| 430 | ${resp}= Redfish.Get /redfish/v1/Systems/system/LogServices/CELog/Entries |
| 431 | Should Be True ${resp.dict["Members@odata.count"]} == ${1} |
| 432 | |
| 433 | # Wait for the BMC dump to become available and verify its presence. |
| 434 | Wait Until Keyword Succeeds 2 min 10 sec Is BMC Dump Available |
| 435 | ${dump_entries}= Get BMC Dump Entries |
| 436 | ${length}= Get length ${dump_entries} |
| 437 | Should Be Equal As Integers ${length} ${1} |
| 438 | |
| 439 | |
manashsarma | 3524741 | 2023-02-06 02:14:04 -0600 | [diff] [blame] | 440 | Verify Core Watchdog Initiated BMC Dump |
| 441 | [Documentation] Verify core watchdog timeout initiated BMC dump. |
| 442 | [Tags] Verify_Core_Watchdog_Initiated_BMC_Dump |
| 443 | |
| 444 | Redfish Delete All BMC Dumps |
| 445 | Redfish Power Off stack_mode=skip |
| 446 | |
| 447 | # Trigger watchdog timeout. |
| 448 | Redfish Initiate Auto Reboot 2000 |
| 449 | |
| 450 | # Wait for BMC dump to get generated after injecting watchdog timeout. |
| 451 | Wait Until Keyword Succeeds 4 min 20 sec Is BMC Dump Available |
| 452 | |
| 453 | # Verify that only one BMC dump is available. |
| 454 | ${dump_entry_list}= Get BMC Dump Entries |
| 455 | ${length}= Get length ${dump_entry_list} |
| 456 | Should Be Equal As Integers ${length} ${1} |
| 457 | |
| 458 | |
manashsarma | 1202ffe | 2023-09-22 09:01:18 -0500 | [diff] [blame^] | 459 | Verify User Initiated BMC Dump Type |
| 460 | [Documentation] Download user initiate BMC dump and validates its type. |
| 461 | [Tags] Verify_User_Initiated_BMC_Dump_Type |
| 462 | |
| 463 | ${dump_id}= Create User Initiated BMC Dump Via Redfish |
| 464 | |
| 465 | # Download BMC dump and verify its size. |
| 466 | ${resp}= Redfish.Get /redfish/v1/Managers/bmc/LogServices/Dump/Entries/${dump_id} |
| 467 | ${redfish_bmc_dump_size}= Set Variable ${resp.dict["AdditionalDataSizeBytes"]} |
| 468 | ${redfish_dump_creation_timestamp}= Set Variable ${resp.dict["Created"]} |
| 469 | |
| 470 | Initialize OpenBMC |
| 471 | ${headers}= Create Dictionary Content-Type=application/octet-stream X-Auth-Token=${XAUTH_TOKEN} |
| 472 | |
| 473 | ${ret}= Get Request openbmc /redfish/v1/Managers/bmc/LogServices/Dump/Entries/${dump_id}/attachment headers=${headers} |
| 474 | |
| 475 | Should Be Equal As Numbers ${ret.status_code} 200 |
| 476 | |
| 477 | Create Binary File BMC_dump.tar.gz ${ret.content} |
| 478 | ${downloaded_dump_size}= Get File Size BMC_dump.tar.gz |
| 479 | Should Be Equal ${downloaded_dump_size} ${redfish_bmc_dump_size} |
| 480 | |
| 481 | # Extract dump and verify type of dump from summary.log content: |
| 482 | # Wed Aug 30 17:23:29 UTC 2023 Name: BMCDUMP.XXXXXXX.0001005.20230830172329 |
| 483 | # Wed Aug 30 17:23:29 UTC 2023 Epochtime: 1693416209 |
| 484 | # Wed Aug 30 17:23:29 UTC 2023 ID: 0001005 |
| 485 | # Wed Aug 30 17:23:29 UTC 2023 Type: user |
| 486 | ${extracted_dump_folder}= Extract BMC Dump BMC_dump.tar.gz ${redfish_dump_creation_timestamp} |
| 487 | ${contents}= OperatingSystem.Get File ${extracted_dump_folder}/summary.log |
| 488 | Should Match Regexp ${contents} Type:[ ]*user |
| 489 | |
| 490 | # Clean extracted dump files. |
| 491 | Remove Files output output.zst |
| 492 | Remove Directory ${extracted_dump_folder} True |
| 493 | |
| 494 | |
Rahul Maheshwari | 2568f87 | 2020-09-23 01:03:39 -0500 | [diff] [blame] | 495 | *** Keywords *** |
| 496 | |
manashsarma | 1202ffe | 2023-09-22 09:01:18 -0500 | [diff] [blame^] | 497 | Extract BMC Dump |
| 498 | [Documentation] Extract BMC dump from the tar file and returns the name of |
| 499 | ... extracted folder like BMCDUMP.XXXXXXX.0000070.20230706063841. |
| 500 | [Arguments] ${filename} ${bmc_dump_timestamp} |
| 501 | |
| 502 | # Description of argument(s): |
| 503 | # filename name of BMC dump tar file. |
| 504 | # bmc_dump_timestamp timestamp of generated BMC dump. |
| 505 | |
| 506 | File Should Exist ${filename} |
| 507 | ${rc}= Run And Return RC dd if=${filename} of=output.zst bs=1 skip=628 |
| 508 | Should Be True 0 == ${rc} |
| 509 | |
| 510 | ${rc}= Run And Return RC zstd -d output.zst |
| 511 | Should Be True 0 == ${rc} |
| 512 | |
| 513 | ${rc}= Run And Return RC tar -xvf output |
| 514 | Should Be True 0 == ${rc} |
| 515 | |
| 516 | # Find the extracted dump folder identified with BMCDUMP as prefix and |
| 517 | # timestamp of dump generation where timestamp format is : 2023-09-27T08:30:17.000000+00:00. |
| 518 | ${var}= Fetch From Left ${bmc_dump_timestamp} . |
| 519 | ${var}= Remove String ${var} - T : |
| 520 | ${bmc_extraction_folders}= OperatingSystem.List Directories In Directory . BMCDUMP*${var} |
| 521 | ${cnt}= Get length ${bmc_extraction_folders} |
| 522 | should be equal as numbers ${cnt} 1 |
| 523 | |
| 524 | [Return] ${bmc_extraction_folders}[0] |
| 525 | |
| 526 | |
Rahul Maheshwari | bcefdf2 | 2020-10-16 07:51:34 -0500 | [diff] [blame] | 527 | Get BMC Dump Entries |
| 528 | [Documentation] Return BMC dump ids list. |
| 529 | |
ganesanb | 4d43028 | 2023-04-27 14:33:23 +0000 | [diff] [blame] | 530 | ${dump_uris}= redfish_utils.get_member_list /redfish/v1/Managers/${MANAGER_ID}/LogServices/Dump/Entries |
Rahul Maheshwari | bcefdf2 | 2020-10-16 07:51:34 -0500 | [diff] [blame] | 531 | ${dump_ids}= Create List |
| 532 | |
| 533 | FOR ${dump_uri} IN @{dump_uris} |
| 534 | ${dump_id}= Fetch From Right ${dump_uri} / |
| 535 | Append To List ${dump_ids} ${dump_id} |
| 536 | END |
| 537 | |
| 538 | [Return] ${dump_ids} |
| 539 | |
| 540 | |
manashsarma | 633cac9 | 2022-11-25 03:27:03 -0600 | [diff] [blame] | 541 | Is BMC Dump Available |
| 542 | [Documentation] Verify if BMC dump is available. |
| 543 | |
| 544 | ${dump_entries}= Get BMC Dump Entries |
| 545 | |
| 546 | # Verifying that BMC dump is available. |
| 547 | ${length}= Get length ${dump_entries} |
| 548 | Should Be True 0 < ${length} |
| 549 | |
| 550 | |
Rahul Maheshwari | d709c8d | 2020-11-01 23:03:51 -0600 | [diff] [blame] | 551 | Get Disk Usage For Dumps |
| 552 | [Documentation] Return disk usage in kilobyte for BMC dumps. |
| 553 | |
George Keishing | 952bb0f | 2022-12-12 03:13:35 -0600 | [diff] [blame] | 554 | ${usage_output} ${stderr} ${rc}= BMC Execute Command du -s ${BMC_DUMP_COLLECTOR_PATH} |
Rahul Maheshwari | d709c8d | 2020-11-01 23:03:51 -0600 | [diff] [blame] | 555 | |
| 556 | # Example of output from above BMC cli command. |
| 557 | # $ du -s /var/lib/phosphor-debug-collector/dumps |
| 558 | # 516 /var/lib/phosphor-debug-collector/dumps |
| 559 | |
| 560 | ${usage_output}= Fetch From Left ${usage_output} / |
| 561 | ${usage_output}= Convert To Integer ${usage_output} |
| 562 | |
| 563 | [return] ${usage_output} |
| 564 | |
| 565 | |
Rahul Maheshwari | 2568f87 | 2020-09-23 01:03:39 -0500 | [diff] [blame] | 566 | Test Teardown Execution |
| 567 | [Documentation] Do test teardown operation. |
| 568 | |
| 569 | FFDC On Test Case Fail |
| 570 | Close All Connections |