Rahul Maheshwari | ad676bf | 2017-06-22 15:06:05 -0500 | [diff] [blame] | 1 | *** Settings *** |
| 2 | |
| 3 | Documentation Test dump functionality of OpenBMC. |
| 4 | |
George Keishing | fb3f9a5 | 2019-07-18 13:13:58 -0500 | [diff] [blame] | 5 | Resource ../../lib/openbmc_ffdc.robot |
| 6 | Resource ../../lib/rest_client.robot |
| 7 | Resource ../../lib/dump_utils.robot |
| 8 | Resource ../../lib/boot_utils.robot |
| 9 | Resource ../../lib/utils.robot |
| 10 | Library ../../lib/bmc_ssh_utils.py |
Rahul Maheshwari | ad676bf | 2017-06-22 15:06:05 -0500 | [diff] [blame] | 11 | |
| 12 | Test Setup Open Connection And Log In |
George Keishing | 40b7ee0 | 2017-11-13 05:12:41 -0600 | [diff] [blame] | 13 | Test Teardown Test Teardown Execution |
Rahul Maheshwari | ad676bf | 2017-06-22 15:06:05 -0500 | [diff] [blame] | 14 | |
| 15 | *** Test Cases *** |
| 16 | |
Sivas SRR | c095741 | 2017-10-06 13:02:08 -0500 | [diff] [blame] | 17 | Pre Dump BMC Performance Test |
| 18 | [Documentation] Check performance of memory, CPU & file system of BMC. |
| 19 | [Tags] Pre_Dump_BMC_Performance_Test |
| 20 | |
| 21 | Open Connection And Log In |
| 22 | Check BMC Performance |
| 23 | |
George Keishing | 40b7ee0 | 2017-11-13 05:12:41 -0600 | [diff] [blame] | 24 | |
Rahul Maheshwari | 355f270 | 2017-10-17 09:15:23 -0500 | [diff] [blame] | 25 | Verify User Initiated BMC Dump When Powered Off |
| 26 | [Documentation] Create user initiated BMC dump at host off state and |
| 27 | ... verify dump entry for it. |
| 28 | [Tags] Verify_User_Initiated_BMC_Dump_When_Powered_Off |
Rahul Maheshwari | ad676bf | 2017-06-22 15:06:05 -0500 | [diff] [blame] | 29 | |
George Keishing | fb3f9a5 | 2019-07-18 13:13:58 -0500 | [diff] [blame] | 30 | Redfish Power Off stack_mode=skip |
Sivas SRR | 7aabd79 | 2017-10-18 21:28:23 -0500 | [diff] [blame] | 31 | ${dump_id}= Create User Initiated Dump |
| 32 | Check Existence of BMC Dump file ${dump_id} |
Rahul Maheshwari | ad676bf | 2017-06-22 15:06:05 -0500 | [diff] [blame] | 33 | |
Steven Sombar | a011c02 | 2019-01-16 18:13:52 -0600 | [diff] [blame] | 34 | |
George Keishing | a2dab9f | 2017-09-05 13:54:01 -0500 | [diff] [blame] | 35 | Verify Dump Persistency On Service Restart |
| 36 | [Documentation] Create user dump, restart BMC service and verify dump |
| 37 | ... persistency. |
| 38 | [Tags] Verify_Dump_Persistency_On_Service_Restart |
| 39 | |
George Keishing | fee18cf | 2017-09-20 09:46:00 -0500 | [diff] [blame] | 40 | Delete All BMC Dump |
Sivas SRR | 7aabd79 | 2017-10-18 21:28:23 -0500 | [diff] [blame] | 41 | ${dump_id}= Create User Initiated Dump |
George Keishing | a2dab9f | 2017-09-05 13:54:01 -0500 | [diff] [blame] | 42 | BMC Execute Command |
| 43 | ... systemctl restart xyz.openbmc_project.Dump.Manager.service |
| 44 | Sleep 10s reason=Wait for BMC dump service to restart properly. |
| 45 | |
Steven Sombar | aaaab22 | 2018-12-19 13:16:23 -0600 | [diff] [blame] | 46 | ${resp}= OpenBMC Get Request ${DUMP_ENTRY_URI}list |
George Keishing | a2dab9f | 2017-09-05 13:54:01 -0500 | [diff] [blame] | 47 | Should Be Equal As Strings ${resp.status_code} ${HTTP_OK} |
Sivas SRR | 7aabd79 | 2017-10-18 21:28:23 -0500 | [diff] [blame] | 48 | Check Existence of BMC Dump file ${dump_id} |
| 49 | |
| 50 | |
| 51 | Verify Dump Persistency On Reset |
| 52 | [Documentation] Create user dump, reset BMC and verify dump persistency. |
| 53 | [Tags] Verify_Dump_Persistency_On_Reset |
| 54 | |
| 55 | Delete All BMC Dump |
| 56 | ${dump_id}= Create User Initiated Dump |
| 57 | OBMC Reboot (off) |
Steven Sombar | aaaab22 | 2018-12-19 13:16:23 -0600 | [diff] [blame] | 58 | ${resp}= OpenBMC Get Request ${DUMP_ENTRY_URI}list |
Sivas SRR | 7aabd79 | 2017-10-18 21:28:23 -0500 | [diff] [blame] | 59 | Should Be Equal As Strings ${resp.status_code} ${HTTP_OK} |
| 60 | Check Existence of BMC Dump file ${dump_id} |
George Keishing | a2dab9f | 2017-09-05 13:54:01 -0500 | [diff] [blame] | 61 | |
| 62 | |
Rahul Maheshwari | ad676bf | 2017-06-22 15:06:05 -0500 | [diff] [blame] | 63 | Delete User Initiated BMC Dump And Verify |
| 64 | [Documentation] Delete user initiated dump and verify. |
| 65 | [Tags] Delete_User_Initiated_Dump_And_Verify |
| 66 | |
| 67 | ${dump_id}= Create User Initiated Dump |
Sivas SRR | 7aabd79 | 2017-10-18 21:28:23 -0500 | [diff] [blame] | 68 | Check Existence of BMC Dump file ${dump_id} |
Rahul Maheshwari | ad676bf | 2017-06-22 15:06:05 -0500 | [diff] [blame] | 69 | |
| 70 | Delete BMC Dump ${dump_id} |
| 71 | |
| 72 | |
| 73 | Verify User Initiated Dump Size |
George Keishing | 6db5202 | 2017-08-22 00:39:22 -0500 | [diff] [blame] | 74 | [Documentation] Verify user Initiated BMC dump size is under 200k. |
Rahul Maheshwari | ad676bf | 2017-06-22 15:06:05 -0500 | [diff] [blame] | 75 | [Tags] Verify_User_Initiated_Dump_Size |
| 76 | |
| 77 | ${dump_id}= Create User Initiated Dump |
| 78 | |
Steven Sombar | aaaab22 | 2018-12-19 13:16:23 -0600 | [diff] [blame] | 79 | ${dump_size}= Read Attribute ${DUMP_ENTRY_URI}${dump_id} Size |
George Keishing | 6db5202 | 2017-08-22 00:39:22 -0500 | [diff] [blame] | 80 | # Max size for dump is 200k = 200x1024 |
| 81 | Should Be True 0 < ${dump_size} < 204800 |
Sivas SRR | 7aabd79 | 2017-10-18 21:28:23 -0500 | [diff] [blame] | 82 | Check Existence of BMC Dump file ${dump_id} |
Rahul Maheshwari | ad676bf | 2017-06-22 15:06:05 -0500 | [diff] [blame] | 83 | |
| 84 | |
Rahul Maheshwari | ad676bf | 2017-06-22 15:06:05 -0500 | [diff] [blame] | 85 | Create Two User Initiated Dump And Delete One |
| 86 | [Documentation] Create two dumps and delete the first. |
| 87 | [Tags] Create_Two_User_Initiated_Dump_And_Delete_One |
| 88 | |
| 89 | ${dump_id_1}= Create User Initiated Dump |
| 90 | ${dump_id_2}= Create User Initiated Dump |
| 91 | |
| 92 | Delete BMC Dump ${dump_id_1} |
| 93 | |
Steven Sombar | aaaab22 | 2018-12-19 13:16:23 -0600 | [diff] [blame] | 94 | ${resp}= OpenBMC Get Request ${DUMP_ENTRY_URI}${dump_id_1} |
Rahul Maheshwari | ad676bf | 2017-06-22 15:06:05 -0500 | [diff] [blame] | 95 | Should Be Equal As Strings ${resp.status_code} ${HTTP_NOT_FOUND} |
| 96 | |
Steven Sombar | aaaab22 | 2018-12-19 13:16:23 -0600 | [diff] [blame] | 97 | ${resp}= OpenBMC Get Request ${DUMP_ENTRY_URI}${dump_id_2} |
Rahul Maheshwari | ad676bf | 2017-06-22 15:06:05 -0500 | [diff] [blame] | 98 | Should Be Equal As Strings ${resp.status_code} ${HTTP_OK} |
Sivas SRR | 7aabd79 | 2017-10-18 21:28:23 -0500 | [diff] [blame] | 99 | Check Existence of BMC Dump file ${dump_id_2} |
Rahul Maheshwari | ad676bf | 2017-06-22 15:06:05 -0500 | [diff] [blame] | 100 | |
| 101 | |
Rahul Maheshwari | 0dafbb4 | 2017-08-22 13:17:52 -0500 | [diff] [blame] | 102 | Create And Delete BMC Dump Multiple Times |
| 103 | [Documentation] Create and delete BMC dump multiple times. |
| 104 | [Tags] Create_And_Delete_BMC_Dump_Multiple_Times |
| 105 | |
| 106 | :FOR ${INDEX} IN RANGE 1 5 |
| 107 | \ ${dump_id}= Create User Initiated Dump |
Rahul Maheshwari | 0dafbb4 | 2017-08-22 13:17:52 -0500 | [diff] [blame] | 108 | \ Delete BMC Dump ${dump_id} |
| 109 | |
| 110 | |
| 111 | Delete All BMC Dumps And Verify |
| 112 | [Documentation] Delete all BMC dumps and verify. |
| 113 | [Tags] Delete_All_BMC_Dumps_And_Verify |
| 114 | |
| 115 | # Create some dump. |
| 116 | Create User Initiated Dump |
| 117 | Create User Initiated Dump |
| 118 | |
George Keishing | fee18cf | 2017-09-20 09:46:00 -0500 | [diff] [blame] | 119 | Delete All BMC Dump |
Steven Sombar | aaaab22 | 2018-12-19 13:16:23 -0600 | [diff] [blame] | 120 | ${resp}= OpenBMC Get Request ${DUMP_ENTRY_URI}list |
Rahul Maheshwari | 0dafbb4 | 2017-08-22 13:17:52 -0500 | [diff] [blame] | 121 | Should Be Equal As Strings ${resp.status_code} ${HTTP_NOT_FOUND} |
| 122 | |
George Keishing | 40b7ee0 | 2017-11-13 05:12:41 -0600 | [diff] [blame] | 123 | |
| 124 | Verify User Initiated BMC Dump When Host Booted |
| 125 | [Documentation] Create user initiated BMC dump at host booted state and |
| 126 | ... verify dump entry for it. |
| 127 | [Tags] Verify_User_Initiated_BMC_Dump_When_Host_Booted |
| 128 | |
George Keishing | fb3f9a5 | 2019-07-18 13:13:58 -0500 | [diff] [blame] | 129 | Redfish Power On stack_mode=skip |
George Keishing | 40b7ee0 | 2017-11-13 05:12:41 -0600 | [diff] [blame] | 130 | Create User Initiated Dump |
| 131 | |
| 132 | |
Rahul Maheshwari | 1612ac9 | 2017-08-30 14:42:32 -0500 | [diff] [blame] | 133 | Verify Core Dump Size |
| 134 | [Documentation] Verify BMC core dump size is under 200k. |
| 135 | [Tags] Verify_Core_Dump_Size |
| 136 | |
| 137 | Delete All Dumps |
| 138 | Trigger Core Dump |
George Keishing | 40efd76 | 2018-10-22 04:04:00 -0500 | [diff] [blame] | 139 | Wait Until Keyword Succeeds 2 min 10 sec Get Dump Entries |
Rahul Maheshwari | 1612ac9 | 2017-08-30 14:42:32 -0500 | [diff] [blame] | 140 | |
| 141 | ${dump_entries}= Get URL List ${DUMP_ENTRY_URI} |
| 142 | ${dump_size}= Read Attribute ${dump_entries[0]} Size |
| 143 | |
| 144 | # Max size for dump is 200k = 200x1024 |
| 145 | Should Be True 0 < ${dump_size} < 204800 msg=Size of dump is incorrect. |
| 146 | |
| 147 | |
Steven Sombar | a011c02 | 2019-01-16 18:13:52 -0600 | [diff] [blame] | 148 | Dump Out Of Space Test |
| 149 | [Documentation] Verify out of dump space is reported when attempt |
| 150 | ... to create too many dumps. |
| 151 | [Tags] Dump_Out_Of_Space_Test |
| 152 | |
| 153 | # Systems typically hold 8-14 dumps before running out of dump space. |
| 154 | # Attempt to create too_many_dumps. Expect to run out of space |
| 155 | # before this. |
| 156 | ${too_many_dumps} Set Variable ${100} |
| 157 | |
| 158 | # Should be able to create at least this many dumps. |
| 159 | ${minimum_number_of_dumps} Set Variable ${7} |
| 160 | |
| 161 | # Loop, creating a dump each iteration. Will either get dump_id or |
| 162 | # will get EMPTY when out of dump space. |
| 163 | :FOR ${n} IN RANGE ${too_many_dumps} |
| 164 | \ ${dump_id}= Create User Initiated Dump check_out_of_space=${True} |
| 165 | \ Exit For Loop If '${dump_id}' == '${EMPTY}' |
| 166 | \ Check Existence of BMC Dump file ${dump_id} |
| 167 | |
| 168 | Run Keyword If '${dump_id}' != '${EMPTY}' Fail |
| 169 | ... msg=Did not run out of dump space as expected. |
| 170 | |
| 171 | Run Keyword If ${n} < ${minimum_number_of_dumps} Fail |
| 172 | ... msg=Insufficient space for at least ${minimum_number_of_dumps} dumps. |
| 173 | |
| 174 | |
Sivas SRR | c095741 | 2017-10-06 13:02:08 -0500 | [diff] [blame] | 175 | Post Dump BMC Performance Test |
| 176 | [Documentation] Check performance of memory, CPU & file system of BMC. |
| 177 | [Tags] Post_Dump_BMC_Performance_Test |
| 178 | |
| 179 | Open Connection And Log In |
| 180 | Check BMC Performance |
| 181 | |
Steven Sombar | a011c02 | 2019-01-16 18:13:52 -0600 | [diff] [blame] | 182 | |
Sivas SRR | c095741 | 2017-10-06 13:02:08 -0500 | [diff] [blame] | 183 | Post Dump Core Dump Check |
| 184 | [Documentation] Check core dump existence on BMC after code update. |
| 185 | [Tags] Post_Dump_Core_Dump_Check |
| 186 | |
| 187 | Check For Core Dumps |
Rahul Maheshwari | 0dafbb4 | 2017-08-22 13:17:52 -0500 | [diff] [blame] | 188 | |
Rahul Maheshwari | 1612ac9 | 2017-08-30 14:42:32 -0500 | [diff] [blame] | 189 | |
Anusha Dathatri | eda0949 | 2019-02-26 08:30:13 -0600 | [diff] [blame] | 190 | Verify Dump After Host Watchdog Error Injection |
| 191 | [Documentation] Inject host watchdog error and verify whether dump is generated. |
| 192 | [Tags] Verify_Dump_After_Host_Watchdog_Error_Injection |
| 193 | |
George Keishing | fb3f9a5 | 2019-07-18 13:13:58 -0500 | [diff] [blame] | 194 | Redfish Power On |
Anusha Dathatri | eda0949 | 2019-02-26 08:30:13 -0600 | [diff] [blame] | 195 | |
| 196 | Run Keyword And Ignore Error Delete All Dumps |
| 197 | |
| 198 | # Enable auto reboot |
| 199 | Set Auto Reboot ${1} |
| 200 | |
| 201 | Trigger Host Watchdog Error 2000 30 |
| 202 | |
| 203 | Wait Until Keyword Succeeds 300 sec 20 sec Is Host Rebooted |
| 204 | |
| 205 | #Get dump details |
| 206 | @{dump_entry_list}= Read Properties ${DUMP_ENTRY_URI} |
| 207 | |
| 208 | # Verifing that there is only one dump |
| 209 | ${length}= Get length ${dump_entry_list} |
| 210 | Should Be Equal As Integers ${length} ${1} |
| 211 | |
| 212 | # Get dump id |
| 213 | ${value}= Get From List ${dump_entry_list} 0 |
| 214 | @{split_value}= Split String ${value} / |
| 215 | ${dump_id}= Get From List ${split_value} -1 |
| 216 | |
| 217 | # Max size for dump is 200k = 200x1024 |
| 218 | ${dump_size}= Read Attribute ${DUMP_ENTRY_URI}${dump_id} Size |
| 219 | Should Be True 0 < ${dump_size} < 204800 |
| 220 | |
| 221 | |
Rahul Maheshwari | ad676bf | 2017-06-22 15:06:05 -0500 | [diff] [blame] | 222 | *** Keywords *** |
| 223 | |
George Keishing | 40b7ee0 | 2017-11-13 05:12:41 -0600 | [diff] [blame] | 224 | Test Teardown Execution |
Rahul Maheshwari | ad676bf | 2017-06-22 15:06:05 -0500 | [diff] [blame] | 225 | [Documentation] Do the post test teardown. |
| 226 | |
Rahul Maheshwari | 953038b | 2017-10-17 05:08:59 -0500 | [diff] [blame] | 227 | Wait Until Keyword Succeeds 3 min 15 sec Verify No Dump In Progress |
Rahul Maheshwari | ad676bf | 2017-06-22 15:06:05 -0500 | [diff] [blame] | 228 | FFDC On Test Case Fail |
Rahul Maheshwari | bf9024f | 2017-10-25 10:51:39 -0500 | [diff] [blame] | 229 | Delete All BMC Dump |
Rahul Maheshwari | ad676bf | 2017-06-22 15:06:05 -0500 | [diff] [blame] | 230 | Close All Connections |