George Keishing | 5e870cd | 2016-08-24 10:05:47 -0500 | [diff] [blame] | 1 | *** Settings *** |
| 2 | Documentation Trigger code update to a target BMC. |
| 3 | ... Execution Method : |
| 4 | ... python -m robot -v OPENBMC_HOST:<hostname> |
| 5 | ... -v FILE_PATH:<path/*all.tar> update_bmc.robot |
| 6 | ... |
| 7 | ... Code update method BMC using REST |
| 8 | ... Update work flow sequence: |
| 9 | ... - User input BMC File existence check |
| 10 | ... - Ping Test and REST authentication |
George Keishing | 8db0e1b | 2016-10-20 13:46:54 -0500 | [diff] [blame] | 11 | ... - Set Host Power host setting Policy to RESTORE_LAST_STATE |
| 12 | ... On reboot this policy would ensure the BMC comes |
| 13 | ... online and stays at HOST_POWERED_OFF state. |
George Keishing | dc5aa58e9 | 2016-09-27 01:35:11 -0500 | [diff] [blame] | 14 | ... - Issue poweroff |
George Keishing | 0bd59f1 | 2016-12-14 10:39:21 -0600 | [diff] [blame] | 15 | ... - Prune archived journal logs |
George Keishing | dc5aa58e9 | 2016-09-27 01:35:11 -0500 | [diff] [blame] | 16 | ... - Prepare for Update |
George Keishing | dc5aa58e9 | 2016-09-27 01:35:11 -0500 | [diff] [blame] | 17 | ... - Wait for BMC to come online clean |
George Keishing | 1b15020 | 2016-09-29 08:51:58 -0500 | [diff] [blame] | 18 | ... - Wait for BMC_READY state |
George Keishing | 5e870cd | 2016-08-24 10:05:47 -0500 | [diff] [blame] | 19 | ... - Apply preserve BMC Network setting |
| 20 | ... - SCP image to BMC |
| 21 | ... - Activate the flash image |
| 22 | ... - Warm Reset BMC to activate code |
| 23 | ... - Wait for BMC to come online time out 30 minutes |
| 24 | ... - Version check post update |
Rahul Maheshwari | 724e591 | 2017-04-20 06:04:37 -0500 | [diff] [blame] | 25 | ... - Identify REST url post update |
George Keishing | 5e870cd | 2016-08-24 10:05:47 -0500 | [diff] [blame] | 26 | |
Michael Walsh | 5ce9805 | 2017-04-12 16:40:31 -0500 | [diff] [blame] | 27 | Library ../../lib/gen_robot_keyword.py |
| 28 | Library String |
Sweta Potthuri | 3a8ae26 | 2017-03-10 00:02:09 -0600 | [diff] [blame] | 29 | |
Michael Walsh | 5ce9805 | 2017-04-12 16:40:31 -0500 | [diff] [blame] | 30 | Resource code_update_utils.robot |
| 31 | Resource ../../lib/boot/boot_resource_master.robot |
| 32 | Resource ../../lib/state_manager.robot |
| 33 | Resource ../../lib/utils.robot |
Rahul Maheshwari | 724e591 | 2017-04-20 06:04:37 -0500 | [diff] [blame] | 34 | Resource ../../lib/list_utils.robot |
Michael Walsh | 5ce9805 | 2017-04-12 16:40:31 -0500 | [diff] [blame] | 35 | Resource ../../lib/openbmc_ffdc.robot |
| 36 | Resource ../../extended/obmc_boot_test_resource.robot |
| 37 | |
| 38 | Test Teardown Run Key FFDC On Test Case Fail |
George Keishing | 5e870cd | 2016-08-24 10:05:47 -0500 | [diff] [blame] | 39 | |
| 40 | *** Variables *** |
| 41 | |
Michael Walsh | 5ce9805 | 2017-04-12 16:40:31 -0500 | [diff] [blame] | 42 | ${QUIET} ${1} |
| 43 | # Boot failures are not acceptable so we set the threshold to 0. |
| 44 | ${boot_fail_threshold} ${0} |
| 45 | |
| 46 | ${FILE_PATH} ${EMPTY} |
| 47 | ${FORCE_UPDATE} ${0} |
Rahul Maheshwari | 724e591 | 2017-04-20 06:04:37 -0500 | [diff] [blame] | 48 | ${REST_URL_FILE_PATH} ${EMPTY} |
George Keishing | 80dd5af | 2017-02-08 07:01:25 -0600 | [diff] [blame] | 49 | |
| 50 | # There are two reboots issued by code update. |
Michael Walsh | 5ce9805 | 2017-04-12 16:40:31 -0500 | [diff] [blame] | 51 | ${MAX_BOOT_COUNT} ${2} |
George Keishing | 5e870cd | 2016-08-24 10:05:47 -0500 | [diff] [blame] | 52 | |
Gunnar Mills | 7c8923f | 2016-12-12 21:19:52 -0600 | [diff] [blame] | 53 | *** Test Cases *** |
George Keishing | 5e870cd | 2016-08-24 10:05:47 -0500 | [diff] [blame] | 54 | |
Sivas SRR | aca5571 | 2016-12-21 04:32:35 -0600 | [diff] [blame] | 55 | Test Basic BMC Performance Before Code Update |
| 56 | [Documentation] Check performance of memory, CPU & file system of BMC. |
| 57 | [Tags] Test_Basic_BMC_Performance_Before_Code_Update |
| 58 | Open Connection And Log In |
| 59 | Check BMC CPU Performance |
| 60 | Check BMC Mem Performance |
| 61 | Check BMC File System Performance |
| 62 | |
Sivas SRR | 01262bd | 2017-03-19 10:00:23 -0500 | [diff] [blame] | 63 | Check Core Dump Exist Before Code Update |
| 64 | [Documentation] Check core dump existence on BMC before code update. |
| 65 | [Tags] Check_Core_Dump_Exist_Before_Code_Update |
| 66 | Check For Core Dumps |
| 67 | |
Rahul Maheshwari | 724e591 | 2017-04-20 06:04:37 -0500 | [diff] [blame] | 68 | Check URLs Before Code Update |
| 69 | [Documentation] Check available URLs before code update. |
| 70 | [Tags] Check_URLs_Before_Code_Update |
Rahul Maheshwari | 13256d9 | 2017-06-15 10:05:40 -0500 | [diff] [blame] | 71 | |
Rahul Maheshwari | 724e591 | 2017-04-20 06:04:37 -0500 | [diff] [blame] | 72 | ${url_list}= Get URL List ${OPENBMC_BASE_URI} |
| 73 | Set Global Variable ${URL_BEFORE_CU} ${url_list} |
| 74 | |
Rahul Maheshwari | 13256d9 | 2017-06-15 10:05:40 -0500 | [diff] [blame] | 75 | ${bmc_version}= Get BMC Version |
| 76 | Set Suite Variable ${bmc_version_before} ${bmc_version} |
| 77 | |
Sivas SRR | aca5571 | 2016-12-21 04:32:35 -0600 | [diff] [blame] | 78 | Initiate Code Update BMC |
Michael Walsh | 5ce9805 | 2017-04-12 16:40:31 -0500 | [diff] [blame] | 79 | [Documentation] Initiate a code update on the BMC. |
Sivas SRR | aca5571 | 2016-12-21 04:32:35 -0600 | [diff] [blame] | 80 | [Tags] Initiate_Code_Update_BMC |
George Keishing | 5e870cd | 2016-08-24 10:05:47 -0500 | [diff] [blame] | 81 | |
George Keishing | 830211e | 2017-07-19 08:44:07 -0500 | [diff] [blame] | 82 | ${status}= Run Keyword If '${LAST_KNOWN_GOOD_VERSION}' != '${EMPTY}' |
| 83 | ... Run Keyword And Return Status |
| 84 | ... Validate BMC Version ${LAST_KNOWN_GOOD_VERSION} |
George Keishing | fb10f49 | 2017-03-07 21:34:18 -0600 | [diff] [blame] | 85 | |
George Keishing | 830211e | 2017-07-19 08:44:07 -0500 | [diff] [blame] | 86 | Run Keyword if '${status}' == '${True}' |
| 87 | ... Pass Execution The BMC already has the requested build loaded so no further action will be taken. |
George Keishing | 50787fc | 2016-09-21 03:44:26 -0500 | [diff] [blame] | 88 | |
George Keishing | 5e78582 | 2017-04-04 10:15:50 -0500 | [diff] [blame] | 89 | # Enable user to bypass prerequisite operations. |
| 90 | # Use cases for if BMC is not in working state. |
| 91 | Run Keyword If ${FORCE_UPDATE} == ${0} |
| 92 | ... Prepare BMC For Update |
George Keishing | 2582bee | 2016-11-17 21:41:49 -0600 | [diff] [blame] | 93 | |
Michael Walsh | 5ce9805 | 2017-04-12 16:40:31 -0500 | [diff] [blame] | 94 | Run Key U Preserve BMC Network Setting |
| 95 | Run Key SCP Tar Image File To BMC \ ${FILE_PATH} |
| 96 | Run Key Activate BMC Flash Image |
| 97 | Run Key U OBMC Boot Test \ OBMC Reboot (off) |
| 98 | Run Key U Check Boot Count And Time |
George Keishing | 80dd5af | 2017-02-08 07:01:25 -0600 | [diff] [blame] | 99 | Run Keyword If ${BOOT_COUNT} == ${1} |
Michael Walsh | 5ce9805 | 2017-04-12 16:40:31 -0500 | [diff] [blame] | 100 | ... Log Boot time not updated by kernel. level=WARN |
Sivas SRR | ea85d1f | 2016-11-13 22:44:28 -0600 | [diff] [blame] | 101 | |
George Keishing | a126e21 | 2017-03-24 08:07:17 -0500 | [diff] [blame] | 102 | Install BMC Debug Tarball |
| 103 | [Documentation] Install the downloaded debug tarball on BMC. |
| 104 | [Tags] Install_BMC_Debug_Tarball |
| 105 | Run Keyword If '${DEBUG_TARBALL_PATH}' != '${EMPTY}' |
| 106 | ... Install Debug Tarball On BMC ${DEBUG_TARBALL_PATH} |
| 107 | |
Rahul Maheshwari | 724e591 | 2017-04-20 06:04:37 -0500 | [diff] [blame] | 108 | Compare URLs Before And After Code Update |
| 109 | [Documentation] Compare URLs before and after code update. |
| 110 | [Tags] Compare_URLs_Before_And_After_Code_Update |
| 111 | |
Rahul Maheshwari | 13256d9 | 2017-06-15 10:05:40 -0500 | [diff] [blame] | 112 | ${bmc_version}= Get BMC Version |
| 113 | Set Suite Variable ${bmc_version_after} ${bmc_version} |
| 114 | |
| 115 | # Exit test for same firmware version. |
| 116 | Pass Execution If '${bmc_version_before}' == '${bmc_version_after}' |
| 117 | ... Same BMC firmware version found |
| 118 | |
Rahul Maheshwari | 724e591 | 2017-04-20 06:04:37 -0500 | [diff] [blame] | 119 | ${url_after_cu}= Get URL List ${OPENBMC_BASE_URI} |
| 120 | Compare URL List After Code Update ${URL_BEFORE_CU} ${url_after_cu} |
George Keishing | a126e21 | 2017-03-24 08:07:17 -0500 | [diff] [blame] | 121 | |
Sivas SRR | aca5571 | 2016-12-21 04:32:35 -0600 | [diff] [blame] | 122 | Test Basic BMC Performance At Ready State |
| 123 | [Documentation] Check performance of memory, CPU & file system of BMC. |
| 124 | [Tags] Test_Basic_BMC_Performance_At_Ready_State |
George Keishing | 128643a | 2016-12-15 11:36:46 -0600 | [diff] [blame] | 125 | Open Connection And Log In |
Sivas SRR | ea85d1f | 2016-11-13 22:44:28 -0600 | [diff] [blame] | 126 | Check BMC CPU Performance |
| 127 | Check BMC Mem Performance |
Sivas SRR | aca5571 | 2016-12-21 04:32:35 -0600 | [diff] [blame] | 128 | Check BMC File System Performance |
| 129 | |
Sivas SRR | 01262bd | 2017-03-19 10:00:23 -0500 | [diff] [blame] | 130 | Check Core Dump Exist After Code Update |
| 131 | [Documentation] Check core dump existence on BMC after code update. |
| 132 | [Tags] Check_Core_Dump_Exist_After_Code_Update |
| 133 | Check For Core Dumps |
| 134 | |
| 135 | Enable Core Dump File Size To Be Unlimited |
| 136 | [Documentation] Set core dump file size to unlimited. |
| 137 | [Tags] Enable_Core_Dump_File_size_To_Be_unlimited |
| 138 | Set Core Dump File Size Unlimited |
| 139 | |
George Keishing | 80dd5af | 2017-02-08 07:01:25 -0600 | [diff] [blame] | 140 | *** Keywords *** |
| 141 | |
George Keishing | 5e78582 | 2017-04-04 10:15:50 -0500 | [diff] [blame] | 142 | Prepare BMC For Update |
| 143 | [Documentation] Prerequisite operation before code update. |
| 144 | Check Boot Count And Time |
| 145 | Prune Journal Log |
| 146 | Power Off Request |
Michael Walsh | fdc5ced | 2017-08-17 13:15:15 -0500 | [diff] [blame] | 147 | Set BMC Power Policy RESTORE_LAST_STATE |
George Keishing | d85b28f | 2017-08-17 00:31:54 -0500 | [diff] [blame] | 148 | |
George Keishing | 5e78582 | 2017-04-04 10:15:50 -0500 | [diff] [blame] | 149 | Prepare For Update |
| 150 | Check If BMC is Up 20 min 10 sec |
| 151 | Check Boot Count And Time |
| 152 | |
| 153 | # Temporary fix for lab migration for driver which is booted with |
| 154 | # BMC state "/xyz/openbmc_project/state/BMC0/". |
| 155 | ${status}= Run Keyword And Return Status Temp BMC URI Check |
| 156 | Run Keyword If '${status}' == '${False}' |
| 157 | ... Wait For BMC Ready |
| 158 | ... ELSE Wait For Temp BMC Ready |
| 159 | |
George Keishing | 80dd5af | 2017-02-08 07:01:25 -0600 | [diff] [blame] | 160 | Check Boot Count And Time |
| 161 | [Documentation] Check for unexpected reboots. |
| 162 | Set BMC Reset Reference Time |
| 163 | Log To Console \n Boot Count: ${BOOT_COUNT} |
| 164 | Log To Console \n Boot Time: ${BOOT_TIME} |
| 165 | Run Keyword If ${BOOT_COUNT} > ${MAX_BOOT_COUNT} |
| 166 | ... Log Phantom Reboot!!! Unexpected reboot detected level=WARN |
| 167 | |
George Keishing | 3cb1ade | 2017-03-23 09:03:06 -0500 | [diff] [blame] | 168 | Temp BMC URI Check |
| 169 | [Documentation] Check for transient "BMC0" interface. |
| 170 | ${resp}= Openbmc Get Request /xyz/openbmc_project/state/BMC0/ |
| 171 | Should Be Equal As Strings ${resp.status_code} ${HTTP_OK} |
| 172 | |
| 173 | Check Temp BMC State |
| 174 | [Documentation] BMC state should be "Ready". |
| 175 | # quiet - Suppress REST output logging to console. |
| 176 | ${state}= |
| 177 | ... Read Attribute /xyz/openbmc_project/state/BMC0/ CurrentBMCState |
| 178 | Should Be Equal Ready ${state.rsplit('.', 1)[1]} |
| 179 | |
| 180 | Wait For Temp BMC Ready |
| 181 | [Documentation] Check for BMC "Ready" until timedout. |
| 182 | Wait Until Keyword Succeeds |
| 183 | ... 10 min 10 sec Check Temp BMC State |
| 184 | |
Rahul Maheshwari | 724e591 | 2017-04-20 06:04:37 -0500 | [diff] [blame] | 185 | Compare URL List After Code Update |
| 186 | [Documentation] Compare URL list before and after code update. |
| 187 | [Arguments] ${url_before_cu} ${url_after_cu} |
| 188 | # Description of arguments: |
| 189 | # url_before_cu List of URLs available before code update. |
| 190 | # url_after_cu List of URLs available after code update. |
| 191 | |
| 192 | ${url_removed_list}= Subtract Lists ${url_before_cu} ${url_after_cu} |
| 193 | ${url_added_list}= Subtract Lists ${url_after_cu} ${url_before_cu} |
| 194 | Log ${url_removed_list} |
| 195 | Log ${url_added_list} |
| 196 | |
| 197 | Return From Keyword If '${REST_URL_FILE_PATH}' == '${EMPTY}' |
| 198 | |
Rahul Maheshwari | 13256d9 | 2017-06-15 10:05:40 -0500 | [diff] [blame] | 199 | # Create file with BMC firmware version before and after code update. |
| 200 | # i.e. <Before_Version>--<After_Version> |
| 201 | # Example v1.99.6-141-ge662190--v1.99.6-141-ge664242 |
| 202 | |
| 203 | ${file_name}= Catenate SEPARATOR=-- |
| 204 | ... ${bmc_version_before} ${bmc_version_after} |
| 205 | ${REST_URL_FILE}= Catenate ${REST_URL_FILE_PATH}/${file_name} |
| 206 | Create File ${REST_URL_FILE} URL Removed${\n} |
| 207 | |
Rahul Maheshwari | 724e591 | 2017-04-20 06:04:37 -0500 | [diff] [blame] | 208 | Return From Keyword If |
| 209 | ... ${url_removed_list} == [] and ${url_added_list} == [] |
| 210 | |
Rahul Maheshwari | 13256d9 | 2017-06-15 10:05:40 -0500 | [diff] [blame] | 211 | Create File ${REST_URL_FILE} URL Removed${\n} |
| 212 | Append To File ${REST_URL_FILE} [${url_removed_list}] |
| 213 | Append To File ${REST_URL_FILE} ${\n}URL Added${\n} |
| 214 | Append To File ${REST_URL_FILE} [${url_added_list}] |