| George Keishing | 78ea2dc | 2016-11-18 04:33:45 -0600 | [diff] [blame] | 1 | *** Settings *** | 
 | 2 | Documentation     This module will take whatever action is necessary | 
 | 3 | ...               to bring the BMC to a stable, standby state.  For our | 
 | 4 | ...               purposes, a stable state is defined as: | 
 | 5 | ...                  - BMC is communicating | 
 | 6 | ...                   (pinging, sshing and REST commands working) | 
 | 7 | ...                  - Power state is 0 (off) | 
| Rahul Maheshwari | 2c72504 | 2017-01-29 22:55:28 -0600 | [diff] [blame] | 8 | ...                  - BMC state is "Ready" | 
 | 9 | ...                  - HOST state is "Off" | 
| George Keishing | efc3ff2 | 2017-12-12 11:49:25 -0600 | [diff] [blame] | 10 | ...                  - Boot policy is "ALWAYS_POWER_OFF" | 
| George Keishing | 34d4b8d | 2016-12-07 08:35:29 -0600 | [diff] [blame] | 11 | ...               Power cycle system via PDU if specified | 
| George Keishing | 0bd59f1 | 2016-12-14 10:39:21 -0600 | [diff] [blame] | 12 | ...               Prune archived journal logs | 
| George Keishing | 78ea2dc | 2016-11-18 04:33:45 -0600 | [diff] [blame] | 13 |  | 
| George Keishing | 78ea2dc | 2016-11-18 04:33:45 -0600 | [diff] [blame] | 14 | Resource          ../lib/utils.robot | 
| George Keishing | 34d4b8d | 2016-12-07 08:35:29 -0600 | [diff] [blame] | 15 | Resource          ../lib/pdu/pdu.robot | 
| Rahul Maheshwari | 9bd3abc | 2017-02-07 00:51:30 -0600 | [diff] [blame] | 16 | Resource          ../lib/state_manager.robot | 
| Michael Walsh | 1b0ceb2 | 2017-05-31 16:03:01 -0500 | [diff] [blame] | 17 | Resource          ../lib/bmc_network_utils.robot | 
| Sunil M | ec6fe31 | 2017-06-19 04:21:25 -0500 | [diff] [blame] | 18 | Resource          ../lib/bmc_cleanup.robot | 
| George Keishing | 27a1f20 | 2017-08-30 22:40:26 -0500 | [diff] [blame] | 19 | Resource          ../lib/dump_utils.robot | 
| George Keishing | caa718b | 2019-03-10 00:08:33 -0600 | [diff] [blame] | 20 | Resource          ../lib/bmc_redfish_resource.robot | 
 | 21 | Resource          ../lib/bmc_redfish_utils.robot | 
| George Keishing | c7748be | 2018-07-12 05:40:37 -0500 | [diff] [blame] | 22 | Library           ../lib/gen_misc.py | 
| George Keishing | 78ea2dc | 2016-11-18 04:33:45 -0600 | [diff] [blame] | 23 |  | 
| George Keishing | 8dd04af | 2021-03-23 10:36:40 -0500 | [diff] [blame] | 24 | # Force the test to timedout to prevent test hanging. | 
 | 25 | Test Timeout      10 minutes | 
 | 26 |  | 
 | 27 |  | 
| George Keishing | 78ea2dc | 2016-11-18 04:33:45 -0600 | [diff] [blame] | 28 | *** Variables *** | 
 | 29 | ${HOST_SETTING}      /org/openbmc/settings/host0 | 
 | 30 |  | 
| George Keishing | c7748be | 2018-07-12 05:40:37 -0500 | [diff] [blame] | 31 | ${ERROR_REGEX}  xyz.openbmc_project.Software.BMC.Updater.service: Failed with result 'core-dump' | 
 | 32 |  | 
| Gunnar Mills | 7c8923f | 2016-12-12 21:19:52 -0600 | [diff] [blame] | 33 | *** Test Cases *** | 
| George Keishing | 78ea2dc | 2016-11-18 04:33:45 -0600 | [diff] [blame] | 34 |  | 
| George Keishing | 62f9430 | 2017-01-31 07:00:52 -0600 | [diff] [blame] | 35 | Get To Stable State | 
| Rahul Maheshwari | 2c72504 | 2017-01-29 22:55:28 -0600 | [diff] [blame] | 36 |     [Documentation]  BMC cleanup drive to stable state | 
 | 37 |     ...              1. PDU powercycle if specified | 
 | 38 |     ...              1. Ping Test | 
 | 39 |     ...              2. SSH Connection session Test | 
 | 40 |     ...              3. REST Connection session Test | 
 | 41 |     ...              4. Reboot BMC if REST Test failed | 
 | 42 |     ...              5. Get BMC in Ready state if its not in this state | 
 | 43 |     ...              6. Get Host in Off state if its not in this state | 
 | 44 |     ...              7. Update restore policy | 
| Sunil M | 89fb0f5 | 2017-05-26 05:25:33 -0500 | [diff] [blame] | 45 |     ...              8. Verify and Update MAC address. | 
| George Keishing | 62f9430 | 2017-01-31 07:00:52 -0600 | [diff] [blame] | 46 |     [Tags]  Get_To_Stable_State | 
| George Keishing | 78ea2dc | 2016-11-18 04:33:45 -0600 | [diff] [blame] | 47 |  | 
| Rahul Maheshwari | 2c72504 | 2017-01-29 22:55:28 -0600 | [diff] [blame] | 48 |     Run Keyword And Ignore Error  Powercycle System Via PDU | 
| George Keishing | 34d4b8d | 2016-12-07 08:35:29 -0600 | [diff] [blame] | 49 |  | 
| George Keishing | 13497eb | 2018-11-07 06:35:14 -0600 | [diff] [blame] | 50 |     ${ping_status}=  Run Keyword And Return Status | 
 | 51 |     ...  Wait For Host To Ping  ${OPENBMC_HOST}  2 mins | 
 | 52 |  | 
 | 53 |     # Check if the ping works using 1400 MTU. | 
| George Keishing | d5e8d86 | 2025-05-13 21:59:19 +0530 | [diff] [blame] | 54 |     #IF  ${ping_status} == ${True}  MTU Ping Test | 
| George Keishing | 13497eb | 2018-11-07 06:35:14 -0600 | [diff] [blame] | 55 |  | 
| George Keishing | d5e8d86 | 2025-05-13 21:59:19 +0530 | [diff] [blame] | 56 |     IF  ${ping_status} == ${False}  Fail  ${OPENBMC_HOST} ping test failed. | 
| George Keishing | 13497eb | 2018-11-07 06:35:14 -0600 | [diff] [blame] | 57 |  | 
 | 58 |     Open Connection And Log In  host=${OPENBMC_HOST} | 
| George Keishing | 78ea2dc | 2016-11-18 04:33:45 -0600 | [diff] [blame] | 59 |  | 
| George Keishing | d5e8d86 | 2025-05-13 21:59:19 +0530 | [diff] [blame] | 60 |     IF  ${REDFISH_SUPPORTED} | 
 | 61 |         Redfish Clean Up | 
 | 62 |     ELSE | 
 | 63 |         REST Clean Up | 
 | 64 |     END | 
| George Keishing | 8dd04af | 2021-03-23 10:36:40 -0500 | [diff] [blame] | 65 |  | 
| George Keishing | 78ea2dc | 2016-11-18 04:33:45 -0600 | [diff] [blame] | 66 |  | 
| George Keishing | 0bd59f1 | 2016-12-14 10:39:21 -0600 | [diff] [blame] | 67 |     Prune Journal Log | 
| Charles Paul Hofer | 7cdb8b4 | 2018-09-10 12:12:04 -0500 | [diff] [blame] | 68 |     Check For Current Boot Application Failures | 
| George Keishing | 78ea2dc | 2016-11-18 04:33:45 -0600 | [diff] [blame] | 69 |  | 
 | 70 | *** Keywords *** | 
 | 71 |  | 
| George Keishing | 8dd04af | 2021-03-23 10:36:40 -0500 | [diff] [blame] | 72 |  | 
 | 73 | REST Clean Up | 
 | 74 |     [Documentation]  Check states, reboot if needed and poweroff. | 
 | 75 |  | 
 | 76 |     Wait Until Keyword Succeeds  1 min  30 sec  Initialize OpenBMC | 
 | 77 |  | 
 | 78 |     ${ready_status}=  Run Keyword And Return Status  Is BMC Ready | 
| George Keishing | d5e8d86 | 2025-05-13 21:59:19 +0530 | [diff] [blame] | 79 |     IF  '${ready_status}' == '${False}' | 
 | 80 |         Put BMC State  Ready | 
 | 81 |     ELSE | 
 | 82 |         REST Power Off  stack_mode=skip | 
 | 83 |     END | 
| George Keishing | 8dd04af | 2021-03-23 10:36:40 -0500 | [diff] [blame] | 84 |  | 
 | 85 |     Run Keyword And Ignore Error  Set BMC Power Policy  ${ALWAYS_POWER_OFF} | 
 | 86 |     Run Keyword And Ignore Error  Delete All Error Logs | 
 | 87 |     Run Keyword And Ignore Error  Delete All Dumps | 
 | 88 |  | 
 | 89 |  | 
 | 90 | Redfish Clean Up | 
 | 91 |     [Documentation]  Check states, reboot if needed and poweroff. | 
 | 92 |  | 
 | 93 |     Wait Until Keyword Succeeds  1 min  30 sec  Redfish.Login | 
 | 94 |  | 
 | 95 |     Redfish Power Off  stack_mode=skip | 
 | 96 |  | 
 | 97 |     Run Keyword And Ignore Error  Redfish Set Power Restore Policy  AlwaysOff | 
 | 98 |     Run Keyword And Ignore Error  Redfish Purge Event Log | 
 | 99 |     Run Keyword And Ignore Error  Redfish Delete All BMC Dumps | 
| George Keishing | 2ef6a7d | 2021-05-19 09:05:32 -0500 | [diff] [blame] | 100 |     Run Keyword And Ignore Error  Redfish Delete All System Dumps | 
| Anusha Dathatri | 34298bf | 2021-06-17 00:00:01 -0500 | [diff] [blame] | 101 |     Run Keyword And Ignore Error  Clear All Subscriptions | 
| George Keishing | 8dd04af | 2021-03-23 10:36:40 -0500 | [diff] [blame] | 102 |     Run Keyword And Ignore Error  Delete All Redfish Sessions | 
 | 103 |  | 
 | 104 |  | 
| George Keishing | 78ea2dc | 2016-11-18 04:33:45 -0600 | [diff] [blame] | 105 | BMC Online Test | 
 | 106 |     [Documentation]   BMC ping, SSH, REST connection Test | 
 | 107 |  | 
| Gunnar Mills | 597ffa0 | 2016-12-06 11:26:53 -0600 | [diff] [blame] | 108 |     ${l_status}=   Run Keyword and Return Status | 
| George Keishing | 78ea2dc | 2016-11-18 04:33:45 -0600 | [diff] [blame] | 109 |     ...   Verify Ping and REST Authentication | 
| George Keishing | d5e8d86 | 2025-05-13 21:59:19 +0530 | [diff] [blame] | 110 |     IF  '${l_status}' == '${False}' | 
 | 111 |         Fail  msg=System not in ideal state to continue [ERROR] | 
 | 112 |     END | 
| George Keishing | 78ea2dc | 2016-11-18 04:33:45 -0600 | [diff] [blame] | 113 |  | 
 | 114 |  | 
| George Keishing | 78ea2dc | 2016-11-18 04:33:45 -0600 | [diff] [blame] | 115 | Update Policy Setting | 
 | 116 |     [Documentation]   Update the given restore policy | 
| Gunnar Mills | 3803280 | 2016-12-12 13:43:40 -0600 | [diff] [blame] | 117 |     [Arguments]   ${policy} | 
| George Keishing | 78ea2dc | 2016-11-18 04:33:45 -0600 | [diff] [blame] | 118 |  | 
| Gunnar Mills | 597ffa0 | 2016-12-06 11:26:53 -0600 | [diff] [blame] | 119 |     ${valueDict}=     create dictionary  data=${policy} | 
| George Keishing | 78ea2dc | 2016-11-18 04:33:45 -0600 | [diff] [blame] | 120 |     Write Attribute    ${HOST_SETTING}    power_policy   data=${valueDict} | 
 | 121 |     ${currentPolicy}=  Read Attribute     ${HOST_SETTING}   power_policy | 
 | 122 |     Should Be Equal    ${currentPolicy}   ${policy} | 
 | 123 |  | 
| George Keishing | d5123f7 | 2016-12-07 09:54:51 -0600 | [diff] [blame] | 124 |  | 
 | 125 | Trigger Warm Reset via Reboot | 
 | 126 |     [Documentation]    Execute reboot command on the remote BMC and | 
 | 127 |     ...                returns immediately. This keyword "Start Command" | 
 | 128 |     ...                returns nothing and does not wait for the command | 
 | 129 |     ...                execution to be finished. | 
 | 130 |     Open Connection And Log In | 
 | 131 |  | 
 | 132 |     Start Command   /sbin/reboot | 
| George Keishing | 34d4b8d | 2016-12-07 08:35:29 -0600 | [diff] [blame] | 133 |  | 
 | 134 |  | 
 | 135 | Powercycle System Via PDU | 
| Joy Onyerikwu | f4a807b | 2018-06-20 08:43:54 -0500 | [diff] [blame] | 136 |     [Documentation]   AC cycle the system via PDU. | 
| George Keishing | 34d4b8d | 2016-12-07 08:35:29 -0600 | [diff] [blame] | 137 |  | 
 | 138 |     Validate Parameters | 
 | 139 |     PDU Power Cycle | 
 | 140 |     Check If BMC is Up   5 min    10 sec | 
 | 141 |  | 
 | 142 |  | 
| Charles Paul Hofer | 7cdb8b4 | 2018-09-10 12:12:04 -0500 | [diff] [blame] | 143 | Check For Current Boot Application Failures | 
| George Keishing | c7748be | 2018-07-12 05:40:37 -0500 | [diff] [blame] | 144 |     [Documentation]  Parse the journal log and check for failures. | 
 | 145 |     [Arguments]  ${error_regex}=${ERROR_REGEX} | 
 | 146 |  | 
 | 147 |     ${error_regex}=  Escape Bash Quotes  ${error_regex} | 
 | 148 |     ${journal_log}  ${stderr}  ${rc}=  BMC Execute Command | 
| Charles Paul Hofer | bcae08b | 2018-08-09 11:33:33 -0500 | [diff] [blame] | 149 |     ...  journalctl -b --no-pager | egrep '${error_regex}'  ignore_err=1 | 
| George Keishing | c7748be | 2018-07-12 05:40:37 -0500 | [diff] [blame] | 150 |  | 
 | 151 |     Should Be Empty  ${journal_log} | 
 | 152 |  | 
 | 153 |  | 
| George Keishing | 34d4b8d | 2016-12-07 08:35:29 -0600 | [diff] [blame] | 154 | Validate Parameters | 
| Joy Onyerikwu | f4a807b | 2018-06-20 08:43:54 -0500 | [diff] [blame] | 155 |     [Documentation]  Validate PDU parameters. | 
| George Keishing | 34d4b8d | 2016-12-07 08:35:29 -0600 | [diff] [blame] | 156 |     Should Not Be Empty   ${PDU_IP} | 
 | 157 |     Should Not Be Empty   ${PDU_TYPE} | 
 | 158 |     Should Not Be Empty   ${PDU_SLOT_NO} | 
 | 159 |     Should Not Be Empty   ${PDU_USERNAME} | 
 | 160 |     Should Not Be Empty   ${PDU_PASSWORD} | 
| George Keishing | 13497eb | 2018-11-07 06:35:14 -0600 | [diff] [blame] | 161 |  | 
 | 162 |  | 
 | 163 | MTU Ping Test | 
 | 164 |     [Documentation]  Ping test using MTU. | 
 | 165 |     [Arguments]  ${mtu}=${1400} | 
 | 166 |  | 
 | 167 |     # Description of argument(s): | 
 | 168 |     # mtu   The maximum transmission unit (MTU) of a network interface. | 
 | 169 |  | 
 | 170 |     ${rc}  ${output}=  Run And Return RC And Output | 
 | 171 |     ...  ping -M do -s ${mtu} -c 10 ${OPENBMC_HOST} | 
 | 172 |     Should Be Equal As Integers  ${rc}  0 | 
 | 173 |     Should Not Contain  ${output}  100% packet loss | 
| Anusha Dathatri | 34298bf | 2021-06-17 00:00:01 -0500 | [diff] [blame] | 174 |  | 
 | 175 |  | 
 | 176 | Clear All Subscriptions | 
 | 177 |     [Documentation]  Delete all subscriptions. | 
 | 178 |  | 
 | 179 |     ${subscriptions}=  Redfish.Get Attribute  /redfish/v1/EventService/Subscriptions  Members | 
 | 180 |     FOR  ${subscription}  IN  @{subscriptions} | 
 | 181 |         Redfish.Delete  ${subscription['@odata.id']} | 
 | 182 |     END |