George Keishing | eefc07a | 2016-07-26 03:25:35 -0500 | [diff] [blame] | 1 | *** Settings *** |
George Keishing | 0a46d02 | 2017-01-30 08:11:43 -0600 | [diff] [blame] | 2 | Documentation Test power on for HW CI. |
George Keishing | eefc07a | 2016-07-26 03:25:35 -0500 | [diff] [blame] | 3 | |
George Keishing | 22e57e1 | 2018-04-18 09:39:31 -0500 | [diff] [blame] | 4 | Library DateTime |
| 5 | |
George Keishing | d55a4be | 2016-08-26 03:28:17 -0500 | [diff] [blame] | 6 | Resource ../lib/openbmc_ffdc.robot |
George Keishing | 97e9653 | 2016-10-06 12:44:52 -0500 | [diff] [blame] | 7 | Resource ../lib/utils.robot |
George Keishing | 0a46d02 | 2017-01-30 08:11:43 -0600 | [diff] [blame] | 8 | Resource ../lib/state_manager.robot |
George Keishing | 766c623 | 2018-02-02 11:42:55 -0600 | [diff] [blame] | 9 | Resource ../lib/open_power_utils.robot |
Michael Tritz | 96ed277 | 2018-01-17 15:38:38 -0600 | [diff] [blame] | 10 | Resource ../lib/ipmi_client.robot |
George Keishing | 78ce8dc | 2018-03-30 11:49:06 -0500 | [diff] [blame] | 11 | Resource ../lib/boot_utils.robot |
George Keishing | 82462bf | 2018-09-20 23:48:43 -0500 | [diff] [blame] | 12 | Resource ../lib/remote_logging_utils.robot |
George Keishing | 97e9653 | 2016-10-06 12:44:52 -0500 | [diff] [blame] | 13 | |
George Keishing | 3c004dc | 2018-10-10 07:16:47 -0500 | [diff] [blame] | 14 | Suite Setup Set REST Logging Policy ${True} |
George Keishing | bde0915 | 2018-06-14 14:53:08 -0500 | [diff] [blame] | 15 | Test Teardown FFDC On Test Case Fail |
George Keishing | 3c004dc | 2018-10-10 07:16:47 -0500 | [diff] [blame] | 16 | Suite Teardown Set REST Logging Policy ${False} |
George Keishing | 83ada4f | 2016-08-09 03:15:08 -0500 | [diff] [blame] | 17 | |
causten | 147f575 | 2016-08-11 16:24:45 -0500 | [diff] [blame] | 18 | Force Tags chassisboot |
George Keishing | eefc07a | 2016-07-26 03:25:35 -0500 | [diff] [blame] | 19 | |
George Keishing | 0af2441 | 2017-03-10 13:33:23 -0600 | [diff] [blame] | 20 | *** Variables *** |
| 21 | |
| 22 | # User may pass LOOP_COUNT. |
| 23 | # By default 2 cycle for CI/CT. |
| 24 | ${LOOP_COUNT} ${2} |
| 25 | |
George Keishing | 32ed412 | 2017-05-13 03:06:39 -0500 | [diff] [blame] | 26 | # Error strings to check from journald. |
George Keishing | 49dc0c0 | 2019-03-15 08:43:34 -0500 | [diff] [blame^] | 27 | ${ERROR_REGEX} SEGV|core-dump|FAILURE|Failed to start |
George Keishing | 22e57e1 | 2018-04-18 09:39:31 -0500 | [diff] [blame] | 28 | ${STANDBY_REGEX} Startup finished in |
| 29 | |
| 30 | # 3 minutes standby boot time. |
| 31 | ${startup_time_threshold} 180 |
George Keishing | 32ed412 | 2017-05-13 03:06:39 -0500 | [diff] [blame] | 32 | |
Gunnar Mills | 7c8923f | 2016-12-12 21:19:52 -0600 | [diff] [blame] | 33 | *** Test Cases *** |
George Keishing | eefc07a | 2016-07-26 03:25:35 -0500 | [diff] [blame] | 34 | |
George Keishing | 82462bf | 2018-09-20 23:48:43 -0500 | [diff] [blame] | 35 | Test Remote Logging Configuration |
| 36 | [Documentation] Configure remote log server and verify configuration. |
| 37 | [Tags] Test_Remote_Logging_Configuration |
| 38 | |
| 39 | # Dummy IP and port to update /etc/rsyslog.d/server.conf |
| 40 | Configure Remote Log Server With Parameters |
| 41 | ... remote_host=10.10.10.10 remote_port=514 |
| 42 | |
| 43 | Verify Rsyslog Config On BMC remote_host=10.10.10.10 remote_port=514 |
| 44 | |
| 45 | # Reset to default configuration. |
| 46 | Configure Remote Log Server With Parameters |
| 47 | ... remote_host=${EMPTY} remote_port=0 |
| 48 | |
| 49 | |
George Keishing | 78ce8dc | 2018-03-30 11:49:06 -0500 | [diff] [blame] | 50 | Verify Front And Rear LED At Standby |
| 51 | [Documentation] Front and Rear LED should be off at standby. |
| 52 | [Tags] Verify_Front_And_Rear_LED_At_Standby |
| 53 | |
| 54 | REST Power Off stack_mode=skip quiet=1 |
| 55 | Verify Identify LED State Off |
| 56 | |
George Keishing | c5fef58 | 2018-07-18 08:41:28 -0500 | [diff] [blame] | 57 | |
George Keishing | c3d2343 | 2018-08-28 00:36:38 -0500 | [diff] [blame] | 58 | Verify Application Services Running At Standby |
| 59 | [Documentation] Check if there are services that have not completed. |
| 60 | [Tags] Verify_Application_Services_Running_At_Standby |
| 61 | |
| 62 | # Application services running on the BMC are not tightly coupled. |
| 63 | # At standby, there shouldn't be any pending job waiting to complete. |
| 64 | # Examples: |
| 65 | # Failure o/p: |
| 66 | # root@witherspoon:~# systemctl list-jobs --no-pager | cat |
| 67 | # JOB UNIT TYPE STATE |
| 68 | # 35151 xyz.openbmc_project.ObjectMapper.service start running |
| 69 | # 1 jobs listed. |
| 70 | # |
| 71 | # Success o/p: |
| 72 | # root@witherspoon:~# systemctl list-jobs --no-pager | cat |
| 73 | # No jobs running. |
| 74 | |
| 75 | REST Power Off stack_mode=skip quiet=1 |
| 76 | ${stdout} ${stderr} ${rc}= BMC Execute Command |
| 77 | ... systemctl list-jobs --no-pager | cat |
| 78 | Should Be Equal As Strings ${stdout} No jobs running. |
| 79 | |
| 80 | |
George Keishing | 0a46d02 | 2017-01-30 08:11:43 -0600 | [diff] [blame] | 81 | Power On Test |
| 82 | [Documentation] Power off and on. |
| 83 | [Tags] Power_On_Test |
George Keishing | bde0915 | 2018-06-14 14:53:08 -0500 | [diff] [blame] | 84 | [Setup] Test Setup Execution |
| 85 | [Teardown] Test Teardown Execution |
George Keishing | eefc07a | 2016-07-26 03:25:35 -0500 | [diff] [blame] | 86 | |
George Keishing | 0af2441 | 2017-03-10 13:33:23 -0600 | [diff] [blame] | 87 | Repeat Keyword ${LOOP_COUNT} times Host Off And On |
George Keishing | 97e9653 | 2016-10-06 12:44:52 -0500 | [diff] [blame] | 88 | |
George Keishing | c5fef58 | 2018-07-18 08:41:28 -0500 | [diff] [blame] | 89 | |
George Keishing | 32ed412 | 2017-05-13 03:06:39 -0500 | [diff] [blame] | 90 | Check For Application Failures |
| 91 | [Documentation] Parse the journal log and check for failures. |
| 92 | [Tags] Check_For_Application_Failures |
| 93 | |
George Keishing | b4adfc7 | 2018-10-26 09:39:32 -0500 | [diff] [blame] | 94 | Check For Regex In Journald ${ERROR_REGEX} error_check=${0} boot=-b |
George Keishing | 32ed412 | 2017-05-13 03:06:39 -0500 | [diff] [blame] | 95 | |
George Keishing | 3e53aa0 | 2018-07-12 10:07:52 -0500 | [diff] [blame] | 96 | |
Michael Tritz | cb64a7c | 2018-02-05 15:25:17 -0600 | [diff] [blame] | 97 | Verify Uptime Average Against Threshold |
| 98 | [Documentation] Compare BMC average boot time to a constant threshold. |
| 99 | [Tags] Verify_Uptime_Average_Against_Threshold |
| 100 | |
| 101 | OBMC Reboot (off) |
George Keishing | 22e57e1 | 2018-04-18 09:39:31 -0500 | [diff] [blame] | 102 | |
George Keishing | 3e53aa0 | 2018-07-12 10:07:52 -0500 | [diff] [blame] | 103 | Wait Until Keyword Succeeds |
| 104 | ... 1 min 30 sec Check BMC Uptime Journald |
George Keishing | 22e57e1 | 2018-04-18 09:39:31 -0500 | [diff] [blame] | 105 | |
Michael Tritz | cb64a7c | 2018-02-05 15:25:17 -0600 | [diff] [blame] | 106 | |
George Keishing | 8a5f147 | 2018-04-04 10:08:47 -0500 | [diff] [blame] | 107 | Test SSH And IPMI Connections |
| 108 | [Documentation] Try SSH and IPMI commands to verify each connection. |
| 109 | [Tags] Test_SSH_And_IPMI_Connections |
| 110 | |
| 111 | Check If BMC Is Up 3 min 20 sec |
| 112 | Wait Until Keyword Succeeds |
| 113 | ... 3 min 30 sec Wait for BMC state Ready |
| 114 | |
| 115 | BMC Execute Command true |
| 116 | Run IPMI Standard Command chassis status |
| 117 | |
George Keishing | 97e9653 | 2016-10-06 12:44:52 -0500 | [diff] [blame] | 118 | *** Keywords *** |
George Keishing | 0a46d02 | 2017-01-30 08:11:43 -0600 | [diff] [blame] | 119 | |
George Keishing | fcc6d78 | 2018-01-24 00:44:03 -0600 | [diff] [blame] | 120 | Test Setup Execution |
| 121 | [Documentation] Do test case setup tasks. |
| 122 | Start SOL Console Logging |
| 123 | Set Auto Reboot ${0} |
| 124 | |
George Keishing | 2f0f4b3 | 2018-01-21 23:25:47 -0600 | [diff] [blame] | 125 | Test Teardown Execution |
George Keishing | 0a46d02 | 2017-01-30 08:11:43 -0600 | [diff] [blame] | 126 | [Documentation] Collect FFDC and SOL log. |
Gunnar Mills | eac1af2 | 2016-11-14 15:30:09 -0600 | [diff] [blame] | 127 | FFDC On Test Case Fail |
George Keishing | 97e9653 | 2016-10-06 12:44:52 -0500 | [diff] [blame] | 128 | ${sol_log}= Stop SOL Console Logging |
| 129 | Log ${sol_log} |
George Keishing | 2f0f4b3 | 2018-01-21 23:25:47 -0600 | [diff] [blame] | 130 | Set Auto Reboot ${1} |
George Keishing | 0af2441 | 2017-03-10 13:33:23 -0600 | [diff] [blame] | 131 | |
| 132 | Host Off And On |
| 133 | [Documentation] Verify power off and on. |
| 134 | |
| 135 | Initiate Host PowerOff |
George Keishing | 0af2441 | 2017-03-10 13:33:23 -0600 | [diff] [blame] | 136 | |
| 137 | Initiate Host Boot |
George Keishing | 766c623 | 2018-02-02 11:42:55 -0600 | [diff] [blame] | 138 | Verify OCC State |
George Keishing | 383d29b | 2017-03-25 11:10:52 -0500 | [diff] [blame] | 139 | |
George Keishing | 5f2f7f5 | 2017-06-30 09:17:53 -0500 | [diff] [blame] | 140 | # TODO: Host shutdown race condition. |
| 141 | # Wait 30 seconds before Powering Off. |
| 142 | Sleep 30s |
Michael Tritz | cb64a7c | 2018-02-05 15:25:17 -0600 | [diff] [blame] | 143 | |
George Keishing | 3e53aa0 | 2018-07-12 10:07:52 -0500 | [diff] [blame] | 144 | |
| 145 | Check BMC Uptime Journald |
| 146 | [Documentation] Check BMC journald uptime entry. |
| 147 | |
| 148 | # Example output: |
| 149 | # Startup finished in 10.074s (kernel) + 2min 23.506s (userspace) = 2min 33.581s. |
| 150 | ${startup_time} ${stderr} ${rc}= BMC Execute Command |
| 151 | ... journalctl --no-pager | egrep '${STANDBY_REGEX}' | tail -1 |
| 152 | Should Not Be Empty ${startup_time} |
| 153 | |
| 154 | # Example time conversion: |
| 155 | # Get the "2min 33.581s" string total time taken to reach standby. |
| 156 | # Convert time "2min 33.581s" to unit 153.581. |
| 157 | ${startup_time}= Convert Time ${startup_time.split("= ",1)[1].strip(".")} |
| 158 | |
| 159 | Should Be True ${startup_time} < ${startup_time_threshold} |
| 160 | ... msg=${startup_time} greater than threshold value of ${startup_time_threshold}. |