blob: 0b08f7ec77deca972b67efc7d87c899c3b0e8dbd [file] [log] [blame]
Chris Austenb29d2e82016-06-07 12:25:35 -05001*** Settings ***
Steven Sombar0acb3412018-01-24 09:38:42 -06002
3Documentation Utilities for Robot keywords.
4
Chris Austenb29d2e82016-06-07 12:25:35 -05005Resource ../lib/resource.txt
6Resource ../lib/rest_client.robot
George Keishing5e870cd2016-08-24 10:05:47 -05007Resource ../lib/connection_client.robot
Michael Walsh391ba9b2017-08-24 11:36:24 -05008Library String
George Keishing30c12ff2016-09-02 10:25:29 -05009Library DateTime
10Library Process
Chris Austenb29d2e82016-06-07 12:25:35 -050011Library OperatingSystem
Michael Walsha6723f22016-11-22 11:12:01 -060012Library gen_print.py
13Library gen_robot_print.py
Michael Walsh5f3f4142017-05-22 17:09:47 -050014Library gen_cmd.py
Sunil Macd110a2017-05-23 04:14:32 -050015Library gen_robot_keyword.py
Michael Walshe53e47a2017-06-30 17:03:24 -050016Library bmc_ssh_utils.py
Michael Walshfdc5ced2017-08-17 13:15:15 -050017Library utils.py
Charles Paul Hofer35816152017-10-02 11:55:11 -050018Library var_funcs.py
Sridevi Ramesh0d88ab32017-09-21 11:07:28 -050019Library SCPLibrary WITH NAME scp
Chris Austenb29d2e82016-06-07 12:25:35 -050020
21*** Variables ***
Steven Sombar0acb3412018-01-24 09:38:42 -060022
23${pflash_cmd} /usr/sbin/pflash -r /dev/stdout -P VERSION
24${SYSTEM_SHUTDOWN_TIME} ${5}
25
Michael Walsha6723f22016-11-22 11:12:01 -060026${dbuscmdBase}
George Keishing85ca05e2016-11-30 09:47:18 -060027... dbus-send --system --print-reply --dest=${OPENBMC_BASE_DBUS}.settings.Host
Michael Walsha6723f22016-11-22 11:12:01 -060028${dbuscmdGet}
George Keishing9485af92017-01-04 10:21:43 -060029... ${SETTINGS_URI}host0 org.freedesktop.DBus.Properties.Get
Steven Sombar0acb3412018-01-24 09:38:42 -060030${dbuscmdString}= string:"xyz.openbmc_project.settings.Host" string:
Michael Walsha6723f22016-11-22 11:12:01 -060031
32# Assign default value to QUIET for programs which may not define it.
33${QUIET} ${0}
Steven Sombar0acb3412018-01-24 09:38:42 -060034
Sivas SRRea85d1f2016-11-13 22:44:28 -060035${bmc_mem_free_cmd}= free | tr -s ' ' | sed '/^Mem/!d' | cut -d" " -f4
36${bmc_mem_total_cmd}= free | tr -s ' ' | sed '/^Mem/!d' | cut -d" " -f2
37${bmc_cpu_usage_cmd}= top -n 1 | grep CPU: | cut -c 7-9
Sridevi Ramesh1699d372016-12-06 00:20:22 -060038${HOST_SETTING} ${SETTINGS_URI}host0
Steven Sombar0acb3412018-01-24 09:38:42 -060039
Sivas SRR352bc162017-02-13 22:12:29 -060040# /run/initramfs/ro associate filesystem should be 100% full always
Sivas SRR5ff4d692017-11-23 02:53:52 -060041${bmc_file_system_usage_cmd}= df -h | cut -c 52-54 | grep 100 | wc -l
42${total_pnor_ro_file_system_cmd}= df -h | grep /media/pnor-ro | wc -l
43${total_bmc_ro_file_system_cmd}= df -h | grep /media/rofs | wc -l
Chris Austenb29d2e82016-06-07 12:25:35 -050044
George Keishingcce9df22017-01-24 06:19:33 -060045${BOOT_TIME} ${0}
46${BOOT_COUNT} ${0}
Sweta Potthuri025e0122017-02-21 00:42:48 -060047${count} ${0}
Sivas SRR8254db62017-02-07 09:39:46 -060048${devicetree_base} /sys/firmware/devicetree/base/model
George Keishingcce9df22017-01-24 06:19:33 -060049
George Keishingb7c9b542017-05-23 08:19:58 -050050# Initialize default debug value to 0.
51${DEBUG} ${0}
52
Steven Sombar0acb3412018-01-24 09:38:42 -060053${probe_cpu_tool_path} ${EXECDIR}/tools/ras/probe_cpus.sh
54${scom_addrs_tool_path} ${EXECDIR}/tools/ras/scom_addr_p9.sh
55${target_file_path} /root/
56
57${default_tarball} ${EXECDIR}/obmc-phosphor-debug-tarball-witherspoon.tar.xz
58
Michael Walshfdc5ced2017-08-17 13:15:15 -050059# These variables are used to straddle between new and old methods of setting
60# values.
Steven Sombar0acb3412018-01-24 09:38:42 -060061${boot_prog_method} ${EMPTY}
Michael Walshfdc5ced2017-08-17 13:15:15 -050062${power_policy_setup} ${0}
63${bmc_power_policy_method} ${EMPTY}
64@{valid_power_policy_vars} RESTORE_LAST_STATE ALWAYS_POWER_ON
65... ALWAYS_POWER_OFF
66
Sridevi Ramesh0d88ab32017-09-21 11:07:28 -050067
Chris Austenb29d2e82016-06-07 12:25:35 -050068*** Keywords ***
Sridevi Ramesh1699d372016-12-06 00:20:22 -060069
Sivas SRRc0957412017-10-06 13:02:08 -050070Check BMC Performance
71 [Documentation] Check BMC basic CPU Mem File system performance.
72
73 Check BMC CPU Performance
74 Check BMC Mem Performance
75 Check BMC File System Performance
76
Steven Sombar0acb3412018-01-24 09:38:42 -060077
Sivas SRRd97aa9c2017-05-22 11:19:25 -050078Verify PNOR Update
79 [Documentation] Verify that the PNOR is not corrupted.
80 # Example:
81 # FFS: Flash header not found. Code: 100
82 # Error 100 opening ffs !
83
84 Open Connection And Log In
85 ${pnor_info}= Execute Command On BMC ${pflash_cmd}
86 Should Not Contain Any ${pnor_info} Flash header not found Error
87
Steven Sombar0acb3412018-01-24 09:38:42 -060088
Sivas SRR8254db62017-02-07 09:39:46 -060089Get BMC System Model
Steven Sombar0acb3412018-01-24 09:38:42 -060090 [Documentation] Get the BMC model from the device tree and return it.
Sivas SRR8254db62017-02-07 09:39:46 -060091
Michael Walshe53e47a2017-06-30 17:03:24 -050092 ${bmc_model} ${stderr} ${rc}= BMC Execute Command
George Keishing90ab1772017-02-24 05:40:55 -060093 ... cat ${devicetree_base} | cut -d " " -f 1 return_stderr=True
Michael Walsh71182ab2017-10-20 14:44:25 -050094 ... test_mode=0
Sivas SRR8254db62017-02-07 09:39:46 -060095 Should Be Empty ${stderr}
Steven Sombar0acb3412018-01-24 09:38:42 -060096 Should Not Be Empty ${bmc_model} msg=BMC model is empty.
Sivas SRR8254db62017-02-07 09:39:46 -060097 [Return] ${bmc_model}
98
Steven Sombar0acb3412018-01-24 09:38:42 -060099
Sivas SRR8254db62017-02-07 09:39:46 -0600100Verify BMC System Model
101 [Documentation] Verify the BMC model with ${OPENBMC_MODEL}.
102 [Arguments] ${bmc_model}
103
Steven Sombar0acb3412018-01-24 09:38:42 -0600104 # Description of argument(s):
105 # bmc_model System model (e.g. "witherspoon").
106
Sivas SRR8254db62017-02-07 09:39:46 -0600107 ${tmp_bmc_model}= Fetch From Right ${OPENBMC_MODEL} /
108 ${tmp_bmc_model}= Fetch From Left ${tmp_bmc_model} .
109 ${ret}= Run Keyword And Return Status Should Contain ${bmc_model}
110 ... ${tmp_bmc_model} ignore_case=True
111 [Return] ${ret}
112
Steven Sombar0acb3412018-01-24 09:38:42 -0600113
Chris Austenb29d2e82016-06-07 12:25:35 -0500114Wait For Host To Ping
root442f0ef2016-08-04 20:23:05 +0000115 [Arguments] ${host} ${timeout}=${OPENBMC_REBOOT_TIMEOUT}min
116 ... ${interval}=5 sec
117
Steven Sombar0acb3412018-01-24 09:38:42 -0600118 # Description of argument(s):
119 # host The host name or IP of the host to ping.
120 # timeout The amount of time after which ping attempts cease.
121 # This should be expressed in Robot Framework's time format
122 # (e.g. "10 seconds").
root442f0ef2016-08-04 20:23:05 +0000123 # interval The amount of time in between attempts to ping.
Steven Sombar0acb3412018-01-24 09:38:42 -0600124 # This should be expressed in Robot Framework's time format
125 # (e.g. "5 seconds").
root442f0ef2016-08-04 20:23:05 +0000126
127 Wait Until Keyword Succeeds ${timeout} ${interval} Ping Host ${host}
Chris Austenb29d2e82016-06-07 12:25:35 -0500128
Steven Sombar0acb3412018-01-24 09:38:42 -0600129
Chris Austenb29d2e82016-06-07 12:25:35 -0500130Ping Host
131 [Arguments] ${host}
Steven Sombar0acb3412018-01-24 09:38:42 -0600132
133 # Description of argument(s):
134 # host The host name or IP of the host to ping.
135
George Keishing8a84f952016-08-25 04:54:53 -0500136 Should Not Be Empty ${host} msg=No host provided
Michael Walsha6723f22016-11-22 11:12:01 -0600137 ${RC} ${output}= Run and return RC and Output ping -c 4 ${host}
Chris Austenb29d2e82016-06-07 12:25:35 -0500138 Log RC: ${RC}\nOutput:\n${output}
139 Should be equal ${RC} ${0}
140
Steven Sombar0acb3412018-01-24 09:38:42 -0600141
Chris Austenb29d2e82016-06-07 12:25:35 -0500142Get Boot Progress
Michael Walshc972d4d2017-08-16 15:38:03 -0500143 [Documentation] Get the boot progress and return it.
Michael Walsha6723f22016-11-22 11:12:01 -0600144 [Arguments] ${quiet}=${QUIET}
145
Michael Walshc972d4d2017-08-16 15:38:03 -0500146 # Description of argument(s):
147 # quiet Indicates whether this keyword should run without any output to
148 # the console.
149
150 Set Boot Progress Method
151 ${state}= Run Keyword If '${boot_prog_method}' == 'New'
152 ... New Get Boot Progress quiet=${quiet}
153 ... ELSE
154 ... Old Get Boot Progress quiet=${quiet}
155
156 [Return] ${state}
157
Steven Sombar0acb3412018-01-24 09:38:42 -0600158
Michael Walshc972d4d2017-08-16 15:38:03 -0500159Set Boot Progress Method
160 [Documentation] Set the boot_prog_method to either 'Old' or 'New'.
161
162 # The boot progress data has moved from an 'org' location to an 'xyz'
163 # location. This keyword will determine whether the new method of getting
164 # the boot progress is valid and will set the global boot_prog_method
165 # variable accordingly. If boot_prog_method is already set (either by a
166 # prior call to this function or via a -v parm), this keyword will simply
167 # return.
168
Michael Walsh98d8c9e2017-10-04 12:24:29 -0500169 # Note: There are interim builds that contain boot_progress in both the
170 # old and the new location values. It is nearly impossible for this
171 # keyword to determine whether the old boot_progress or the new one is
172 # active. When using such builds where the old boot_progress is active,
173 # the only recourse users will have is that they may specify
174 # -v boot_prog_method:Old to force old behavior on such builds.
Michael Walshc972d4d2017-08-16 15:38:03 -0500175
176 Run Keyword If '${boot_prog_method}' != '${EMPTY}' Return From Keyword
177
178 ${new_status} ${new_value}= Run Keyword And Ignore Error
179 ... New Get Boot Progress
180 # If the new style read fails, the method must necessarily be "Old".
181 Run Keyword If '${new_status}' == 'PASS'
182 ... Run Keywords
183 ... Set Global Variable ${boot_prog_method} New AND
184 ... Rqpvars boot_prog_method AND
185 ... Return From Keyword
186
187 # Default method is "Old".
188 Set Global Variable ${boot_prog_method} Old
189 Rqpvars boot_prog_method
190
Steven Sombar0acb3412018-01-24 09:38:42 -0600191
Michael Walshc972d4d2017-08-16 15:38:03 -0500192Old Get Boot Progress
193 [Documentation] Get the boot progress the old way (via org location).
194 [Arguments] ${quiet}=${QUIET}
195
196 # Description of argument(s):
197 # quiet Indicates whether this keyword should run without any output to
198 # the console.
199
George Keishing85ca05e2016-11-30 09:47:18 -0600200 ${state}= Read Attribute ${OPENBMC_BASE_URI}sensors/host/BootProgress
Michael Walsha6723f22016-11-22 11:12:01 -0600201 ... value quiet=${quiet}
Michael Walshc972d4d2017-08-16 15:38:03 -0500202
Gunnar Millsc9ea9362016-12-13 16:21:13 -0600203 [Return] ${state}
Chris Austenb29d2e82016-06-07 12:25:35 -0500204
Steven Sombar0acb3412018-01-24 09:38:42 -0600205
Michael Walshc972d4d2017-08-16 15:38:03 -0500206New Get Boot Progress
207 [Documentation] Get the boot progress the new way (via xyz location).
208 [Arguments] ${quiet}=${QUIET}
209
210 # Description of argument(s):
211 # quiet Indicates whether this keyword should run without any output to
212 # the console.
213
214 ${state}= Read Attribute ${HOST_STATE_URI} BootProgress quiet=${quiet}
215
216 [Return] ${state.rsplit('.', 1)[1]}
217
Steven Sombar0acb3412018-01-24 09:38:42 -0600218
Chris Austenb29d2e82016-06-07 12:25:35 -0500219Is Power On
root442f0ef2016-08-04 20:23:05 +0000220 ${state}= Get Power State
221 Should be equal ${state} ${1}
Chris Austenb29d2e82016-06-07 12:25:35 -0500222
Steven Sombar0acb3412018-01-24 09:38:42 -0600223
Chris Austenb29d2e82016-06-07 12:25:35 -0500224Is Power Off
root442f0ef2016-08-04 20:23:05 +0000225 ${state}= Get Power State
226 Should be equal ${state} ${0}
Chris Austenb29d2e82016-06-07 12:25:35 -0500227
Steven Sombar0acb3412018-01-24 09:38:42 -0600228
root442f0ef2016-08-04 20:23:05 +0000229Initiate Power On
230 [Documentation] Initiates the power on and waits until the Is Power On
231 ... keyword returns that the power state has switched to on.
Michael Walsha6723f22016-11-22 11:12:01 -0600232 [Arguments] ${wait}=${1}
233
Steven Sombar0acb3412018-01-24 09:38:42 -0600234 # Description of argument(s):
235 # wait Indicates whether to wait for a powered on state after issuing
236 # the power on command.
237
Chris Austenb29d2e82016-06-07 12:25:35 -0500238 @{arglist}= Create List
239 ${args}= Create Dictionary data=@{arglist}
George Keishing85ca05e2016-11-30 09:47:18 -0600240 ${resp}= Call Method ${OPENBMC_BASE_URI}control/chassis0/ powerOn
Michael Walsha6723f22016-11-22 11:12:01 -0600241 ... data=${args}
Chris Austenb29d2e82016-06-07 12:25:35 -0500242 should be equal as strings ${resp.status_code} ${HTTP_OK}
Chris Austenb29d2e82016-06-07 12:25:35 -0500243
Michael Walsha6723f22016-11-22 11:12:01 -0600244 # Does caller want to wait for power on status?
245 Run Keyword If '${wait}' == '${0}' Return From Keyword
246 Wait Until Keyword Succeeds 3 min 10 sec Is Power On
Rahul Maheshwarif684ba72016-10-25 07:24:41 -0500247
Steven Sombar0acb3412018-01-24 09:38:42 -0600248
root442f0ef2016-08-04 20:23:05 +0000249Initiate Power Off
250 [Documentation] Initiates the power off and waits until the Is Power Off
251 ... keyword returns that the power state has switched to off.
Steven Sombar0acb3412018-01-24 09:38:42 -0600252
Chris Austenb29d2e82016-06-07 12:25:35 -0500253 @{arglist}= Create List
254 ${args}= Create Dictionary data=@{arglist}
George Keishing85ca05e2016-11-30 09:47:18 -0600255 ${resp}= Call Method ${OPENBMC_BASE_URI}control/chassis0/ powerOff
Michael Walsha6723f22016-11-22 11:12:01 -0600256 ... data=${args}
Chris Austenb29d2e82016-06-07 12:25:35 -0500257 should be equal as strings ${resp.status_code} ${HTTP_OK}
Michael Walsha6723f22016-11-22 11:12:01 -0600258 Wait Until Keyword Succeeds 1 min 10 sec Is Power Off
Chris Austenb29d2e82016-06-07 12:25:35 -0500259
Steven Sombar0acb3412018-01-24 09:38:42 -0600260
Michael Walsh81843772017-03-02 14:44:44 -0600261Initiate OS Host Power Off
262 [Documentation] Initiate an OS reboot.
263 [Arguments] ${os_host}=${OS_HOST} ${os_username}=${OS_USERNAME}
Michael Walshdfc75372018-01-11 11:47:25 -0600264 ... ${os_password}=${OS_PASSWORD} ${hard}=${0}
Michael Walsh81843772017-03-02 14:44:44 -0600265
Michael Walshdfc75372018-01-11 11:47:25 -0600266 # Description of argument(s):
Michael Walsh81843772017-03-02 14:44:44 -0600267 # os_host The DNS name or IP of the OS.
268 # os_username The username to be used to sign in to the OS.
269 # os_password The password to be used to sign in to the OS.
Michael Walshdfc75372018-01-11 11:47:25 -0600270 # hard Indicates whether to do a hard vs. soft power off.
271
272 ${time_string}= Run Keyword If ${hard} Set Variable ${SPACE}now
273 ... ELSE Set Variable ${EMPTY}
Michael Walsh81843772017-03-02 14:44:44 -0600274
Michael Walsh4549cd12017-09-07 18:02:36 -0500275 ${cmd_buf}= Run Keyword If '${os_username}' == 'root'
Michael Walshdfc75372018-01-11 11:47:25 -0600276 ... Set Variable shutdown${time_string}
Michael Walsh4549cd12017-09-07 18:02:36 -0500277 ... ELSE
Michael Walshdfc75372018-01-11 11:47:25 -0600278 ... Set Variable echo ${os_password} | sudo -S shutdown${time_string}
Michael Walsh4549cd12017-09-07 18:02:36 -0500279
280 ${output} ${stderr} ${rc}= OS Execute Command
281 ... ${cmd_buf} fork=${1}
Michael Walsh81843772017-03-02 14:44:44 -0600282
Steven Sombar0acb3412018-01-24 09:38:42 -0600283
Michael Walsh81843772017-03-02 14:44:44 -0600284Initiate OS Host Reboot
285 [Documentation] Initiate an OS reboot.
286 [Arguments] ${os_host}=${OS_HOST} ${os_username}=${OS_USERNAME}
287 ... ${os_password}=${OS_PASSWORD}
288
Michael Walshf1a59fb2017-09-01 17:07:13 -0500289 # Description of argument(s):
290 # os_host The host name or IP address of the OS.
Michael Walsh81843772017-03-02 14:44:44 -0600291 # os_username The username to be used to sign in to the OS.
292 # os_password The password to be used to sign in to the OS.
293
Michael Walshf1a59fb2017-09-01 17:07:13 -0500294 ${cmd_buf}= Run Keyword If '${os_username}' == 'root'
295 ... Set Variable reboot
296 ... ELSE
297 ... Set Variable echo ${os_password} | sudo -S reboot
298
299 ${output} ${stderr} ${rc}= OS Execute Command
300 ... ${cmd_buf} fork=${1}
Michael Walsh81843772017-03-02 14:44:44 -0600301
Steven Sombar0acb3412018-01-24 09:38:42 -0600302
Michael Walsh81843772017-03-02 14:44:44 -0600303Initiate Auto Reboot
304 [Documentation] Initiate an auto reboot.
305
306 # Set the auto reboot policy.
Sweta Potthuri62a23572017-07-28 04:05:13 -0500307 Set Auto Reboot ${1}
Michael Walsh81843772017-03-02 14:44:44 -0600308 # Set the watchdog timer. Note: 5000 = milliseconds which is 5 seconds.
George Keishing37cfa6b2017-06-15 10:25:55 -0500309 Trigger Host Watchdog Error 5000
Michael Walsh81843772017-03-02 14:44:44 -0600310
Steven Sombar0acb3412018-01-24 09:38:42 -0600311
Chris Austenb29d2e82016-06-07 12:25:35 -0500312Trigger Warm Reset
Steven Sombar0acb3412018-01-24 09:38:42 -0600313 [Documentation] Initiate a warm reset.
314
Chris Austenb29d2e82016-06-07 12:25:35 -0500315 log to console "Triggering warm reset"
Michael Walsha6723f22016-11-22 11:12:01 -0600316 ${data}= create dictionary data=@{EMPTY}
George Keishing85ca05e2016-11-30 09:47:18 -0600317 ${resp}= openbmc post request
318 ... ${OPENBMC_BASE_URI}control/bmc0/action/warmReset data=${data}
Chris Austenb29d2e82016-06-07 12:25:35 -0500319 Should Be Equal As Strings ${resp.status_code} ${HTTP_OK}
George Keishingb3700812016-08-31 03:03:30 -0500320 ${session_active}= Check If warmReset is Initiated
321 Run Keyword If '${session_active}' == '${True}'
322 ... Fail msg=warm reset didn't occur
323
Chris Austenb29d2e82016-06-07 12:25:35 -0500324 Sleep ${SYSTEM_SHUTDOWN_TIME}min
Rahul Maheshwari5f253c42017-01-30 05:19:51 -0600325 Check If BMC Is Up
Michael Walsh49ab0f42016-07-20 11:44:33 -0500326
Steven Sombar0acb3412018-01-24 09:38:42 -0600327
Michael Walsh49ab0f42016-07-20 11:44:33 -0500328Check OS
root442f0ef2016-08-04 20:23:05 +0000329 [Documentation] Attempts to ping the host OS and then checks that the host
330 ... OS is up by running an SSH command.
Michael Walsh49ab0f42016-07-20 11:44:33 -0500331
332 [Arguments] ${os_host}=${OS_HOST} ${os_username}=${OS_USERNAME}
Michael Walsha6723f22016-11-22 11:12:01 -0600333 ... ${os_password}=${OS_PASSWORD} ${quiet}=${QUIET}
334 ... ${print_string}=${EMPTY}
Michael Walshac275512017-03-07 11:39:28 -0600335 [Teardown] SSHLibrary.Close Connection
Michael Walsh49ab0f42016-07-20 11:44:33 -0500336
Steven Sombar0acb3412018-01-24 09:38:42 -0600337 # Description of argument(s):
Michael Walsh49ab0f42016-07-20 11:44:33 -0500338 # os_host The DNS name/IP of the OS host associated with our BMC.
339 # os_username The username to be used to sign on to the OS host.
340 # os_password The password to be used to sign on to the OS host.
Michael Walsha6723f22016-11-22 11:12:01 -0600341 # quiet Indicates whether this keyword should write to console.
342 # print_string A string to be printed before checking the OS.
343
344 rprint ${print_string}
Michael Walsh49ab0f42016-07-20 11:44:33 -0500345
root442f0ef2016-08-04 20:23:05 +0000346 # Attempt to ping the OS. Store the return code to check later.
347 ${ping_rc}= Run Keyword and Return Status Ping Host ${os_host}
348
Michael Walshac275512017-03-07 11:39:28 -0600349 SSHLibrary.Open connection ${os_host}
Michael Walsh49ab0f42016-07-20 11:44:33 -0500350
Michael Walsha6723f22016-11-22 11:12:01 -0600351 ${status} ${msg}= Run Keyword And Ignore Error Login ${os_username}
352 ... ${os_password}
353 ${err_msg1}= Sprint Error ${msg}
354 ${err_msg}= Catenate SEPARATOR= \n ${err_msg1}
355 Run Keyword If '${status}' == 'FAIL' Fail msg=${err_msg}
Michael Walsh49ab0f42016-07-20 11:44:33 -0500356 ${output} ${stderr} ${rc}= Execute Command uptime return_stderr=True
357 ... return_rc=True
358
Michael Walsha6723f22016-11-22 11:12:01 -0600359 ${temp_msg}= Catenate Could not execute a command on the operating
360 ... system.\n
361 ${err_msg1}= Sprint Error ${temp_msg}
362 ${err_msg}= Catenate SEPARATOR= \n ${err_msg1}
363
364 # If the return code returned by "Execute Command" is non-zero, this
365 # keyword will fail.
366 Should Be Equal ${rc} ${0} msg=${err_msg}
Michael Walsh49ab0f42016-07-20 11:44:33 -0500367 # We will likewise fail if there is any stderr data.
368 Should Be Empty ${stderr}
369
Michael Walsha6723f22016-11-22 11:12:01 -0600370 ${temp_msg}= Set Variable Could not ping the operating system.\n
371 ${err_msg1}= Sprint Error ${temp_msg}
372 ${err_msg}= Catenate SEPARATOR= \n ${err_msg1}
373 # We will likewise fail if the OS did not ping, as we could SSH but not
374 # ping
375 Should Be Equal As Strings ${ping_rc} ${TRUE} msg=${err_msg}
root442f0ef2016-08-04 20:23:05 +0000376
Steven Sombar0acb3412018-01-24 09:38:42 -0600377
Michael Walsh49ab0f42016-07-20 11:44:33 -0500378Wait for OS
379 [Documentation] Waits for the host OS to come up via calls to "Check OS".
380 [Arguments] ${os_host}=${OS_HOST} ${os_username}=${OS_USERNAME}
381 ... ${os_password}=${OS_PASSWORD} ${timeout}=${OS_WAIT_TIMEOUT}
Michael Walsha6723f22016-11-22 11:12:01 -0600382 ... ${quiet}=${0}
383 [Teardown] rprintn
Michael Walsh49ab0f42016-07-20 11:44:33 -0500384
Steven Sombar0acb3412018-01-24 09:38:42 -0600385 # Description of argument(s):
Michael Walsh49ab0f42016-07-20 11:44:33 -0500386 # os_host The DNS name or IP of the OS host associated with our
387 # BMC.
388 # os_username The username to be used to sign on to the OS host.
389 # os_password The password to be used to sign on to the OS host.
390 # timeout The timeout in seconds indicating how long you're
391 # willing to wait for the OS to respond.
Michael Walsha6723f22016-11-22 11:12:01 -0600392 # quiet Indicates whether this keyword should write to console.
Michael Walsh49ab0f42016-07-20 11:44:33 -0500393
394 # The interval to be used between calls to "Check OS".
395 ${interval}= Set Variable 5
396
Michael Walsha6723f22016-11-22 11:12:01 -0600397 ${message}= Catenate Checking every ${interval} seconds for up to
398 ... ${timeout} seconds for the operating system to communicate.
399 rqprint_timen ${message}
400
Michael Walsh49ab0f42016-07-20 11:44:33 -0500401 Wait Until Keyword Succeeds ${timeout} sec ${interval} Check OS
402 ... ${os_host} ${os_username} ${os_password}
Michael Walsha6723f22016-11-22 11:12:01 -0600403 ... print_string=\#
404
405 rqprintn
406
407 rqprint_timen The operating system is now communicating.
root442f0ef2016-08-04 20:23:05 +0000408
root442f0ef2016-08-04 20:23:05 +0000409
410Get Power State
411 [Documentation] Returns the power state as an integer. Either 0 or 1.
Michael Walsha6723f22016-11-22 11:12:01 -0600412 [Arguments] ${quiet}=${QUIET}
413
Steven Sombar0acb3412018-01-24 09:38:42 -0600414 # Description of argument(s):
415 # quiet Indicates whether this keyword should run without any output to
416 # the console.
417
root442f0ef2016-08-04 20:23:05 +0000418 @{arglist}= Create List
419 ${args}= Create Dictionary data=@{arglist}
Michael Walsha6723f22016-11-22 11:12:01 -0600420
George Keishing85ca05e2016-11-30 09:47:18 -0600421 ${resp}= Call Method ${OPENBMC_BASE_URI}control/chassis0/ getPowerState
Michael Walsha6723f22016-11-22 11:12:01 -0600422 ... data=${args} quiet=${quiet}
root442f0ef2016-08-04 20:23:05 +0000423 Should be equal as strings ${resp.status_code} ${HTTP_OK}
424 ${content}= to json ${resp.content}
Steven Sombar0acb3412018-01-24 09:38:42 -0600425
Gunnar Millsc9ea9362016-12-13 16:21:13 -0600426 [Return] ${content["data"]}
Jay Azurine4c52eb2016-08-16 20:51:10 -0500427
George Keishinga8d68e22017-12-18 09:30:30 -0600428
429Clear BMC Gard Record
430 [Documentation] Clear gard records from the system.
431
432 @{arglist}= Create List
433 ${args}= Create Dictionary data=@{arglist}
George Keishing85ca05e2016-11-30 09:47:18 -0600434 ${resp}= Call Method
George Keishinga8d68e22017-12-18 09:30:30 -0600435 ... ${OPENPOWER_CONTROL}gard Reset data=${args}
436 Should Be Equal As Strings ${resp.status_code} ${HTTP_OK}
437
Jay Azurine4c52eb2016-08-16 20:51:10 -0500438
439Copy PNOR to BMC
440 Import Library SCPLibrary WITH NAME scp
441 Open Connection for SCP
442 Log Copying ${PNOR_IMAGE_PATH} to /tmp
443 scp.Put File ${PNOR_IMAGE_PATH} /tmp
444
Steven Sombar0acb3412018-01-24 09:38:42 -0600445
Jay Azurine4c52eb2016-08-16 20:51:10 -0500446Flash PNOR
447 [Documentation] Calls flash bios update method to flash PNOR image
Gunnar Mills38032802016-12-12 13:43:40 -0600448 [Arguments] ${pnor_image}
Steven Sombar0acb3412018-01-24 09:38:42 -0600449
450 # Description of argument(s):
451 # pnor_image The filename and path of the PNOR image
452 # (e.g. "/home/image/zaius.pnor").
453
Jay Azurine4c52eb2016-08-16 20:51:10 -0500454 @{arglist}= Create List ${pnor_image}
455 ${args}= Create Dictionary data=@{arglist}
George Keishinge24fdbc2017-10-31 13:41:35 -0500456 ${resp}= Call Method /org/openbmc/control/flash/bios/ update
Michael Walsha6723f22016-11-22 11:12:01 -0600457 ... data=${args}
Jay Azurine4c52eb2016-08-16 20:51:10 -0500458 should be equal as strings ${resp.status_code} ${HTTP_OK}
459 Wait Until Keyword Succeeds 2 min 10 sec Is PNOR Flashing
460
Steven Sombar0acb3412018-01-24 09:38:42 -0600461
Jay Azurine4c52eb2016-08-16 20:51:10 -0500462Get Flash BIOS Status
463 [Documentation] Returns the status of the flash BIOS API as a string. For
464 ... example 'Flashing', 'Flash Done', etc
George Keishinge24fdbc2017-10-31 13:41:35 -0500465 ${data}= Read Properties /org/openbmc/control/flash/bios
Gunnar Millsc9ea9362016-12-13 16:21:13 -0600466 [Return] ${data['status']}
Jay Azurine4c52eb2016-08-16 20:51:10 -0500467
Steven Sombar0acb3412018-01-24 09:38:42 -0600468
Jay Azurine4c52eb2016-08-16 20:51:10 -0500469Is PNOR Flashing
470 [Documentation] Get BIOS 'Flashing' status. This indicates that PNOR
471 ... flashing has started.
472 ${status}= Get Flash BIOS Status
Rahul Maheshwari449dfb72017-05-08 02:06:48 -0500473 Should Contain ${status} Flashing
Jay Azurine4c52eb2016-08-16 20:51:10 -0500474
Steven Sombar0acb3412018-01-24 09:38:42 -0600475
Jay Azurine4c52eb2016-08-16 20:51:10 -0500476Is PNOR Flash Done
477 [Documentation] Get BIOS 'Flash Done' status. This indicates that the
478 ... PNOR flashing has completed.
Leah McNutt9ba32272016-11-17 15:48:39 +0000479 ${status}= Get Flash BIOS Status
Jay Azurine4c52eb2016-08-16 20:51:10 -0500480 should be equal as strings ${status} Flash Done
481
George Keishing5e870cd2016-08-24 10:05:47 -0500482
Rahul Maheshwari2c725042017-01-29 22:55:28 -0600483Is OS Starting
484 [Documentation] Check if boot progress is OS starting.
485 ${boot_progress}= Get Boot Progress
Michael Walsh01975fa2017-08-20 20:51:36 -0500486 Should Be Equal ${boot_progress} OSStart
Rahul Maheshwari2c725042017-01-29 22:55:28 -0600487
Steven Sombar0acb3412018-01-24 09:38:42 -0600488
George Keishing0af24412017-03-10 13:33:23 -0600489Is OS Off
490 [Documentation] Check if boot progress is "Off".
491 ${boot_progress}= Get Boot Progress
492 Should Be Equal ${boot_progress} Off
493
Steven Sombar0acb3412018-01-24 09:38:42 -0600494
Rahul Maheshwariffd30de2017-06-27 05:57:45 -0500495Get Boot Progress To OS Starting State
496 [Documentation] Get the system to a boot progress state of 'FW Progress,
497 ... Starting OS'.
498
499 ${boot_progress}= Get Boot Progress
Michael Walsh01975fa2017-08-20 20:51:36 -0500500 Run Keyword If '${boot_progress}' == 'OSStart'
Rahul Maheshwariffd30de2017-06-27 05:57:45 -0500501 ... Log Host is already in OS starting state
502 ... ELSE
503 ... Run Keywords Initiate Host PowerOff AND Initiate Host Boot
504 ... AND Wait Until Keyword Succeeds 10 min 10 sec Is OS Starting
505
Steven Sombar0acb3412018-01-24 09:38:42 -0600506
George Keishing5e870cd2016-08-24 10:05:47 -0500507Verify Ping and REST Authentication
Michael Walsha6723f22016-11-22 11:12:01 -0600508 ${l_ping}= Run Keyword And Return Status
George Keishing5e870cd2016-08-24 10:05:47 -0500509 ... Ping Host ${OPENBMC_HOST}
George Keishingc4d3dc02016-09-19 03:45:55 -0500510 Run Keyword If '${l_ping}' == '${False}'
511 ... Fail msg=Ping Failed
George Keishing5e870cd2016-08-24 10:05:47 -0500512
Michael Walsha6723f22016-11-22 11:12:01 -0600513 ${l_rest}= Run Keyword And Return Status
George Keishing5e870cd2016-08-24 10:05:47 -0500514 ... Initialize OpenBMC
George Keishingc4d3dc02016-09-19 03:45:55 -0500515 Run Keyword If '${l_rest}' == '${False}'
516 ... Fail msg=REST Authentication Failed
George Keishing5e870cd2016-08-24 10:05:47 -0500517
518 # Just to make sure the SSH is working for SCP
519 Open Connection And Log In
520 ${system} ${stderr}= Execute Command hostname return_stderr=True
521 Should Be Empty ${stderr}
522
Steven Sombar0acb3412018-01-24 09:38:42 -0600523
George Keishingc4d3dc02016-09-19 03:45:55 -0500524Check If BMC is Up
525 [Documentation] Wait for Host to be online. Checks every X seconds
526 ... interval for Y minutes and fails if timed out.
527 ... Default MAX timedout is 10 min, interval 10 seconds.
Gunnar Mills38032802016-12-12 13:43:40 -0600528 [Arguments] ${max_timeout}=${OPENBMC_REBOOT_TIMEOUT} min
George Keishingc4d3dc02016-09-19 03:45:55 -0500529 ... ${interval}=10 sec
530
Steven Sombar0acb3412018-01-24 09:38:42 -0600531 # Description of argument(s):
532 # max_timeout Maximum time to wait.
533 # This should be expressed in Robot Framework's time format
534 # (e.g. "10 minutes").
535 # interfal Interval to wait between status checks.
536 # This should be expressed in Robot Framework's time format
537 # (e.g. "5 seconds").
538
George Keishingc4d3dc02016-09-19 03:45:55 -0500539 Wait Until Keyword Succeeds
540 ... ${max_timeout} ${interval} Verify Ping and REST Authentication
541
George Keishingb3700812016-08-31 03:03:30 -0500542
543Check If warmReset is Initiated
544 [Documentation] Ping would be still alive, so try SSH to connect
545 ... if fails the ports are down indicating reboot
546 ... is in progress
George Keishing3c05d352016-12-13 06:54:50 -0600547
548 # Warm reset adds 3 seconds delay before forcing reboot
549 # To minimize race conditions, we wait for 7 seconds
550 Sleep 7s
George Keishingb3700812016-08-31 03:03:30 -0500551 ${alive}= Run Keyword and Return Status
552 ... Open Connection And Log In
553 Return From Keyword If '${alive}' == '${False}' ${False}
Gunnar Millsc9ea9362016-12-13 16:21:13 -0600554 [Return] ${True}
George Keishingb3700812016-08-31 03:03:30 -0500555
Steven Sombar0acb3412018-01-24 09:38:42 -0600556
George Keishing06ae4aa2016-08-30 01:41:28 -0500557Flush REST Sessions
558 [Documentation] Removes all the active session objects
559 Delete All Sessions
George Keishingb3700812016-08-31 03:03:30 -0500560
Steven Sombar0acb3412018-01-24 09:38:42 -0600561
Sivas SRRe1143ae2016-08-26 22:31:02 -0500562Initialize DBUS cmd
563 [Documentation] Initialize dbus string with property string to extract
Gunnar Mills38032802016-12-12 13:43:40 -0600564 [Arguments] ${boot_property}
Steven Sombar0acb3412018-01-24 09:38:42 -0600565
566 # Description of argument(s):
567 # boot_property Property string.
568
Michael Walsha6723f22016-11-22 11:12:01 -0600569 ${cmd}= Catenate ${dbuscmdBase} ${dbuscmdGet} ${dbuscmdString}
570 ${cmd}= Catenate ${cmd}${boot_property}
Sivas SRRe1143ae2016-08-26 22:31:02 -0500571 Set Global Variable ${dbuscmd} ${cmd}
572
Steven Sombar0acb3412018-01-24 09:38:42 -0600573
David Shawba2d2c22017-01-23 16:56:38 -0600574Create OS Console File Path
575 [Documentation] Create OS console file path name and return it.
576 [Arguments] ${log_file_path}=${EMPTY}
George Keishing30c12ff2016-09-02 10:25:29 -0500577
Gunnar Mills28e403b2017-10-25 16:16:38 -0500578 # Description of arguments:
David Shawba2d2c22017-01-23 16:56:38 -0600579 # file_path The caller's candidate value. If this value is ${EMPTY}, this
580 # keyword will compose a file path name. Otherwise, this
581 # keyword will use the caller's file_path value. In either
582 # case, the value will be returned.
Gunnar Mills9c760342016-12-08 10:12:06 -0600583
George Keishing3deb7d12017-09-14 07:50:00 -0500584 ${default_file_path}= Catenate /tmp/${OPENBMC_HOST}_os_console.txt
David Shawba2d2c22017-01-23 16:56:38 -0600585 ${log_file_path}= Set Variable If '${log_file_path}' == '${EMPTY}'
586 ... ${default_file_path} ${log_file_path}
Michael Walsh01bd3682017-01-10 11:21:05 -0600587
David Shawba2d2c22017-01-23 16:56:38 -0600588 [Return] ${log_file_path}
Gunnar Mills9c760342016-12-08 10:12:06 -0600589
Steven Sombar0acb3412018-01-24 09:38:42 -0600590
David Shawba2d2c22017-01-23 16:56:38 -0600591Create OS Console Command String
592 [Documentation] Return a command string to start OS console logging.
Gunnar Mills9c760342016-12-08 10:12:06 -0600593
David Shawba2d2c22017-01-23 16:56:38 -0600594 # First make sure that the ssh_pw program is available.
Steven Sombar0acb3412018-01-24 09:38:42 -0600595 ${cmd}= Catenate which ssh_pw 2>/dev/null || find
596 ... ${EXECDIR} -name 'ssh_pw'
597
598 Rdpissuing ${cmd}
599 ${rc} ${output}= Run And Return Rc And Output ${cmd}
David Shawba2d2c22017-01-23 16:56:38 -0600600 Rdpvars rc output
Gunnar Mills9c760342016-12-08 10:12:06 -0600601
Michael Walsh98d8c9e2017-10-04 12:24:29 -0500602 Should Be Equal As Integers 0 ${rc} msg=Could not find ssh_pw.
603
604 ${ssh_pw_file_path}= Set Variable ${output}
George Keishingb45d9522017-06-29 04:20:59 -0500605
Steven Sombar0acb3412018-01-24 09:38:42 -0600606 ${cmd}= Catenate ${ssh_pw_file_path} ${OPENBMC_PASSWORD} -p 2200
George Keishing1f9ec7f2017-06-06 08:59:19 -0500607 ... -o "StrictHostKeyChecking no" ${OPENBMC_USERNAME}@${OPENBMC_HOST}
George Keishing30c12ff2016-09-02 10:25:29 -0500608
Steven Sombar0acb3412018-01-24 09:38:42 -0600609 [Return] ${cmd}
610
George Keishing30c12ff2016-09-02 10:25:29 -0500611
Michael Walshcadf1342017-08-28 13:49:25 -0500612Get SOL Console Pid
Michael Walsh70419172017-12-06 14:20:52 -0600613 [Documentation] Get the pid of the active SOL console job.
614 [Arguments] ${expect_running}=${0}
615
616 # Description of argument(s):
617 # expect_running If set and if no SOL console job is found, print debug
618 # info and fail.
Michael Walshcadf1342017-08-28 13:49:25 -0500619
620 # Find the pid of the active system console logging session (if any).
621 ${search_string}= Create OS Console Command String
622 # At least in some cases, ps output does not show double quotes so we must
623 # replace them in our search string with the regexes to indicate that they
624 # are optional.
625 ${search_string}= Replace String ${search_string} " ["]?
Michael Walsh70419172017-12-06 14:20:52 -0600626 ${ps_cmd}= Catenate ps axwwo user,pid,cmd
627 ${cmd_buf}= Catenate echo $(${ps_cmd} | egrep '${search_string}' |
628 ... egrep -v grep | cut -c10-14)
Michael Walshcadf1342017-08-28 13:49:25 -0500629 Rdpissuing ${cmd_buf}
630 ${rc} ${os_con_pid}= Run And Return Rc And Output ${cmd_buf}
631 Rdpvars os_con_pid
632 # If rc is not zero it just means that there is no OS Console process
633 # running.
634
Michael Walsh70419172017-12-06 14:20:52 -0600635 Return From Keyword If '${os_con_pid}' != '${EMPTY}' ${os_con_pid}
636 Return From Keyword If '${expect_running}' == '${0}' ${os_con_pid}
637
638 Cmd Fnc cat ${log_file_path} ; echo ; ${ps_cmd} quiet=${0}
639 ... print_output=${1} show_err=${1}
640
641 Should Not Be Empty ${os_con_pid}
Michael Walshcadf1342017-08-28 13:49:25 -0500642
643
George Keishing30c12ff2016-09-02 10:25:29 -0500644Stop SOL Console Logging
David Shawba2d2c22017-01-23 16:56:38 -0600645 [Documentation] Stop system console logging and return log output.
George Keishing72992e42017-06-16 06:42:36 -0500646 [Arguments] ${log_file_path}=${EMPTY}
647 ... ${targ_file_path}=${EXECDIR}${/}logs${/}
Michael Walsh5f3f4142017-05-22 17:09:47 -0500648 ... ${return_data}=${1}
David Shawba2d2c22017-01-23 16:56:38 -0600649
650 # If there are muliple system console processes, they will all be stopped.
651 # If there is no existing log file this keyword will return an error
652 # message to that effect (and write that message to targ_file_path, if
653 # specified).
654 # NOTE: This keyword will not fail if there is no running system console
655 # process.
Michael Walsh01bd3682017-01-10 11:21:05 -0600656
657 # Description of arguments:
David Shawba2d2c22017-01-23 16:56:38 -0600658 # log_file_path The file path that was used to call "Start SOL
659 # Console Logging". See that keyword (above) for details.
660 # targ_file_path If specified, the file path to which the source
661 # file path (i.e. "log_file_path") should be copied.
Michael Walsh5f3f4142017-05-22 17:09:47 -0500662 # return_data If this is set to ${1}, this keyword will return the SOL
663 # data to the caller as a unicode string.
George Keishing30c12ff2016-09-02 10:25:29 -0500664
David Shawba2d2c22017-01-23 16:56:38 -0600665 ${log_file_path}= Create OS Console File Path ${log_file_path}
Michael Walshcadf1342017-08-28 13:49:25 -0500666
667 ${os_con_pid}= Get SOL Console Pid
George Keishing30c12ff2016-09-02 10:25:29 -0500668
David Shawba2d2c22017-01-23 16:56:38 -0600669 ${cmd_buf}= Catenate kill -9 ${os_con_pid}
670 Run Keyword If '${os_con_pid}' != '${EMPTY}' Rdpissuing ${cmd_buf}
671 ${rc} ${output}= Run Keyword If '${os_con_pid}' != '${EMPTY}'
672 ... Run And Return Rc And Output ${cmd_buf}
673 Run Keyword If '${os_con_pid}' != '${EMPTY}' Rdpvars rc output
George Keishing30c12ff2016-09-02 10:25:29 -0500674
Michael Walsh01bd3682017-01-10 11:21:05 -0600675 Run Keyword If '${targ_file_path}' != '${EMPTY}'
David Shawba2d2c22017-01-23 16:56:38 -0600676 ... Run Keyword And Ignore Error
677 ... Copy File ${log_file_path} ${targ_file_path}
George Keishing30c12ff2016-09-02 10:25:29 -0500678
Michael Walsh5f3f4142017-05-22 17:09:47 -0500679 ${output}= Set Variable ${EMPTY}
680 ${loc_quiet}= Evaluate ${debug}^1
681 ${rc} ${output}= Run Keyword If '${return_data}' == '${1}'
Michael Walsh98d8c9e2017-10-04 12:24:29 -0500682 ... Cmd Fnc cat ${log_file_path} 2>/dev/null quiet=${loc_quiet}
683 ... print_output=${0} show_err=${0}
Michael Walsh5f3f4142017-05-22 17:09:47 -0500684
David Shawba2d2c22017-01-23 16:56:38 -0600685 [Return] ${output}
686
Steven Sombar0acb3412018-01-24 09:38:42 -0600687
David Shawba2d2c22017-01-23 16:56:38 -0600688Start SOL Console Logging
689 [Documentation] Start system console log to file.
Michael Walsh5f3f4142017-05-22 17:09:47 -0500690 [Arguments] ${log_file_path}=${EMPTY} ${return_data}=${1}
David Shawba2d2c22017-01-23 16:56:38 -0600691
692 # This keyword will first call "Stop SOL Console Logging". Only then will
693 # it start SOL console logging. The data returned by "Stop SOL Console
694 # Logging" will in turn be returned by this keyword.
695
696 # Description of arguments:
Michael Walsh5f3f4142017-05-22 17:09:47 -0500697 # log_file_path The file path to which system console log data should be
698 # written. Note that this path is taken to be a location
699 # on the machine where this program is running rather than
700 # on the Open BMC system.
701 # return_data If this is set to ${1}, this keyword will return any SOL
702 # data to the caller as a unicode string.
David Shawba2d2c22017-01-23 16:56:38 -0600703
704 ${log_file_path}= Create OS Console File Path ${log_file_path}
705
706 ${log_output}= Stop SOL Console Logging ${log_file_path}
Michael Walsh5f3f4142017-05-22 17:09:47 -0500707 ... return_data=${return_data}
David Shawba2d2c22017-01-23 16:56:38 -0600708
709 # Validate by making sure we can create the file. Problems creating the
710 # file would not be noticed by the subsequent ssh command because we fork
711 # the command.
712 Create File ${log_file_path}
713 ${sub_cmd_buf}= Create OS Console Command String
714 # Routing stderr to stdout so that any startup error text will go to the
715 # output file.
George Keishing8c3c1df2017-09-13 13:50:20 -0500716 # TODO: Doesn't work with tox so reverting temporarily.
Michael Walsh460042e2017-09-05 16:55:49 -0500717 # nohup detaches the process completely from our pty.
George Keishing8c3c1df2017-09-13 13:50:20 -0500718 #${cmd_buf}= Catenate nohup ${sub_cmd_buf} &> ${log_file_path} &
719 ${cmd_buf}= Catenate ${sub_cmd_buf} > ${log_file_path} 2>&1 &
David Shawba2d2c22017-01-23 16:56:38 -0600720 Rdpissuing ${cmd_buf}
721 ${rc} ${output}= Run And Return Rc And Output ${cmd_buf}
722 # Because we are forking this command, we essentially will never get a
723 # non-zero return code or any output.
724 Should Be Equal ${rc} ${0}
725
Michael Walsh70419172017-12-06 14:20:52 -0600726 Wait Until Keyword Succeeds 10 seconds 0 seconds
727 ... Get SOL Console Pid ${1}
Michael Walshcadf1342017-08-28 13:49:25 -0500728
David Shawba2d2c22017-01-23 16:56:38 -0600729 [Return] ${log_output}
George Keishing30c12ff2016-09-02 10:25:29 -0500730
Michael Walsh70419172017-12-06 14:20:52 -0600731
George Keishing30c12ff2016-09-02 10:25:29 -0500732Get Time Stamp
733 [Documentation] Get the current time stamp data
734 ${cur_time}= Get Current Date result_format=%Y%m%d%H%M%S%f
Sridevi Rameshb96a0c32016-10-19 05:09:52 -0500735 [Return] ${cur_time}
George Keishing30c12ff2016-09-02 10:25:29 -0500736
George Keishing1b150202016-09-29 08:51:58 -0500737
Rahul Maheshwarif684ba72016-10-25 07:24:41 -0500738Start Journal Log
739 [Documentation] Start capturing journal log to a file in /tmp using
740 ... journalctl command. By default journal log is collected
741 ... at /tmp/journal_log else user input location.
742 ... The File is appended with datetime.
743 [Arguments] ${file_path}=/tmp/journal_log
744
Steven Sombar0acb3412018-01-24 09:38:42 -0600745 # Description of arguments:
746 # file_path The file path of the journal file.
Rahul Maheshwarif684ba72016-10-25 07:24:41 -0500747
748 ${cur_time}= Get Time Stamp
749 Set Global Variable ${LOG_TIME} ${cur_time}
750 Start Command
751 ... journalctl -f > ${file_path}-${LOG_TIME}
752 Log Journal Log Started: ${file_path}-${LOG_TIME}
753
David Shawba2d2c22017-01-23 16:56:38 -0600754
Rahul Maheshwarif684ba72016-10-25 07:24:41 -0500755Stop Journal Log
756 [Documentation] Stop journalctl process if its running.
757 ... By default return log from /tmp/journal_log else
758 ... user input location.
759 [Arguments] ${file_path}=/tmp/journal_log
760
Steven Sombar0acb3412018-01-24 09:38:42 -0600761 # Description of arguments:
762 # file_path The file path of the journal file.
763
Rahul Maheshwarif684ba72016-10-25 07:24:41 -0500764 Open Connection And Log In
765
766 ${rc}=
767 ... Execute Command
768 ... ps ax | grep journalctl | grep -v grep
769 ... return_stdout=False return_rc=True
770
771 Return From Keyword If '${rc}' == '${1}'
772 ... No journal log process running
773
774 ${output} ${stderr}=
775 ... Execute Command killall journalctl
776 ... return_stderr=True
777 Should Be Empty ${stderr}
778
779 ${journal_log} ${stderr}=
780 ... Execute Command
781 ... cat ${file_path}-${LOG_TIME}
782 ... return_stderr=True
783 Should Be Empty ${stderr}
784
785 Log ${journal_log}
786
787 Execute Command rm ${file_path}-${LOG_TIME}
788
789 [Return] ${journal_log}
Rahul Maheshwari757d80c2016-10-17 01:09:55 -0500790
Steven Sombar0acb3412018-01-24 09:38:42 -0600791
Rahul Maheshwari757d80c2016-10-17 01:09:55 -0500792Mac Address To Hex String
793 [Documentation] Converts MAC address into hex format.
794 ... Example
795 ... Given the following MAC: 00:01:6C:80:02:78
796 ... This keyword will return: 0x00 0x01 0x6C 0x80 0x02 0x78
797 ... Description of arguments:
Michael Walsh01bd3682017-01-10 11:21:05 -0600798 ... i_macaddress MAC address in the following format
799 ... 00:01:6C:80:02:78
Rahul Maheshwari757d80c2016-10-17 01:09:55 -0500800 [Arguments] ${i_macaddress}
801
Steven Sombar0acb3412018-01-24 09:38:42 -0600802 # Description of arguments:
803 # i_macaddress The MAC address.
804
Rahul Maheshwari757d80c2016-10-17 01:09:55 -0500805 ${mac_hex}= Catenate 0x${i_macaddress.replace(':', ' 0x')}
Gunnar Millsc9ea9362016-12-13 16:21:13 -0600806 [Return] ${mac_hex}
Rahul Maheshwari757d80c2016-10-17 01:09:55 -0500807
Steven Sombar0acb3412018-01-24 09:38:42 -0600808
Rahul Maheshwari757d80c2016-10-17 01:09:55 -0500809IP Address To Hex String
810 [Documentation] Converts IP address into hex format.
811 ... Example:
812 ... Given the following IP: 10.3.164.100
813 ... This keyword will return: 0xa 0x3 0xa4 0xa0
Rahul Maheshwari757d80c2016-10-17 01:09:55 -0500814 [Arguments] ${i_ipaddress}
815
Steven Sombar0acb3412018-01-24 09:38:42 -0600816 # Description of arguments:
817 # i_macaddress The IP address in the format 10.10.10.10.
818
Rahul Maheshwari757d80c2016-10-17 01:09:55 -0500819 @{ip}= Split String ${i_ipaddress} .
820 ${index}= Set Variable ${0}
821
822 :FOR ${item} IN @{ip}
823 \ ${hex}= Convert To Hex ${item} prefix=0x lowercase=yes
824 \ Set List Value ${ip} ${index} ${hex}
825 \ ${index}= Set Variable ${index + 1}
826 ${ip_hex}= Catenate @{ip}
Steven Sombar0acb3412018-01-24 09:38:42 -0600827
Gunnar Millsc9ea9362016-12-13 16:21:13 -0600828 [Return] ${ip_hex}
Sivas SRRea85d1f2016-11-13 22:44:28 -0600829
Steven Sombar0acb3412018-01-24 09:38:42 -0600830
Sivas SRRea85d1f2016-11-13 22:44:28 -0600831BMC CPU Performance Check
832 [Documentation] Minimal 10% of proc should be free in this instance
833
Sivas SRRa13eb202017-11-03 10:01:56 -0500834 ${bmc_cpu_usage_output} ${stderr} ${rc}= BMC Execute Command
835 ... ${bmc_cpu_usage_cmd}
Sivas SRR5ff4d692017-11-23 02:53:52 -0600836 ${bmc_cpu_usage_output} ${stderr} ${rc}= BMC Execute Command
837 ... ${bmc_cpu_usage_cmd}
Sivas SRRea85d1f2016-11-13 22:44:28 -0600838 ${bmc_cpu_percentage}= Fetch From Left ${bmc_cpu_usage_output} %
839 Should be true ${bmc_cpu_percentage} < 90
840
Steven Sombar0acb3412018-01-24 09:38:42 -0600841
Sivas SRRea85d1f2016-11-13 22:44:28 -0600842BMC Mem Performance Check
843 [Documentation] Minimal 10% of memory should be free in this instance
844
Sivas SRRa13eb202017-11-03 10:01:56 -0500845 ${bmc_mem_free_output} ${stderr} ${rc}= BMC Execute Command
846 ... ${bmc_mem_free_cmd}
Sivas SRRea85d1f2016-11-13 22:44:28 -0600847
Sivas SRRa13eb202017-11-03 10:01:56 -0500848 ${bmc_mem_total_output} ${stderr} ${rc}= BMC Execute Command
849 ... ${bmc_mem_total_cmd}
Sivas SRR5ff4d692017-11-23 02:53:52 -0600850 ${bmc_mem_free_output} ${stderr} ${rc}= BMC Execute Command
851 ... ${bmc_mem_free_cmd}
Sivas SRRea85d1f2016-11-13 22:44:28 -0600852
Sivas SRR5ff4d692017-11-23 02:53:52 -0600853 ${bmc_mem_total_output} ${stderr} ${rc}= BMC Execute Command
854 ... ${bmc_mem_total_cmd}
855
856 ${bmc_mem_percentage}= Evaluate ${bmc_mem_free_output}*100
Sivas SRRea85d1f2016-11-13 22:44:28 -0600857 ${bmc_mem_percentage}= Evaluate
858 ... ${bmc_mem_percentage}/${bmc_mem_total_output}
859 Should be true ${bmc_mem_percentage} > 10
860
Steven Sombar0acb3412018-01-24 09:38:42 -0600861
Sivas SRRaca55712016-12-21 04:32:35 -0600862BMC File System Usage Check
Sivas SRRc0957412017-10-06 13:02:08 -0500863 [Documentation] Check the file system space. 4 file system should be
864 ... 100% full which is expected
Steven Sombar0acb3412018-01-24 09:38:42 -0600865 # Filesystem Size Used Available Use% Mounted on
866 # /dev/root 14.4M 14.4M 0 100% /
867 # /dev/ubiblock0_0 14.4M 14.4M 0 100% /media/rofs-c9249b0e
868 # /dev/ubiblock8_0 19.6M 19.6M 0 100% /media/pnor-ro-8764baa3
869 # /dev/ubiblock4_0 14.4M 14.4M 0 100% /media/rofs-407816c
870 # /dev/ubiblock8_4 21.1M 21.1M 0 100% /media/pnor-ro-cecc64c4
Sivas SRR5ff4d692017-11-23 02:53:52 -0600871 ${bmc_fs_usage_output} ${stderr} ${rc}= BMC Execute Command
872 ... ${bmc_file_system_usage_cmd}
873 ${bmc_pnor_fs_usage_output} ${stderr} ${rc}= BMC Execute Command
874 ... ${total_pnor_ro_file_system_cmd}
875 ${bmc_bmc_fs_usage_output} ${stderr} ${rc}= BMC Execute Command
876 ... ${total_bmc_ro_file_system_cmd}
877 ${total_bmc_pnor_image}= Evaluate
878 ... ${bmc_pnor_fs_usage_output}+${bmc_bmc_fs_usage_output}
879 # Considering /dev/root also in total 100% used file system
880 ${total_full_fs}= Evaluate ${total_bmc_pnor_image}+1
881 Should Be True ${bmc_fs_usage_output}==${total_full_fs}
Sivas SRRaca55712016-12-21 04:32:35 -0600882
Steven Sombar0acb3412018-01-24 09:38:42 -0600883
Sivas SRRea85d1f2016-11-13 22:44:28 -0600884Check BMC CPU Performance
885 [Documentation] Minimal 10% of proc should be free in 3 sample
886 :FOR ${var} IN Range 1 4
887 \ BMC CPU Performance check
888
Steven Sombar0acb3412018-01-24 09:38:42 -0600889
Sivas SRRea85d1f2016-11-13 22:44:28 -0600890Check BMC Mem Performance
891 [Documentation] Minimal 10% of memory should be free
892
893 :FOR ${var} IN Range 1 4
894 \ BMC Mem Performance check
895
Steven Sombar0acb3412018-01-24 09:38:42 -0600896
Sivas SRRaca55712016-12-21 04:32:35 -0600897Check BMC File System Performance
898 [Documentation] Check for file system usage for 4 times
899
900 :FOR ${var} IN Range 1 4
901 \ BMC File System Usage check
902
Steven Sombar0acb3412018-01-24 09:38:42 -0600903
Rahul Maheshwari63c105b2017-05-12 05:13:52 -0500904Get URL List
905 [Documentation] Return list of URLs under given URL.
906 [Arguments] ${openbmc_url}
Steven Sombar0acb3412018-01-24 09:38:42 -0600907
Rahul Maheshwari63c105b2017-05-12 05:13:52 -0500908 # Description of argument(s):
Michael Walsh5f3f4142017-05-22 17:09:47 -0500909 # openbmc_url URL for list operation (e.g.
910 # /xyz/openbmc_project/inventory).
Rahul Maheshwari63c105b2017-05-12 05:13:52 -0500911
George Keishing335f5362017-06-30 15:11:20 -0500912 ${url_list}= Read Properties ${openbmc_url}/list quiet=${1}
George Keishing184a89a2017-06-27 07:12:16 -0500913 Sort List ${url_list}
Steven Sombar0acb3412018-01-24 09:38:42 -0600914
Rahul Maheshwari63c105b2017-05-12 05:13:52 -0500915 [Return] ${url_list}
916
Steven Sombar0acb3412018-01-24 09:38:42 -0600917
Rahul Maheshwari9a8d3b12016-12-05 04:06:16 -0600918Get Endpoint Paths
919 [Documentation] Returns all url paths ending with given endpoint
920 ... Example:
921 ... Given the following endpoint: cpu
Michael Walsh01bd3682017-01-10 11:21:05 -0600922 ... This keyword will return: list of all urls ending with
923 ... cpu -
Rahul Maheshwari9a8d3b12016-12-05 04:06:16 -0600924 ... /org/openbmc/inventory/system/chassis/motherboard/cpu0,
925 ... /org/openbmc/inventory/system/chassis/motherboard/cpu1
Rahul Maheshwari9a8d3b12016-12-05 04:06:16 -0600926 [Arguments] ${path} ${endpoint}
927
Steven Sombar0acb3412018-01-24 09:38:42 -0600928 # Description of arguments:
929 # path URL path for enumeration.
930 # endpoint Endpoint string (url path ending).
931
George Keishingc7b161b2017-12-01 09:58:12 -0600932 ${resp}= Read Properties ${path}/enumerate timeout=30
933 Log Dictionary ${resp}
Rahul Maheshwari9a8d3b12016-12-05 04:06:16 -0600934
George Keishingc7b161b2017-12-01 09:58:12 -0600935 ${list}= Get Dictionary Keys ${resp}
936 # For a given string, look for prefix and suffix for matching expression.
937 # Start of string followed by zero or more of any character followed by
938 # any digit or lower case character.
939 ${resp}= Get Matches ${list} regexp=^.*[0-9a-z_].${endpoint}[0-9a-z]*$
Steven Sombar0acb3412018-01-24 09:38:42 -0600940
George Keishingc7b161b2017-12-01 09:58:12 -0600941 [Return] ${resp}
Sridevi Rameshb96a0c32016-10-19 05:09:52 -0500942
Steven Sombar0acb3412018-01-24 09:38:42 -0600943
Sridevi Rameshb96a0c32016-10-19 05:09:52 -0500944Check Zombie Process
945 [Documentation] Check if any defunct process exist or not on BMC
Michael Walsh01bd3682017-01-10 11:21:05 -0600946 ${count} ${stderr} ${rc}= Execute Command ps -o stat | grep Z | wc -l
Sridevi Rameshb96a0c32016-10-19 05:09:52 -0500947 ... return_stderr=True return_rc=True
948 Should Be True ${count}==0
949 Should Be Empty ${stderr}
George Keishing5327d012016-12-08 08:43:29 -0600950
Steven Sombar0acb3412018-01-24 09:38:42 -0600951
George Keishing5327d012016-12-08 08:43:29 -0600952Prune Journal Log
953 [Documentation] Prune archived journal logs.
954 [Arguments] ${vacuum_size}=1M
955
956 # This keyword can be used to prevent the journal
957 # log from filling up the /run filesystem.
958 # This command will retain only the latest logs
959 # of the user specified size.
960
Steven Sombar0acb3412018-01-24 09:38:42 -0600961 # Description of argument(s):
962 # vacuum_size Size of journal.
963
George Keishing5327d012016-12-08 08:43:29 -0600964 Open Connection And Log In
965 ${output} ${stderr} ${rc}=
966 ... Execute Command
967 ... journalctl --vacuum-size=${vacuum_size}
968 ... return_stderr=True return_rc=True
969
970 Should Be Equal ${rc} ${0} msg=${stderr}
Sridevi Ramesh1699d372016-12-06 00:20:22 -0600971
Steven Sombar0acb3412018-01-24 09:38:42 -0600972
Sridevi Ramesh1699d372016-12-06 00:20:22 -0600973Set BMC Power Policy
974 [Documentation] Set the given BMC power policy.
Michael Walshfdc5ced2017-08-17 13:15:15 -0500975 [Arguments] ${policy}
976
977 # Note that this function will translate the old style "RESTORE_LAST_STATE"
978 # policy to the new style "xyz.openbmc_project.Control.Power.RestorePolicy.
979 # Policy.Restore" for you.
Sridevi Ramesh1699d372016-12-06 00:20:22 -0600980
George Keishing1c3a9662017-08-11 10:16:36 -0500981 # Description of argument(s):
Michael Walshfdc5ced2017-08-17 13:15:15 -0500982 # policy Power restore policy (e.g "RESTORE_LAST_STATE",
983 # ${RESTORE_LAST_STATE}).
984
985 # Set the bmc_power_policy_method to either 'Old' or 'New'.
986 Set Power Policy Method
987 # This translation helps bridge between old and new method for calling.
988 ${policy}= Translate Power Policy Value ${policy}
989 # Run the appropriate keyword.
990 Run Key ${bmc_power_policy_method} Set Power Policy \ ${policy}
991 ${currentPolicy}= Get System Power Policy
992 Should Be Equal ${currentPolicy} ${policy}
993
Steven Sombar0acb3412018-01-24 09:38:42 -0600994
Michael Walshfdc5ced2017-08-17 13:15:15 -0500995New Set Power Policy
996 [Documentation] Set the given BMC power policy (new method).
997 [Arguments] ${policy}
998
999 # Description of argument(s):
George Keishingefc3ff22017-12-12 11:49:25 -06001000 # policy Power restore policy (e.g. ${ALWAYS_POWER_OFF}).
George Keishing1c3a9662017-08-11 10:16:36 -05001001
1002 ${valueDict}= Create Dictionary data=${policy}
1003 Write Attribute
Michael Walshfdc5ced2017-08-17 13:15:15 -05001004 ... ${POWER_RESTORE_URI} PowerRestorePolicy data=${valueDict}
1005
Steven Sombar0acb3412018-01-24 09:38:42 -06001006
Michael Walshfdc5ced2017-08-17 13:15:15 -05001007Old Set Power Policy
1008 [Documentation] Set the given BMC power policy (old method).
1009 [Arguments] ${policy}
1010
1011 # Description of argument(s):
George Keishingefc3ff22017-12-12 11:49:25 -06001012 # policy Power restore policy (e.g. "ALWAYS_POWER_OFF").
Michael Walshfdc5ced2017-08-17 13:15:15 -05001013
1014 ${valueDict}= create dictionary data=${policy}
1015 Write Attribute ${HOST_SETTING} power_policy data=${valueDict}
George Keishingcce9df22017-01-24 06:19:33 -06001016
Steven Sombar0acb3412018-01-24 09:38:42 -06001017
George Keishing4d6f3d62017-02-06 09:50:41 -06001018Get System Power Policy
Steven Sombar0acb3412018-01-24 09:38:42 -06001019 [Documentation] Returns the BMC power policy.
Michael Walshfdc5ced2017-08-17 13:15:15 -05001020
1021 # Set the bmc_power_policy_method to either 'Old' or 'New'.
1022 Set Power Policy Method
1023 ${cmd_buf}= Create List ${bmc_power_policy_method} Get Power Policy
1024 # Run the appropriate keyword.
1025 ${currentPolicy}= Run Keyword @{cmd_buf}
Steven Sombar0acb3412018-01-24 09:38:42 -06001026
Michael Walshfdc5ced2017-08-17 13:15:15 -05001027 [Return] ${currentPolicy}
1028
Steven Sombar0acb3412018-01-24 09:38:42 -06001029
Michael Walshfdc5ced2017-08-17 13:15:15 -05001030New Get Power Policy
Steven Sombar0acb3412018-01-24 09:38:42 -06001031 [Documentation] Returns the BMC power policy (new method).
George Keishing1c3a9662017-08-11 10:16:36 -05001032 ${currentPolicy}= Read Attribute ${POWER_RESTORE_URI} PowerRestorePolicy
Steven Sombar0acb3412018-01-24 09:38:42 -06001033
George Keishing4d6f3d62017-02-06 09:50:41 -06001034 [Return] ${currentPolicy}
George Keishingcce9df22017-01-24 06:19:33 -06001035
Steven Sombar0acb3412018-01-24 09:38:42 -06001036
Michael Walshfdc5ced2017-08-17 13:15:15 -05001037Old Get Power Policy
Steven Sombar0acb3412018-01-24 09:38:42 -06001038 [Documentation] Returns the BMC power policy (old method).
Michael Walshfdc5ced2017-08-17 13:15:15 -05001039 ${currentPolicy}= Read Attribute ${HOST_SETTING} power_policy
Steven Sombar0acb3412018-01-24 09:38:42 -06001040
Michael Walshfdc5ced2017-08-17 13:15:15 -05001041 [Return] ${currentPolicy}
1042
Steven Sombar0acb3412018-01-24 09:38:42 -06001043
Rahul Maheshwari1cbf0042017-02-13 05:17:37 -06001044Get Auto Reboot
1045 [Documentation] Returns auto reboot setting.
Sweta Potthuri62a23572017-07-28 04:05:13 -05001046 ${setting}= Read Attribute ${CONTROL_HOST_URI}/auto_reboot AutoReboot
Steven Sombar0acb3412018-01-24 09:38:42 -06001047
Rahul Maheshwari1cbf0042017-02-13 05:17:37 -06001048 [Return] ${setting}
1049
Steven Sombar0acb3412018-01-24 09:38:42 -06001050
Rahul Maheshwari1cbf0042017-02-13 05:17:37 -06001051Set Auto Reboot
1052 [Documentation] Set the given auto reboot setting.
1053 [Arguments] ${setting}
Steven Sombar0acb3412018-01-24 09:38:42 -06001054
1055 # Description of argument(s):
1056 # setting The reboot setting, 1 for enabling and 0 for disabling.
Rahul Maheshwari1cbf0042017-02-13 05:17:37 -06001057
1058 ${valueDict}= Set Variable ${setting}
1059 ${data}= Create Dictionary data=${valueDict}
Sweta Potthuri62a23572017-07-28 04:05:13 -05001060 Write Attribute ${CONTROL_HOST_URI}/auto_reboot AutoReboot data=${data}
Rahul Maheshwari1cbf0042017-02-13 05:17:37 -06001061 ${current_setting}= Get Auto Reboot
Sweta Potthuri62a23572017-07-28 04:05:13 -05001062 Should Be Equal As Integers ${current_setting} ${setting}
Rahul Maheshwari1cbf0042017-02-13 05:17:37 -06001063
1064
George Keishingcce9df22017-01-24 06:19:33 -06001065Set BMC Reset Reference Time
1066 [Documentation] Set current boot time as a reference and increment
George Keishing80dd5af2017-02-08 07:01:25 -06001067 ... boot count.
George Keishingcce9df22017-01-24 06:19:33 -06001068
1069 ${cur_btime}= Get BMC Boot Time
George Keishing80dd5af2017-02-08 07:01:25 -06001070 Run Keyword If ${BOOT_TIME} == ${0} and ${BOOT_COUNT} == ${0}
George Keishingcce9df22017-01-24 06:19:33 -06001071 ... Set Global Variable ${BOOT_TIME} ${cur_btime}
George Keishing80dd5af2017-02-08 07:01:25 -06001072 ... ELSE IF ${cur_btime} > ${BOOT_TIME}
1073 ... Run Keywords Set Global Variable ${BOOT_TIME} ${cur_btime}
George Keishingcce9df22017-01-24 06:19:33 -06001074 ... AND
1075 ... Set Global Variable ${BOOT_COUNT} ${BOOT_COUNT + 1}
1076
Steven Sombar0acb3412018-01-24 09:38:42 -06001077
George Keishingcce9df22017-01-24 06:19:33 -06001078Get BMC Boot Time
Steven Sombar0acb3412018-01-24 09:38:42 -06001079 [Documentation] Returns boot time from /proc/stat.
George Keishingcce9df22017-01-24 06:19:33 -06001080
1081 Open Connection And Log In
1082 ${output} ${stderr}=
1083 ... Execute Command egrep '^btime ' /proc/stat | cut -f 2 -d ' '
1084 ... return_stderr=True
1085 Should Be Empty ${stderr}
1086 ${btime}= Convert To Integer ${output}
1087 [Return] ${btime}
George Keishingede30002017-02-02 09:33:07 -06001088
Steven Sombar0acb3412018-01-24 09:38:42 -06001089
George Keishingede30002017-02-02 09:33:07 -06001090Execute Command On BMC
1091 [Documentation] Execute given command on BMC and return output.
1092 [Arguments] ${command}
Steven Sombar0acb3412018-01-24 09:38:42 -06001093
1094 # Description of argument(s):
1095 # command The command to execute on the BMC.
1096
George Keishingede30002017-02-02 09:33:07 -06001097 ${stdout} ${stderr}= Execute Command ${command} return_stderr=True
1098 Should Be Empty ${stderr}
1099 [Return] ${stdout}
David Shawba2d2c22017-01-23 16:56:38 -06001100
Sridevi Rameshea36b412017-03-09 04:08:02 -06001101
George Keishing7a520222017-02-27 09:44:30 -06001102Enable Core Dump On BMC
1103 [Documentation] Enable core dump collection.
1104 Open Connection And Log In
1105 ${core_pattern}= Execute Command On BMC
1106 ... echo '/tmp/core_%e.%p' | tee /proc/sys/kernel/core_pattern
1107 Should Be Equal As Strings ${core_pattern} /tmp/core_%e.%p
Rahul Maheshwarie95622c2017-02-24 10:04:29 -06001108
Steven Sombar0acb3412018-01-24 09:38:42 -06001109
Sivas SRR01262bd2017-03-19 10:00:23 -05001110Get Number Of BMC Core Dump Files
Steven Sombar0acb3412018-01-24 09:38:42 -06001111 [Documentation] Returns number of core dump files on BMC.
Sivas SRR01262bd2017-03-19 10:00:23 -05001112 Open Connection And Log In
1113 ${num_of_core_dump}= Execute Command
1114 ... ls /tmp/core* 2>/dev/null | wc -l
1115 [Return] ${num_of_core_dump}
1116
Steven Sombar0acb3412018-01-24 09:38:42 -06001117
Sivas SRR01262bd2017-03-19 10:00:23 -05001118Set Core Dump File Size Unlimited
1119 [Documentation] Set core dump file size to unlimited.
1120 Open Connection And Log In
1121 Execute Command On BMC
1122 ... ulimit -c unlimited
1123
Steven Sombar0acb3412018-01-24 09:38:42 -06001124
Sivas SRR01262bd2017-03-19 10:00:23 -05001125Check For Core Dumps
1126 [Documentation] Check for any core dumps exist.
1127 ${output}= Get Number Of BMC Core Dump Files
1128 Run Keyword If ${output} > 0
1129 ... Log **Warning** BMC core dump files exist level=WARN
1130
Steven Sombar0acb3412018-01-24 09:38:42 -06001131
Rahul Maheshwarie95622c2017-02-24 10:04:29 -06001132Trigger Host Watchdog Error
George Keishing37cfa6b2017-06-15 10:25:55 -05001133 [Documentation] Inject host watchdog timeout error via REST.
Rahul Maheshwarie95622c2017-02-24 10:04:29 -06001134 [Arguments] ${milliseconds}=1000 ${sleep_time}=5s
Steven Sombar0acb3412018-01-24 09:38:42 -06001135
Michael Walshb5839d02017-04-12 16:11:20 -05001136 # Description of argument(s):
1137 # milliseconds The time watchdog timer value in milliseconds (e.g. 1000 =
1138 # 1 second).
Rahul Maheshwarie95622c2017-02-24 10:04:29 -06001139 # sleep_time Time delay for host watchdog error to get injected.
1140 # Default is 5 seconds.
1141
George Keishing37cfa6b2017-06-15 10:25:55 -05001142 ${data}= Create Dictionary data=${True}
1143 Write Attribute /xyz/openbmc_project/watchdog/host0 Enabled data=${data}
1144
1145 ${data}= Create Dictionary data=${milliseconds}
Michael Walsh98d8c9e2017-10-04 12:24:29 -05001146 Write Attribute /xyz/openbmc_project/watchdog/host0 TimeRemaining
1147 ... data=${data}
George Keishing37cfa6b2017-06-15 10:25:55 -05001148
Rahul Maheshwarie95622c2017-02-24 10:04:29 -06001149 Sleep ${sleep_time}
Prashanth Kattiae7c2282017-03-15 07:43:46 -05001150
Steven Sombar0acb3412018-01-24 09:38:42 -06001151
Prashanth Kattiae7c2282017-03-15 07:43:46 -05001152Login To OS Host
Steven Sombar0acb3412018-01-24 09:38:42 -06001153 [Documentation] Login to OS Host and return the Login response code.
Prashanth Kattiae7c2282017-03-15 07:43:46 -05001154 [Arguments] ${os_host}=${OS_HOST} ${os_username}=${OS_USERNAME}
1155 ... ${os_password}=${OS_PASSWORD}
Steven Sombar0acb3412018-01-24 09:38:42 -06001156
Gunnar Mills28e403b2017-10-25 16:16:38 -05001157 # Description of arguments:
Prashanth Kattiae7c2282017-03-15 07:43:46 -05001158 # ${os_host} IP address of the OS Host.
1159 # ${os_username} OS Host Login user name.
1160 # ${os_password} OS Host Login passwrd.
1161
1162 ${os_state}= Run Keyword And Return Status Ping Host ${os_host}
1163 Run Keyword If '${os_state}' == 'False'
1164 ... Run Keywords Initiate Host Reboot AND
1165 ... Is Host Running AND
1166 ... Wait for OS ${os_host} ${os_username} ${os_password}
1167
George Keishing3f223b82017-06-28 03:50:35 -05001168 SSHLibrary.Open Connection ${os_host}
Sridevi Rameshea36b412017-03-09 04:08:02 -06001169 ${resp}= Login ${os_username} ${os_password}
1170 [Return] ${resp}
Prashanth Katti884ee062017-03-16 05:05:03 -05001171
Steven Sombar0acb3412018-01-24 09:38:42 -06001172
Prashanth Katti884ee062017-03-16 05:05:03 -05001173Configure Initial Settings
1174 [Documentation] Restore old IP and route.
1175 ... This keyword requires initial settings viz IP address,
1176 ... Network Mask, default gatway and serial console IP and port
1177 ... information which should be provided in command line.
1178
Michael Walshb5839d02017-04-12 16:11:20 -05001179 [Arguments] ${host}=${OPENBMC_HOST} ${mask}=${NET_MASK}
1180 ... ${gw_ip}=${GW_IP}
Prashanth Katti884ee062017-03-16 05:05:03 -05001181
Steven Sombar0acb3412018-01-24 09:38:42 -06001182 # Description of arguments:
1183 # host IP address of the OS Host.
1184 # mask Network mask.
1185 # gu_ip Gateway IP address or hostname.
1186
Michael Walshb5839d02017-04-12 16:11:20 -05001187 # Open telnet connection and ignore the error, in case telnet session is
1188 # already opened by the program calling this keyword.
Prashanth Katti884ee062017-03-16 05:05:03 -05001189 Run Keyword And Ignore Error Open Telnet Connection to BMC Serial Console
1190 Telnet.write ifconfig eth0 ${host} netmask ${mask}
1191 Telnet.write route add default gw ${gw_ip}
George Keishingd05b1082017-03-24 05:27:32 -05001192
Steven Sombar0acb3412018-01-24 09:38:42 -06001193
George Keishingd05b1082017-03-24 05:27:32 -05001194Install Debug Tarball On BMC
George Keishing55441822017-05-18 14:17:59 -05001195 [Documentation] Copy the debug tar file to BMC and install.
Steven Sombar0acb3412018-01-24 09:38:42 -06001196 [Arguments] ${tarball_file_path}=${default_tarball}
1197 ... ${targ_tarball_dir_path}=/tmp/tarball/
George Keishingd05b1082017-03-24 05:27:32 -05001198
George Keishing55441822017-05-18 14:17:59 -05001199 # Description of arguments:
1200 # tarball_file_path Path of the debug tarball file.
1201 # The tar file is downloaded from the build page
1202 # https://openpower.xyz/job/openbmc-build/
1203 # obmc-phosphor-debug-tarball-witherspoon.tar.xz
1204 #
1205 # targ_tarball_dir_path The directory path where the tarball is to be
1206 # installed.
1207
1208 OperatingSystem.File Should Exist ${tarball_file_path}
1209 ... msg=${tarball_file_path} doesn't exist.
1210
1211 # Upload the file to BMC.
George Keishingd05b1082017-03-24 05:27:32 -05001212 Import Library SCPLibrary WITH NAME scp
1213 Open Connection for SCP
George Keishing55441822017-05-18 14:17:59 -05001214 scp.Put File ${tarball_file_path} /tmp/debug-tarball.tar.xz
1215
1216 # Create tarball directory and install.
George Keishing95c371a2017-09-14 15:42:33 -05001217 BMC Execute Command mkdir -p ${targ_tarball_dir_path}
1218 BMC Execute Command
George Keishing55441822017-05-18 14:17:59 -05001219 ... tar -xf /tmp/debug-tarball.tar.xz -C ${targ_tarball_dir_path}
1220
George Keishing95c371a2017-09-14 15:42:33 -05001221 # Remove the tarball file from BMC.
1222 BMC Execute Command rm -f /tmp/debug-tarball.tar.xz
George Keishingd05b1082017-03-24 05:27:32 -05001223
George Keishing55441822017-05-18 14:17:59 -05001224
Sweta Potthuri025e0122017-02-21 00:42:48 -06001225Get BMC Boot Count
Steven Sombar0acb3412018-01-24 09:38:42 -06001226 [Documentation] Returns BMC boot count based on boot time.
Sweta Potthuri025e0122017-02-21 00:42:48 -06001227 ${cur_btime}= Get BMC Boot Time
1228
1229 # Set global variable BOOT_TIME to current boot time if current boot time
1230 # is changed. Also increase value of global variable BOOT_COUNT by 1.
1231 Run Keyword If ${cur_btime} > ${BOOT_TIME}
1232 ... Run Keywords Set Global Variable ${BOOT_TIME} ${cur_btime}
1233 ... AND
1234 ... Set Global Variable ${BOOT_COUNT} ${BOOT_COUNT + 1}
Steven Sombar0acb3412018-01-24 09:38:42 -06001235
Sweta Potthuri025e0122017-02-21 00:42:48 -06001236 [Return] ${BOOT_COUNT}
1237
Steven Sombar0acb3412018-01-24 09:38:42 -06001238
Sweta Potthuri025e0122017-02-21 00:42:48 -06001239Set BMC Boot Count
1240 [Documentation] Set BMC boot count to given value.
1241 [Arguments] ${count}
1242
1243 # Description of arguments:
1244 # count boot count value.
1245 ${cur_btime}= Get BMC Boot Time
1246
1247 # Set global variable BOOT_COUNT to given value.
1248 Set Global Variable ${BOOT_COUNT} ${count}
1249
1250 # Set BOOT_TIME variable to current boot time.
1251 Set Global Variable ${BOOT_COUNT} ${count}
Michael Walsh689fbbe2017-04-04 17:58:03 -05001252
Steven Sombar0acb3412018-01-24 09:38:42 -06001253
Sweta Potthurie5a9fc72017-05-03 07:02:46 -05001254Get System LED State
1255 [Documentation] Return the state of given system LED.
1256 [Arguments] ${led_name}
1257
1258 # Description of argument(s):
1259 # led_name System LED name (e.g. heartbeat, identify, beep).
1260
1261 ${state}= Read Attribute ${LED_PHYSICAL_URI}${led_name} State
1262 [Return] ${state.rsplit('.', 1)[1]}
1263
George Keishing3987a1e2017-09-20 09:13:02 -05001264
Steven Sombar5f3af442018-02-14 08:30:15 -06001265Set System LED State
1266 [Documentation] Set given system LED via REST.
1267 [Arguments] ${led_name} ${led_state}
1268 # Description of argument(s):
1269 # led_name System LED name (e.g. heartbeat, identify, beep).
1270 # led_state LED state to be set (e.g. On, Off).
1271
1272 ${args}= Create Dictionary
1273 ... data=xyz.openbmc_project.Led.Physical.Action.${led_state}
1274 Write Attribute ${LED_PHYSICAL_URI}${led_name} State data=${args}
1275
1276 Verify LED State ${led_name} ${led_state}
1277
1278
1279Verify LED State
1280 [Documentation] Checks if LED is in given state.
1281 [Arguments] ${led_name} ${led_state}
1282 # Description of argument(s):
1283 # led_name System LED name (e.g. heartbeat, identify, beep).
1284 # led_state LED state to be verified (e.g. On, Off).
1285
1286 ${state}= Get System LED State ${led_name}
1287 Should Be Equal ${state} ${led_state}
1288
1289
Steven Sombarb89d5802018-02-15 14:48:49 -06001290Get LED State XYZ
1291 [Documentation] Returns state of given LED.
1292 [Arguments] ${led_name}
1293
1294 # Description of argument(s):
1295 # led_name Name of LED.
1296
1297 ${state}= Read Attribute ${LED_GROUPS_URI}${led_name} Asserted
1298 # Returns the state of the LED, either On or Off.
1299 [Return] ${state}
1300
1301
George Keishingd2d90af2017-06-27 08:17:34 -05001302Delete Error Logs
Michael Walsh689fbbe2017-04-04 17:58:03 -05001303 [Documentation] Delete error logs.
1304
George Keishingd2d90af2017-06-27 08:17:34 -05001305 # Check if error logs entries exist, if not return.
George Keishing335f5362017-06-30 15:11:20 -05001306 ${resp}= OpenBMC Get Request ${BMC_LOGGING_ENTRY}${/}list quiet=${1}
George Keishingd2d90af2017-06-27 08:17:34 -05001307 Return From Keyword If ${resp.status_code} == ${HTTP_NOT_FOUND}
1308
1309 # Get the list of error logs entries and delete them all.
George Keishing335f5362017-06-30 15:11:20 -05001310 ${elog_entries}= Get URL List ${BMC_LOGGING_ENTRY}
George Keishingd2d90af2017-06-27 08:17:34 -05001311 :FOR ${entry} IN @{elog_entries}
1312 \ Delete Error Log Entry ${entry}
George Keishingdb34c2f2017-04-18 13:20:25 -05001313
George Keishing3987a1e2017-09-20 09:13:02 -05001314
George Keishingd2d90af2017-06-27 08:17:34 -05001315Delete Error Log Entry
George Keishingdb34c2f2017-04-18 13:20:25 -05001316 [Documentation] Delete error log entry.
1317 [Arguments] ${entry_path}
1318
1319 # Description of argument(s):
1320 # entry_path Delete an error log entry.
1321 # Ex. /xyz/openbmc_project/logging/entry/1
1322
George Keishing6a8ed0f2017-07-11 22:54:42 -05001323 # Skip delete if entry URI is a callout.
1324 # Example: /xyz/openbmc_project/logging/entry/1/callout
1325 Return From Keyword If '${entry_path.rsplit('/', 1)[1]}' == 'callout'
1326
George Keishingdb34c2f2017-04-18 13:20:25 -05001327 ${data}= Create Dictionary data=@{EMPTY}
1328 ${resp}= Openbmc Delete Request ${entry_path} data=${data}
1329 Should Be Equal As Strings ${resp.status_code} ${HTTP_OK}
Rahul Maheshwari43be6932017-05-01 02:37:06 -05001330
1331
George Keishingcc71c3c2017-09-20 09:38:03 -05001332Delete All Error Logs
George Keishing3987a1e2017-09-20 09:13:02 -05001333 [Documentation] Delete all error log entries using "DeleteAll" interface.
1334
1335 ${data}= Create Dictionary data=@{EMPTY}
1336 ${resp}= Openbmc Post Request ${BMC_LOGGING_URI}action/DeleteAll
1337 ... data=${data}
1338 Should Be Equal As Strings ${resp.status_code} ${HTTP_OK}
1339
1340
Rahul Maheshwari13256d92017-06-15 10:05:40 -05001341Get BMC Version
1342 [Documentation] Returns BMC version from /etc/os-release.
1343 ... e.g. "v1.99.6-141-ge662190"
1344
1345 Open Connection And Log In
1346 ${cmd}= Set Variable grep ^VERSION_ID= /etc/os-release | cut -f 2 -d '='
1347 ${output}= Execute Command On BMC ${cmd}
1348 [Return] ${output}
1349
George Keishingad5817a2017-06-28 10:19:03 -05001350
Charles Paul Hofere43fb2f2017-09-26 15:36:18 -05001351Get PNOR Version
Steven Sombar0acb3412018-01-24 09:38:42 -06001352 [Documentation] Returns the PNOR version from the BMC.
Charles Paul Hofere43fb2f2017-09-26 15:36:18 -05001353
1354 ${pnor_attrs}= Get PNOR Attributes
1355 [Return] ${pnor_attrs['version']}
1356
1357
1358Get PNOR Attributes
1359 [Documentation] Return PNOR software attributes as a dictionary.
1360
Michael Walsh98d8c9e2017-10-04 12:24:29 -05001361 # This keyword parses /var/lib/phosphor-software-manager/pnor/ro/pnor.toc
1362 # into key/value pairs.
Charles Paul Hofere43fb2f2017-09-26 15:36:18 -05001363
1364 ${outbuf} ${stderr} ${rc}= BMC Execute Command
1365 ... cat /var/lib/phosphor-software-manager/pnor/ro/pnor.toc
1366 ${pnor_attrs}= Key Value Outbuf To Dict ${outbuf} delim==
1367
1368 [Return] ${pnor_attrs}
1369
1370
George Keishingad5817a2017-06-28 10:19:03 -05001371Get Elog URL List
1372 [Documentation] Return error log entry list of URLs.
1373
1374 ${url_list}= Read Properties /xyz/openbmc_project/logging/entry/
1375 Sort List ${url_list}
1376 [Return] ${url_list}
1377
1378
Steven Sombaraaf72c42017-09-05 12:02:00 -05001379Read Turbo Setting Via REST
1380 [Documentation] Return turbo setting via REST.
1381
1382 ${resp}= OpenBMC Get Request ${SENSORS_URI}host/TurboAllowed
1383 ${jsondata}= To JSON ${resp.content}
1384 Should Be Equal As Strings ${resp.status_code} ${HTTP_OK}
1385 [Return] ${jsondata["data"]["value"]}
1386
1387
Steven Sombaraaf72c42017-09-05 12:02:00 -05001388Set Turbo Setting Via REST
1389 [Documentation] Set turbo setting via REST.
1390 [Arguments] ${setting}
Steven Sombar0acb3412018-01-24 09:38:42 -06001391
Steven Sombaraaf72c42017-09-05 12:02:00 -05001392 # Description of argument(s):
Steven Sombar0acb3412018-01-24 09:38:42 -06001393 # setting Value which needs to be set.(i.e. False or True).
Steven Sombaraaf72c42017-09-05 12:02:00 -05001394
1395 ${valueDict}= Create Dictionary data=${setting}
1396 Write Attribute ${SENSORS_URI}host/TurboAllowed value data=${valueDict}
George Keishingef74a8c2017-10-20 10:23:52 -05001397
Steven Sombar0acb3412018-01-24 09:38:42 -06001398
George Keishingef74a8c2017-10-20 10:23:52 -05001399Set Control Boot Mode
1400 [Documentation] Set given boot mode on the boot object path attribute.
1401 [Arguments] ${boot_path} ${boot_mode}
1402
1403 # Description of argument(s):
1404 # boot_path Boot object path.
1405 # Example:
1406 # /xyz/openbmc_project/control/host0/boot
1407 # /xyz/openbmc_project/control/host0/boot/one_time
1408 # boot_mode Boot mode which need to be set.
1409 # Example:
1410 # "xyz.openbmc_project.Control.Boot.Mode.Modes.Regular"
1411
1412 ${valueDict}= Create Dictionary data=${boot_mode}
1413 Write Attribute ${boot_path} BootMode data=${valueDict}
Sridevi Ramesh0d88ab32017-09-21 11:07:28 -05001414
Steven Sombar0acb3412018-01-24 09:38:42 -06001415
Sridevi Ramesh0d88ab32017-09-21 11:07:28 -05001416Copy Address Translation Utils To HOST OS
1417 [Documentation] Copy address translation utils to host OS.
1418
1419 OperatingSystem.File Should Exist ${probe_cpu_tool_path}
1420 ... msg=${probe_cpu_tool_path} doesn't exist.
1421 OperatingSystem.File Should Exist ${probe_cpu_tool_path}
1422 ... msg=${probe_cpu_tool_path} doesn't exist.
1423
1424 scp.Open connection ${OS_HOST} username=${OS_USERNAME}
1425 ... password=${OS_PASSWORD}
1426 scp.Put File ${probe_cpu_tool_path} ${target_file_path}
1427 scp.Put File ${scom_addrs_tool_path} ${target_file_path}
George Keishing034902d2017-11-10 13:15:38 -06001428
1429
1430Verify BMC RTC And UTC Time Drift
1431 [Documentation] Verify that the RTC and UTC time difference is less than
1432 ... the given time_drift_max.
1433 [Arguments] ${time_diff_max}=${10}
1434
1435 # Description of argument(s):
1436 # time_diff_max The max allowable RTC and UTC time difference in seconds.
1437
1438 # Example:
1439 # time_dict:
1440 # [local_time]: Fri 2017-11-03 152756 UTC
1441 # [local_time_seconds]: 1509740876
1442 # [universal_time]: Fri 2017-11-03 152756 UTC
1443 # [universal_time_seconds]: 1509740876
1444 # [rtc_time]: Fri 2016-05-20 163403
1445 # [rtc_time_seconds]: 1463780043
1446 # [time_zone]: n/a (UTC, +0000)
1447 # [network_time_on]: yes
1448 # [ntp_synchronized]: no
1449 # [rtc_in_local_tz]: no
1450
Steven Sombar0acb3412018-01-24 09:38:42 -06001451 ${time}= Get BMC Date Time
George Keishing034902d2017-11-10 13:15:38 -06001452 ${time_diff}= Evaluate
Steven Sombar0acb3412018-01-24 09:38:42 -06001453 ... ${time['universal_time_seconds']} - ${time['rtc_time_seconds']}
George Keishing034902d2017-11-10 13:15:38 -06001454 Should Be True ${time_diff} < ${time_diff_max}
Sweta Potthuri7d3af3c2018-01-29 03:07:07 -06001455
Sweta Potthurif39022d2018-02-06 03:40:07 -06001456
Sweta Potthuri7d3af3c2018-01-29 03:07:07 -06001457Watchdog Object Should Exist
1458 [Documentation] Check that watchdog object exists.
1459
1460 ${resp}= OpenBMC Get Request ${WATCHDOG_URI}host0
1461 Should Be Equal As Strings ${resp.status_code} ${HTTP_OK}
1462 ... msg=Expected watchdog object does not exist.
Sweta Potthurif39022d2018-02-06 03:40:07 -06001463
1464
1465Validate IP On BMC
1466 [Documentation] Validate IP address is present in set of IP addresses.
1467 [Arguments] ${ip_address} ${ip_data}
1468
1469 # Description of argument(s):
1470 # ip_address IP address to check (e.g. xx.xx.xx.xx).
1471 # ip_data Set of the IP addresses present.
1472
1473 Should Contain Match ${ip_data} ${ip_address}/*
1474 ... msg=${ip_address} not found in the list provided.
George Keishingb9f407b2018-02-20 00:07:46 -06001475
1476
1477Remove Journald Logs
1478 [Documentation] Remove all journald logs and restart service.
1479
1480 ${cmd}= Catenate systemctl stop systemd-journald.service &&
1481 ... rm -rf /var/log/journal && systemctl start systemd-journald.service
1482
1483 BMC Execute Command ${cmd}
1484