blob: c1f5399d8875a02bc5d791b6f462438063b7e643 [file] [log] [blame]
Leah McNuttc9c9cde2016-10-07 16:53:52 +00001*** Settings ***
Michael Walsh0bbd8602016-11-22 11:31:49 -06002Documentation Do random repeated boots based on the state of the BMC machine.
3... The number of repetitions is designated by ${max_num_tests}. Keyword
4... names that are listed in @{AVAIL_BOOTS} become the selection of possible
5... boots for the test.
Leah McNuttc9c9cde2016-10-07 16:53:52 +00006
Leah McNuttc9c9cde2016-10-07 16:53:52 +00007Resource ../lib/dvt/obmc_driver_vars.txt
8Resource ../lib/list_utils.robot
Michael Walsh0bbd8602016-11-22 11:31:49 -06009Resource ../lib/openbmc_ffdc.robot
10
11Library ../lib/gen_robot_print.py
Michael Walsh55302292017-01-10 11:43:02 -060012Library ../lib/gen_misc.py
Michael Walsh0bbd8602016-11-22 11:31:49 -060013Library ../lib/gen_robot_plug_in.py
14Library ../lib/gen_robot_valid.py
15Library ../lib/state.py
16Library ../lib/boot/powerons.py
17Library ../lib/boot/poweroffs.py
18Library ../lib/obmc_boot_test.py
19
20# WITH NAME boot_results
21
22*** Variables ***
23# Initialize program parameters variables.
24# Create parm_list containing all of our program parameters. This is used by
25# 'Rqprint Pgm Header'
26@{parm_list} openbmc_nickname openbmc_host openbmc_username
27... openbmc_password os_host os_username os_password pdu_host
28... pdu_username pdu_password pdu_slot_no openbmc_serial_host
29... openbmc_serial_port boot_stack boot_list max_num_tests
30... plug_in_dir_paths status_file_path openbmc_model boot_pass boot_fail
Michael Walsh55302292017-01-10 11:43:02 -060031... ffdc_dir_path_style ffdc_check state_change_timeout power_on_timeout
32... power_on_timeout test_mode quiet debug
Michael Walsh0bbd8602016-11-22 11:31:49 -060033
34# Initialize each program parameter.
35${openbmc_nickname} ${EMPTY}
36${openbmc_host} ${EMPTY}
37${openbmc_username} root
38${openbmc_password} 0penBmc
39${os_host} ${EMPTY}
40${os_username} root
41${os_password} P@ssw0rd
42${pdu_host} ${EMPTY}
43${pdu_username} admin
44${pdu_password} admin
45${pdu_slot_no} ${EMPTY}
46${openbmc_serial_host} ${EMPTY}
47${openbmc_serial_port} ${EMPTY}
48${boot_stack} ${EMPTY}
49${boot_list} ${EMPTY}
50${max_num_tests} 0
51${plug_in_dir_paths} ${EMPTY}
52${status_file_path} ${EMPTY}
53${openbmc_model} ${EMPTY}
54# The reason boot_pass and boot_fail are parameters is that it is possible to
55# be called by a program that has already done some tests. This allows us to
56# keep the grand total.
57${boot_pass} ${0}
58${boot_fail} ${0}
Michael Walshb4125952016-12-13 15:49:50 -060059${ffdc_dir_path_style} ${EMPTY}
60${ffdc_check} ${EMPTY}
Michael Walsh55302292017-01-10 11:43:02 -060061${state_change_timeout} 1 min
62${power_on_timeout} 14 mins
63${power_off_timeout} 2 mins
Michael Walsh0bbd8602016-11-22 11:31:49 -060064${test_mode} 0
65${quiet} 0
66${debug} 0
67
Michael Walshb4125952016-12-13 15:49:50 -060068
Michael Walsh0bbd8602016-11-22 11:31:49 -060069# Plug-in variables.
70${shell_rc} 0x00000000
71${fail_on_plug_in_failure} 1
72${return_on_non_zero_rc} 0
73
74${next_boot} ${EMPTY}
75# State dictionary. Initializing to a realistic state for testing in
76# test_mode.
77&{default_state} power=1
78... bmc=HOST_BOOTED
79... boot_progress=FW Progress, Starting OS
80... os_ping=1
81... os_login=1
82... os_run_cmd=1
83&{state} &{default_state}
84
85# Flag variables.
86${cp_setup_called} ${0}
87# test_really_running is needed by DB_Logging plug-in.
88${test_really_running} ${1}
Leah McNuttc9c9cde2016-10-07 16:53:52 +000089
90*** Test Cases ***
Michael Walsh0bbd8602016-11-22 11:31:49 -060091Randomized Boot Testing
92 [Documentation] Performs random, repeated boots.
93 [Tags] Randomized_boot_testing
Leah McNuttc9c9cde2016-10-07 16:53:52 +000094
95 # Call the Main keyword to prevent any dots from appearing in the console
96 # due to top level keywords.
97 Main
98
99*** Keywords ***
Michael Walsh0bbd8602016-11-22 11:31:49 -0600100###############################################################################
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000101Main
Michael Walsh0bbd8602016-11-22 11:31:49 -0600102 [Teardown] Program Teardown
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000103
Michael Walsh0bbd8602016-11-22 11:31:49 -0600104 Setup
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000105
Michael Walsh0bbd8602016-11-22 11:31:49 -0600106 :For ${BOOT_COUNT} IN RANGE ${max_num_tests}
107 \ Test Loop Body ${BOOT_COUNT}
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000108
Michael Walsh0bbd8602016-11-22 11:31:49 -0600109 Rprint Timen Completed all requested boot tests.
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000110
Michael Walsh0bbd8602016-11-22 11:31:49 -0600111###############################################################################
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000112
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000113
Michael Walsh0bbd8602016-11-22 11:31:49 -0600114###############################################################################
115Setup
116 [Documentation] Do general program setup tasks.
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000117
Michael Walsh0bbd8602016-11-22 11:31:49 -0600118 Rprintn
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000119
Michael Walsh0bbd8602016-11-22 11:31:49 -0600120 Validate Parms
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000121
Michael Walsh0bbd8602016-11-22 11:31:49 -0600122 Rqprint Pgm Header
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000123
Michael Walsh0bbd8602016-11-22 11:31:49 -0600124 Create Boot Results Table
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000125
Michael Walsh0bbd8602016-11-22 11:31:49 -0600126 # Preserve the values of boot_pass/boot_fail that were passed in.
127 Set Global Variable ${initial_boot_pass} ${boot_pass}
128 Set Global Variable ${initial_boot_fail} ${boot_fail}
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000129
Michael Walsh0bbd8602016-11-22 11:31:49 -0600130 # Call "setup" plug-ins, if any.
131 Plug In Setup
132 ${rc} ${shell_rc} ${failed_plug_in_name}= Rprocess Plug In Packages
133 ... call_point=setup
134 Should Be Equal '${rc}' '${0}'
135
136 # Keyword "FFDC" will fail if TEST_MESSAGE is not set.
137 Set Global Variable ${TEST_MESSAGE} ${EMPTY}
138
139 # Setting cp_setup_called lets our Teardown know that it needs to call
140 # the cleanup plug-in call point.
141 Set Global Variable ${cp_setup_called} ${1}
142
143 Rqprint Timen Getting system state.
144 # The state dictionary must be primed before calling Test Loop Body.
145 ${temp_state}= Run Keyword If '${test_mode}' == '0' Get State
146 ... ELSE Create Dictionary &{default_state}
147 Set Global Variable &{state} &{temp_state}
Michael Walsh55302292017-01-10 11:43:02 -0600148 Rpvars state
Michael Walsh0bbd8602016-11-22 11:31:49 -0600149
150###############################################################################
151
152
153###############################################################################
154Validate Parms
155 [Documentation] Validate all program parameters.
156
157 rprintn
158
159 Rvalid Value AVAIL_BOOTS
160 Rvalid Value openbmc_host
161 Rvalid Value openbmc_username
162 Rvalid Value openbmc_password
163 # os_host is optional so no validation is being done.
Michael Walshb4125952016-12-13 15:49:50 -0600164 Run Keyword If '${OS_HOST}' != '${EMPTY}' Run Keywords
165 ... Rvalid Value os_username AND
166 ... Rvalid Value os_password
Michael Walsh0bbd8602016-11-22 11:31:49 -0600167 Rvalid Value pdu_host
168 Rvalid Value pdu_username
169 Rvalid Value pdu_password
170 Rvalid Integer pdu_slot_no
171 Rvalid Value openbmc_serial_host
172 Rvalid Integer openbmc_serial_port
173 Rvalid Integer max_num_tests
174 Rvalid Value openbmc_model
175 Rvalid Integer boot_pass
176 Rvalid Integer boot_fail
177
178 ${boot_pass_temp}= Convert To Integer ${boot_pass}
179 Set Global Variable ${boot_pass} ${boot_pass_temp}
180 ${boot_fail_temp}= Convert To Integer ${boot_fail}
181 Set Global Variable ${boot_fail} ${boot_fail_temp}
182
183 ${temp_arr}= Rvalidate Plug Ins ${plug_in_dir_paths}
184 Set Global Variable @{plug_in_packages_list} @{temp_arr}
185
Michael Walsh55302292017-01-10 11:43:02 -0600186 Run Keyword If '${openbmc_nickname}' == '${EMPTY}'
187 ... Set Global Variable ${openbmc_nickname} ${openbmc_host}
188
Michael Walshb4125952016-12-13 15:49:50 -0600189 Set FFDC Dir Path Style
190
191###############################################################################
192
193
194###############################################################################
195Set FFDC Dir Path Style
196
197 Run Keyword If '${ffdc_dir_path_style}' != '${EMPTY}' Return from Keyword
198
199 ${temp}= Run Keyword and Continue On Failure Get Environment Variable
200 ... FFDC_DIR_PATH_STYLE ${0}
201
202 Set Global Variable ${ffdc_dir_path_style} ${temp}
203
Michael Walsh0bbd8602016-11-22 11:31:49 -0600204###############################################################################
205
206
207###############################################################################
208Program Teardown
209 [Documentation] Clean up after this program.
210
211 Run Keyword If '${cp_setup_called}' == '1' Run Keywords
212 ... Plug In Setup AND
213 ... Rprocess Plug In Packages call_point=cleanup
214 ... stop_on_plug_in_failure=1
215
216 Rqprint Pgm Footer
217
218###############################################################################
219
220
221###############################################################################
222Test Loop Body
223 [Documentation] The main loop body for the loop in "main".
224 [Arguments] ${BOOT_COUNT}
225
226 Rqprintn
227 Rqprint Timen Starting boot ${BOOT_COUNT+1} of ${max_num_tests}.
228
229 ${loc_next_boot}= Select Boot ${state['power']}
230 Set Global Variable ${next_boot} ${loc_next_boot}
231
Michael Walsh55302292017-01-10 11:43:02 -0600232 # Clear this file. Plug-ins may now write to it.
233 Remove File ${FFDC_LIST_FILE_PATH}
234
Michael Walsh0bbd8602016-11-22 11:31:49 -0600235 ${status} ${msg}= Run Keyword And Ignore Error Run Boot ${next_boot}
236 Run Keyword If '${status}' == 'FAIL' rprint ${msg}
237
238 rprintn
239 Run Keyword If '${BOOT_STATUS}' == 'PASS' Run Keywords
240 ... Set Global Variable ${boot_success} ${1} AND
241 ... Rqprint Timen BOOT_SUCCESS: "${next_boot}" succeeded.
242 ... ELSE Run Keywords
243 ... Set Global Variable ${boot_success} ${0} AND
244 ... Rqprint Timen BOOT_FAILED: ${next_boot} failed.
245
246 Update Boot Results Table ${next_boot} ${BOOT_STATUS}
247
248 # NOTE: A post_test_case call point failure is NOT counted as a boot
249 # failure.
250 Plug In Setup
251 ${rc} ${shell_rc} ${failed_plug_in_name}= Rprocess Plug In Packages
252 ... call_point=post_test_case stop_on_plug_in_failure=1
253
Michael Walsh55302292017-01-10 11:43:02 -0600254 ${rc} ${shell_rc} ${failed_plug_in_name}= Rprocess Plug In Packages
255 ... call_point=ffdc_check shell_rc=${0x00000200}
256 ... stop_on_plug_in_failure=1 stop_on_non_zero_rc=1
257
258 Run Keyword If
259 ... '${BOOT_STATUS}' != 'PASS' or '${FFDC_CHECK}' == 'All' or '${shell_rc}' == '${0x00000200}'
Michael Walsh0bbd8602016-11-22 11:31:49 -0600260 ... Run Keyword and Continue On Failure My FFDC
261
262 # Run plug-ins to see if we ought to stop execution.
263 Plug In Setup
264 ${rc} ${shell_rc} ${failed_plug_in_name}= Rprocess Plug In Packages
265 ... call_point=stop_check
266 Run Keyword If '${rc}' != '${0}' Run Keywords
267 ... Rprint Error Report Stopping as requested by user.
268 ... Fail
269
270 Print Boot Results Table
271 Rqprint Timen Finished boot ${BOOT_COUNT+1} of ${max_num_tests}.
272
273 Rqprint Timen Getting system state.
274 # The state must be refreshed before calling Test Loop Body again.
275 ${temp_state}= Run Keyword If '${test_mode}' == '0' Get State
276 ... quiet=${1}
277 ... ELSE Create Dictionary &{default_state}
278 Set Global Variable &{state} &{temp_state}
279 rpvars state
280
281###############################################################################
282
283
284###############################################################################
285Select Boot
286 [Documentation] Select a boot test to be run based on our current state.
287 ... Return the chosen boot type.
288 [Arguments] ${power}
289
290 # power The power state of the machine, either zero or one.
291
292 ${boot}= Run Keyword If ${power} == ${0} Select Power On
293 ... ELSE Run Keyword If ${power} == ${1} Select Power Off
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000294 ... ELSE Run Keywords Log to Console
Michael Walsh0bbd8602016-11-22 11:31:49 -0600295 ... **ERROR** BMC not in state to power on or off: "${power}" AND
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000296 ... Fatal Error
297
Michael Walsh55302292017-01-10 11:43:02 -0600298 [return] ${boot}
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000299
Michael Walsh0bbd8602016-11-22 11:31:49 -0600300###############################################################################
301
302
303###############################################################################
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000304Select Power On
Michael Walsh0bbd8602016-11-22 11:31:49 -0600305 [Documentation] Randomly chooses a boot from the list of Power On boots.
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000306
Michael Walsh0bbd8602016-11-22 11:31:49 -0600307 @{power_on_choices}= Intersect Lists ${VALID_POWER_ON} ${AVAIL_BOOTS}
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000308
309 ${length}= Get Length ${power_on_choices}
310
Michael Walsh0bbd8602016-11-22 11:31:49 -0600311 # Currently selects the first boot in the list of options, rather than
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000312 # selecting randomly.
313 ${chosen}= Set Variable @{power_on_choices}[0]
314
Michael Walsh55302292017-01-10 11:43:02 -0600315 [return] ${chosen}
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000316
Michael Walsh0bbd8602016-11-22 11:31:49 -0600317###############################################################################
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000318
Michael Walsh0bbd8602016-11-22 11:31:49 -0600319
320###############################################################################
321Select Power Off
322 [Documentation] Randomly chooses an boot from the list of Power Off boots.
323
324 @{power_off_choices}= Intersect Lists ${VALID_POWER_OFF} ${AVAIL_BOOTS}
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000325
326 ${length}= Get Length ${power_off_choices}
327
Michael Walsh0bbd8602016-11-22 11:31:49 -0600328 # Currently selects the first boot in the list of options, rather than
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000329 # selecting randomly.
330 ${chosen}= Set Variable @{power_off_choices}[0]
331
Michael Walsh55302292017-01-10 11:43:02 -0600332 [return] ${chosen}
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000333
Michael Walsh0bbd8602016-11-22 11:31:49 -0600334###############################################################################
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000335
Michael Walsh0bbd8602016-11-22 11:31:49 -0600336
337###############################################################################
338Run Boot
339 [Documentation] Run the selected boot and mark the status when complete.
340 [Arguments] ${boot_keyword}
341 [Teardown] Set Global Variable ${BOOT_STATUS} ${KEYWORD STATUS}
342
343 # boot_keyword The name of the boot to run, which corresponds to the
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000344 # keyword to run. (i.e "BMC Power On")
345
Michael Walsh0bbd8602016-11-22 11:31:49 -0600346 Print Test Start Message ${boot_keyword}
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000347
Michael Walsh0bbd8602016-11-22 11:31:49 -0600348 Plug In Setup
349 ${rc} ${shell_rc} ${failed_plug_in_name}= Rprocess Plug In Packages
350 ... call_point=pre_boot
351 Should Be Equal '${rc}' '${0}'
352
353 @{cmd_buf}= Create List ${boot_keyword}
354 rqpissuing_keyword ${cmd_buf} ${test_mode}
355 Run Keyword If '${test_mode}' == '0' Run Keyword @{cmd_buf}
356
357 Plug In Setup
358 ${rc} ${shell_rc} ${failed_plug_in_name}= Rprocess Plug In Packages
359 ... call_point=post_boot
360 Should Be Equal '${rc}' '${0}'
361
362###############################################################################
363
364
365###############################################################################
366Print Test Start Message
367 [Documentation] Print a message indicating what boot test is about to run.
368 [Arguments] ${boot_keyword}
369
370 ${doing_msg}= Sprint Timen Doing "${boot_keyword}".
371 rqprint ${doing_msg}
372
373 Append to List ${LAST_TEN} ${doing_msg}
374 ${length}= Get Length ${LAST_TEN}
375
376 Run Keyword If '${length}' > '${10}' Remove From List ${LAST_TEN} 0
377
378###############################################################################
379
380
381###############################################################################
Michael Walsh55302292017-01-10 11:43:02 -0600382Print Defect Report
383 [Documentation] Print a defect report.
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000384
Michael Walsh0bbd8602016-11-22 11:31:49 -0600385 Rqprintn
386 # indent=0, width=90, linefeed=1, char="="
387 Rqprint Dashes ${0} ${90} ${1} =
388 Rqprintn Copy this data to the defect:
389 Rqprintn
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000390
Michael Walsh0bbd8602016-11-22 11:31:49 -0600391 Rqpvars @{parm_list}
392 Print Last Ten Boots
Michael Walsh55302292017-01-10 11:43:02 -0600393 Rqprintn
394 Rqpvars state
Michael Walsh0bbd8602016-11-22 11:31:49 -0600395
Michael Walsh55302292017-01-10 11:43:02 -0600396 # At some point I'd like to have the 'Call FFDC Methods' return a list
397 # of files it has collected. In that case, the following "ls" command
398 # would no longer be needed. For now, however, ls shows the files
399 # named in FFDC_LIST_FILE_PATH so I will refrain from printing those
400 # out (so we don't see duplicates in the list).
401
402 ${rc} ${output}= Run and return RC and Output ls ${LOG_PREFIX}*
Michael Walsh0bbd8602016-11-22 11:31:49 -0600403
404 Run Keyword If '${rc}' != '${0}' and '${rc}' != 'None' rqpvars rc
Michael Walsh55302292017-01-10 11:43:02 -0600405 ${status} ${ffdc_list}= Run Keyword and Ignore Error
406 ... OperatingSystem.Get File ${FFDC_LIST_FILE_PATH}
Michael Walsh0bbd8602016-11-22 11:31:49 -0600407
408 Rqprintn
409 Rqprintn FFDC data files:
Michael Walsh55302292017-01-10 11:43:02 -0600410 Run Keyword If '${status_file_path}' != '${EMPTY}'
411 ... Rqprintn ${status_file_path}
Michael Walsh0bbd8602016-11-22 11:31:49 -0600412 Rqprintn ${output}
Michael Walsh55302292017-01-10 11:43:02 -0600413 # Run Keyword If '${status}' == 'PASS' Rqprintn ${ffdc_list}
Michael Walsh0bbd8602016-11-22 11:31:49 -0600414 Rqprintn
Michael Walsh55302292017-01-10 11:43:02 -0600415
Michael Walsh0bbd8602016-11-22 11:31:49 -0600416 Rqprint Dashes ${0} ${90} ${1} =
417
418###############################################################################
419
420
421###############################################################################
422Print Last Ten Boots
423 [Documentation] Logs the last ten boots that were performed with their
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000424 ... starting time stamp.
425
Michael Walsh0bbd8602016-11-22 11:31:49 -0600426 # indent 0, 90 chars wide, linefeed, char is "="
427 Rqprint Dashes ${0} ${90}
428 Rqprintn Last 10 boots:
429 Rqprintn
430 :FOR ${boot_entry} IN @{LAST_TEN}
431 \ rqprint ${boot_entry}
432 Rqprint Dashes ${0} ${90}
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000433
Michael Walsh0bbd8602016-11-22 11:31:49 -0600434###############################################################################
Leah McNuttc9c9cde2016-10-07 16:53:52 +0000435
Michael Walsh0bbd8602016-11-22 11:31:49 -0600436