blob: fd2fea093ae80d0d81d8a2c2eeaa91d19280d903 [file] [log] [blame]
leetde9356e2022-03-23 01:39:29 +00001*** Settings ***
2
3Documentation Module to test out of band IPMI watchdog functionality.
4Resource ../lib/ipmi_client.robot
5Resource ../lib/openbmc_ffdc.robot
6Resource ../lib/boot_utils.robot
7Library ../lib/ipmi_utils.py
8Library String
9Library Collections
10Variables ../data/ipmi_raw_cmd_table.py
11
George Keishing87dc4422023-10-20 12:56:30 +053012Force Tags IPMI_Watchdog
13
leetde9356e2022-03-23 01:39:29 +000014*** Variables ***
15
16${TIMER_DELAY} 3
Nagarjun B011c5f32023-07-17 17:14:59 +053017${POWER_STATE_CHANGE} 3
leetde9356e2022-03-23 01:39:29 +000018
19*** Test Cases ***
20
21Test IPMI Watchdog Timer Does Not Log Bit
22 [Documentation] Execute out of band set/get do not log bit for watchdog timer.
23 [Tags] Test_IPMI_Watchdog_Timer_Does_Not_Log_Bit
24 [Template] Execute IPMI Raw Command And Verify Response Data
25
26 # set_raw_cmd get_raw_cmd resp_expect
27 ${IPMI_RAW_CMD['Watchdog']['Set'][0]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
28 ... ${IPMI_RAW_CMD['Watchdog']['Get'][1]}
29 ${IPMI_RAW_CMD['Watchdog']['Set'][3]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
30 ... ${IPMI_RAW_CMD['Watchdog']['Get'][3]}
31
32Test IPMI Watchdog Timer Stop Bit
33 [Documentation] Execute out of band set/get stop/resume timer stop bit for watchdog timer.
34 [Tags] Test_IPMI_Watchdog_Timer_Stop_Bit
35 [Template] Execute IPMI Raw Command And Verify Response Data After Watchdog Expires
36
37 # set_raw_cmd get_raw_cmd resp_expect
38 ${IPMI_RAW_CMD['Watchdog']['Set'][6]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
39 ... ${IPMI_RAW_CMD['Watchdog']['Get'][5]}
40 ${IPMI_RAW_CMD['Watchdog']['Set'][9]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
41 ... ${IPMI_RAW_CMD['Watchdog']['Get'][7]}
42
43Test IPMI Watchdog Timer Use Bits
44 [Documentation] Execute out of band set/get timer use bits for watchdog timer.
45 [Tags] Test_IPMI_Watchdog_Timer_Use_Bits
46 [Template] Execute IPMI Raw Command And Verify Response Data
47
48 # set_raw_cmd get_raw_cmd resp_expect
49 ${IPMI_RAW_CMD['Watchdog']['Set'][12]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
50 ... ${IPMI_RAW_CMD['Watchdog']['Get'][9]}
51 ${IPMI_RAW_CMD['Watchdog']['Set'][15]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
52 ... ${IPMI_RAW_CMD['Watchdog']['Get'][11]}
53 ${IPMI_RAW_CMD['Watchdog']['Set'][18]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
54 ... ${IPMI_RAW_CMD['Watchdog']['Get'][13]}
55 ${IPMI_RAW_CMD['Watchdog']['Set'][21]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
56 ... ${IPMI_RAW_CMD['Watchdog']['Get'][15]}
57 ${IPMI_RAW_CMD['Watchdog']['Set'][24]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
58 ... ${IPMI_RAW_CMD['Watchdog']['Get'][17]}
59
60Test IPMI Watchdog Timer Pre-Timeout Interrupt Bits
61 [Documentation] Execute out of band set/get pre-timeout interrupt bits for watchdog timer.
62 [Tags] Test_IPMI_Watchdog_Timer_Pre-Timeout_Interrupt_Bits
63 [Template] Execute IPMI Raw Command And Verify Response Data
64
65 # set_raw_cmd get_raw_cmd resp_expect
66 ${IPMI_RAW_CMD['Watchdog']['Set'][27]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
67 ... ${IPMI_RAW_CMD['Watchdog']['Get'][19]}
68 ${IPMI_RAW_CMD['Watchdog']['Set'][30]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
69 ... ${IPMI_RAW_CMD['Watchdog']['Get'][21]}
70
71Test IPMI Watchdog Timer Timeout Action Bits
72 [Documentation] Execute out of band set/get timer timeout bits for watchdog timer.
73 [Tags] Test_IPMI_Watchdog_Timer_Timeout_Action_Bits
74 [Template] Execute IPMI Raw Command And Verify Response Data
75
76 # set_raw_cmd get_raw_cmd resp_expect
77 ${IPMI_RAW_CMD['Watchdog']['Set'][33]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
78 ... ${IPMI_RAW_CMD['Watchdog']['Get'][23]}
79 ${IPMI_RAW_CMD['Watchdog']['Set'][36]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
80 ... ${IPMI_RAW_CMD['Watchdog']['Get'][25]}
81 ${IPMI_RAW_CMD['Watchdog']['Set'][39]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
82 ... ${IPMI_RAW_CMD['Watchdog']['Get'][27]}
83 ${IPMI_RAW_CMD['Watchdog']['Set'][42]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
84 ... ${IPMI_RAW_CMD['Watchdog']['Get'][29]}
85
86Test IPMI Watchdog Timer Timeout Flag Bits
87 [Documentation] Execute out of band set/get timer timeout flag bits for watchdog timer.
88 [Tags] Test_IPMI_Watchdog_Timer_Timeout_Flag_Bits
89 [Template] Execute IPMI Raw Command And Verify Timer Expiration Data
90
91 # set_raw_cmd get_raw_cmd resp_expect
92 ${IPMI_RAW_CMD['Watchdog']['Set'][45]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
93 ... ${IPMI_RAW_CMD['Watchdog']['Get'][31]}
94 ${IPMI_RAW_CMD['Watchdog']['Set'][48]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
95 ... ${IPMI_RAW_CMD['Watchdog']['Get'][33]}
96 ${IPMI_RAW_CMD['Watchdog']['Set'][51]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
97 ... ${IPMI_RAW_CMD['Watchdog']['Get'][35]}
98 ${IPMI_RAW_CMD['Watchdog']['Set'][54]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
99 ... ${IPMI_RAW_CMD['Watchdog']['Get'][37]}
100 ${IPMI_RAW_CMD['Watchdog']['Set'][57]} ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
101 ... ${IPMI_RAW_CMD['Watchdog']['Get'][39]}
102
103
104Verify Timer Action For State Change
105 [Documentation] Set Watchdog via IPMI raw command and verify timer actions.
George Keishing71fd6712022-03-28 09:56:14 -0500106 [Tags] Verify_Timer_Action_For_State_Change
leetde9356e2022-03-23 01:39:29 +0000107 [Template] Validate Watchdog Timer Actions And SEL Events
108
109 # set action command power state SEL event
ganesanbdf75fea2023-04-28 18:06:02 +0000110 ${IPMI_RAW_CMD['Watchdog']['Set'][60]} ['off'] Power down
111 ${IPMI_RAW_CMD['Watchdog']['Set'][63]} ['on'] Hard reset
leetde9356e2022-03-23 01:39:29 +0000112 ${IPMI_RAW_CMD['Watchdog']['Set'][66]} ['on'] Power cycle
113 ${IPMI_RAW_CMD['Watchdog']['Set'][69]} ['on'] Timer expired
114
115
116Verify Reset Timer
117 [Documentation] Set Watchdog via IPMI raw command and verify Reset Timer functions as expected.
118 [Tags] Verify_Reset_Timer
119
120 # Check the chassis status.
121 Power On Host And Verify
122
123 # Set Watchdog Timer initCount(0x3530).
ishwaryamathim372cd862023-11-14 18:12:23 +0000124 Run IPMI Command ${IPMI_RAW_CMD['Watchdog']['Set'][72]}
leetde9356e2022-03-23 01:39:29 +0000125
126 # Get Watchdog Timer.
ishwaryamathim372cd862023-11-14 18:12:23 +0000127 ${resp}= Run IPMI Command ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
leetde9356e2022-03-23 01:39:29 +0000128 Should Contain ${resp} ${IPMI_RAW_CMD['Watchdog']['Get'][41]}
129
130 @{start_timer_value}= Split String ${resp}
131
132 # Convert start value to integer.
133 # Example: Get watchdog response is 0x06 0x24 0x05 0x00 0x64 0x00 0x64 0x00.
134 # Start_timer_value is bits 6 - 7; set to 0x64 0x00 (100 ms decimal).
135 # Reverse bits 6 - 7 due to BMC being little endian; new value is 0x00 0x64.
136 # Convert hex value 0x00 0x64 to integer; start_timer_integer = 100.
137 ${value}= Get Slice From List ${start_timer_value} 6
138 Reverse List ${value}
139 ${start_timer_string}= Evaluate "".join(${value})
140 ${start_timer_integer} = Convert To Integer ${start_timer_string} 16
141
142 # Delay.
143 Sleep ${TIMER_DELAY}
144
145 # Get Watchdog Timer before reset watchdog timer.
ishwaryamathim372cd862023-11-14 18:12:23 +0000146 ${resp}= Run IPMI Command ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
leetde9356e2022-03-23 01:39:29 +0000147 Should Contain ${resp} ${IPMI_RAW_CMD['Watchdog']['Get'][41]}
148
149 FOR ${1} IN ${3}
150
151 # Reset Watchdog Timer.
ishwaryamathim372cd862023-11-14 18:12:23 +0000152 Run IPMI Command ${IPMI_RAW_CMD['Watchdog']['Reset'][0]}
leetde9356e2022-03-23 01:39:29 +0000153 # Delay.
154 Sleep ${TIMER_DELAY}
155 Get Watchdog Timer And Compare To Start Value ${start_timer_integer}
156
157 END
158
159
160Verify Pre-timeout Values
161 [Documentation] Set Watchdog Pre-timeout via IPMI raw command and verify via Get Watchdog Timer.
162 [Tags] Verify_Pre-timeout_Values
163 [Template] Validate Watchdog Pre-timeout
164
165 # command response
166 ${IPMI_RAW_CMD['Watchdog']['Set'][75]} ${EMPTY}
167 ${IPMI_RAW_CMD['Watchdog']['Set'][81]} ${EMPTY}
168 ${IPMI_RAW_CMD['Watchdog']['Get'][0]} ${IPMI_RAW_CMD['Watchdog']['Get'][43]}
169
170Verify Failure For Pre-Timeout Interval Greater Than Initial Count
171 [Documentation] Set Watchdog Pre-timeout via IPMI raw command and verify via Get Watchdog Timer.
172 [Tags] Verify_Failure_For_Pre-Timeout_Interval_Greater_Than_Initial_Count
173
174 # Expected to fail: pre-timeout interval (4000) > initial count (1000).
175 Run Keyword and Expect Error *Invalid data field*
ishwaryamathim372cd862023-11-14 18:12:23 +0000176 ... Run IPMI Command ${IPMI_RAW_CMD['Watchdog']['Set'][78]}
leetde9356e2022-03-23 01:39:29 +0000177
178Verify Invalid Request Data Length
179 [Documentation] Set Watchdog via IPMI raw command and verify via Get Watchdog Timer.
180 [Tags] Verify_Invalid_Request_Data_Length
181 [Template] Watchdog Invalid Request Data Length
182
183 # command
184 ${IPMI_RAW_CMD['Watchdog']['Set'][84]}
185 ${IPMI_RAW_CMD['Watchdog']['Set'][87]}
186 ${IPMI_RAW_CMD['Watchdog']['Get'][45]}
187
188Verify Invalid Reset Timer Request Data
189 [Documentation] Set Watchdog via IPMI raw command and verify via Get Watchdog Timer.
190 [Tags] Verify_Invalid_Reset_Timer_Request_Data
191
192 # Reset Watchdog Timer with one extra byte.
193 Run Keyword and Expect Error *Request data length*
ishwaryamathim372cd862023-11-14 18:12:23 +0000194 ... Run IPMI Command ${IPMI_RAW_CMD['Watchdog']['Reset'][3]}
leetde9356e2022-03-23 01:39:29 +0000195
196 # Reset BMC.
ishwaryamathim372cd862023-11-14 18:12:23 +0000197 Run IPMI Standard Command mc reset cold -N 10 -R 1
Nagarjun B011c5f32023-07-17 17:14:59 +0530198 Wait Until Keyword Succeeds ${OPENBMC_REBOOT_TIMEOUT} min 10 sec
199 ... Is BMC Operational
leetde9356e2022-03-23 01:39:29 +0000200
201 # Reset Watchdog Timer without initialized watchdog.
202 Run Keyword and Expect Error *Unknown*
ishwaryamathim372cd862023-11-14 18:12:23 +0000203 ... Run IPMI Command ${IPMI_RAW_CMD['Watchdog']['Reset'][6]}
leetde9356e2022-03-23 01:39:29 +0000204
205*** Keywords ***
206
207Execute IPMI Raw Command And Verify Response Data After Watchdog Expires
208 [Documentation] Execute out of band IPMI raw command and verify response data after watchdog expires.
209 [Arguments] ${set_raw_cmd} ${get_raw_cmd} ${resp_expect}
210 # Description of argument(s):
211 # set_raw_cmd The request bytes for the command.
212 # get_raw_cmd The response bytes for the command.
213 # resp_expect The expected response bytes for the command.
214
ishwaryamathim372cd862023-11-14 18:12:23 +0000215 Run IPMI Command ${set_raw_cmd}
216 Run IPMI Command ${IPMI_RAW_CMD['Watchdog']['Reset'][0]}
217 Run IPMI Command ${set_raw_cmd}
218 ${resp}= Run IPMI Command ${get_raw_cmd}
leetde9356e2022-03-23 01:39:29 +0000219 Should Contain ${resp} ${resp_expect} msg=Expecting ${resp_expect} but got ${resp}.
220
221Execute IPMI Raw Command And Verify Response Data
222 [Documentation] Execute out of band IPMI raw command and verify response data.
223 [Arguments] ${set_raw_cmd} ${get_raw_cmd} ${resp_expect}
224
225 # Description of argument(s):
226 # set_raw_cmd The request bytes for the command.
227 # get_raw_cmd The response bytes for the command.
228 # resp_expect The expected response bytes for the command.
229
ishwaryamathim372cd862023-11-14 18:12:23 +0000230 Run IPMI Command ${set_raw_cmd}
231 ${resp}= Run IPMI Command ${get_raw_cmd}
leetde9356e2022-03-23 01:39:29 +0000232 Should Contain ${resp} ${resp_expect} msg=Expecting ${resp_expect} but got ${resp}.
233
234Execute IPMI Raw Command And Verify Timer Expiration Data
235 [Documentation] Execute out of band IPMI raw command and verify timer expiration response data.
236 [Arguments] ${set_raw_cmd} ${get_raw_cmd} ${resp_expect}
237
238 # Description of argument(s):
239 # set_raw_cmd The request bytes for the command.
240 # get_raw_cmd The response bytes for the command.
241 # resp_expect The expected response bytes for the command.
242
ishwaryamathim372cd862023-11-14 18:12:23 +0000243 Run IPMI Command ${set_raw_cmd}
244 Run IPMI Command ${get_raw_cmd}
245 Run IPMI Command ${IPMI_RAW_CMD['Watchdog']['Reset'][0]}
246 ${resp}= Run IPMI Command ${get_raw_cmd}
leetde9356e2022-03-23 01:39:29 +0000247 Should Contain ${resp} ${resp_expect} msg=Expecting ${resp_expect} but got ${resp}.
248
249Validate Watchdog Timer Actions And SEL Events
250 [Documentation] Verify the watchdog timer actions and the associated SEL events.
251 [Arguments] ${set_raw_cmd} ${power_state} ${sel_event}
252
253 # Description of argument(s):
254 # set_raw_cmd The set timeout action request bytes for the command.
255 # power_state The expected power state of the host.
256 # sel_event The response bytes for the command.
257
258 # Check the chassis status.
259 Power On Host And Verify
260
261 # Clear SEL.
262 Run IPMI Standard Command sel clear
263
264 # Set watchdog timer action to perform action.
ishwaryamathim372cd862023-11-14 18:12:23 +0000265 Run IPMI Command ${set_raw_cmd}
leetde9356e2022-03-23 01:39:29 +0000266
267 # Reset Watchdog Timer.
ishwaryamathim372cd862023-11-14 18:12:23 +0000268 Run IPMI Command ${IPMI_RAW_CMD['Watchdog']['Reset'][0]}
leetde9356e2022-03-23 01:39:29 +0000269
270 # Delay for power state.
271 Sleep ${POWER_STATE_CHANGE}
272
Nagarjun B011c5f32023-07-17 17:14:59 +0530273 IF '${sel_event}' == 'Hard reset' or '${sel_event}' == 'Power cycle'
274 # If timer expire action is 'Hard Reset' then the host/chassis immediately powering on after
275 # timer expires. So, verify host is rebooting by expecting that host is unpingable.
276 Run Keyword If '${sel_event}' == 'Power cycle'
277 ... Wait Until Keyword Succeeds 1 min 10 sec Verify Host Power State ['off']
278 ... ELSE
279 ... Wait Until Keyword Succeeds 20 sec 5 sec Is Host Unpingable ${OS_HOST}
280 Wait Until Keyword Succeeds 5 min 30 sec Verify Host Power State ['on']
281 Wait Until Keyword Succeeds 7 min 30 sec OS Execute Command uptime
282 ELSE
283 Wait Until Keyword Succeeds 1 min 20 sec Verify Host Power State ${power_state}
284 END
285
leetde9356e2022-03-23 01:39:29 +0000286 Verify Watchdog Timer Action SEL Event ${sel_event}
287
288
289Verify Host Power State
290 [Documentation] Get host power state using external IPMI command and verify.
291 [Arguments] ${power_state}
292
293 # Description of argument(s):
294 # power_state Value of Host power state: "on" or "off".
295
296 ${ipmi_state}= Get Host State Via External IPMI
297 Valid Value ipmi_state ${power_state}
298
299Verify Watchdog Timer Action SEL Event
300 [Documentation] Verify_Watchdog_Timer_Action_SEL_Event.
301 [Arguments] ${sel_event}
302
303 # Description of argument(s):
304 # sel_event Text of SEL event after timer action.
305
306 ${resp}= Run IPMI Standard Command sel elist
307 ${power_status}= Get Lines Containing String ${resp} Watchdog
308 Should Contain ${power_status} ${sel_event}
309
310Power On Host And Verify
311 [Documentation] Power the host on and verify.
312
313 IPMI Power On stack_mode=skip quiet=1
314 ${ipmi_state}= Get Host State Via External IPMI
315 Valid Value ipmi_state ['on']
316
317Watchdog Invalid Request Data Length
318 [Documentation] Verify invalid request bytes for set watchdog returns correct error.
319 [Arguments] ${watchdog_command}
320
321 # Description of argument(s):
322 # watchdog_command The raw watchdog IPMI command request bytes.
323
324 Run Keyword and Expect Error *Request data length*
ishwaryamathim372cd862023-11-14 18:12:23 +0000325 ... Run IPMI Command ${watchdog_command}
leetde9356e2022-03-23 01:39:29 +0000326
327Validate Watchdog Pre-timeout
328 [Documentation] Verify watchdog pre-timeout valid request bytes.
329 [Arguments] ${watchdog_command} ${response}
330
331 # Description of argument(s):
332 # watchdog_command The raw watchdog IPMI command request bytes.
333 # response The expected response bytes.
334
ishwaryamathim372cd862023-11-14 18:12:23 +0000335 ${resp}= Run IPMI Command ${watchdog_command}
leetde9356e2022-03-23 01:39:29 +0000336 Should Contain ${resp} ${response}
337
338Get Watchdog Timer And Compare To Start Value
339 [Documentation] Get watchdog value, convert to integer, and compare to original start value.
340 [Arguments] ${start_timer_integer}
341
342 # Description of argument(s):
343 # start_timer_integer The initial value for the watchdog timer.
344
345 # Get Watchdog Timer.
ishwaryamathim372cd862023-11-14 18:12:23 +0000346 ${resp}= Run IPMI Command ${IPMI_RAW_CMD['Watchdog']['Get'][0]}
leetde9356e2022-03-23 01:39:29 +0000347 @{timer_value}= Split String ${resp}
348
349 # Convert to integer and compare with start value.
350 # Example: Get watchdog response is 0x06 0x24 0x05 0x00 0x64 0x00 0x64 0x00.
351 # Start_timer_value is bits 6 - 7; set to 0x64 0x00 (100 ms decimal).
352 # Reverse bits 6 - 7 due to BMC being little endian; new value is 0x00 0x64.
353 # Convert hex value 0x00 0x64 to integer; start_timer_integer = 100.
354 ${value}= Get Slice From List ${timer_value} 6
355 Reverse List ${value}
356 ${timer_string}= Evaluate "".join(${value})
357 ${current_timer_integer}= Convert To Integer ${timer_string} 16
George Keishing71fd6712022-03-28 09:56:14 -0500358 Should Be True ${current_timer_integer} < ${start_timer_integer}