George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 1 | *** Settings *** |
| 2 | Documentation Methods to execute commands on BMC and collect |
| 3 | ... data to a list of FFDC files |
| 4 | |
| 5 | Resource openbmc_ffdc_utils.robot |
George Keishing | 8e47f53 | 2016-10-05 08:41:45 -0500 | [diff] [blame] | 6 | Resource rest_client.robot |
Gunnar Mills | aca140d | 2016-10-26 13:05:10 -0500 | [diff] [blame] | 7 | Resource utils.robot |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 8 | Resource list_utils.robot |
George Keishing | ab7a907 | 2017-02-24 13:03:24 -0600 | [diff] [blame] | 9 | Library SSHLibrary |
Sivas SRR | 0d657e8 | 2017-03-13 01:00:34 -0500 | [diff] [blame] | 10 | Library OperatingSystem |
George Keishing | 16333ec | 2017-03-21 06:18:29 -0500 | [diff] [blame] | 11 | Library Collections |
Michael Walsh | e844e9a | 2017-04-20 16:51:10 -0500 | [diff] [blame] | 12 | Library String |
Michael Walsh | e53e47a | 2017-06-30 17:03:24 -0500 | [diff] [blame] | 13 | Library gen_print.py |
Michael Walsh | 02c7060 | 2018-10-04 15:24:30 -0500 | [diff] [blame^] | 14 | Library gen_cmd.py |
Michael Walsh | e844e9a | 2017-04-20 16:51:10 -0500 | [diff] [blame] | 15 | Library gen_robot_keyword.py |
Sweta Potthuri | a82efd6 | 2017-10-18 05:34:59 -0500 | [diff] [blame] | 16 | Library dump_utils.py |
Michael Walsh | 02c7060 | 2018-10-04 15:24:30 -0500 | [diff] [blame^] | 17 | Library logging_utils.py |
Gunnar Mills | 7e2cda2 | 2016-10-11 15:37:34 -0500 | [diff] [blame] | 18 | |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 19 | *** Keywords *** |
| 20 | |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 21 | # Method : Call FFDC Methods # |
| 22 | # Execute the user define keywords from the FFDC List # |
| 23 | # Unlike any other keywords this will call into the # |
| 24 | # list of keywords defined in the FFDC list at one go # |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 25 | |
| 26 | Call FFDC Methods |
Michael Walsh | e844e9a | 2017-04-20 16:51:10 -0500 | [diff] [blame] | 27 | [Documentation] Call into FFDC Keyword index list. |
| 28 | [Arguments] ${ffdc_function_list}=${EMPTY} |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 29 | |
Michael Walsh | e844e9a | 2017-04-20 16:51:10 -0500 | [diff] [blame] | 30 | # Description of argument(s): |
| 31 | # ffdc_function_list A colon-delimited list naming the kinds of FFDC that |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 32 | # are to be collected |
Michael Walsh | e844e9a | 2017-04-20 16:51:10 -0500 | [diff] [blame] | 33 | # (e.g. "FFDC Generic Report:BMC Specific Files"). |
| 34 | # Acceptable values can be found in the description |
| 35 | # field of FFDC_METHOD_CALL in |
| 36 | # lib/openbmc_ffdc_list.py. Those values can be |
| 37 | # obtained via a call to 'Get FFDC Method Desc' (also |
| 38 | # from lib/openbmc_ffdc_list.py). |
| 39 | |
| 40 | @{entries}= Get FFDC Method Index |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 41 | # Example entries: |
| 42 | # entries: |
| 43 | # entries[0]: BMC LOGS |
| 44 | |
| 45 | @{ffdc_file_list}= Create List |
Michael Walsh | e844e9a | 2017-04-20 16:51:10 -0500 | [diff] [blame] | 46 | :FOR ${index} IN @{entries} |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 47 | \ ${ffdc_file_sub_list}= Method Call Keyword List ${index} |
| 48 | ... ${ffdc_function_list} |
| 49 | \ ${ffdc_file_list}= Smart Combine Lists ${ffdc_file_list} |
| 50 | ... ${ffdc_file_sub_list} |
| 51 | |
Michael Walsh | 6bad841 | 2017-06-27 14:09:36 -0500 | [diff] [blame] | 52 | Run Key U SSHLibrary.Close All Connections |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 53 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 54 | [Return] ${ffdc_file_list} |
| 55 | |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 56 | Method Call Keyword List |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 57 | [Documentation] Process FFDC request and return a list of generated files. |
Michael Walsh | e844e9a | 2017-04-20 16:51:10 -0500 | [diff] [blame] | 58 | [Arguments] ${index} ${ffdc_function_list}=${EMPTY} |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 59 | |
Michael Walsh | e844e9a | 2017-04-20 16:51:10 -0500 | [diff] [blame] | 60 | # Description of argument(s): |
| 61 | # index The index into the FFDC_METHOD_CALL dictionary (e.g. |
| 62 | # 'BMC LOGS'). |
| 63 | # ffdc_function_list See ffdc_function_list description in |
| 64 | # "Call FFDC Methods" (above). |
| 65 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 66 | @{method_list}= Get FFDC Method Call ${index} |
| 67 | # Example method_list: |
| 68 | # method_list: |
| 69 | # method_list[0]: |
| 70 | # method_list[0][0]: FFDC Generic Report |
| 71 | # method_list[0][1]: BMC FFDC Manifest |
| 72 | # method_list[1]: |
| 73 | # method_list[1][0]: Get Request FFDC |
| 74 | # method_list[1][1]: BMC FFDC Get Requests |
| 75 | # (etc.) |
Michael Walsh | e844e9a | 2017-04-20 16:51:10 -0500 | [diff] [blame] | 76 | |
| 77 | # If function list is empty assign default (i.e. a list of all allowable |
| 78 | # values). In either case, convert ffdc_function_list from a string to |
| 79 | # a list. |
| 80 | @{ffdc_function_list}= |
| 81 | ... Run Keyword If '${ffdc_function_list}' == '${EMPTY}' |
| 82 | ... Get FFDC Method Desc ${index} |
| 83 | ... ELSE |
| 84 | ... Split String ${ffdc_function_list} separator=: |
| 85 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 86 | @{ffdc_file_list}= Create List |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 87 | :FOR ${method} IN @{method_list} |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 88 | \ ${ffdc_file_sub_list}= Execute Keyword Method ${method[0]} |
| 89 | ... ${method[1]} @{ffdc_function_list} |
| 90 | \ ${ffdc_file_list}= Smart Combine Lists ${ffdc_file_list} |
| 91 | ... ${ffdc_file_sub_list} |
| 92 | |
| 93 | [Return] ${ffdc_file_list} |
| 94 | |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 95 | |
| 96 | Execute Keyword Method |
Michael Walsh | e844e9a | 2017-04-20 16:51:10 -0500 | [diff] [blame] | 97 | [Documentation] Call into BMC method keywords. Don't let one |
| 98 | ... failure skip the remaining. Get whatever data |
| 99 | ... it could gather at worse case scenario. |
| 100 | [Arguments] ${description} ${keyword_name} @{ffdc_function_list} |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 101 | |
Michael Walsh | e844e9a | 2017-04-20 16:51:10 -0500 | [diff] [blame] | 102 | # Description of argument(s): |
| 103 | # description The description of the FFDC to be collected. This |
| 104 | # would be any value returned by |
| 105 | # 'Get FFDC Method Desc' (e.g. "FFDC Generic Report"). |
| 106 | # keyword_name The name of the keyword to call to collect the FFDC |
| 107 | # data (again, see FFDC_METHOD_CALL). |
| 108 | # ffdc_function_list See ffdc_function_list description in |
| 109 | # "Call FFDC Methods" (above). The only difference is |
| 110 | # in this case, it should be a list rather than a |
| 111 | # colon-delimited value. |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 112 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 113 | @{ffdc_file_list}= Create List |
Michael Walsh | e844e9a | 2017-04-20 16:51:10 -0500 | [diff] [blame] | 114 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 115 | ${index}= Get Index From List ${ffdc_function_list} ${description} |
| 116 | Run Keyword If '${index}' == '${-1}' Return from Keyword |
| 117 | ... ${ffdc_file_list} |
| 118 | |
| 119 | ${status} ${ffdc_file_list}= Run Key ${keyword_name} ignore=1 |
| 120 | [Return] ${ffdc_file_list} |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 121 | |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 122 | # Method : BMC FFDC Manifest # |
| 123 | # Execute command on BMC and write to ffdc_report.txt # |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 124 | |
| 125 | BMC FFDC Manifest |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 126 | [Documentation] Run the ssh commands from FFDC_BMC_CMD and return a list |
| 127 | ... of generated files. |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 128 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 129 | @{ffdc_file_list}= Create List ${FFDC_FILE_PATH} |
| 130 | @{entries}= Get FFDC Cmd Index |
| 131 | :FOR ${index} IN @{entries} |
| 132 | \ Iterate BMC Command List Pairs ${index} |
| 133 | |
| 134 | [Return] ${ffdc_file_list} |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 135 | |
| 136 | |
| 137 | Iterate BMC Command List Pairs |
| 138 | [Documentation] Feed in key pair list from dictionary to execute |
| 139 | [Arguments] ${key_index} |
| 140 | |
| 141 | @{cmd_list}= Get ffdc bmc cmd ${key_index} |
| 142 | Set Suite Variable ${ENTRY_INDEX} ${key_index} |
| 143 | :FOR ${cmd} IN @{cmd_list} |
| 144 | \ Execute Command and Write FFDC ${cmd[0]} ${cmd[1]} |
| 145 | |
| 146 | |
| 147 | Execute Command and Write FFDC |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 148 | [Documentation] Run a command on the BMC or OS, write the output to the |
| 149 | ... specified file and return a list of generated files. |
| 150 | [Arguments] ${key_index} ${cmd} ${logpath}=${FFDC_FILE_PATH} |
| 151 | ... ${target}=BMC |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 152 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 153 | Run Keyword If '${logpath}' == '${FFDC_FILE_PATH}' |
| 154 | ... Write Cmd Output to FFDC File ${key_index} ${cmd} |
| 155 | |
| 156 | @{ffdc_file_list}= Create List ${log_path} |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 157 | |
Michael Walsh | e53e47a | 2017-06-30 17:03:24 -0500 | [diff] [blame] | 158 | ${cmd_buf}= Catenate ${target} Execute Command \ ${cmd} \ ignore_err=${1} |
| 159 | ${status} ${ret_values}= Run Key ${cmd_buf} ignore=${1} |
| 160 | |
| 161 | ${stdout}= Set Variable @{ret_values}[0] |
| 162 | ${stderr}= Set Variable @{ret_values}[1] |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 163 | |
Jeff Whitlock | 7d805b2 | 2017-01-20 14:04:33 -0600 | [diff] [blame] | 164 | # Write stdout on success and stderr/stdout to the file on failure. |
| 165 | Run Keyword If $stderr == '${EMPTY}' |
Sweta Potthuri | 15e6d2f | 2017-02-28 03:10:02 -0600 | [diff] [blame] | 166 | ... Write Data To File ${stdout}${\n} ${logpath} |
| 167 | ... ELSE Write Data To File |
Jeff Whitlock | 7d805b2 | 2017-01-20 14:04:33 -0600 | [diff] [blame] | 168 | ... ERROR output:${\n}${stderr}${\n}Output:${\n}${stdout}${\n} |
| 169 | ... ${logpath} |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 170 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 171 | [Return] ${ffdc_file_list} |
| 172 | |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 173 | |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 174 | # Method : BMC FFDC Files # |
| 175 | # Execute command on BMC and write to individual file # |
| 176 | # based on the file name pre-defined in the list # |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 177 | |
| 178 | BMC FFDC Files |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 179 | [Documentation] Run the commands from FFDC_BMC_FILE and return a list of |
| 180 | ... generated files. |
| 181 | |
| 182 | @{entries}= Get FFDC File Index |
| 183 | # Example of entries: |
| 184 | # entries: |
| 185 | # entries[0]: BMC FILES |
| 186 | |
| 187 | @{ffdc_file_list}= Create List |
| 188 | :FOR ${index} IN @{entries} |
| 189 | \ ${ffdc_file_sub_list}= Create File and Write Data ${index} |
| 190 | \ ${ffdc_file_list}= Smart Combine Lists ${ffdc_file_list} |
| 191 | ... ${ffdc_file_sub_list} |
| 192 | |
| 193 | [Return] ${ffdc_file_list} |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 194 | |
| 195 | |
| 196 | Create File and Write Data |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 197 | [Documentation] Run commands from FFDC_BMC_FILE to create FFDC files and |
| 198 | ... return a list of generated files. |
| 199 | [Arguments] ${key_index} |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 200 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 201 | # Description of argument(s): |
| 202 | # key_index The index into the FFDC_BMC_FILE dictionary. |
| 203 | |
| 204 | @{ffdc_file_list}= Create List |
| 205 | @{cmd_list}= Get FFDC BMC File ${key_index} |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 206 | :FOR ${cmd} IN @{cmd_list} |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 207 | \ ${logpath}= Catenate SEPARATOR= ${LOG_PREFIX} ${cmd[0]}.txt |
| 208 | \ ${ffdc_file_sub_list}= Execute Command and Write FFDC ${cmd[0]} |
| 209 | ... ${cmd[1]} ${logpath} |
| 210 | \ ${ffdc_file_list}= Smart Combine Lists ${ffdc_file_list} |
| 211 | ... ${ffdc_file_sub_list} |
Sivas SRR | 0d657e8 | 2017-03-13 01:00:34 -0500 | [diff] [blame] | 212 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 213 | [Return] ${ffdc_file_list} |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 214 | |
| 215 | |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 216 | # Method : Log Test Case Status # |
| 217 | # Creates test result history footprint for reference # |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 218 | |
| 219 | Log Test Case Status |
Jeff Whitlock | 7d805b2 | 2017-01-20 14:04:33 -0600 | [diff] [blame] | 220 | [Documentation] Test case execution result history. |
| 221 | ... Create once and append to this file |
| 222 | ... logs/test_history.txt |
| 223 | ... Format Date:Test suite:Test case:Status |
| 224 | ... 20160909214053719992:Test Warmreset:Test WarmReset via REST:FAIL |
Michael Walsh | fa9f70f | 2017-04-21 16:00:18 -0500 | [diff] [blame] | 225 | |
| 226 | ${FFDC_DIR_PATH_STYLE}= Get Variable Value ${FFDC_DIR_PATH_STYLE} |
| 227 | ... ${EMPTY} |
| 228 | ${FFDC_DIR_PATH}= Get Variable Value ${FFDC_DIR_PATH} ${EMPTY} |
| 229 | |
| 230 | Run Keyword If '${FFDC_DIR_PATH}' == '${EMPTY}' Set FFDC Defaults |
| 231 | |
| 232 | Run Keyword If '${FFDC_DIR_PATH_STYLE}' == '${1}' Run Keywords |
| 233 | ... Set Global Variable ${FFDC_LOG_PATH} ${FFDC_DIR_PATH} AND |
| 234 | ... Set Global Variable ${TEST_HISTORY} ${FFDC_DIR_PATH}test_history.txt |
| 235 | |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 236 | Create Directory ${FFDC_LOG_PATH} |
| 237 | |
| 238 | ${exist}= Run Keyword and Return Status |
| 239 | ... OperatingSystem.File Should Exist ${TEST_HISTORY} |
| 240 | |
| 241 | Run Keyword If '${exist}' == '${False}' |
| 242 | ... Create File ${TEST_HISTORY} |
| 243 | |
Michael Walsh | fa9f70f | 2017-04-21 16:00:18 -0500 | [diff] [blame] | 244 | Rpvars TEST_HISTORY |
| 245 | |
George Keishing | 69e6f71 | 2016-09-12 06:30:09 -0500 | [diff] [blame] | 246 | ${cur_time}= Get Current Time Stamp |
| 247 | |
| 248 | Append To File ${TEST_HISTORY} |
| 249 | ... ${cur_time}:${SUITE_NAME}:${TEST_NAME}:${TEST_STATUS}${\n} |
| 250 | |
Gunnar Mills | a812e0f | 2016-09-29 20:30:03 -0500 | [diff] [blame] | 251 | |
| 252 | Log FFDC Get Requests |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 253 | [Documentation] Run the get requests associated with the key and return a |
| 254 | ... list of generated files. |
| 255 | [Arguments] ${key_index} |
Gunnar Mills | a812e0f | 2016-09-29 20:30:03 -0500 | [diff] [blame] | 256 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 257 | # Note: Output will be in JSON pretty_print format. |
| 258 | |
| 259 | # Description of argument(s): |
| 260 | # key_index The key to the FFDC_GET_REQUEST dictionary that contains the |
| 261 | # get requests that are to be run. |
| 262 | |
| 263 | @{ffdc_file_list}= Create List |
| 264 | @{cmd_list}= Get FFDC Get Request ${key_index} |
Gunnar Mills | a812e0f | 2016-09-29 20:30:03 -0500 | [diff] [blame] | 265 | :FOR ${cmd} IN @{cmd_list} |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 266 | \ ${logpath}= Catenate SEPARATOR= ${LOG_PREFIX} ${cmd[0]}.txt |
| 267 | \ ${resp}= OpenBMC Get Request ${cmd[1]} quiet=${1} |
| 268 | \ ${status}= Run Keyword and Return Status |
| 269 | ... Should Be Equal As Strings ${resp.status_code} ${HTTP_OK} |
| 270 | \ Run Keyword If '${status}' == '${False}' Continue For Loop |
| 271 | \ ${jsondata}= to json ${resp.content} pretty_print=True |
| 272 | \ Write Data To File ${\n}${jsondata}${\n} ${logpath} |
| 273 | \ Append To List ${ffdc_file_list} ${logpath} |
Gunnar Mills | a812e0f | 2016-09-29 20:30:03 -0500 | [diff] [blame] | 274 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 275 | [Return] ${ffdc_file_list} |
Gunnar Mills | a812e0f | 2016-09-29 20:30:03 -0500 | [diff] [blame] | 276 | |
| 277 | BMC FFDC Get Requests |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 278 | [Documentation] Iterate over get request list and return a list of |
| 279 | ... generated files. |
| 280 | |
| 281 | @{ffdc_file_list}= Create List |
| 282 | |
Gunnar Mills | a812e0f | 2016-09-29 20:30:03 -0500 | [diff] [blame] | 283 | @{entries}= Get ffdc get request index |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 284 | # Example of entries: |
| 285 | # entries: |
| 286 | # entries[0]: GET REQUESTS |
Gunnar Mills | a812e0f | 2016-09-29 20:30:03 -0500 | [diff] [blame] | 287 | :FOR ${index} IN @{entries} |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 288 | \ ${ffdc_file_sub_list}= Log FFDC Get Requests ${index} |
| 289 | \ ${ffdc_file_list}= Smart Combine Lists ${ffdc_file_list} |
| 290 | ... ${ffdc_file_sub_list} |
Gunnar Mills | 7e2cda2 | 2016-10-11 15:37:34 -0500 | [diff] [blame] | 291 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 292 | [Return] ${ffdc_file_list} |
Gunnar Mills | 7e2cda2 | 2016-10-11 15:37:34 -0500 | [diff] [blame] | 293 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 294 | Log OS All distros FFDC |
| 295 | [Documentation] Run commands from FFDC_OS_ALL_DISTROS_FILE to create FFDC |
| 296 | ... files and return a list of generated files. |
| 297 | [Arguments] ${key_index} |
Gunnar Mills | 7e2cda2 | 2016-10-11 15:37:34 -0500 | [diff] [blame] | 298 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 299 | # Description of argument(s): |
| 300 | # key_index The index into the FFDC_OS_ALL_DISTROS_FILE dictionary. |
| 301 | |
| 302 | @{ffdc_file_list}= Create List |
| 303 | |
| 304 | @{cmd_list}= Get FFDC OS All Distros Call ${key_index} |
Gunnar Mills | 7e2cda2 | 2016-10-11 15:37:34 -0500 | [diff] [blame] | 305 | :FOR ${cmd} IN @{cmd_list} |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 306 | \ ${logpath}= Catenate SEPARATOR= ${LOG_PREFIX} ${cmd[0]}.txt |
| 307 | \ ${ffdc_file_sub_list}= Execute Command and Write FFDC ${cmd[0]} |
| 308 | ... ${cmd[1]} ${logpath} target=OS |
| 309 | \ ${ffdc_file_list}= Smart Combine Lists ${ffdc_file_list} |
| 310 | ... ${ffdc_file_sub_list} |
| 311 | |
| 312 | [Return] ${ffdc_file_list} |
Gunnar Mills | 7e2cda2 | 2016-10-11 15:37:34 -0500 | [diff] [blame] | 313 | |
| 314 | |
Gunnar Mills | cce185d | 2016-10-17 17:04:15 -0500 | [diff] [blame] | 315 | Log OS SPECIFIC DISTRO FFDC |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 316 | [Documentation] Run commands from the FFDC_OS_<distro>_FILE to create FFDC |
| 317 | ... files and return a list of generated files. |
| 318 | [Arguments] ${key_index} ${linux_distro} |
Gunnar Mills | cce185d | 2016-10-17 17:04:15 -0500 | [diff] [blame] | 319 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 320 | # Description of argument(s): |
| 321 | # key_index The index into the FFDC_OS_<distro>_FILE dictionary. |
| 322 | # linux_distro Your OS's linux distro (e.g. "UBUNTU", "RHEL", etc). |
| 323 | |
| 324 | @{ffdc_file_list}= Create List |
| 325 | |
| 326 | @{cmd_list}= Get FFDC OS Distro Call ${key_index} ${linux_distro} |
Gunnar Mills | cce185d | 2016-10-17 17:04:15 -0500 | [diff] [blame] | 327 | :FOR ${cmd} IN @{cmd_list} |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 328 | \ ${logpath}= Catenate SEPARATOR= ${LOG_PREFIX} ${cmd[0]}.txt |
| 329 | \ ${ffdc_file_sub_list}= Execute Command and Write FFDC ${cmd[0]} |
| 330 | ... ${cmd[1]} ${logpath} target=OS |
| 331 | \ ${ffdc_file_list}= Smart Combine Lists ${ffdc_file_list} |
| 332 | ... ${ffdc_file_sub_list} |
| 333 | |
| 334 | [Return] ${ffdc_file_list} |
Gunnar Mills | cce185d | 2016-10-17 17:04:15 -0500 | [diff] [blame] | 335 | |
| 336 | |
George Keishing | ca8c4c6 | 2016-10-14 10:08:40 -0500 | [diff] [blame] | 337 | OS FFDC Files |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 338 | [Documentation] Run the commands from FFDC_OS_ALL_DISTROS_FILE to create |
| 339 | ... FFDC files and return a list of generated files. |
Gunnar Mills | aca140d | 2016-10-26 13:05:10 -0500 | [diff] [blame] | 340 | [Arguments] ${OS_HOST}=${OS_HOST} ${OS_USERNAME}=${OS_USERNAME} |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 341 | ... ${OS_PASSWORD}=${OS_PASSWORD} |
Gunnar Mills | 7e2cda2 | 2016-10-11 15:37:34 -0500 | [diff] [blame] | 342 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 343 | @{ffdc_file_list}= Create List |
Gunnar Mills | aca140d | 2016-10-26 13:05:10 -0500 | [diff] [blame] | 344 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 345 | Run Keyword If '${OS_HOST}' == '${EMPTY}' Run Keywords |
| 346 | ... Print Timen No OS Host provided so no OS FFDC will be done. AND |
| 347 | ... Return From Keyword ${ffdc_file_list} |
Gunnar Mills | aca140d | 2016-10-26 13:05:10 -0500 | [diff] [blame] | 348 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 349 | ${match_state}= Create Dictionary os_ping=^1$ os_login=^1$ |
| 350 | ... os_run_cmd=^1$ |
| 351 | ${status} ${ret_values}= Run Keyword and Ignore Error Check State |
| 352 | ... ${match_state} quiet=0 |
| 353 | |
| 354 | Run Keyword If '${status}' == 'FAIL' Run Keywords |
Michael Walsh | c3b28d7 | 2017-11-16 10:43:14 -0600 | [diff] [blame] | 355 | ... Print Timen The OS is not communicating so no OS FFDC will be done.\n |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 356 | ... AND |
| 357 | ... Return From Keyword ${ffdc_file_list} |
Gunnar Mills | cce185d | 2016-10-17 17:04:15 -0500 | [diff] [blame] | 358 | |
Michael Walsh | e53e47a | 2017-06-30 17:03:24 -0500 | [diff] [blame] | 359 | ${stdout} ${stderr} ${rc}= OS Execute Command |
| 360 | ... . /etc/os-release; echo $ID ignore_err=${0} |
| 361 | Set Global Variable ${linux_distro} ${stdout} |
Gunnar Mills | aca140d | 2016-10-26 13:05:10 -0500 | [diff] [blame] | 362 | |
Michael Walsh | e53e47a | 2017-06-30 17:03:24 -0500 | [diff] [blame] | 363 | Rpvars linux_distro |
Gunnar Mills | aca140d | 2016-10-26 13:05:10 -0500 | [diff] [blame] | 364 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 365 | @{entries}= Get FFDC OS All Distros Index |
Gunnar Mills | 7e2cda2 | 2016-10-11 15:37:34 -0500 | [diff] [blame] | 366 | :FOR ${index} IN @{entries} |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 367 | \ ${ffdc_file_sub_list}= Log OS All distros FFDC ${index} |
| 368 | \ ${ffdc_file_list}= Smart Combine Lists ${ffdc_file_list} |
| 369 | ... ${ffdc_file_sub_list} |
Gunnar Mills | cce185d | 2016-10-17 17:04:15 -0500 | [diff] [blame] | 370 | |
Jeff Whitlock | 7d805b2 | 2017-01-20 14:04:33 -0600 | [diff] [blame] | 371 | Return From Keyword If |
| 372 | ... '${linux_distro}' == '${EMPTY}' or '${linux_distro}' == 'None' |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 373 | ... ${ffdc_file_list} |
Gunnar Mills | cce185d | 2016-10-17 17:04:15 -0500 | [diff] [blame] | 374 | |
| 375 | @{entries}= Get ffdc os distro index ${linux_distro} |
| 376 | :FOR ${index} IN @{entries} |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 377 | \ ${ffdc_file_sub_list}= Log OS SPECIFIC DISTRO FFDC ${index} |
| 378 | ... ${linux_distro} |
| 379 | \ ${ffdc_file_list}= Smart Combine Lists ${ffdc_file_list} |
| 380 | ... ${ffdc_file_sub_list} |
| 381 | |
| 382 | [Return] ${ffdc_file_list} |
George Keishing | 7a52022 | 2017-02-27 09:44:30 -0600 | [diff] [blame] | 383 | |
| 384 | |
Steven Sombar | 96de573 | 2017-08-17 09:04:52 -0500 | [diff] [blame] | 385 | System Inventory Files |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 386 | [Documentation] Copy systest os_inventory files and return a list of |
| 387 | ... generated files.. |
Steven Sombar | 96de573 | 2017-08-17 09:04:52 -0500 | [diff] [blame] | 388 | # The os_inventory files are the result of running |
| 389 | # systest/htx_hardbootme_test. If these files exist |
| 390 | # they are copied to the FFDC directory. |
| 391 | # Global variable ffdc_dir_path is the path name of the |
| 392 | # directory they are copied to. |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 393 | |
| 394 | @{ffdc_file_list}= Create List |
| 395 | |
| 396 | ${globex}= Set Variable os_inventory_*.json |
| 397 | |
| 398 | @{file_list}= OperatingSystem.List Files In Directory . ${globex} |
| 399 | |
| 400 | Copy Files ${globex} ${ffdc_dir_path} |
| 401 | |
| 402 | : FOR ${file_name} IN @{file_list} |
| 403 | \ Append To List ${ffdc_file_list} ${ffdc_dir_path}${file_name} |
| 404 | |
| 405 | Run Keyword and Ignore Error Remove Files ${globex} |
| 406 | |
| 407 | [Return] ${ffdc_file_list} |
Steven Sombar | 96de573 | 2017-08-17 09:04:52 -0500 | [diff] [blame] | 408 | |
| 409 | |
George Keishing | 7a52022 | 2017-02-27 09:44:30 -0600 | [diff] [blame] | 410 | SCP Coredump Files |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 411 | [Documentation] Copy core dump files from BMC to local system and return a |
| 412 | ... list of generated file names. |
| 413 | |
| 414 | @{ffdc_file_list}= Create List |
Michael Walsh | e53e47a | 2017-06-30 17:03:24 -0500 | [diff] [blame] | 415 | |
George Keishing | 7a52022 | 2017-02-27 09:44:30 -0600 | [diff] [blame] | 416 | # Check if core dump exist in the /tmp |
Michael Walsh | e53e47a | 2017-06-30 17:03:24 -0500 | [diff] [blame] | 417 | ${core_files} ${stderr} ${rc}= BMC Execute Command ls /tmp/core_* |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 418 | ... ignore_err=${1} |
| 419 | Run Keyword If '${rc}' != '${0}' Return From Keyword ${ffdc_file_list} |
| 420 | |
| 421 | @{core_list}= Split String ${core_files} |
George Keishing | 7a52022 | 2017-02-27 09:44:30 -0600 | [diff] [blame] | 422 | # Copy the core files |
Michael Walsh | 6bad841 | 2017-06-27 14:09:36 -0500 | [diff] [blame] | 423 | Run Key U Open Connection for SCP |
George Keishing | 7a52022 | 2017-02-27 09:44:30 -0600 | [diff] [blame] | 424 | :FOR ${index} IN @{core_list} |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 425 | \ ${ffdc_file_path}= Catenate ${LOG_PREFIX}${index.lstrip("/tmp/")} |
| 426 | \ ${status}= Run Keyword and Return Status |
| 427 | ... scp.Get File ${index} ${ffdc_file_path} |
| 428 | \ Run Keyword If '${status}' == '${False}' Continue For Loop |
| 429 | \ Append To List ${ffdc_file_list} ${ffdc_file_path} |
George Keishing | 7a52022 | 2017-02-27 09:44:30 -0600 | [diff] [blame] | 430 | # Remove the file from remote to avoid re-copying on next FFDC call |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 431 | \ BMC Execute Command rm ${index} ignore_err=${1} |
Michael Walsh | e53e47a | 2017-06-30 17:03:24 -0500 | [diff] [blame] | 432 | # I can't find a way to do this: scp.Close Connection |
George Keishing | 7a52022 | 2017-02-27 09:44:30 -0600 | [diff] [blame] | 433 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 434 | [Return] ${ffdc_file_list} |
| 435 | |
Sweta Potthuri | a82efd6 | 2017-10-18 05:34:59 -0500 | [diff] [blame] | 436 | SCP Dump Files |
| 437 | [Documentation] Copy all dump files from BMC to local system. |
| 438 | |
| 439 | # Check if dumps exist |
| 440 | ${ffdc_file_list}= Scp Dumps ${FFDC_DIR_PATH} ${FFDC_PREFIX} |
| 441 | |
| 442 | [Return] ${ffdc_file_list} |
| 443 | |
| 444 | Collect Dump Log |
| 445 | [Documentation] Collect dumps from dump entry. |
| 446 | [Arguments] ${log_prefix_path}=${LOG_PREFIX} |
| 447 | |
| 448 | ${data}= Read Properties ${DUMP_ENTRY_URI}/enumerate quiet=${1} |
| 449 | |
| 450 | # Grab the list of entries from dump/entry/ |
| 451 | # The data shown below is the result of the "Get Dictionary Keys". |
| 452 | # Example: |
| 453 | # /xyz/openbmc_project/dump/entry/1 |
| 454 | # /xyz/openbmc_project/dump/entry/2 |
| 455 | |
| 456 | ${dump_list}= Get Dictionary Keys ${data} |
George Keishing | eb7b4bc | 2017-03-13 12:04:28 -0500 | [diff] [blame] | 457 | |
Michael Walsh | 02c7060 | 2018-10-04 15:24:30 -0500 | [diff] [blame^] | 458 | |
George Keishing | eb7b4bc | 2017-03-13 12:04:28 -0500 | [diff] [blame] | 459 | Collect eSEL Log |
Michael Walsh | 02c7060 | 2018-10-04 15:24:30 -0500 | [diff] [blame^] | 460 | [Documentation] Create raw and formatted eSEL files. |
Sridevi Ramesh | b180c9f | 2017-08-06 10:27:41 -0500 | [diff] [blame] | 461 | [Arguments] ${log_prefix_path}=${LOG_PREFIX} |
| 462 | |
Michael Walsh | 02c7060 | 2018-10-04 15:24:30 -0500 | [diff] [blame^] | 463 | # NOTE: If no eSEL.pl program can be located, then no formatted eSEL file |
| 464 | # will be generated. |
| 465 | |
| 466 | # Description of argument(s): |
| 467 | # log_prefix_path The path prefix to be used in creating |
| 468 | # eSEL file path names. For example, if |
| 469 | # log_prefix_path is |
| 470 | # "/tmp/user1/dummy.181001.120000.", then |
| 471 | # this keyword will create |
| 472 | # /tmp/user1/dummy.181001.120000.esel (raw) |
| 473 | # and |
| 474 | # /tmp/user1/dummy.181001.120000.esel.txt |
| 475 | # (formatted). |
| 476 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 477 | @{ffdc_file_list}= Create List |
| 478 | |
Michael Walsh | 02c7060 | 2018-10-04 15:24:30 -0500 | [diff] [blame^] | 479 | ${esels}= Get Esels |
| 480 | ${num_esels}= Evaluate len(${esels}) |
| 481 | Rprint Vars num_esels |
| 482 | Return From Keyword If ${num_esels} == ${0} ${ffdc_file_list} |
George Keishing | eb7b4bc | 2017-03-13 12:04:28 -0500 | [diff] [blame] | 483 | |
Sridevi Ramesh | b180c9f | 2017-08-06 10:27:41 -0500 | [diff] [blame] | 484 | ${logpath}= Catenate SEPARATOR= ${log_prefix_path} esel |
George Keishing | eb7b4bc | 2017-03-13 12:04:28 -0500 | [diff] [blame] | 485 | Create File ${logpath} |
George Keishing | eb7b4bc | 2017-03-13 12:04:28 -0500 | [diff] [blame] | 486 | |
Michael Walsh | 02c7060 | 2018-10-04 15:24:30 -0500 | [diff] [blame^] | 487 | :FOR ${esel} IN @{esels} |
| 488 | \ Write Data To File "${esel}"${\n} ${logpath} |
George Keishing | b1ca69b | 2017-03-31 14:36:42 -0500 | [diff] [blame] | 489 | |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 490 | Append To List ${ffdc_file_list} ${logpath} |
Michael Walsh | 02c7060 | 2018-10-04 15:24:30 -0500 | [diff] [blame^] | 491 | |
| 492 | ${rc} ${output}= Shell Cmd which eSEL.pl show_err=0 |
| 493 | Return From Keyword If ${rc} != ${0} ${ffdc_file_list} |
| 494 | |
| 495 | Convert eSEL To Elog Format ${logpath} |
Michael Walsh | 0192e9b | 2017-11-14 15:14:15 -0600 | [diff] [blame] | 496 | Append To List ${ffdc_file_list} ${logpath}.txt |
Michael Walsh | 63b70c0 | 2017-10-30 15:02:09 -0500 | [diff] [blame] | 497 | |
| 498 | [Return] ${ffdc_file_list} |
| 499 | |
George Keishing | b1ca69b | 2017-03-31 14:36:42 -0500 | [diff] [blame] | 500 | |
George Keishing | b1ca69b | 2017-03-31 14:36:42 -0500 | [diff] [blame] | 501 | Convert eSEL To Elog Format |
| 502 | [Documentation] Execute parser tool on the eSEL data file to generate |
Michael Walsh | a30dac7 | 2017-04-04 17:56:23 -0500 | [diff] [blame] | 503 | ... formatted error log. |
George Keishing | b1ca69b | 2017-03-31 14:36:42 -0500 | [diff] [blame] | 504 | [Arguments] ${esel_file_path} |
Michael Walsh | 02c7060 | 2018-10-04 15:24:30 -0500 | [diff] [blame^] | 505 | # Description of argument(s): |
| 506 | # esel_file_path The path to the file containg raw eSEL |
| 507 | # data (e.g. |
| 508 | # "/tmp/user1/dummy.181001.120000.esel"). |
George Keishing | b1ca69b | 2017-03-31 14:36:42 -0500 | [diff] [blame] | 509 | |
Michael Walsh | a30dac7 | 2017-04-04 17:56:23 -0500 | [diff] [blame] | 510 | # Note: The only way to get eSEL.pl to put the output in a particular |
| 511 | # directory is to cd to that directory. |
| 512 | ${cmd_buf}= Catenate cd $(dirname ${esel_file_path}) ; eSEL.pl -l |
| 513 | ... ${esel_file_path} -p decode_obmc_data |
Michael Walsh | 02c7060 | 2018-10-04 15:24:30 -0500 | [diff] [blame^] | 514 | Qprint Issuing ${cmd_buf} |
Michael Walsh | a30dac7 | 2017-04-04 17:56:23 -0500 | [diff] [blame] | 515 | Run ${cmd_buf} |
Michael Walsh | 02c7060 | 2018-10-04 15:24:30 -0500 | [diff] [blame^] | 516 | # The .binary file, which is generated by eSEL.pl, is of no use to us. |
| 517 | Remove File ${esel_file_path}.binary |