Chris Austen | b29d2e8 | 2016-06-07 12:25:35 -0500 | [diff] [blame] | 1 | *** Settings *** |
George Keishing | 4d6c1da | 2016-07-15 05:51:22 -0500 | [diff] [blame] | 2 | Documentation This module is for IPMI client for copying ipmitool to |
Sivas SRR | a2dab3c | 2016-07-25 05:08:18 -0500 | [diff] [blame] | 3 | ... openbmc box and execute ipmitool IPMI standard |
| 4 | ... command. IPMI raw command will use dbus-send command |
Chris Austen | b29d2e8 | 2016-06-07 12:25:35 -0500 | [diff] [blame] | 5 | Resource ../lib/resource.txt |
George Keishing | 4d6c1da | 2016-07-15 05:51:22 -0500 | [diff] [blame] | 6 | Resource ../lib/connection_client.robot |
Prashanth Katti | ae7c228 | 2017-03-15 07:43:46 -0500 | [diff] [blame] | 7 | Resource ../lib/utils.robot |
| 8 | Resource ../lib/state_manager.robot |
| 9 | |
Sivas SRR | a2dab3c | 2016-07-25 05:08:18 -0500 | [diff] [blame] | 10 | Library String |
| 11 | |
| 12 | *** Variables *** |
George Keishing | d7f11f7 | 2017-01-04 10:58:19 -0600 | [diff] [blame] | 13 | ${dbusHostIpmicmd1}= dbus-send --system ${OPENBMC_BASE_URI}HostIpmi/1 |
| 14 | ${dbusHostIpmiCmdReceivedMsg}= ${OPENBMC_BASE_DBUS}.HostIpmi.ReceivedMessage |
Gunnar Mills | 59204d2 | 2016-12-06 11:05:19 -0600 | [diff] [blame] | 15 | ${netfnByte}= ${EMPTY} |
George Keishing | d7f11f7 | 2017-01-04 10:58:19 -0600 | [diff] [blame] | 16 | ${cmdByte}= ${EMPTY} |
Gunnar Mills | 59204d2 | 2016-12-06 11:05:19 -0600 | [diff] [blame] | 17 | ${arrayByte}= array:byte: |
Michael Walsh | a791389 | 2017-04-20 16:29:01 -0500 | [diff] [blame] | 18 | ${IPMI_EXT_CMD}= ipmitool -I lanplus -C 3 |
| 19 | ${IPMI_USER_OPTIONS} ${EMPTY} |
Prashanth Katti | ae7c228 | 2017-03-15 07:43:46 -0500 | [diff] [blame] | 20 | ${IPMI_INBAND_CMD}= ipmitool -C 3 |
Gunnar Mills | 59204d2 | 2016-12-06 11:05:19 -0600 | [diff] [blame] | 21 | ${HOST}= -H |
| 22 | ${RAW}= raw |
Chris Austen | b29d2e8 | 2016-06-07 12:25:35 -0500 | [diff] [blame] | 23 | |
| 24 | *** Keywords *** |
Rahul Maheshwari | 7a212a0 | 2016-09-05 21:17:30 -0500 | [diff] [blame] | 25 | |
Chris Austen | b29d2e8 | 2016-06-07 12:25:35 -0500 | [diff] [blame] | 26 | Run IPMI Command |
Prashanth Katti | ae7c228 | 2017-03-15 07:43:46 -0500 | [diff] [blame] | 27 | [Arguments] ${args} |
| 28 | ${resp}= Run Keyword If '${IPMI_COMMAND}' == 'External' |
| 29 | ... Run External IPMI RAW Command ${args} |
| 30 | ... ELSE IF '${IPMI_COMMAND}' == 'Inband' |
| 31 | ... Run Inband IPMI RAW Command ${args} |
| 32 | ... ELSE IF '${IPMI_COMMAND}' == 'Dbus' |
| 33 | ... Run Dbus IPMI RAW Command ${args} |
| 34 | ... ELSE Fail msg=Invalid IPMI Command type provided : ${IPMI_COMMAND} |
| 35 | [Return] ${resp} |
Rahul Maheshwari | 7a212a0 | 2016-09-05 21:17:30 -0500 | [diff] [blame] | 36 | |
| 37 | Run IPMI Standard Command |
Prashanth Katti | ae7c228 | 2017-03-15 07:43:46 -0500 | [diff] [blame] | 38 | [Arguments] ${args} |
| 39 | ${resp}= Run Keyword If '${IPMI_COMMAND}' == 'External' |
| 40 | ... Run External IPMI Standard Command ${args} |
| 41 | ... ELSE IF '${IPMI_COMMAND}' == 'Inband' |
| 42 | ... Run Inband IPMI Standard Command ${args} |
| 43 | ... ELSE IF '${IPMI_COMMAND}' == 'Dbus' |
| 44 | ... Run Dbus IPMI Standard Command ${args} |
| 45 | ... ELSE Fail msg=Invalid IPMI Command type provided : ${IPMI_COMMAND} |
Rahul Maheshwari | 7a212a0 | 2016-09-05 21:17:30 -0500 | [diff] [blame] | 46 | |
Prashanth Katti | ae7c228 | 2017-03-15 07:43:46 -0500 | [diff] [blame] | 47 | [Return] ${resp} |
Rahul Maheshwari | 7a212a0 | 2016-09-05 21:17:30 -0500 | [diff] [blame] | 48 | |
| 49 | Run Dbus IPMI RAW Command |
Gunnar Mills | 3803280 | 2016-12-12 13:43:40 -0600 | [diff] [blame] | 50 | [Arguments] ${args} |
Gunnar Mills | 59204d2 | 2016-12-06 11:05:19 -0600 | [diff] [blame] | 51 | ${valueinBytes}= Byte Conversion ${args} |
| 52 | ${cmd}= Catenate ${dbushostipmicmd1} ${dbusHostIpmiCmdReceivedMsg} |
| 53 | ${cmd}= Catenate ${cmd} ${valueinBytes} |
Sivas SRR | a2dab3c | 2016-07-25 05:08:18 -0500 | [diff] [blame] | 54 | ${output} ${stderr}= Execute Command ${cmd} return_stderr=True |
| 55 | Should Be Empty ${stderr} |
Chris Austen | b29d2e8 | 2016-06-07 12:25:35 -0500 | [diff] [blame] | 56 | set test variable ${OUTPUT} "${output}" |
| 57 | |
Rahul Maheshwari | 7a212a0 | 2016-09-05 21:17:30 -0500 | [diff] [blame] | 58 | Run Dbus IPMI Standard Command |
Gunnar Mills | 3803280 | 2016-12-12 13:43:40 -0600 | [diff] [blame] | 59 | [Arguments] ${args} |
George Keishing | 4d6c1da | 2016-07-15 05:51:22 -0500 | [diff] [blame] | 60 | Copy ipmitool |
Rahul Maheshwari | 7a212a0 | 2016-09-05 21:17:30 -0500 | [diff] [blame] | 61 | ${stdout} ${stderr} ${output}= Execute Command |
| 62 | ... /tmp/ipmitool -I dbus ${args} return_stdout=True |
| 63 | ... return_stderr= True return_rc=True |
Chris Austen | b29d2e8 | 2016-06-07 12:25:35 -0500 | [diff] [blame] | 64 | Should Be Equal ${output} ${0} msg=${stderr} |
Gunnar Mills | c9ea936 | 2016-12-13 16:21:13 -0600 | [diff] [blame] | 65 | [Return] ${stdout} |
Chris Austen | b29d2e8 | 2016-06-07 12:25:35 -0500 | [diff] [blame] | 66 | |
Prashanth Katti | ae7c228 | 2017-03-15 07:43:46 -0500 | [diff] [blame] | 67 | Run Inband IPMI Raw Command |
| 68 | [Documentation] Run Inband IPMI raw command. |
| 69 | [Arguments] ${args} ${os_host}=${OS_HOST} ${os_username}=${OS_USERNAME} |
| 70 | ... ${os_password}=${OS_PASSWORD} |
| 71 | |
| 72 | # Desription of arguments: |
| 73 | # ${args} parameters to IPMI command. |
| 74 | # ${os_host} IP address of the OS Host. |
| 75 | # ${os_username} OS Host Login user name. |
| 76 | # ${os_password} OS Host Login passwrd. |
| 77 | |
| 78 | Login To OS Host ${os_host} ${os_username} ${os_password} |
| 79 | Check If IPMI Tool Exist |
| 80 | |
| 81 | ${inband_raw_cmd}= Catenate ${IPMI_INBAND_CMD} ${RAW} ${args} |
| 82 | ${stdout} ${stderr}= Execute Command ${inband_raw_cmd} return_stderr=True |
| 83 | Should Be Empty ${stderr} msg=${stdout} |
| 84 | [Return] ${stdout} |
| 85 | |
| 86 | Run Inband IPMI Standard Command |
| 87 | [Documentation] Run Inband IPMI standard command. |
| 88 | [Arguments] ${args} ${os_host}=${OS_HOST} ${os_username}=${OS_USERNAME} |
| 89 | ... ${os_password}=${OS_PASSWORD} |
| 90 | |
| 91 | # Desription of arguments: |
| 92 | # ${args} parameters to IPMI command. |
| 93 | # ${os_host} IP address of the OS Host. |
| 94 | # ${os_username} OS Host Login user name. |
| 95 | # ${os_password} OS Host Login passwrd. |
| 96 | |
| 97 | Login To OS Host ${os_host} ${os_username} ${os_password} |
| 98 | Check If IPMI Tool Exist |
| 99 | |
| 100 | ${inband_std_cmd}= Catenate ${IPMI_INBAND_CMD} ${args} |
| 101 | ${stdout} ${stderr}= Execute Command ${inband_std_cmd} return_stderr=True |
| 102 | Should Be Empty ${stderr} msg=${stdout} |
| 103 | [Return] ${stdout} |
| 104 | |
Rahul Maheshwari | 7a212a0 | 2016-09-05 21:17:30 -0500 | [diff] [blame] | 105 | Run External IPMI RAW Command |
Gunnar Mills | 3803280 | 2016-12-12 13:43:40 -0600 | [diff] [blame] | 106 | [Arguments] ${args} |
Rahul Maheshwari | 7a212a0 | 2016-09-05 21:17:30 -0500 | [diff] [blame] | 107 | ${ipmi_raw_cmd}= Catenate SEPARATOR= |
Michael Walsh | a791389 | 2017-04-20 16:29:01 -0500 | [diff] [blame] | 108 | ... ${IPMI_EXT_CMD} -P${SPACE}${IPMI_PASSWORD}${SPACE} |
Rahul Maheshwari | 7a212a0 | 2016-09-05 21:17:30 -0500 | [diff] [blame] | 109 | ... ${HOST}${SPACE}${OPENBMC_HOST}${SPACE}${RAW}${SPACE}${args} |
| 110 | ${rc} ${output}= Run and Return RC and Output ${ipmi_raw_cmd} |
| 111 | Should Be Equal ${rc} ${0} msg=${output} |
Gunnar Mills | c9ea936 | 2016-12-13 16:21:13 -0600 | [diff] [blame] | 112 | [Return] ${output} |
Rahul Maheshwari | 7a212a0 | 2016-09-05 21:17:30 -0500 | [diff] [blame] | 113 | |
| 114 | Run External IPMI Standard Command |
Gunnar Mills | 3803280 | 2016-12-12 13:43:40 -0600 | [diff] [blame] | 115 | [Arguments] ${args} |
Michael Walsh | a791389 | 2017-04-20 16:29:01 -0500 | [diff] [blame] | 116 | |
| 117 | ${ipmi_cmd}= Catenate SEPARATOR= |
| 118 | ... ${IPMI_EXT_CMD} ${IPMI_USER_OPTIONS} -P${SPACE}${IPMI_PASSWORD} |
| 119 | ... ${SPACE}${HOST}${SPACE}${OPENBMC_HOST}${SPACE}${args} |
Rahul Maheshwari | 04a2204 | 2017-05-02 01:54:31 -0500 | [diff] [blame] | 120 | ${rc} ${output}= Run And Return RC and Output ${ipmi_cmd} |
Michael Walsh | a791389 | 2017-04-20 16:29:01 -0500 | [diff] [blame] | 121 | Should Be Equal ${rc} ${0} msg=${output} |
| 122 | [Return] ${output} |
Rahul Maheshwari | 7a212a0 | 2016-09-05 21:17:30 -0500 | [diff] [blame] | 123 | |
Prashanth Katti | ae7c228 | 2017-03-15 07:43:46 -0500 | [diff] [blame] | 124 | Check If IPMI Tool Exist |
| 125 | [Documentation] Check if IPMI Tool installed or not. |
| 126 | ${output}= Execute Command which ipmitool |
| 127 | Should Not Be Empty ${output} msg=ipmitool not installed. |
Rahul Maheshwari | 7a212a0 | 2016-09-05 21:17:30 -0500 | [diff] [blame] | 128 | |
Rahul Maheshwari | 04a2204 | 2017-05-02 01:54:31 -0500 | [diff] [blame] | 129 | |
| 130 | Activate SOL Via IPMI |
| 131 | [Documentation] Start SOL using IPMI and route output to a file. |
| 132 | [Arguments] ${file_path}=/tmp/sol_${OPENBMC_HOST} |
| 133 | # Description of argument(s): |
| 134 | # file_path The file path on the local machine (vs OBMC) to collect SOL |
| 135 | # output. By default SOL output is collected at |
| 136 | # /tmp/sol_<BMC_IP> else user input location. |
| 137 | |
| 138 | ${ipmi_cmd}= Catenate SEPARATOR= |
| 139 | ... ${IPMI_EXT_CMD} -P${SPACE}${IPMI_PASSWORD}${SPACE} |
| 140 | ... ${HOST}${SPACE}${OPENBMC_HOST}${SPACE}sol activate |
| 141 | |
| 142 | Run Process ${ipmi_cmd} shell=True stdout=${file_path} |
| 143 | ... timeout=5s on_timeout=continue |
| 144 | |
| 145 | |
| 146 | Deactivate SOL Via IPMI |
| 147 | [Documentation] Stop SOL using IPMI and return SOL output. |
| 148 | [Arguments] ${file_path}=/tmp/sol_${OPENBMC_HOST} |
| 149 | # Description of argument(s): |
| 150 | # file_path The file path on the local machine to copy SOL output |
| 151 | # collected by above "Activate SOL Via IPMI" keyword. |
| 152 | # By default it copies log from /tmp/sol_<BMC_IP>. |
| 153 | |
| 154 | ${ipmi_cmd}= Catenate SEPARATOR= |
| 155 | ... ${IPMI_EXT_CMD} -P${SPACE}${IPMI_PASSWORD}${SPACE} |
| 156 | ... ${HOST}${SPACE}${OPENBMC_HOST}${SPACE}sol deactivate |
| 157 | |
| 158 | ${rc} ${output}= Run and Return RC and Output ${ipmi_cmd} |
| 159 | Run Keyword If ${rc} > 0 Return From Keyword ${output} |
| 160 | |
| 161 | ${rc} ${output}= Run and Return RC and Output cat ${file_path} |
| 162 | Should Be Equal ${rc} ${0} msg=${output} |
| 163 | |
| 164 | # Logging SOL output for debug purpose. |
| 165 | Log ${output} |
| 166 | |
| 167 | [Return] ${output} |
| 168 | |
| 169 | |
Sivas SRR | a2dab3c | 2016-07-25 05:08:18 -0500 | [diff] [blame] | 170 | Byte Conversion |
| 171 | [Documentation] Byte Conversion method receives IPMI RAW commands as |
| 172 | ... argument in string format. |
| 173 | ... Sample argument is as follows |
| 174 | ... "0x04 0x30 9 0x01 0x00 0x35 0x00 0x00 0x00 0x00 0x00 |
| 175 | ... 0x00" |
| 176 | ... IPMI RAW command format is as follows |
| 177 | ... <netfn Byte> <cmd Byte> <Data Bytes..> |
| 178 | ... This method converts IPMI command format into |
| 179 | ... dbus command format as follows |
| 180 | ... <byte:seq-id> <byte:netfn> <byte:lun> <byte:cmd> |
| 181 | ... <array:byte:data> |
| 182 | ... Sample dbus Host IPMI Received Message argument |
| 183 | ... byte:0x00 byte:0x04 byte:0x00 byte:0x30 |
| 184 | ... array:byte:9,0x01,0x00,0x35,0x00,0x00,0x00,0x00,0x00,0x00 |
Gunnar Mills | 3803280 | 2016-12-12 13:43:40 -0600 | [diff] [blame] | 185 | [Arguments] ${args} |
Gunnar Mills | 59204d2 | 2016-12-06 11:05:19 -0600 | [diff] [blame] | 186 | ${argLength}= Get Length ${args} |
Sivas SRR | a2dab3c | 2016-07-25 05:08:18 -0500 | [diff] [blame] | 187 | Set Global Variable ${arrayByte} array:byte: |
Gunnar Mills | 59204d2 | 2016-12-06 11:05:19 -0600 | [diff] [blame] | 188 | @{listargs}= Split String ${args} |
| 189 | ${index}= Set Variable ${0} |
Sivas SRR | a2dab3c | 2016-07-25 05:08:18 -0500 | [diff] [blame] | 190 | :FOR ${word} in @{listargs} |
| 191 | \ Run Keyword if ${index} == 0 Set NetFn Byte ${word} |
| 192 | \ Run Keyword if ${index} == 1 Set Cmd Byte ${word} |
| 193 | \ Run Keyword if ${index} > 1 Set Array Byte ${word} |
Gunnar Mills | 59204d2 | 2016-12-06 11:05:19 -0600 | [diff] [blame] | 194 | \ ${index}= Set Variable ${index + 1} |
| 195 | ${length}= Get Length ${arrayByte} |
| 196 | ${length}= Evaluate ${length} - 1 |
| 197 | ${arrayByteLocal}= Get Substring ${arrayByte} 0 ${length} |
Sivas SRR | a2dab3c | 2016-07-25 05:08:18 -0500 | [diff] [blame] | 198 | Set Global Variable ${arrayByte} ${arrayByteLocal} |
Gunnar Mills | 59204d2 | 2016-12-06 11:05:19 -0600 | [diff] [blame] | 199 | ${valueinBytesWithArray}= Catenate byte:0x00 ${netfnByte} byte:0x00 |
| 200 | ${valueinBytesWithArray}= Catenate ${valueinBytesWithArray} ${cmdByte} |
| 201 | ${valueinBytesWithArray}= Catenate ${valueinBytesWithArray} ${arrayByte} |
| 202 | ${valueinBytesWithoutArray}= Catenate byte:0x00 ${netfnByte} byte:0x00 |
| 203 | ${valueinBytesWithoutArray}= Catenate ${valueinBytesWithoutArray} ${cmdByte} |
Sivas SRR | a2dab3c | 2016-07-25 05:08:18 -0500 | [diff] [blame] | 204 | # To Check scenario for smaller IPMI raw commands with only 2 arguments |
| 205 | # instead of usual 12 arguments. |
| 206 | # Sample small IPMI raw command: Run IPMI command 0x06 0x36 |
| 207 | # If IPMI raw argument length is only 9 then return value in bytes without |
| 208 | # array population. |
| 209 | # Equivalent dbus-send argument for smaller IPMI raw command: |
| 210 | # byte:0x00 byte:0x06 byte:0x00 byte:0x36 |
| 211 | Run Keyword if ${argLength} == 9 Return from Keyword ${valueinBytesWithoutArray} |
Gunnar Mills | c9ea936 | 2016-12-13 16:21:13 -0600 | [diff] [blame] | 212 | [Return] ${valueinBytesWithArray} |
Sivas SRR | a2dab3c | 2016-07-25 05:08:18 -0500 | [diff] [blame] | 213 | |
| 214 | |
| 215 | Set NetFn Byte |
Gunnar Mills | 3803280 | 2016-12-12 13:43:40 -0600 | [diff] [blame] | 216 | [Arguments] ${word} |
Gunnar Mills | 59204d2 | 2016-12-06 11:05:19 -0600 | [diff] [blame] | 217 | ${netfnByteLocal}= Catenate byte:${word} |
Sivas SRR | a2dab3c | 2016-07-25 05:08:18 -0500 | [diff] [blame] | 218 | Set Global Variable ${netfnByte} ${netfnByteLocal} |
| 219 | |
| 220 | Set Cmd Byte |
Gunnar Mills | 3803280 | 2016-12-12 13:43:40 -0600 | [diff] [blame] | 221 | [Arguments] ${word} |
Gunnar Mills | 59204d2 | 2016-12-06 11:05:19 -0600 | [diff] [blame] | 222 | ${cmdByteLocal}= Catenate byte:${word} |
Sivas SRR | a2dab3c | 2016-07-25 05:08:18 -0500 | [diff] [blame] | 223 | Set Global Variable ${cmdByte} ${cmdByteLocal} |
| 224 | |
| 225 | Set Array Byte |
Gunnar Mills | 3803280 | 2016-12-12 13:43:40 -0600 | [diff] [blame] | 226 | [Arguments] ${word} |
Gunnar Mills | 59204d2 | 2016-12-06 11:05:19 -0600 | [diff] [blame] | 227 | ${arrayByteLocal}= Catenate SEPARATOR= ${arrayByte} ${word} |
| 228 | ${arrayByteLocal}= Catenate SEPARATOR= ${arrayByteLocal} , |
Sivas SRR | a2dab3c | 2016-07-25 05:08:18 -0500 | [diff] [blame] | 229 | Set Global Variable ${arrayByte} ${arrayByteLocal} |
| 230 | |
Chris Austen | b29d2e8 | 2016-06-07 12:25:35 -0500 | [diff] [blame] | 231 | Copy ipmitool |
| 232 | OperatingSystem.File Should Exist tools/ipmitool msg=The ipmitool program could not be found in the tools directory. It is not part of the automation code by default. You must manually copy or link the correct openbmc version of the tool in to the tools directory in order to run this test suite. |
| 233 | |
| 234 | Import Library SCPLibrary WITH NAME scp |
| 235 | scp.Open connection ${OPENBMC_HOST} username=${OPENBMC_USERNAME} password=${OPENBMC_PASSWORD} |
| 236 | scp.Put File tools/ipmitool /tmp |
| 237 | SSHLibrary.Open Connection ${OPENBMC_HOST} |
| 238 | Login ${OPENBMC_USERNAME} ${OPENBMC_PASSWORD} |
| 239 | Execute Command chmod +x /tmp/ipmitool |
Sridevi Ramesh | 0c34f69 | 2017-04-25 02:40:32 -0500 | [diff] [blame] | 240 | |
| 241 | Initiate Host Boot Via External IPMI |
| 242 | [Documentation] Initiate host power on using external IPMI. |
Rahul Maheshwari | 982fee4 | 2017-05-03 00:33:15 -0500 | [diff] [blame] | 243 | [Arguments] ${wait}=${1} |
| 244 | # Description of argument(s): |
| 245 | # wait Indicates that this keyword should wait for host running state. |
| 246 | |
Sridevi Ramesh | 0c34f69 | 2017-04-25 02:40:32 -0500 | [diff] [blame] | 247 | ${output}= Run External IPMI Standard Command chassis power on |
| 248 | Should Not Contain ${output} Error |
Rahul Maheshwari | 982fee4 | 2017-05-03 00:33:15 -0500 | [diff] [blame] | 249 | |
| 250 | Run Keyword If '${wait}' == '${0}' Return From Keyword |
| 251 | Wait Until Keyword Succeeds 10 min 10 sec Is Host Running |
Sridevi Ramesh | 0c34f69 | 2017-04-25 02:40:32 -0500 | [diff] [blame] | 252 | |
| 253 | Initiate Host PowerOff Via External IPMI |
| 254 | [Documentation] Initiate host power off using external IPMI. |
Rahul Maheshwari | 982fee4 | 2017-05-03 00:33:15 -0500 | [diff] [blame] | 255 | [Arguments] ${wait}=${1} |
| 256 | # Description of argument(s): |
| 257 | # wait Indicates that this keyword should wait for host off state. |
| 258 | |
Sridevi Ramesh | 0c34f69 | 2017-04-25 02:40:32 -0500 | [diff] [blame] | 259 | ${output}= Run External IPMI Standard Command chassis power off |
| 260 | Should Not Contain ${output} Error |
Rahul Maheshwari | 982fee4 | 2017-05-03 00:33:15 -0500 | [diff] [blame] | 261 | |
| 262 | Run Keyword If '${wait}' == '${0}' Return From Keyword |
| 263 | Wait Until Keyword Succeeds 3 min 10 sec Is Host Off |
| 264 | |
| 265 | Get Host State Via External IPMI |
| 266 | [Documentation] Returns host state using external IPMI. |
| 267 | |
| 268 | ${output}= Run External IPMI Standard Command chassis power status |
| 269 | Should Not Contain ${output} Error |
| 270 | ${output}= Fetch From Right ${output} ${SPACE} |
| 271 | |
| 272 | [Return] ${output} |