MCTP LPC Hardware Binding for ASPEED BMC Systems

This design provides an efficient method to transfer MCTP packets between the host and BMC over the LPC bus on ASPEED BMC platforms.

The basic components used for the transfer are:

  • A window of the LPC FW address space, where reads and writes are forwarded to BMC memory, using the LPC2AHB hardware
  • An interrupt mechanism using the IPMI KCS interface

In order to transfer a packet, either side of the channel (BMC or host) will:

  1. Write the packet to the LPC FW window
    • The BMC will perform writes by writing to the memory backing the LPC window
    • The host will perform writes by writing to the LPC bus, at predefined addresses
  2. Trigger an interrupt on the remote side, by writing to the KCS data buffer

On this indication, the remote side will:

  1. Read from the KCS status register, which shows that the single-byte KCS data buffer is full
  2. Read the MCTP packet from the LPC FW window
  3. Read from the KCS buffer, to clear the 'buffer full' state.

LPC FW Window Layout

The window of BMC-memory-backed LPC FW address space has a predefined format, consisting of:

  • A control descriptor, describing static data about the rest of the window
  • A receive area for BMC-to-host packets
  • A transmit area, for host-to-BMC packets

The control descriptor contains a version, and offset and size data for the transmit and receive areas. These offsets are relative to the start of the LPC FW window.

Full definition of the control area is defined below, and it will be the base for all future versions.

struct mctp_lpcmap_hdr {
   uint32_t magic;

   uint16_t bmc_ver_min;
   uint16_t bmc_ver_cur;
   uint16_t host_ver_min;
   uint16_t host_ver_cur;
   uint16_t negotiated_ver;
   uint16_t pad0;

   uint32_t rx_offset;
   uint32_t rx_size;
   uint32_t tx_offset;
   uint32_t tx_size;
} __attribute__((packed));

Where the magic value marking the beginning of the control area is the ASCII encoding of "MCTP":

#define LPC_MAGIC 0x4d435450

The transmit and receive areas contain a length field, followed by the actual MCTP packet to be transferred. At version 1, only a single MCTP packet is present in the Rx and Tx areas. This may change for future versions of the protocol.

All control data is in big-endian format. MCTP packet data is transferred exactly as is presented, and no data escaping is performed.

KCS Control

The KCS hardware on the ASPEED BMCs is used as a method of indicating, to the remote side, that a packet is ready to be transferred through the LPC FW mapping.

The KCS hardware consists of two single-byte buffers: the Output Data Register (ODR) and the Input Data Register (IDR). The ODR is written by the BMC and read by the host. The IDR is the obverse.

The KCS unit also contains a status register, allowing both host and BMC to determine if there is data in the ODR or IDR. These are single-bit flags, designated Input/Output Buffer Full (IBF/OBF), and are automatically set by hardware when data has been written to the corresponding ODR/IDR buffer (and cleared when data has been read).

We use these flags to determine whether data in the LPC window is available to be consumed.

KCS Status Register Layout

BitManaged ByDescription
7Software(MSB) BMC Active
6SoftwareChannel active, version negotiated
5SoftwareUnused
4SoftwareUnused
3HardwareCommand / Data
2SoftwareUnused
1HardwareInput Buffer Full
0Hardware(LSB) Output Buffer Full

KCS Data Register Commands

CommandDescription
0x00Initialise
0x01Tx Begin
0x02Rx Complete
0xffDummy Value

General Protocol Behaviours

  • The BMC writes to the status register
    • The hardware triggers a host interrupt
    • The host reads the status register for BMC operating state transitions
  • The host writes to the data register
    • The hardware triggers a BMC interrupt
    • The BMC reads the status register for IBF (this is the only bit that may change)
    • If IBF is set, the BMC reads the data register for buffer state transitions
  • The BMC writes to the data register
    • The hardware triggers a Host interrupt
    • The host reads the status register for OBF (this is the only bit that may change)
    • If OBF is set, the host reads the data register for buffer state transitions
  • Some KCS hardware implementations may only trigger an interrupt from ODR events (and not status update). The 0xff dummy value allows either side of the KCS interface to trigger a data-register interrupt by performing a dummy write

LPC Window Ownership and Synchronisation

Because the LPC FW window is shared between the host and the BMC we need strict rules on which entity is allowed to access it at specific times.

Firstly, we have rules for modification:

  • The control data is only written during initialisation. Only the BMC may write to the control area, except the host-version fields. The control area is never modified once the channel is active.
  • Only the BMC may write to the Rx buffer described in the control area
  • Only the host may write to the Tx buffer described in the control area

During packet transmission, the follow sequence occurs:

  • The Tx side writes the packet to its Tx buffer
  • The Tx side sends a Tx Begin message, indicating that the buffer ownership is transferred
  • The Rx side now owns the buffer, and reads the message from its Rx area
  • The Rx side sends a Rx Complete once done, indicating that the buffer ownership is transferred back to the Tx side.

LPC Binding Operation

The binding operation is not symmetric as the BMC is the only side that can drive the status register. Each side's initialisation sequence is outlined below.

BMC Initialisation Sequence

StepDescription
1The BMC initialises the control area: magic value, BMC versions and buffer parameters
2The BMC sets the BMC active bit and triggers the host interrupt

Host initialisation Sequence

StepDescription
1Wait for the BMC to indicate active via the KCS status register
2Populate the host version fields
3Send the Initialise message via KCS
4The hardware sets the IBF flag in the status register
5The KCS interrupt is triggered on the BMC
6The BMC reads the KCS status and data registers
7The hardware clears IBF and de-asserts the KCS IRQ
8The BMC calculates the negotiated version
9The BMC sets the Channel Active bit in the KCS status register
10The KCS interrupt is triggered on the host
11The host reads the KCS status and data registers
12The hardware clears OBF and de-asserts the host KCS IRQ
13The host observes that Channel Active is set in the KCS status register
14The host reads the negotiated version

Host Packet Transmission Sequence

StepDescription
1The host waits on the previous Rx Complete message
2The host waits on BMC Active and Channel Active in the KCS status register
3The host writes the packet to its Tx area (BMC Rx area)
4The host sends the Tx Begin command via the KCS interface, transferring ownership of its Tx buffer to the BMC
5The hardware sets the IBF flag in the KCS status register
6The KCS interrupt is triggered on the BMC
7The BMC reads the KCS status and data registers
8The hardware clears IBF and de-asserts the KCS IRQ
9The BMC observes IBF is set and the command is Tx Begin
10The BMC reads the packet from the BMC Rx area (host Tx area)
11The BMC sends the Rx Complete command via the KCS interface
12The hardware sets the OBF flag in the KCS status register
13The KCS interrupt is triggered on the host
14The host reads the KCS status and data registers
15The hardware clears OBF and de-asserts the host KCS IRQ
16The host observes OBF is set and the command is Rx Complete
17The host regains ownership of its Tx buffer

BMC Packet Transmission Sequence

StepDescription
1The BMC waits on the previous Rx Complete message
2The BMC writes the packet to its Tx area (host Rx area)
3The BMC sends the Tx Begin command via the KCS interface, transferring ownership of its Tx buffer to the host
4The hardware sets the OBF flag in the KCS status register
5The KCS interrupt is triggered on the host
6The host reads the KCS status and data registers
7The hardware clears OBF and de-asserts the KCS IRQ
8The host observes OBF is set and the command is Tx Begin
9The host reads the packet from the host Rx area (BMC Tx area)
10The host sends the Rx Complete command via the KCS interface
11The hardware sets the IBF flag in the KCS status register
12The KCS interrupt is triggered on the BMC
13The BMC reads the KCS status and data registers
14The hardware clears IBF and de-asserts the BMC KCS IRQ
15The BMC observes IBF is set and the command is Rx Complete.
16The BMC regains ownership of its Tx buffer

Alternatives Considered

The KCS MCTP Binding (DSP0254)

The KCS hardware (used as the full transfer channel) can be used to transfer arbitrarily-sized MCTP messages. However, there are much larger overheads in synchronisation between host and BMC for every byte transferred.

Using the MCTP Serial Binding (DSP0253)

We could use the VUART hardware to transfer the MCTP packets according to the existing MCTP Serial Binding. However, the VUART device is already used for console data. Multiplexing both MCTP and console would be an alternative, but the complexity introduced would make low-level debugging both more difficult and less reliable.

The BT interface

The BT interface allows for block-at-time transfers. However, the BT buffer size is only 64 bytes on the AST2500 hardware, which does not allow us to comply with the MCTP Base Specification (DSP0236) that requires a 64-byte payload size as the minimum. The 64-byte BT buffer does not allow for MCTP and transport headers.

Additionally, we would like to develop the MCTP channel alongside the existing IPMI interfaces, to allow a gradual transition from IPMI to MCTP. As the BT channel is already used on OpenPOWER systems for IPMI transfers, we would not be able to support both in parallel.

Using the AST2500 LPC Mailbox

This would require enabling the SuperIO interface, which allows the host to access the entire BMC address space, and so introduces security vulnerabilities.