[docs]: Add D-Bus based MCTP daemon
In order to support Bus owner concept in MCTP to
handle Add-in-card use case, need a standard way of exposing
discovered devices, and it's MCTP properties in standard way.
This update proposes secondary approach using D-Bus based
interface support, instead of UNIX style socket concept.
Signed-off-by: Sumanth Bhat <sumanth.bhat@linux.intel.com>
Signed-off-by: Richard Marian Thomaiyar <richard.marian.thomaiyar@linux.intel.com>
Change-Id: Ic5d28268bd1d431a1e468c8278547806af64f30c
diff --git a/designs/mctp.md b/designs/mctp.md
index 2d794b8..f8ef1ca 100644
--- a/designs/mctp.md
+++ b/designs/mctp.md
@@ -2,33 +2,24 @@
Author: Jeremy Kerr <jk@ozlabs.org> <jk>
+Other contributors: Sumanth Bhat, Richard Thomaiyar
+
## Problem Description
-Currently, we have a few different methods of communication between host
-and BMC. This is primarily IPMI-based, but also includes a few
-hardware-specific side-channels, like hiomap. On OpenPOWER hardware at
-least, we've definitely started to hit some of the limitations of IPMI
-(for example, we have need for >255 sensors), as well as the hardware
-channels that IPMI typically uses.
+With IPMI standards body no longer operational, DMTF's PMCI working group
+defines standards to address `inside the box` communication interfaces between
+the components of the platform management subsystem, including host-BMC
+communication.
-This design aims to use the Management Component Transport Protocol
-(MCTP) to provide a common transport layer over the multiple channels
-that OpenBMC platforms provide. Then, on top of MCTP, we have the
+This design aims to implement Management Component Transport Protocol (MCTP)
+to provide a common transport layer protocol for application layer
+protocols providing platform manageability solutions and to provide an
opportunity to move to newer host/BMC messaging protocols to overcome
-some of the limitations we've encountered with IPMI.
+some of the limitations we've encountered with IPMI (Sensor limit, h/w channel
+limitations etc.).
## Background and References
-Separating the "transport" and "messaging protocol" parts of the current
-stack allows us to design these parts separately. Currently, IPMI
-defines both of these; we currently have BT and KCS (both defined as
-part of the IPMI 2.0 standard) as the transports, and IPMI itself as the
-messaging protocol.
-
-Some efforts of improving the hardware transport mechanism of IPMI have
-been attempted, but not in a cross-implementation manner so far. This
-does not address some of the limitations of the IPMI data model.
-
MCTP defines a standard transport protocol, plus a number of separate
physical layer bindings for the actual transport of MCTP packets. These
are defined by the DMTF's Platform Management Working group; standards
@@ -66,9 +57,14 @@
Unit (MTU) are split into individual packets by the transmit
implementation, and reassembled at the receive implementation.
+ - [DSP0236 - MCTP Base specification 1.3.1](https://www.dmtf.org/sites/default/files/standards/documents/DSP0239_1.6.0.pdf)
+ - [DSP0239 - MCTP IDs & Codes 1.6.0](https://www.dmtf.org/sites/default/files/standards/documents/DSP0239_1.6.0.pdf)
+ - [DSP2016 - MCTP Overview white paper 1.0.0a](https://www.dmtf.org/sites/default/files/standards/documents/DSP2016.pdf)
+ - [OpenBMC D-Bus interfaces](https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-dbus-interfaces/+/30139)
+
## Requirements
-Any channel between host and BMC should:
+Any channel supported on BMC should:
- Have a simple serialisation and deserialisation format, to enable
implementations in host firmware, which have widely varying runtime
@@ -82,32 +78,101 @@
- Ideally, integrate with newer messaging protocols
+## Design Principles
+
+When BMC implements the MCTP protocol as a transport service,
+it should:
+
+* Support both MCTP Bus Owner and Endpoint roles.
+* Support multiple physical bindings (PCIe, SMBus, Serial, OEM etc.).
+* Provide a way for the upper layer protocols to send messages to an endpoint
+(EID) and receive messages from endpoints. (Tx/Rx mechanism)
+* Discover MCTP protocol supported devices when BMC is the Bus Owner of the
+physical medium and assign Endpoint IDs (EID)
+* Advertise the supported MCTP types, when BMC is the endpoint device of the
+physical medium.
+
+To enable implementation of MCTP on both host and BMC, platform independent
+library (libmctp) will be written in C. The MCTP service can leverage
+libmctp to provide transport services to upper layer protocols in OpenBMC.
+
## Proposed Design
-The MCTP core specification just provides the packetisation, routing and
+The MCTP core specification just provides the packetization, routing and
addressing mechanisms. The actual transmit/receive of those packets is
up to the hardware binding of the MCTP transport.
For OpenBMC, we would introduce a MCTP daemon, which implements the transport
over a configurable hardware channel (eg., Serial UART, I2C or PCIe), and
-provides a socket-based interface for other processes to send and
+provides a socket / D-Bus based interface for other processes to send and
receive complete MCTP messages. This daemon is responsible for the
-packetisation and routing of MCTP messages from external endpoints, and
+packetization and routing of MCTP messages from external endpoints, and
handling the forwarding these messages to and from individual handler
applications. This includes handling local MCTP-stack configuration,
like local EID assignments.
-This daemon has a few components:
+Following components are proposed.
- 1) the core MCTP stack
+### Core MCTP stack (libmctp)
+which includes the following
+1. MCTP Control commands - contains code to handle all the core MCTP
+control Commands.
+2. MCTP Packet formatter - contains code to do MCTP packet assembly or
+disassembly and verification of MCTP messages.
+3. MCTP binding implementations - contains code for binding implementations,
+which will interact with the hardware channel to transmit / receive MCTP
+packets. This will be implemented as per the specified binding specification
+of that channel.
+4. MCTP device discovery - contains code to discover MCTP capable devices
+on the physical medium bus, when MCTP is the bus owner.
- 2) one or more binding implementations (eg, MCTP-over-serial), which
- interact with the hardware channel(s).
+### Daemons
+Two types of daemons are proposed as follows, and both will rely on Core MCTP
+stack (libmctp).
- 3) an interface to handler applications over a unix-domain socket.
+1. De-mux daemon - UNIX domain socket as backbone
+or
+2. D-Bus based daemon - relying on D-Bus itself.
-The proposed implementation here is to produce an MCTP "library" which
-provides the packetisation and routing functions, between:
+Applications like PLDM / SPDM etc. can either use De-mux daemon or D-Bus based
+daemon to send / receive MCTP messages, with each having it's own pro's & con's.
+The vision is to make the MCTP communication through socket based approach,
+but due to the current limitations on advertising MCTP discovered
+devices & it's properties in standard way, D-Bus approach is provided.
+Based on the usage compile time flag (TBD) option for wrapper
+library can be used for easy switch (i.e. Applications like PLDM / SPDM etc.
+will use this wrapper library to switch between socket / D-Bus based approach).
+
+```
++--------------------------------------------------------------------------+
+| |
+| |
+| MCTP Daemon - De-mux with socket |
+| OR |
+| MCTP Daemon - D-bus based |
+| |
+| +---------------------------------------------------------+ |
+| | +------------------------------------------------+ | |
+| | | Control Commands | | |
+| | +------------------------------------------------+ | |
+| | +------------------------------------------------+ | |
+| | | Device Discovery | | |
+| | +------------------------------------------------+ | |
+| | +------------------------------------------------+ +--------------> libmctp
+| | | Message TX/RX Handlers | | |
+| | +------------------------------------------------+ | |
+| | | |
+| | +-------+ +--------+ +----------+ +----------+ | |
+| | | LPC | | Serial | | SMBus | | PCIe | | |
+| | +-------+ +--------+ +----------+ +----------+ | |
+| +---------------------------------------------------------+ |
+| |
++--------------------------------------------------------------------------+
+
+```
+
+The proposed implementation produces an MCTP "library" component which
+provides the packetization and routing functions, between:
- an "upper" messaging transmit/receive interface, for tx/rx of a full
message to a specific endpoint (ie, (1) above)
@@ -140,9 +205,11 @@
binding for use in low-level host firmware environments may interact
directly with hardware registers to perform packet transfers.
-The application-specific handlers implement the actual functionality
-provided over the MCTP channel, and connect to the central daemon over a
-UNIX domain socket. Each of these would register with the MCTP daemon to
+The application specific handlers can connect to MCTP Daemon over two
+options : either over a UNIX domain socket OR over D-Bus.
+
+### De-mux daemon - MCTP daemon over a UNIX domain socket
+Each of these would register with the MCTP daemon to
receive MCTP messages of a certain type, and would transmit MCTP
messages of that same type.
@@ -159,12 +226,7 @@
this, as the unix-domain socket interface could be fairly easily swapped
out with a new kernel-based socket type.
-MCTP is intended to be an optional component of OpenBMC. Platforms using
-OpenBMC are free to adopt it as they see fit.
-
-### Demultiplexer daemon interface
-
-MCTP handlers (ie, clients of the demultiplexer) connect using a
+MCTP handlers (i.e., clients of the de-multiplexer) connect using a
unix-domain socket, at the abstract socket address:
\0mctp-demux
@@ -176,15 +238,15 @@
greater than zero.
Subsequent messages sent over the socket are MCTP messages sent/received
-by the demultiplexer, that match the specified MCTP message type.
+by the de-multiplexer, that match the specified MCTP message type.
Clients should use the send/recv syscalls to interact with the socket.
Each message has a fixed small header:
`uint8_t eid`
-For messages coming from the demux daemon, this indicates the source EID
-of the outgoing MCTP message. For messages going to the demux daemon,
+For messages coming from the De-mux daemon, this indicates the source EID
+of the outgoing MCTP message. For messages going to the De-mux daemon,
this indicates the destination EID.
The rest of the message data is the complete MCTP message, including
@@ -194,6 +256,52 @@
retrieve values for the tag field in individual MCTP packets.
+### D-Bus daemon - MCTP Daemon over D-Bus interface
+
+The applications can also connect to the MCTP Daemon over D-Bus.
+
+The design principles behind MCTP Daemon:
+
+1. Support all binding compiled in to a single (MCTP Daemon) application.
+
+2. Execute separate instance (of MCTP Daemon) for each physical interface.
+This is to limit any problems of the offending bus isolated to that application.
+This also enables parallel execution as many limitations apply to the physical
+medium.
+
+3. Start MCTP Daemon as user space application, which will be exposing
+D-Bus objects for the MCTP devices discovered.
+
+4. Binding support (SMBus, PCIe) and control commands will be added to libmctp.
+
+5. Needed MCTP daemons can be instantiated by reading the configuration from
+configuration file or using any inventory advertising method like
+`Entity-manager`. Based on this, each MCTP Daemon will work as bus owner /
+endpoint in particular physical interface (SMBus, PCIe, etc.).
+
+6. PLDM, SPDM, PCI VDM, NVME-MI application will run as separate daemon
+interacting with the MCTP Daemons (through D-BUS / Socket – abstracted
+library).
+
+The MCTP Daemon instance can either start in `Endpoint mode` or in `Bus Owner
+mode`. When the MCTP daemon instance comes up in `Endpoint mode`, BMC can be
+configured to work in static EID itself or configured to accept EID from the
+bus owner in that physical medium (in which case BMC will start in special
+EID 0 or static EID). When the MCTP daemon instance comes up in
+`Bus Owner mode`, BMC will discover MCTP capable devices on the physical
+medium and assigns EIDs from a pre-configured EID pool.
+
+#### D-Bus Interfaces
+
+Please [refer for D-Bus Interfaces](https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-dbus-interfaces/+/30139)
+
+An illustration of how D-Bus interfaces would look like on the MCTP is shown
+below (BMC as PCIe endpoint and BMC as SMBus bus owner).
+
+
+
+
+
## Alternatives Considered
There have been two main alternatives to this approach:
@@ -201,7 +309,7 @@
Continue using IPMI, but start making more use of OEM extensions to
suit the requirements of new platforms. However, given that the IPMI
standard is no longer under active development, we would likely end up
-with a large amount of platform-specific customisations. This also does
+with a large amount of platform-specific customisation. This also does
not solve the hardware channel issues in a standard manner.
Redfish between host and BMC. This would mean that host firmware needs a
@@ -225,11 +333,14 @@
Also, this single-process approach would likely need more significant
modifications if/when MCTP protocol support is moved to the kernel.
-The interface between the demultiplexer daemon and clients is currently
-defined as a socket-based interface. However, an alternative here would
-be to pass MCTP messages over dbus instead. The reason for the choice of
-sockets rather than dbus is that the former allows a direct transition
-to a kernel-based socket API when suitable.
+The interface between the de-multiplexer daemon and clients is currently
+defined as a socket-based interface. The reason for the choice of
+sockets rather than D-Bus is that the former allows a direct transition
+to a kernel-based socket API when suitable. However, D-Bus option is also
+provided to enable certain use-cases like Add-In-Cards. Socket interface
+currently does not provide a way to discover addition of a card, removal of a
+card to upper layers, and thus D-Bus based solution is provided until
+kernel-based socket API has these capabilities.
## Impacts
diff --git a/designs/media/MCTP.jpg b/designs/media/MCTP.jpg
new file mode 100644
index 0000000..29be6d9
--- /dev/null
+++ b/designs/media/MCTP.jpg
Binary files differ
diff --git a/designs/media/StateFlow.jpg b/designs/media/StateFlow.jpg
new file mode 100644
index 0000000..852d5ea
--- /dev/null
+++ b/designs/media/StateFlow.jpg
Binary files differ
