HP ML530 RDMA protocol: improving network performance - Page 7

RDMA write, RDMA read, Terminate, Verbs

Page 7 highlights

The send with solicited event message is similar to the send message except that when the send with solicited event message has been placed and delivered, an event (for example, an interrupt) may be generated at the recipient, if the recipient is configured to generate such an event. This allows the recipient to control the amount of interrupt overhead it will encounter. The send with solicited event and invalidate message combines the functionality of the send with invalidate message and the send with solicited event message. RDMA write An RDMA write operation uses an RDMA write message to transfer data from the data source to a previously advertised buffer at the data target. The ULP at the data target enables the data target tagged buffer for access and advertises the buffer's size (length), location (tagged offset), and STag to the data source through a ULP-specific mechanism such as a prior send message. The ULP at the data source initiates the RDMA write operation. The RDMA write message uses the DDP tagged buffer model to transfer the ULP message into the data target's tagged buffer. The STag associated with the tagged buffer remains valid until the ULP at the data target invalidates it or until the ULP at the data source invalidates it through a send with invalidate operation or a send with solicited event with invalidate operation. RDMA read The RDMA read operation transfers data to a tagged buffer at the data target from a tagged buffer at the data source. The ULP at the data source enables the data source tagged buffer for access and advertises the buffer's size (length), location (tagged offset), and STag to the data target through a ULP-specific mechanism such as a prior send message. The ULP at the data target enables the data target tagged buffer for access and initiates the RDMA read operation. The RDMA read operation consists of a single RDMA read request message and a single RDMA read response message, which may be segmented into multiple DDP segments. The RDMA read request message uses the DDP untagged buffer model to deliver to the data source's RDMA read request queue the STag, starting tagged offset, and length for both the data source and the data target tagged buffers. When the data source receives this message, it then processes the message and generates a read response message, which will transfer the data. The RDMA read response message uses the DDP tagged buffer model to deliver the data source's tagged buffer to the data target, without any involvement from the ULP at the data source. The data source STag associated with the tagged buffer remains valid until the ULP at the data source invalidates it or until the ULP at the data target invalidates it through a send with invalidate or send with solicited event and invalidate. The data target STag associated with the tagged buffer remains valid until the ULP at the data target invalidates it. Terminate A terminate operation is included to perform an abortive tear down of the connection when an error is encountered. The terminate operation transfers to the data target information associated with an error that occurred at the data source. The terminate message uses the DDP untagged buffer model to transfer the message into the data target's untagged buffer. Verbs The RDMA Verbs Specification describes an abstract interface to an RNIC. The RNIC implements the RDMA protocol over a reliable transport, such as MPA over TCP. The RDMA verbs provide a semantic definition of the RNIC interface. A verbs consumer uses the capabilities of the RNIC to accomplish some objective. A verbs consumer may be defined as an operating system kernel thread, a non-privileged application, or a special privileged process. RDMA provides verbs consumers the capability to control data placement, 7

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12

The
send with solicited event message
is similar to the send message except that when the send with
solicited event message has been placed and delivered, an event (for example, an interrupt) may be
generated at the recipient, if the recipient is configured to generate such an event. This allows the
recipient to control the amount of interrupt overhead it will encounter.
The
send with solicited event and invalidate
message combines the functionality of the send with
invalidate message and the send with solicited event message.
RDMA write
An RDMA write operation uses an RDMA write message to transfer data from the data source to a
previously advertised buffer at the data target. The ULP at the data target enables the data target
tagged buffer for access and advertises the buffer’s size (length), location (tagged offset), and STag to
the data source through a ULP-specific mechanism such as a prior send message. The ULP at the data
source initiates the RDMA write operation. The RDMA write message uses the DDP tagged buffer
model to transfer the ULP message into the data target’s tagged buffer. The STag associated with the
tagged buffer remains valid until the ULP at the data target invalidates it or until the ULP at the data
source invalidates it through a send with invalidate operation or a send with solicited event with
invalidate operation.
RDMA read
The RDMA read operation transfers data to a tagged buffer at the data target from a tagged buffer at
the data source. The ULP at the data source enables the data source tagged buffer for access and
advertises the buffer’s size (length), location (tagged offset), and STag to the data target through a
ULP-specific mechanism such as a prior send message. The ULP at the data target enables the data
target tagged buffer for access and initiates the RDMA read operation. The RDMA read operation
consists of a single RDMA read request message and a single RDMA read response message, which
may be segmented into multiple DDP segments.
The RDMA read request message uses the DDP untagged buffer model to deliver to the data source’s
RDMA read request queue the STag, starting tagged offset, and length for both the data source and
the data target tagged buffers. When the data source receives this message, it then processes the
message and generates a read response message, which will transfer the data.
The RDMA read response message uses the DDP tagged buffer model to deliver the data source’s
tagged buffer to the data target, without any involvement from the ULP at the data source.
The data source STag associated with the tagged buffer remains valid until the ULP at the data source
invalidates it or until the ULP at the data target invalidates it through a send with invalidate or send
with solicited event and invalidate. The data target STag associated with the tagged buffer remains
valid until the ULP at the data target invalidates it.
Terminate
A terminate operation is included to perform an abortive tear down of the connection when an error
is encountered. The terminate operation transfers to the data target information associated with an
error that occurred at the data source. The terminate message uses the DDP untagged buffer model to
transfer the message into the data target’s untagged buffer.
Verbs
The RDMA Verbs Specification describes an abstract interface to an RNIC. The RNIC implements the
RDMA protocol over a reliable transport, such as MPA over TCP. The RDMA verbs provide a semantic
definition of the RNIC interface.
A verbs consumer uses the capabilities of the RNIC to accomplish some objective. A verbs consumer
may be defined as an operating system kernel thread, a non-privileged application, or a special
privileged process. RDMA provides verbs consumers the capability to control data placement,
7