HP ML530 RDMA protocol: improving network performance - Page 5

RDMA protocol overview A series of protocols are layered to perform RDMA over TCP (Figure 3). The top three layers form the iWARP family of protocols that provide high-speed internet operability. The RDMA protocol converts RDMA write, RDMA read, and sends into Direct Data Placement (DDP) messages. The DDP protocol segments outbound DDP messages into one or more DDP segments, and reassembles one or more DDP segments into a DDP message. The marker-based, protocol-data-unit-aligned (MPA) protocol adds a backward marker at a fixed interval to DDP segments and also adds a length and cyclical redundancy check (CRC) to each MPA segment. TCP schedules outbound TCP segments and satisfies delivery guarantees. IP adds necessary network routing information. Figure 3. Protocol layers for RDMA over TCP TX RDMA RX Converts RDMA write, RDMA read, and sends into DDP messages DDP Segments outbound (TX) DDP messages into one or more DDP segments; reassembles one or more DDP segments into a DDP message MPA Adds a backward marker at a fixed interval to DDP segments; also adds a length and CRC to each MPA segment TCP Schedules outbound TCP segments and satisfies delivery guarantees IP Adds necessary network routing information TCP uses a stream of 8-bit fields (octets) to communicate data, while DDP uses fixed protocol data units (PDUs). To enable RDMA, DDP needs a framing mechanism for the TCP transport protocol. MPA is a marker-based PDU-aligned framing mechanism for the TCP protocol. MPA provides a generalized framing mechanism that enables a network adapter using DDP to locate the DDP header. The adapter can then place the data directly in the application buffer, based on the control information carried in the header. MPA enables this capability even when the packets arrive out of order. By enabling DDP, MPA avoids the memory copy overhead and reduces the memory requirement for handling out-of-order packets and dropped packets. MPA creates a framed PDU (FPDU) by prefixing a header, inserting markers, and appending a CRC after the DDP segment. MPA delivers the FPDU to TCP. The MPA-aware TCP sender puts the FPDUs into the TCP stream and segments the TCP stream so that each TCP segment contains a single FPDU. The MPA receiver locates and assembles complete FPDUs within the stream, verifies their integrity, and removes information that is no longer necessary. MPA then provides the complete DDP segment to DDP. 5