InfiniBand over Ethernet

InfiniBand is a high-bandwidth, low-latency interconnect developed for high-performance computing and storage clusters, specified by the InfiniBand Trade Association. Its defining feature is remote direct memory access (RDMA): one machine reads from or writes to another machine’s memory without passing each transfer through the remote operating system and processor. InfiniBand over Ethernet carries that transport across Ethernet instead of dedicated InfiniBand switches. The usual implementation is RDMA over Converged Ethernet (RoCE): its first version runs directly over Ethernet within one layer 2 domain, and version 2 wraps the traffic in UDP and IP so that it can be routed.

The ISC2 exam outline names it as an example of converged protocols, alongside iSCSI, voice over IP and Compute Express Link, and study sources commonly add Fibre Channel over Ethernet: specialised traffic moved onto a shared, general-purpose network. The security consequence follows from the design. Because RDMA transfers bypass much of the host’s software stack, host-based monitoring may not see them, and RoCE offers little built-in authentication or encryption of its own. Deployments therefore commonly keep this traffic on dedicated, segmented data centre networks.

Exam relevance: questions in this area tend to turn on recognising the term as a converged protocol, and on the trade-off convergence brings: lower cost and simpler cabling, in exchange for sharing a network, with its failures and attack surface, with other traffic.