Write-Optimized and Consistent RDMA-based NVM Systems

Abstract

In order to deliver high performance in cloud computing, we generally exploit and leverage RDMA (Remote Direct Memory Access) in networking and NVM (Non-Volatile Memory) in end systems. Due to no involvement of CPU, one-sided RDMA becomes efficient to access the remote memory, and NVM technologies have the strengths of non-volatility, byte-addressability and DRAM-like latency. In order to achieve end-to-end high performance, many efforts aim to synergize one-sided RDMA and NVM. Due to the need to guarantee Remote Data Atomicity (RDA), we have to consume extra network round-trips, remote CPU participation and double NVM writes. In order to address these problems, we propose a zero-copy log-structured memory design for Efficient Remote Data Atomicity, called Erda. In Erda, clients directly transfer data to the destination address at servers via one-sided RDMA writes without redundant copy and remote CPU consumption. To detect the incompleteness of fetched data, we verify a checksum without client-server coordination. We further ensure metadata consistency by leveraging an 8-byte atomic update in the hash table, which also contains the address information for the stale data. When a failure occurs, the server properly restores to a consistent version. Experimental results show that compared with Redo Logging (a CPU involvement scheme) and Read After Write (a network dominant scheme), Erda reduces NVM writes approximately by 50%, as well as significantly improves throughput and decreases latency.