Low Latency Inter-Thread Communication Over Gigabit Ethernet

The increase in network and processing capacity for commodity platforms has enabled Networks of Workstations to become affordable alternatives to dedicated parallel computers. However layered communication protocols used on commodity Operating Systems (such as TCP/IP) were not originally designed for use in high performance cluster computing, so they give rise to communication bottlenecks due to interrupt generation overheads, in-memory copying, protocol processing, as well as latencies induced by inefficient use of buses and IO devices. In this book we identify such bottlenecks and present a communication system based on Gigabit Ethernet that operates completely at the user level without any OS interaction. Our library integrates with a user-level thread scheduler to provide multi-threaded applications with zero-copy messaging based on CSP channels. We replace interrupts with local polling to reach payload throughputs of 941Mbps whilst reducing CPU load extensively when compared to traditional kernel-based communication. This book should interest anyone researching the areas of high performance cluster computing and distributed systems.