A high speed data acquisition collector for merging and sorting data

Abstract

Distributed data acquisition systems have a need for high throughput and a reliable delivery of data and commands in order to function properly. Connectionless oriented protocols like UDP provides a small overhead, but due to its unreliability it can be adequate to equip the application layer with additional services like flow control, data ordering and error checking. High performance buffers are important and necessary components here in order to not add further delays. When data, transmitted from a number of nodes are merged together in one node, not only serving as a collector of data, but also performing additional tasks like sorting of incoming data etc, high performance buffers and software architectures are equally important for adding as little delay as possible to the transmission chain. Involved in different projects we have faced this challenge and have managed to reach throughputs of 100 MB/s on 1Gb Ethernet with the developed architecture when implemented in Java. We will in our paper share our results when it comes to performance and ease of development, and discuss pros and cons with an equal implementation developed in C. first performance results will be presented.