Special issue for data intensive eScience

Abstract

Data-intensive science is enabled by the data deluge and has been called the “fourth paradigm” of scientific discovery. New fields are being born, such as drug discovery based on large scale study of correlations in published papers and climate implications from data on the accelerating pace of changes in previously quiescent ice sheets. Astronomy and the search for fundamental particles at the Large Hadron Collider drive mainstream aspects of data-intensive science with many petabytes of data derived from large advanced instruments. Medical imagery and genomics have as much data but from a slew of distributed instruments. It is projected that there will be 24 billion devices on the Internet by 2020. Most of this “Internet of Things” will be small sensors that produce streams of information which will be processed and integrated with other streams and turned into knowledge. The deluge and its impact are pervasive. In synergy with moving to a data-driven world, we are also in the midst of an evolution in the compute landscape of hardware systems. We now live in the world of massive multi-core and GPU processing systems, very large main memory systems, fast networking components, fast solid state drive, and large data centers that consume massive amounts of energy. Computing paradigm is changing and suggesting new programming models, new data structures and more attention to fault tolerance while enabling much easier access to computing. It is clear that many aspects of how we have dealt with data processing have to change in this new world.