Generating Big Data repositories for research in medical imaging

Abstract

The production of medical imaging data has grown tremendously in the last decades. The proliferation of digital acquisition equipment has enabled even small institutions to produce considerable amounts of studies. Furthermore, the general trend for new imaging modalities is to produce more data per examinations. As a result, the design and implementation of tomorrow's storage and communication systems must deal with Big Data issues. Moreover, new realities such as the outsourcing of medical imaging infrastructures to the Cloud impose additional pressure on these systems. The research on technologies for coping with Big Data issues on large scale medical imaging environments is still in its early stages. This is mostly due to the difficulty of implementing and validating new technological approaches in real environments, without interfering with clinical practice. Therefore, it is crucial to create test bed environments for research purposes. This article proposes a methodology for creating simulated Big Data repositories. The system is able to use a real world repository to collect data from a representative time window and expand it according to research needs. In addition, the solution provides anonymization tools and allows exporting two types of simulated data: a structured file repository containing DICOM objects and a statistical summary of the archive content based on its characteristics, such as the number of produced studies per day, their modality and their associated data volumes.