Abstract
Recently, with the rapid development of electron microscopy (EM) technology and the increasing demand of neuron circuit reconstruction, the scale of reconstruction data grows significantly. This brings many challenges, one of which is how to effectively manage large-scale data so that researchers can mine valuable information. For this purpose, we developed a data management module equipped with two parts, a storage and retrieval module on the server-side and an image cache module on the client-side. On the server-side, Hadoop and HBase are introduced to resolve massive data storage and retrieval. The pyramid model is adopted to store electron microscope images, which represent multiresolution data of the image. A block storage method is proposed to store volume segmentation results. We design a spatial location-based retrieval method for fast obtaining images and segments by layers rapidly, which achieves a constant time complexity. On the client-side, a three-level image cache module is designed to reduce latency when acquiring data. Through theoretical analysis and practical tests, our tool shows excellent real-time performance when handling large-scale data. Additionally, the server-side can be used as a backend of other similar software or a public database to manage shared datasets, showing strong scalability.
Highlights
The brain is the organ of thought, and as the most advanced part of the nervous system, it dominates all the activities of the body
We make a test on our data management module, and it shows excellent real-time performance when handling large-scale data
A massive data management tool is developed in this paper, based on a distributed architecture
Summary
The brain is the organ of thought, and as the most advanced part of the nervous system, it dominates all the activities of the body. Scientists have done much research and have a basic understanding of its functions. MRI technology is adopted to study the connections between different gray matter areas of the brain on a macro scale (millimeter scale) [1]. The projection path of a single neuron axon and its connection with upstream and downstream neurons are studied at the mesoscopic scale (micron scale) [2,3]. To ensure that all parts of the brain work together and drive the body’s activities, the transmission of signals, and the connection between the large number of neurons is still a mystery. The significant number of neurons and the complex connections between them make the structure and function of the brain extremely complicated
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have