Electron Microscopy Image Stacks Research Articles

Deep residual contextual and subpixel convolution network for automated neuronal structure segmentation in micro-connectomics

Background and Objective: The goal of micro-connectomics research is to reconstruct the connectome and elucidate the mechanisms and functions of the nervous system via electron microscopy (EM). Due to the enormous variety of neuronal structures, neuron segmentation is among most difficult tasks in connectome reconstruction, and neuroanatomists desperately need a reliable neuronal structure segmentation method to reduce the burden of manual labeling and validation. Methods: In this article, we proposed an effective deep learning method based on a deep residual contextual and subpixel convolution network to obtain the neuronal structure segmentation in anisotropic EM image stacks. Furthermore, lifted multicut is used for post-processing to optimize the prediction and obtain the reconstruction results. Results: On the ISBI EM segmentation challenge, the proposed method ranks among the top of the leader board and yields a Rand score of 0.98788. On the public data set of mouse piriform cortex, it achieves a Rand score of 0.9562 and 0.9318 in the different testing stacks. The evaluation scores of our method are significantly improved when compared with those of state-of-the-art methods. Conclusions: The proposed automatic method contributes to the development of micro-connectomics, which improves the accuracy of neuronal structure segmentation and provides neuroanatomists with an effective approach to obtain the segmentation and reconstruction of neurons.

Sequential semi-supervised segmentation for serial electron microscopy image with small number of labels

BackgroundSegmentation of electron microscopic continuous section images by deep learning has attracted attention as a technique to reduce the cost of annotation for researchers attempting to make observations using 3D reconstruction methods. However, when the observed samples are rare, or scanning circumstances are unstable, pursuing generalization performance for newly obtained samples is not appropriate. New methodsWe assume a transductive setting that predicts all labels in a dataset from only partially obtained labels while avoiding the pursuit of generalization performance for unknown data. Then, we propose sequential semi-supervised segmentation (4S), which semi-automatically extracts neural regions from electron microscopy image stacks. This method focuses on the fact that adjacent images have a strong correlation in serial images. Our 4S repeats training, inference, and pseudo-labeling using a minimal number of teacher labels and performs segmentation on all slices. ResultOur experiments using two types of serial section images showed effectiveness in terms of both quality and quantity. In addition, we experimentally clarified the effect of the number and position of teacher labels on performance. Comparison with existing methodsCompared with supervised learning when a small number of labeled data was obtained, the performance of the proposed method was shown to be superior. ConclusionOur 4S leverages a limited number of labeled data and a large amount of unlabeled data to extract neural regions from serial image stacks in a transductive setting. We plan to develop this method as a core module of a general-purpose annotation tool in our future work.

AMST: Alignment to Median Smoothed Template for Focused Ion Beam Scanning Electron Microscopy Image Stacks

Alignment of stacks of serial images generated by Focused Ion Beam Scanning Electron Microscopy (FIB-SEM) is generally performed using translations only, either through slice-by-slice alignments with SIFT or alignment by template matching. However, limitations of these methods are two-fold: the introduction of a bias along the dataset in the z-direction which seriously alters the morphology of observed organelles and a missing compensation for pixel size variations inherent to the image acquisition itself. These pixel size variations result in local misalignments and jumps of a few nanometers in the image data that can compromise downstream image analysis. We introduce a novel approach which enables affine transformations to overcome local misalignments while avoiding the danger of introducing a scaling, rotation or shearing trend along the dataset. Our method first computes a template dataset with an alignment method restricted to translations only. This pre-aligned dataset is then smoothed selectively along the z-axis with a median filter, creating a template to which the raw data is aligned using affine transformations. Our method was applied to FIB-SEM datasets and showed clear improvement of the alignment along the z-axis resulting in a significantly more accurate automatic boundary segmentation using a convolutional neural network.

SynEM, automated synapse detection for connectomics.

Nerve tissue contains a high density of chemical synapses, about 1 per µm3 in the mammalian cerebral cortex. Thus, even for small blocks of nerve tissue, dense connectomic mapping requires the identification of millions to billions of synapses. While the focus of connectomic data analysis has been on neurite reconstruction, synapse detection becomes limiting when datasets grow in size and dense mapping is required. Here, we report SynEM, a method for automated detection of synapses from conventionally en-bloc stained 3D electron microscopy image stacks. The approach is based on a segmentation of the image data and focuses on classifying borders between neuronal processes as synaptic or non-synaptic. SynEM yields 97% precision and recall in binary cortical connectomes with no user interaction. It scales to large volumes of cortical neuropil, plausibly even whole-brain datasets. SynEM removes the burden of manual synapse annotation for large densely mapped connectomes.

Rapid and Semi-automated Extraction of Neuronal Cell Bodies and Nuclei from Electron Microscopy Image Stacks.

Connectomics-the study of how neurons wire together in the brain-is at the forefront of modern neuroscience research. However, many connectomics studies are limited by the time and precision needed to correctly segment large volumes of electron microscopy (EM) image data. We present here a semi-automated segmentation pipeline using freely available software that can significantly decrease segmentation time for extracting both nuclei and cell bodies from EM image volumes.

A workflow for the automatic segmentation of organelles in electron microscopy image stacks.

Electron microscopy (EM) facilitates analysis of the form, distribution, and functional status of key organelle systems in various pathological processes, including those associated with neurodegenerative disease. Such EM data often provide important new insights into the underlying disease mechanisms. The development of more accurate and efficient methods to quantify changes in subcellular microanatomy has already proven key to understanding the pathogenesis of Parkinson's and Alzheimer's diseases, as well as glaucoma. While our ability to acquire large volumes of 3D EM data is progressing rapidly, more advanced analysis tools are needed to assist in measuring precise three-dimensional morphologies of organelles within data sets that can include hundreds to thousands of whole cells. Although new imaging instrument throughputs can exceed teravoxels of data per day, image segmentation and analysis remain significant bottlenecks to achieving quantitative descriptions of whole cell structural organellomes. Here, we present a novel method for the automatic segmentation of organelles in 3D EM image stacks. Segmentations are generated using only 2D image information, making the method suitable for anisotropic imaging techniques such as serial block-face scanning electron microscopy (SBEM). Additionally, no assumptions about 3D organelle morphology are made, ensuring the method can be easily expanded to any number of structurally and functionally diverse organelles. Following the presentation of our algorithm, we validate its performance by assessing the segmentation accuracy of different organelle targets in an example SBEM dataset and demonstrate that it can be efficiently parallelized on supercomputing resources, resulting in a dramatic reduction in runtime.

NeuroMorph: a toolset for the morphometric analysis and visualization of 3D models derived from electron microscopy image stacks.

Serialelectron microscopy imaging is crucial for exploring the structure of cells and tissues. The development of block face scanning electron microscopy methods and their ability to capture large image stacks, some with near isotropic voxels, is proving particularly useful for the exploration of brain tissue. This has led to the creation of numerous algorithms and software for segmenting out different features from the image stacks. However, there are few tools available to view these results and make detailed morphometric analyses on all, or part, of these 3D models. We have addressed this issue by constructing a collection of software tools, called NeuroMorph, with which users can view the segmentation results, in conjunction with the original image stack, manipulate these objects in 3D, and make measurements of any region. This approach to collecting morphometric data provides a faster means of analysing the geometry of structures, such as dendritic spines and axonal boutons. This bridges the gap that currently exists between rapid reconstruction techniques, offered by computer vision research, and the need to collect measurements of shape and form from segmented structures that is currently done using manual segmentation methods.

Automated workflows for the morphological characterization of organelles in electron microscopy image stacks (LB72)

Advances in three‐dimensional electron microscopy (EM) have facilitated the collection of image stacks with a field‐of‐view that is large enough to cover a significant percentage of anatomical subdivisions at nano‐resolution. When coupled with enhanced staining protocols, such techniques produce data that can be mined to establish the morphologies of all organelles across hundreds of whole cells in their in situ environments. Although instrument throughputs are approaching terabytes of data per day, image segmentation and analysis remain significant bottlenecks in achieving quantitative descriptions of whole cell organellomes. Here we describe computational workflows that achieve the automatic segmentation of organelles from regions of the central nervous system by applying supervised machine learning algorithms to slices of serial block‐face scanning EM (SBEM) datasets. We also demonstrate that our workflows can be parallelized on supercomputing resources, resulting in a dramatic reduction of their run times. These methods significantly expedite the development of anatomical models at the subcellular scale and facilitate the study of how these models may be perturbed following pathological insults.Grant Funding Source: Supported by NIGMS under awards number P41 GM103412‐25 and P41 GM103426, the NINDS under award number 1R01NS07531, and NIDA under award number 5T32DA007315‐10.

Automated Detection of Synapses in Serial Section Transmission Electron Microscopy Image Stacks

We describe a method for fully automated detection of chemical synapses in serial electron microscopy images with highly anisotropic axial and lateral resolution, such as images taken on transmission electron microscopes. Our pipeline starts from classification of the pixels based on 3D pixel features, which is followed by segmentation with an Ising model MRF and another classification step, based on object-level features. Classifiers are learned on sparse user labels; a fully annotated data subvolume is not required for training. The algorithm was validated on a set of 238 synapses in 20 serial 7197×7351 pixel images (4.5×4.5×45 nm resolution) of mouse visual cortex, manually labeled by three independent human annotators and additionally re-verified by an expert neuroscientist. The error rate of the algorithm (12% false negative, 7% false positive detections) is better than state-of-the-art, even though, unlike the state-of-the-art method, our algorithm does not require a prior segmentation of the image volume into cells. The software is based on the ilastik learning and segmentation toolkit and the vigra image processing library and is freely available on our website, along with the test data and gold standard annotations (http://www.ilastik.org/synapse-detection/sstem).

WATERSHED MERGE FOREST CLASSIFICATION FOR ELECTRON MICROSCOPY IMAGE STACK SEGMENTATION.

Automated electron microscopy (EM) image analysis techniques can be tremendously helpful for connectomics research. In this paper, we extend our previous work [1] and propose a fully automatic method to utilize inter-section information for intra-section neuron segmentation of EM image stacks. A watershed merge forest is built via the watershed transform with each tree representing the region merging hierarchy of one 2D section in the stack. A section classifier is learned to identify the most likely region correspondence between adjacent sections. The inter-section information from such correspondence is incorporated to update the potentials of tree nodes. We resolve the merge forest using these potentials together with consistency constraints to acquire the final segmentation of the whole stack. We demonstrate that our method leads to notable segmentation accuracy improvement by experimenting with two types of EM image data sets.

The Open Connectome Project Data Cluster: Scalable Analysis and Vision for High-Throughput Neuroscience.

We describe a scalable database cluster for the spatial analysis and annotation of high-throughput brain imaging data, initially for 3-d electron microscopy image stacks, but for time-series and multi-channel data as well. The system was designed primarily for workloads that build connectomes- neural connectivity maps of the brain-using the parallel execution of computer vision algorithms on high-performance compute clusters. These services and open-science data sets are publicly available at openconnecto.me. The system design inherits much from NoSQL scale-out and data-intensive computing architectures. We distribute data to cluster nodes by partitioning a spatial index. We direct I/O to different systems-reads to parallel disk arrays and writes to solid-state storage-to avoid I/O interference and maximize throughput. All programming interfaces are RESTful Web services, which are simple and stateless, improving scalability and usability. We include a performance evaluation of the production system, highlighting the effec-tiveness of spatial data organization.

Supervoxel-Based Segmentation of Mitochondria in EM Image Stacks With Learned Shape Features

It is becoming increasingly clear that mitochondria play an important role in neural function. Recent studies show mitochondrial morphology to be crucial to cellular physiology and synaptic function and a link between mitochondrial defects and neuro-degenerative diseases is strongly suspected. Electron microscopy (EM), with its very high resolution in all three directions, is one of the key tools to look more closely into these issues but the huge amounts of data it produces make automated analysis necessary. State-of-the-art computer vision algorithms designed to operate on natural 2-D images tend to perform poorly when applied to EM data for a number of reasons. First, the sheer size of a typical EM volume renders most modern segmentation schemes intractable. Furthermore, most approaches ignore important shape cues, relying only on local statistics that easily become confused when confronted with noise and textures inherent in the data. Finally, the conventional assumption that strong image gradients always correspond to object boundaries is violated by the clutter of distracting membranes. In this work, we propose an automated graph partitioning scheme that addresses these issues. It reduces the computational complexity by operating on supervoxels instead of voxels, incorporates shape features capable of describing the 3-D shape of the target objects, and learns to recognize the distinctive appearance of true boundaries. Our experiments demonstrate that our approach is able to segment mitochondria at a performance level close to that of a human annotator, and outperforms a state-of-the-art 3-D segmentation technique.