Segmentation Of Image Sequences Research Articles

Monitoring of industrial crystallization processes through image sequence segmentation and characterization

Monitoring of industrial crystallization processes through image sequence segmentation and characterization

Spatio-temporal segmentation of image sequences for non-invasive analysis of cardiovascular structure and function in Whitefish embryos

Label-free quantitative analysis of the cardiovascular structure and function in wild fish is highly important and still unsolved problem. We developed a robust algorithm for bright field microscopy data processing, which allows reliable detection and quantification of circulatory system across the whole animal. We verified this technique by in vivo experiments on broad whitefish (Coregonus nasus) embryos. Calculated morphofunctional features of heart and vessels are quite specific due to low temperature embryonic development of this fish, and correspond well to the reference data obtained by ex vivo techniques. This research highlights the feasibility of non-invasive cardiovascular imaging and measurements valuable for behavioral, environmental, toxicological, drug discovery and many other studies.

Segmentation of ultrasound image sequences by combing a novel deep siamese network with a deformable contour model

Segmentation of ultrasound image sequences by combing a novel deep siamese network with a deformable contour model

Liver segmentation in CT imaging with enhanced mask region-based convolutional neural networks.

BackgroundLiver segmentation in computed tomography (CT) imaging has been widely investigated as a crucial step for analyzing liver characteristics and diagnosing liver diseases. However, obtaining satisfactory liver segmentation performance is highly challenging because of the poor contrast between the liver and its surrounding organs and tissues, the high levels of CT image noise, and the wide variability in liver shapes among patients.MethodsTo overcome these challenges, we propose a novel method for liver segmentation in CT image sequences. This method uses an enhanced mask region-based convolutional neural network (Mask R-CNN) with graph-cut segmentation. Specifically, the k-nearest neighbor (k-NN) algorithm is employed to cluster the target liver pixels in order to get an appropriate aspect ratio. Then, anchors are adapted to the liver size using the ratio information. Thus, high-accuracy liver localization can be achieved using the anchors and rotation-invariant object recognition. Next, a fully convolutional network (FCN) is used to segment the foreground objects, and local fine-grained liver detection is realized by pixel prediction. Finally, a whole liver mask is obtained by Mask R-CNN proposed in this paper.ResultsWe proposed a Mask R-CNN algorithm which achieved superior performance in comparison with the conventional Mask R-CNN algorithms in term of the dice similarity coefficient (DSC), and the Medical Image Computing and Computer-Assisted Intervention (MICCAI) metrics.ConclusionsOur experimental results demonstrate that the improved Mask R-CNN architecture has good performance, accuracy, and robustness for liver segmentation in CT image sequences.

Study on strategy of CT image sequence segmentation for liver and tumor based on U-Net and Bi-ConvLSTM

Accurate segmentation of the liver and tumors in computed tomography (CT) images is critical for intelligent computer-aided diagnosis (CAD). The commonly used segmentation methods based on fully convolutional networks (FCN) only take a single image into consideration but do not make good use of sequence information. In this paper, two more feasible sequence segmentation strategies than 3D U-net which can utilize inter-slice and intra-slice features simultaneously at the lower hardware and time cost are studied to improve the segmentation result. U-net serves as the backbone model of segmentation and Bi-directional convolutional long short-term memory (Bi-ConvLSTM) is chosen to extract and fuse the inter-slice feature. Strategy A corrects the pre-segmented results of U-net in the fusion of sequence information as a post-processing, where Mod-1, Mod-2 and Mod-3 models are built to compare the effects of width, depth, and residual structure on the modified model of sequence segmentation. Strategy B directly integrates the fusion of sequence information into the feature extraction of U-net, and then an end-to-end model called W-net is built based on it. The experiment results show that both strategies improve the liver and tumor segmentation performance in various aspects. The results based on strategy A are closer to the ground truth with less misdiagnose region: Mod-1 achieves better accuracy on liver contour segmentation because of the largest model width; Mod-2 can obtain more accurate tumor contour since the greatest depth of feature extraction process; and Mod-3 is at the average segmentation performance. Therefore, strategy A is recommended in the application of surgery planning of tumors. Strategy B achieves better space coincidence degree and less training time cost, which is more suitable for the early screening of liver cancer.

Mechanics informed fluoroscopy of esophageal transport.

Fluoroscopy is a radiographic procedure for evaluating esophageal disorders such as achalasia, dysphasia and gastroesophageal reflux disease. It performs dynamic imaging of the swallowing process and provides anatomical detail and a qualitative idea of how well swallowed fluid is transported through the esophagus. In this work, we present a method called mechanics informed fluoroscopy (FluoroMech) that derives patient-specific quantitative information about esophageal function. FluoroMech uses a convolutional neural network to perform segmentation of image sequences generated from the fluoroscopy, and the segmented images become input to a one-dimensional model that predicts the flow rate and pressure distribution in fluid transported through the esophagus. We have extended this model to identify and estimate potential physiomarkers such as esophageal wall stiffness and active relaxation ahead of the peristaltic wave in the esophageal musculature. FluoroMech requires minimal computational time and hence can potentially be applied clinically in the diagnosis of esophageal disorders.

Symmetric Algorithm for Direction Tracking of Dribble Directional Shooting

Tracking the trajectory of dribble directional shooting in the video image is beneficial to improve the basketball training skills. In this paper, the trajectory tracking method of dribble directional shooting based on the symmetrical algorithm is proposed. The symmetric differential algorithm is used to segment the video image of the dribble directional shooting. In the segmented image sequence, the range of motion of the dribble directional shooting is detected every three consecutive frames. Through the maximum variance ratio threshold method, the motion area of the dribble directional shot is divided. The initial motion coordinates and feature vectors of the dribble directional shooting are calculated, and the characteristic affine transformation equation of the shooting action pixel information is constructed. The Camshift method is used to mark the trajectory contour of the dribble directional shooting. According to the result of the contour extraction, the gray pixel value feature points of the video sequence are fitted, and the trajectory tracking model of the dribble directional shooting is established to accurately track the trajectory of the dribble directional shooting. It is verified that the trajectory of the dribble directional shooting tracked by the proposed method is consistent with the actual situation, and the tracking error average is about 1.9%, that is the reliable trajectory tracking method of dribble directional shooting.

In vitro Diagnosis of Exercise Related Skin Trauma based on Spatiotemporal Image Segmentation

In vitro diagnosis is a kind of product and service that can get clinical diagnosis information by detecting human samples (blood, body fluid, tissue, etc.) outside human body and then judge disease or body’s functioning. This paper proposes an in vitro diagnosis method based on spatiotemporal image segmentation. Firstly, the image sequence segmentation method based on spatiotemporal Markov random field is used to segment computed tomography image, then the computed tomography image of exercise related skin trauma is segmented. According to the segmented computed tomography image, the in vitro diagnosis of exercise-related skin trauma is realized by the detection method based on the superpixel spatiotemporal characteristics. The results show that the proposed method can effectively detect skin trauma in vitro and has good segmentation effect on skin computed tomography image, as well as high accuracy in trauma detection. It can also be used in physical examination, chronic disease management and severe disease monitoring. Compared with similar detection methods, this method has significant application value.

Learning Event Representations for Temporal Segmentation of Image Sequences by Dynamic Graph Embedding.

Recently, self-supervised learning has proved to be effective to learn representations of events suitable for temporal segmentation in image sequences, where events are understood as sets of temporally adjacent images that are semantically perceived as a whole. However, although this approach does not require expensive manual annotations, it is data hungry and suffers from domain adaptation problems. As an alternative, in this work, we propose a novel approach for learning event representations named Dynamic Graph Embedding (DGE). The assumption underlying our model is that a sequence of images can be represented by a graph that encodes both semantic and temporal similarity. The key novelty of DGE is to learn jointly the graph and its graph embedding. At its core, DGE works by iterating over two steps: 1) updating the graph representing the semantic and temporal similarity of the data based on the current data representation, and 2) updating the data representation to take into account the current data graph structure. The main advantage of DGE over state-of-the-art self-supervised approaches is that it does not require any training set, but instead learns iteratively from the data itself a low-dimensional embedding that reflects their temporal and semantic similarity. Experimental results on two benchmark datasets of real image sequences captured at regular time intervals demonstrate that the proposed DGE leads to event representations effective for temporal segmentation. In particular, it achieves robust temporal segmentation on the EDUBSeg and EDUBSeg-Desc benchmark datasets, outperforming the state of the art. Additional experiments on two Human Motion Segmentation benchmark datasets demonstrate the generalization capabilities of the proposed DGE.

Reliable segmentation of 2D cardiac magnetic resonance perfusion image sequences using time as the 3rd dimension.

Cardiac magnetic resonance (CMR) first-pass perfusion is an established noninvasive diagnostic imaging modality for detecting myocardial ischemia. A CMR perfusion sequence provides a time series of 2D images for dynamic contrast enhancement of the heart. Accurate myocardial segmentation of the perfusion images is essential for quantitative analysis and it can facilitate automated pixel-wise myocardial perfusion quantification. In this study, we compared different deep learning methodologies for CMR perfusion image segmentation. We evaluated the performance of several image segmentation methods using convolutional neural networks, such as the U-Net in 2D and 3D (2D plus time) implementations, with and without additional motion correction image processing step. We also present a modified U-Net architecture with a novel type of temporal pooling layer which results in improved performance. The best DICE scores were 0.86 and 0.90 for LV myocardium and LV cavity, while the best Hausdorff distances were 2.3 and 2.1 pixels for LV myocardium and LV cavity using 5-fold cross-validation. The methods were corroborated in a second independent test set of 20 patients with similar performance (best DICE scores 0.84 for LV myocardium). Our results showed that the LV myocardial segmentation of CMR perfusion images is best performed using a combination of motion correction and 3D convolutional networks which significantly outperformed all tested 2D approaches. Reliable frame-by-frame segmentation will facilitate new and improved quantification methods for CMR perfusion imaging. • Reliable segmentation of the myocardium offers the potential to perform pixel level perfusion assessment. • A deep learning approach in combination with motion correction, 3D (2D + time) methods, and a deep temporal connection module produced reliable segmentation results.

Pixel-Wise Motion Segmentation for SLAM in Dynamic Environments

Visual simultaneous localization and mapping (SLAM) is a key prerequisite for many mobile robotic systems. A common assumption for SLAM methods is a static environment. The interference of dynamic objects can lead to impairment of the camera pose tracking and permanent distortions of the map. This limits the use of many visual SLAM systems in real world scenarios, where dynamic environments are typical. We present a novel method for pixel-wise segmentation of dynamic image sequences based on a scene flow model estimation. We detect and eliminate outlying pixels sparsely by evaluating each pixel motion separately and maintain the most possible area of static scene background for SLAM. The evaluation with the public TUM dataset demonstrates that our proposed method outperforms other comparable state-of-the-art approaches for dynamic removal for SLAM systems.

Weakly supervised segmentation for real-time surgical tool tracking.

Surgical tool tracking has a variety of applications in different surgical scenarios. Electromagnetic (EM) tracking can be utilised for tool tracking, but the accuracy is often limited by magnetic interference. Vision-based methods have also been suggested; however, tracking robustness is limited by specular reflection, occlusions, and blurriness observed in the endoscopic image. Recently, deep learning-based methods have shown competitive performance on segmentation and tracking of surgical tools. The main bottleneck of these methods lies in acquiring a sufficient amount of pixel-wise, annotated training data, which demands substantial labour costs. To tackle this issue, the authors propose a weakly supervised method for surgical tool segmentation and tracking based on hybrid sensor systems. They first generate semantic labellings using EM tracking and laparoscopic image processing concurrently. They then train a light-weight deep segmentation network to obtain a binary segmentation mask that enables tool tracking. To the authors’ knowledge, the proposed method is the first to integrate EM tracking and laparoscopic image processing for generation of training labels. They demonstrate that their framework achieves accurate, automatic tool segmentation (i.e. without any manual labelling of the surgical tool to be tracked) and robust tool tracking in laparoscopic image sequences.

Automatic Left Ventricle Recognition, Segmentation and Tracking in Cardiac Ultrasound Image Sequences

In this study, we propose a novel method incorporating faster region-based convolutional neural network and active shape model to automatically recognize, segment, and track the left ventricle in cardiac ultrasound image sequences, respectively. Ultrasound images typically contain noise and artifacts. The conventional filters cannot preserve the edges of image contours, and thus blurry images are often obtained. In this study, we propose an improved adaptive anisotropic diffusion filter to effectively reduce noise and reinforce image contours. In addition, because of the shape and appearance of the left ventricle vary considerably between adjacent images, conventional methods cannot automatically identify the position of the left ventricle or accurately segment them. A novel method that combines the faster region-based convolutional neural network with the active shape model is proposed to automatically recognize, segment, and track the left ventricle in cardiac ultrasound image sequences. Compared with four state-of-the-art approaches, the method proposed in this study can be applied to accurately segment and track the left ventricle in cardiac ultrasound image sequences. The proposed method produces the most satisfactory results in terms of visual presentation and segmentation quality based on four criteria.

Hierarchical segmentation using equivalence test (HiSET): Application to DCE image sequences.

Dynamical contrast enhanced (DCE) imaging allows non invasive access to tissue micro-vascularization. It appears as a promising tool to build imaging biomarkers for diagnostic, prognosis or anti-angiogenesis treatment monitoring of cancer. However, quantitative analysis of DCE image sequences suffers from low signal to noise ratio (SNR). SNR may be improved by averaging functional information in a large region of interest when it is functionally homogeneous. We propose a novel method for automatic segmentation of DCE image sequences into functionally homogeneous regions, called DCE-HiSET. Using an observation model which depends on one parameter a and is justified a posteriori, DCE-HiSET is a hierarchical clustering algorithm. It uses the p-value of a multiple equivalence test as dissimilarity measure and consists of two steps. The first exploits the spatial neighborhood structure to reduce complexity and takes advantage of the regularity of anatomical features, while the second recovers (spatially) disconnected homogeneous structures at a larger (global) scale. Given a minimal expected homogeneity discrepancy for the multiple equivalence test, both steps stop automatically by controlling the Type I error. This provides an adaptive choice for the number of clusters. Assuming that the DCE image sequence is functionally piecewise constant with signals on each piece sufficiently separated, we prove that DCE-HiSET will retrieve the exact partition with high probability as soon as the number of images in the sequence is large enough. The minimal expected homogeneity discrepancy appears as the tuning parameter controlling the size of the segmentation. DCE-HiSET has been implemented in C++ for 2D and 3D image sequences with competitive speed.

The segmentation of bones in pelvic CT images based on extraction of key frames

BackgroundBone segmentation is important in computed tomography (CT) imaging of the pelvis, which assists physicians in the early diagnosis of pelvic injury, in planning operations, and in evaluating the effects of surgical treatment. This study developed a new algorithm for the accurate, fast, and efficient segmentation of the pelvis.MethodsThe proposed method consists of two main parts: the extraction of key frames and the segmentation of pelvic CT images. Key frames were extracted based on pixel difference, mutual information and normalized correlation coefficient. In the pelvis segmentation phase, skeleton extraction from CT images and a marker-based watershed algorithm were combined to segment the pelvis. To meet the requirements of clinical application, physician’s judgment is needed. Therefore the proposed methodology is semi-automated.ResultsIn this paper, 5 sets of CT data were used to test the overlapping area, and 15 CT images were used to determine the average deviation distance. The average overlapping area of the 5 sets was greater than 94%, and the minimum average deviation distance was approximately 0.58 pixels. In addition, the key frame extraction efficiency and the running time of the proposed method were evaluated on 20 sets of CT data. For each set, approximately 13% of the images were selected as key frames, and the average processing time was approximately 2 min (the time for manual marking was not included).ConclusionsThe proposed method is able to achieve accurate, fast, and efficient segmentation of pelvic CT image sequences. Segmentation results not only provide an important reference for early diagnosis and decisions regarding surgical procedures, they also offer more accurate data for medical image registration, recognition and 3D reconstruction.

Robust Evolution Method of Active Contour Models and Application in Segmentation of Image Sequence

Active contour models are widely used in image segmentation. In order to obtain ideal object boundary, researchers utilize various information to define new models for image segmentation. However, the models could not meet all scenes of image. In this paper, we propose a block evolution method to improve the robustness of contour evolution. A block matrix is consisted of contours of former iterations and contours of shape prior, and a nuclear norm of the matrix is a measure of the similarity of these shapes. The constraint of the nuclear norm minimization is imposed on the evolution of active contour models, which could avoid large deformation of the adjacent curves and keep the shape conformability of contour in the evolution. The shape prior can be integrated into the block evolution method, which is effective in dealing with missing features of images and noise. The proposed method can be applied to image sequence segmentation. Experiments demonstrate that the proposed method improves the robust performance of active contour models and can increase the flexibility of applications in image sequence segmentation.

Automated Segmentation of Whole Cardiac CT Images based on Deep Learning

Segmentation of the whole-cardiac CT image sequence is the key to computer-aided diagnosis and study of lesions in the heart. Due to the dilation, contraction and the flow of the blood, the cardiac CT images are prone to weak boundaries and artifacts. Traditional manual segmentation methods are time-consuming and labor-intensive to produce over-segmentation. Therefore, an automatic cardiac CT image sequence segmentation technique is proposed. This technique was employed using deep learning algorithm to understand the segmentation function from the ground truth data. Using the convolution neural network (CNN) on the central location of the heart, filtering ribs, muscles and other contrasting contrast are not an obvious part of the removal of the heart area. Staked denoising auto-encoders are used to automatically deduce the contours of the heart. Therefore, nine cardiac CT image sequence datasets are used to validate the method. The results showed that the algorithm proposed in this paper has best segmentation impact to such cardiac CT images which have a complex background, the distinctness between the background and the target area which is not obvious; and the internal structure diversification. It can filter out most of the non-heart tissue part, which is more conducive to the doctor observing patient’s heart health.

Modified histogram-based segmentation and adaptive distance tracking of sperm cells image sequences

Proper recognition and tracking of microscopic sperm cells in video images are vital steps of male infertility diagnosis and treatment. The segmentation and detection of sperms in microscopic image analysis is a complicate process as a result of their small sizes, fast movements, and considerable collisions. Histogram-based thresholding schemes are very popular for this purpose, since they are quite fast and provide almost acceptable results. This paper proposes a combined method for sperm cells detection, which consists of a non-linear pre-processing stage, a histogram-based thresholding algorithm, and a tracking method based on an adaptive distance scheme. The results of conducted experiments verify the superiority of the proposed scheme with incorporated Kittler algorithm compared to the other competitive methods in the majority of cases.

A segmentation method for lung nodule image sequences based on superpixels and density-based spatial clustering of applications with noise.

The fast and accurate segmentation of lung nodule image sequences is the basis of subsequent processing and diagnostic analyses. However, previous research investigating nodule segmentation algorithms cannot entirely segment cavitary nodules, and the segmentation of juxta-vascular nodules is inaccurate and inefficient. To solve these problems, we propose a new method for the segmentation of lung nodule image sequences based on superpixels and density-based spatial clustering of applications with noise (DBSCAN). First, our method uses three-dimensional computed tomography image features of the average intensity projection combined with multi-scale dot enhancement for preprocessing. Hexagonal clustering and morphological optimized sequential linear iterative clustering (HMSLIC) for sequence image oversegmentation is then proposed to obtain superpixel blocks. The adaptive weight coefficient is then constructed to calculate the distance required between superpixels to achieve precise lung nodules positioning and to obtain the subsequent clustering starting block. Moreover, by fitting the distance and detecting the change in slope, an accurate clustering threshold is obtained. Thereafter, a fast DBSCAN superpixel sequence clustering algorithm, which is optimized by the strategy of only clustering the lung nodules and adaptive threshold, is then used to obtain lung nodule mask sequences. Finally, the lung nodule image sequences are obtained. The experimental results show that our method rapidly, completely and accurately segments various types of lung nodule image sequences.

Sparse feature competition and shapes similarity based ultrasound image sequence segmentation method

Segmentation of the object region in ultrasound image sequences plays a crucial role in many clinical applications. Active contour models are widely used in medical image segmentation. Yet, the inherent limitations of ultrasound imaging, such as its low signal-to-noise ratio or local fuzzy boundary caused by inhomogeneous distribution of intensity, have prevented the classical active contours from yielding satisfactory results. To cope with these limitations, many methods have been proposed for modeling image features or shapes prior to constraining active contours. However, most features are limited to constructing the object regions image features, and the object shapes priors usually are learnt from a large set of examples. This paper proposes a sparse feature complete and shapes similarity based method for segmentation of ultrasound image sequences. In this method, a novel contour searching strategy, named sparse feature competition, is proposed by exploiting the reconstruction errors based on both the object and the background feature dictionaries, which is used to alleviate the defects in ultrasound images. By proving that the variation in the object shape has the low-rank property in a linear space, the similarity between the object shapes in the image sequence is also exploited as a shapes prior, which can be interpreted as an unsupervised approach to the shapes prior modeling. To validate the performance of our method, sequences of clinical images were used as the training and test set. The proposed method was compared with three well-known methods on the same test set. The results demonstrate that the proposed method can consistently improve the performance of active contour models and increase the robustness against image defects; consequently, it is likely to improve the efficiency of computer-assisted therapy.