Shape Priors Research Articles

Semantically Coherent 4D Scene Flow of Dynamic Scenes

Simultaneous semantically coherent object-based long-term 4D scene flow estimation, co-segmentation and reconstruction is proposed exploiting the coherence in semantic class labels both spatially, between views at a single time instant, and temporally, between widely spaced time instants of dynamic objects with similar shape and appearance. In this paper we propose a framework for spatially and temporally coherent semantic 4D scene flow of general dynamic scenes from multiple view videos captured with a network of static or moving cameras. Semantic coherence results in improved 4D scene flow estimation, segmentation and reconstruction for complex dynamic scenes. Semantic tracklets are introduced to robustly initialize the scene flow in the joint estimation and enforce temporal coherence in 4D flow, semantic labelling and reconstruction between widely spaced instances of dynamic objects. Tracklets of dynamic objects enable unsupervised learning of long-term flow, appearance and shape priors that are exploited in semantically coherent 4D scene flow estimation, co-segmentation and reconstruction. Comprehensive performance evaluation against state-of-the-art techniques on challenging indoor and outdoor sequences with hand-held moving cameras shows improved accuracy in 4D scene flow, segmentation, temporally coherent semantic labelling, and reconstruction of dynamic scenes.

Automatic prior shape selection for image edge detection with modified Mumford–Shah model

Edge detection plays an important role in the field of image processing. In this paper, we propose a novel variational model to automatically and adaptively detect one or more prior shapes from the given dictionary to guide the edge detection process. In that way, we can effectively detect the shapes of interest from the test image. Moreover, an efficient algorithm based on the Alternating Direction Method of Multipliers (ADMM) is proposed to solve this model with guaranteed convergence. A variety of numerical experiments show that the proposed method has achieved ideal performance for edge detection in images with missing information, various types of noise and complicated background, and even multiple objects.

Deep learning based topology guaranteed surface and MME segmentation of multiple sclerosis subjects from retinal OCT.

Optical coherence tomography (OCT) is a noninvasive imaging modality that can be used to obtain depth images of the retina. Patients with multiple sclerosis (MS) have thinning retinal nerve fiber and ganglion cell layers, and approximately 5% of MS patients will develop microcystic macular edema (MME) within the retina. Segmentation of both the retinal layers and MME can provide important information to help monitor MS progression. Graph-based segmentation with machine learning preprocessing is the leading method for retinal layer segmentation, providing accurate surface delineations with the correct topological ordering. However, graph methods are time-consuming and they do not optimally incorporate joint MME segmentation. This paper presents a deep network that extracts continuous, smooth, and topology-guaranteed surfaces and MMEs. The network learns shape priors automatically during training rather than being hard-coded as in graph methods. In this new approach, retinal surfaces and MMEs are segmented together with two cascaded deep networks in a single feed forward propagation. The proposed framework obtains retinal surfaces (separating the layers) with sub-pixel surface accuracy comparable to the best existing graph methods and MMEs with better accuracy than the state-of-the-art method. The full segmentation operation takes only ten seconds for a 3D volume.

Wavefront shaping in complex media with a 350 kHz modulator via a 1D-to-2D transform

Controlling the propagation and interaction of light in complex media has sparked major interest in the last few years. Unfortunately, spatial light modulation devices suffer from limited speed that precludes real-time applications such as imaging in live tissue. To address this critical problem we introduce a phase-control technique to characterize complex media based on the use of fast 1D spatial light modulators and a 1D-to-2D transformation performed by the same medium being analyzed. We implement the concept using a micro-electro-mechanical grating light valve (GLV) with 1088 degrees of freedom modulated at 350 KHz, enabling unprecedented high-speed wavefront measurements. We continuously measure the transmission matrix, calculate the optimal wavefront and project a focus through various dynamic scattering samples in real-time, all within 2.4 ms per cycle. These results improve prior wavefront shaping modulation speed by more than an order of magnitude and open new opportunities for optical processing using 1D-to-2D transformations.

The role perceptions of government professionals: the effects of gender, educational field, and prior job sector

ABSTRACT The public service motivation (PSM) and the call to serve literatures suggest certain individuals are called to public service. At the senior executive level, government professionals may be called to provide a voice for societal interests or to develop new policy agendas. However, we know little about what influences the role perceptions of high-level government professionals. To understand how senior executives perceive their own role in government, we examine survey data across 19 European countries. We find that gender, educational background, and prior job sector socialization shape role perceptions concerning policymaking and providing a voice for society.

Vascular segmentation of neuroimages based on a prior shape and local statistics

Fast and accurate extraction of vascular structures from medical images is fundamental for many clinical procedures. However, most of the vessel segmentation techniques ignore the existence of the isolated and redundant points in the segmentation results. In this study, we propose a vascular segmentation method based on a prior shape and local statistics. It could efficiently eliminate outliers and accurately segment thick and thin vessels. First, an improved vesselness filter is defined. This quantifies the likelihood of each voxel belonging to a bright tubular-shaped structure. A matching and connection process is then performed to obtain a blood-vessel mask. Finally, the region-growing method based on local statistics is implemented on the vessel mask to obtain the whole vascular tree without outliers. Experiments and comparisons with Frangi’s and Yang’s models on real magnetic-resonance-angiography images demonstrate that the proposed method can remove outliers while preserving the connectivity of vessel branches.

Attitude Estimation and Geometry Reconstruction of Satellite Targets Based on ISAR Image Sequence Interpretation

Analysis of the attitude and geometry of space targets with the inverse synthetic aperture radar (ISAR) technique is a significant and difficult task. Most of the existing methods hardly consider the radar observation geometry in the determination of target attitude. This paper proposes a novel approach to estimating the three-dimensional attitude and reconstructing typical component geometry of space targets from an ISAR image sequence. The approach bridges range-Doppler images and target attitude parameters with the accommodation of target trajectory information and the ISAR geometric projection model. By exploring the shape feature within the ISAR sequence, the target attitude and the rectangular component size are estimated through solving an optimization with prior shape constraints. Comparative experiments illustrate the advantages of the proposed method in both feature association and reconstruction feasibility. Moreover, considering practical circumstances, a further analysis is made of the robustness of the proposed algorithm after the attitude estimation experiment.

Subpixel mapping of urban built-up areas using spatial-spectral information from satellite multispectral remote sensing imagery

ABSTRACTA novel method for subpixel mapping of urban built-up areas (SMUBA) using spatial-spectral information from satellite multispectral remote sensing imagery is proposed. SMUBA contains two terms, a spatial term and a spectral term. First, a spatial attraction model is used to produce the spatial term. The Normalized difference built-up index is then utilized to obtain the spectral term. Finally, a particle swarm optimization algorithm is used to optimize the two terms and obtain the final mapping result. Since the spatial-spectral information of the urban built-up areas from multispectral imagery is fully utilized by the two terms, the final mapping result is improved. Experimental results on two Landsat 8 Operational Land Imager data show that SMUBA produces better mapping results than state-of-the-art subpixel mapping methods. Moreover, the proposed method does not need any prior shape information.

Mono3D++: Monocular 3D Vehicle Detection with Two-Scale 3D Hypotheses and Task Priors

We present a method to infer 3D pose and shape of vehicles from a single image. To tackle this ill-posed problem, we optimize two-scale projection consistency between the generated 3D hypotheses and their 2D pseudo-measurements. Specifically, we use a morphable wireframe model to generate a fine-scaled representation of vehicle shape and pose. To reduce its sensitivity to 2D landmarks, we jointly model the 3D bounding box as a coarse representation which improves robustness. We also integrate three task priors, including unsupervised monocular depth, a ground plane constraint as well as vehicle shape priors, with forward projection errors into an overall energy function.

Accurate and robust deep learning-based segmentation of the prostate clinical target volume in ultrasound images.

The goal of this work was to develop a method for accurate and robust automatic segmentation of the prostate clinical target volume in transrectal ultrasound (TRUS) images for brachytherapy. These images can be difficult to segment because of weak or insufficient landmarks or strong artifacts. We devise a method, based on convolutional neural networks (CNNs), that produces accurate segmentations on easy and difficult images alike. We propose two strategies to achieve improved segmentation accuracy on difficult images. First, for CNN training we adopt an adaptive sampling strategy, whereby the training process is encouraged to pay more attention to images that are difficult to segment. Secondly, we train a CNN ensemble and use the disagreement among this ensemble to identify uncertain segmentations and to estimate a segmentation uncertainty map. We improve uncertain segmentations by utilizing the prior shape information in the form of a statistical shape model. Our method achieves Hausdorff distance of 2.7 ± 2.3 mm and Dice score of 93.9 ± 3.5%. Comparisons with several competing methods show that our method achieves significantly better results and reduces the likelihood of committing large segmentation errors. Furthermore, our experiments show that our approach to estimating segmentation uncertainty is better than or on par with recent methods for estimation of prediction uncertainty in deep learning models. Our study demonstrates that estimation of model uncertainty and use of prior shape information can significantly improve the performance of CNN-based medical image segmentation methods, especially on difficult images.

Locality-constrained framework for face alignment

Although the conventional active appearance model (AAM) has achieved some success for face alignment, it still suffers from the generalization problem when be applied to unseen subjects and images. To deal with the generalization problem of AAM, we first reformulate the original AAM as sparsity-regularized AAM, which can achieve more compact/better shape and appearance priors by selecting nearest neighbors as the bases of the shape and appearance model. To speed up the fitting procedure, the sparsity in sparsity-regularized AAM is approximated by using the locality (i.e., K-nearest neighbor), and thus inducing the locality-constrained active appearance-model (LC-AAM). The LC-AAM solves a constrained AAM-like fitting problem with the K-nearest neighbors as the bases of shape and appearance model. To alleviate the adverse influence of inaccurate K-nearest neighbor results, the locality constraint is further embedded in the discriminative fitting method denoted as LC-DFM, which can find better K-nearest neighbor results by employing shape-indexed feature, and can also tolerate some inaccurate neighbors benefited from the regression model rather than the generative model in AAM. Extensive experiments on several datasets demonstrate that our methods outperform the state-of-the-arts in both detection accuracy and generalization ability.

Pseudo-Marginal MCMC Sampling for Image Segmentation Using Nonparametric Shape Priors.

Segmenting images of low quality or with missing data is a challenging problem. In such scenarios, exploiting statistical prior information about the shapes to be segmented can improve the segmentation results significantly. Incorporating prior density of shapes into a Bayesian framework leads to the posterior density of segmenting shapes given the observed data. Most segmentation algorithms that exploit shape priors optimize a cost function based on the posterior density and find a point estimate (e.g., using maximum a posteriori estimation). However, especially when the prior shape density is multimodal leading to a multimodal posterior density, a point estimate does not provide a measure of the degree of confidence in that result, neither does it provide a picture of other probable solutions based on the observed data and the shape priors. With a statistical view, addressing these issues would involve the problem of characterizing the posterior distributions of the shapes of the objects to be segmented. An analytic computation of such posterior distributions is intractable; however, characterization is still possible through their samples. In this paper, we propose an efficient pseudo-marginal Markov chain Monte Carlo (MCMC) sampling approach to draw samples from posterior shape distributions for image segmentation. The computation time of the proposed approach is independent from the training set size. Therefore, it scales well for very large data sets. In addition to better characterization of the statistical structure of the problem, such an approach has the potential to address issues with getting stuck at local optima, suffered by existing shape-based segmentation methods. Our approach is able to characterize the posterior probability density in the space of shapes through its samples, and to return multiple solutions, potentially from different modes of a multimodal probability density, which would be encountered, e.g., in segmenting objects from multiple shape classes. We present promising results on a variety of synthetic and real data sets.

Single-image de-raining using low-rank matrix approximation

Most existing image de-raining methods are based on the assumption that rain has certain shape or chromatic priors, which may be inaccurate for some rain cases and may lead to unsatisfactory de-rained results. In this paper, the proposed image de-raining method is built on the priors of clear images that many self-similar patches exist in natural images. And similar patches satisfy the low-rank property when grouped together. This property has led to many powerful image/video denoising schemes with impressive performance but has never been used for single-image de-raining. We formulate the problem of removing rain as a low-rank matrix approximation problem by adopting this self-similarity image prior with a designed patch matching strategy applicable for rain images. The resulting nuclear norm minimization problem can be efficiently solved by many recently developed methods. Experiments on both real rain images and synthetic rain images show that the proposed method is competitive against other state-of-the-art methods.

Multi-atlas active contour segmentation method using template optimization algorithm

BackgroundBrain image segmentation is the basis and key to brain disease diagnosis, treatment planning and tissue 3D reconstruction. The accuracy of segmentation directly affects the therapeutic effect. Manual segmentation of these images is time-consuming and subjective. Therefore, it is important to research semi-automatic and automatic image segmentation methods. In this paper, we propose a semi-automatic image segmentation method combined with a multi-atlas registration method and an active contour model (ACM).MethodWe propose a multi-atlas active contour segmentation method using a template optimization algorithm. First, a multi-atlas registration method is used to obtain the prior shape information of the target tissue, and then a label fusion algorithm is used to generate the initial template. Second, a template optimization algorithm is used to reduce the multi-atlas registration errors and generate the initial active contour (IAC). Finally, a ACM is used to segment the target tissue.ResultsThe proposed method was applied to the challenging publicly available MR datasets IBSR and MRBrainS13. In the MRBrainS13 datasets, we obtained an average thalamus Dice similarity coefficient of 0.927 ± 0.014 and an average Hausdorff distance (HD) of 2.92 ± 0.53. In the IBSR datasets, we obtained a white matter (WM) average Dice similarity coefficient of 0.827 ± 0.04 and a gray gray matter (GM) average Dice similarity coefficient of 0.853 ± 0.03.ConclusionIn this paper, we propose a semi-automatic brain image segmentation method. The main contributions of this paper are as follows: 1) Our method uses a multi-atlas registration method based on affine transformation, which effectively reduces the multi-atlas registration time compared to the complex nonlinear registration method. The average registration time of each target image in the IBSR datasets is 255 s, and the average registration time of each target image in the MRBrainS13 datasets is 409 s. 2) We used a template optimization algorithm to improve registration error and generate a continuous IAC. 3) Finally, we used a ACM to segment the target tissue and obtain a smooth continuous target contour.

3D Shape Synthesis via Content–Style Revealing Priors

To increase the diversity of created shapes conveniently, we describe a fully automatic synthesis framework based on the content–style analysis of an input set of 3D shapes. To fully characterize the content and style of the input set and enable 3D shape synthesis, we need a set of elements that capture a more complete characterization of the content and the style, which is precisely defined as the set of content–style revealing priors. First, the 3D shape priors extracted from the input collection of shapes are used to compute the global shape descriptors of each shape, and then cluster them to form the different content classes of the input shapes. We pose our search as a content–style analysis problem, where the separation between content and style would lead us to the revealing priors we seek. To this end, we introduce the prior-shape matrix (PSM) which measures how far a specific prior (row) appears in a specific shape (column). This allows us to classify the 3D sparse priors into content and style revealing priors. The results of the content–style separation phase are used to automatically synthesize novel shapes by transferring the style parts of the input shapes to their content parts while maintaining their functionality. Extensive experiments on several datasets of man-made 3D shapes show the identification of the content–style revealing priors and exhibit the power of our approach in synthesizing plausible shapes based on the separated styles and contents. In addition, our method enables a variety of applications such as content–style aware shape retrieval and stylistic suggestions for scene modeling, concluding that the revealing priors are a concrete description of how to grant a specific content or a style to an object.

Segmentation of Overlapping Cytoplasm in Cervical Smear Images via Adaptive Shape Priors Extracted From Contour Fragments

We present a novel approach for segmenting overlapping cytoplasm of cells in cervical smear images by leveraging the adaptive shape priors extracted from cytoplasm's contour fragments and shape statistics. The main challenge of this task is that many occluded boundaries in cytoplasm clumps are extremely difficult to be identified and, sometimes, even visually indistinguishable. Given a clump where multiple cytoplasms overlap, our method starts by cutting its contour into a set of contour fragments. We then locate the corresponding contour fragments of each cytoplasm by a grouping process. For each cytoplasm, according to the grouped fragments and a set of known shape references, we construct its shape and, then, connect the fragments to form a closed contour as the segmentation result, which is explicitly constrained by the constructed shape. We further integrate the intensity and curvature information, which is complementary to the shape priors extracted from contour fragments, into our framework to improve the segmentation accuracy. We propose to iteratively conduct fragments grouping, shape constructing, and fragments connecting for progressively refining the shape priors and improving the segmentation results. We extensively evaluate the effectiveness of our method on two typical cervical smear datasets. The experimental results demonstrate that our approach is highly effective and consistently outperforms the state-of-the-art approaches. The proposed method is general enough to be applied to other similar microscopic image segmentation tasks, where heavily overlapped objects exist.

Effect of hot deformation parameters on flow stress and microstructure in a low carbon microalloyed steel

Hot deformation behavior of low carbon microalloyed steel was investigated by single-pass compression tests using a thermomechanical simulator. After the specimens were subjected to different hot deformation processes, their microstructures were obtained by constant cooling rate and then examined using optical microscopy and electron backscattering diffraction (EBSD) technique. The results showed that the absence of stress peak in the flow stress curves does not necessarily indicate the absence of dynamic recrystallization (DRX) because in some cases the partial DRX phenomenon does occur when the stress peaks are absent in the flow curves. The metallurgical events that occur during hot deformation directly influence the nature of phase transformation products. High dislocation density in the deformed austenite can promote the intragranular nucleation of acicular ferrite, while the DRX austenite grains prefer to form lath bainite/martensite. The EBSD results confirmed that the microstructures that were transformed from prior pancaked austenite grains always contain higher fraction of middle misorientation angle boundaries and larger average local strain as compared to those with prior DRX grain shape.

Directional fast-marching and multi-model strategy to extract coronary artery centerlines

BackgroundComputed tomography angiography (CTA) is a non-invasive technique to image coronary arteries and evaluate coronary artery diseases (CAD). The diagnosis of CAD requires modeling anatomical structures and analyzing the function and pathology of the coronary arteries. Therefore, a robust and automated method for extracting reliable coronary artery centerlines is valuable in clinical practice. MethodWe extracted coronary centerlines using the directional fast marching (DFM) method and improved DFM with a multi-model strategy. The method comprises model guidance, the application of vessel direction, and a multi-model strategy: (1) coronary models are constructed using registration techniques and then used as prior knowledge of the vessels; (2) the vessel direction, modified from the eigenvectors of the Hessian matrix and vesselness, is used to guide the search for the vessel points during fast marching; and (3) the multi-model strategy is applied to identify suboptimal results from the overall outcome as in multi-atlas segmentation. Overlap and accuracy metrics are used to assess the segmentation. The authors evaluated the performance of the proposed method on 32 CT cardiac angiography datasets from the Rotterdam Coronary Artery Algorithm Evaluation Framework (RCAAEF). The authors also studied the effect of models on DFM. ResultsFor the quantitative evaluation, DFM improved the average overlap (OV) from 43.6% of a method without model information to 77.8%. In addition, with the ground truth delineated by experts, multi-model DFM (MM-DFM) obtained 83.5% average overlap (OV) in the training datasets and 86.6% in the test datasets. ConclusionThe authors propose a novel approach to extract coronary centerlines from CTA using DFM and further extend DFM to a multi-model strategy. DFM effectively applies the prior shape of the coronary vessels and vascular features within the target image and has the potential to achieve clinically relevant results.

PedX: Benchmark Dataset for Metric 3-D Pose Estimation of Pedestrians in Complex Urban Intersections

This paper presents a novel dataset titled PedX, a large-scale multimodal collection of pedestrians at complex urban intersections. PedX consists of more than 5,000 pairs of high-resolution (12MP) stereo images and LiDAR data along with providing 2D and 3D labels of pedestrians. We also present a novel 3D model fitting algorithm for automatic 3D labeling harnessing constraints across different modalities and novel shape and temporal priors. All annotated 3D pedestrians are localized into the real-world metric space, and the generated 3D models are validated using a mocap system configured in a controlled outdoor environment to simulate pedestrians in urban intersections. We also show that the manual 2D labels can be replaced by state-of-the-art automated labeling approaches, thereby facilitating automatic generation of large scale datasets.

TeTrIS: Template Transformer Networks for Image Segmentation With Shape Priors.

In this paper, we introduce and compare different approaches for incorporating shape prior information into neural network-based image segmentation. Specifically, we introduce the concept of template transformer networks, where a shape template is deformed to match the underlying structure of interest through an end-to-end trained spatial transformer network. This has the advantage of explicitly enforcing shape priors, and this is free of discretization artifacts by providing a soft partial volume segmentation. We also introduce a simple yet effective way of incorporating priors in the state-of-the-art pixel-wise binary classification methods such as fully convolutional networks and U-net. Here, the template shape is given as an additional input channel, incorporating this information significantly reduces false positives. We report results on synthetic data and sub-voxel segmentation of coronary lumen structures in cardiac computed tomography showing the benefit of incorporating priors in neural network-based image segmentation.