Shape Priors Research Articles

Cascaded face super‐resolution with shape and identity priors

AbstractDespite impressive progress in face super‐resolution (SR), it is an open challenge to reconstruct a reliable SR face that preserves authentic facial characteristics. Here, the problem of super‐resolving low‐resolution (LR) faces to high‐resolution (HR) ones is addressed. To tackle the ill‐posed nature of face SR, the cascaded super‐resolution network (CSRNet) is proposed to utilize shape and identity priors jointly and progressively, the first to explore multiple priors. Specifically, CSRNet adopts a cascaded structure to transform an LR face to HR face progressively via multiple stages. At each stage, CSRNet forces its output face image to match both the shape priors and identity priors extracted from the ground‐truth HR face. The shape priors estimated in one stage are merged into the inputs of its subsequent stage to provide rich information for the face SR. To generate realistic yet discriminative faces, the cascaded super‐resolution generative adversarial network (CSRGAN) is also proposed to incorporate the adversarial loss and identification loss into CSRNet. Extensive experiments on popular benchmarks show that the CSRNet and CSRGAN outperform existing face SR state‐of‐the‐art methods, both quantitatively and qualitatively, and detailed ablation studies show the advantage of this method.

MARF: The Medial Atom Ray Field object representation

We propose Medial Atom Ray Fields (MARFs), a novel neural object representation that enables accurate differentiable surface rendering with a single network evaluation per camera ray. Existing neural ray fields struggle with multi-view consistency and representing surface discontinuities. MARFs address both using a medial shape representation, a dual representation of solid geometry that yields cheap geometrically grounded surface normals, in turn enabling computing analytical curvature despite the network having no second derivative. MARFs map a camera ray to multiple medial intersection candidates, subject to ray-sphere intersection testing. We illustrate how the learned medial shape quantities applies to sub-surface scattering, part segmentation, and aid representing a space of articulated shapes. Able to learn a space of shape priors, MARFs may prove useful for tasks like shape retrieval and shape completion, among others. Code and data can be found at https://github.com/pbsds/MARF.

MCRformer: Morphological constraint reticular transformer for 3D medical image segmentation

Medical image segmentation is essential in medical image analysis since it can provide reliable assistance in computer-aided clinical diagnosis, treatment planning, and intervention. Although deep learning algorithms based on CNNs and Transformers have made notable progress in medical image segmentation, it is still challenging owing to the objects with complex structures, low discrimination and differences between individuals. To alleviate these issues, we propose a novel 3D medical image segmentation network based on Transformers and CNNs combining morphological information and reticular mechanism. Firstly, the morphological constraint stream is designed to learn the prior shape information based on the CNN model and make the attention maps focus on target tasks for enhancing the interpretability. Secondly, the Reticular Transformer is utilized to obtain multi-scale information based on the Transformer, which can bind the local texture information and underlying semantic information to further acquire the feature maps with sufficient details and receptive field. The experiments demonstrate that our proposed method outperforms many existing segmentation models in terms of the performance in metrics DSC and HD (80.46% in DSC on the Synapse dataset and 90.83% in DSC on the ACDC dataset). The code will be released at https://github.com/rocklijun/MCRformer.

Sinogram Inpainting with Generative Adversarial Networks and Shape Priors

X-ray computed tomography is a widely used, non-destructive imaging technique that computes cross-sectional images of an object from a set of X-ray absorption profiles (the so-called sinogram). The computation of the image from the sinogram is an ill-posed inverse problem, which becomes underdetermined when we are only able to collect insufficiently many X-ray measurements. We are here interested in solving X-ray tomography image reconstruction problems where we are unable to scan the object from all directions, but where we have prior information about the object’s shape. We thus propose a method that reduces image artefacts due to limited tomographic measurements by inferring missing measurements using shape priors. Our method uses a Generative Adversarial Network that combines limited acquisition data and shape information. While most existing methods focus on evenly spaced missing scanning angles, we propose an approach that infers a substantial number of consecutive missing acquisitions. We show that our method consistently improves image quality compared to images reconstructed using the previous state-of-the-art sinogram-inpainting techniques. In particular, we demonstrate a 7 dB Peak Signal-to-Noise Ratio improvement compared to other methods.

Segmentation of optic disc in retinal images for glaucoma diagnosis by saliency level set with enhanced active contour model

<p>Glaucoma is an ophthalmic disease which is among the chief causes of visual impairment across the globe. The clarity of the optic disc (OD) is crucial for recognizing glaucoma. Since existing methods are unable to successfully integrate multi-view information derived from shape and appearance to precisely explain OD for segmentation, this paper proposes a saliency-based level set with an enhanced active contour method (SL-EACM), a modified locally statistical active contour model, and entropy-based optical disc localization. The significant contributions are that i) the SL-EACM is introduced to address the often noticed problem of intensity inhomogeneity brought on by defects in imaging equipment or fluctuations in lighting; ii) to prevent the integrity of the OD structures from being compromised by pathological alterations and artery blockage, local image probability data is included from a multi-dimensional feature space around the region of interest in the model; and iii) the model incorporates prior shape information into the technique, for enhancing the accuracy in identifying the OD structures from surrounding regions. Public databases such as CHASE_DB, DRIONS-DB, and Drishti-GS are used to evaluate the proposed model. The findings from numerous trials demonstrate that the proposed model outperforms state-of-the-art approaches in terms of qualitative and quantitative outcomes.</p>

BSMNet: Boundary-salience multi-branch network for intima-media identification in carotid ultrasound images

Carotid artery intima-media thickness (CIMT) is an essential factor in signaling the risk of cardiovascular diseases, which is commonly evaluated using ultrasound imaging. However, automatic intima-media segmentation and thickness measurement are still challenging due to the boundary ambiguity of intima-media and inherent speckle noises in ultrasound images. In this work, we propose an end-to-end boundary-salience multi-branch network, BSMNet, to tackle the carotid intima-media identification from ultrasound images, where the prior shape knowledge and anatomical dependence are exploited using a parallel linear structure learning modules followed by a boundary refinement module. Moreover, we design a strip attention model to boost the thin strip region segmentation with shape priors, in which an anisotropic kernel shape captures long-range global relations and scrutinizes meaningful local salient contexts simultaneously. Extensive experimental results on an in-house carotid ultrasound (US) dataset demonstrate the promising performance of our method, which achieves about 0.02 improvement in Dice and HD95 than other state-of-the-art methods. Our method is promising in advancing the analysis of systemic arterial disease with ultrasound imaging.

A new method incorporating deep learning with shape priors for left ventricular segmentation in myocardial perfusion SPECT images

Accurate segmentation of the left ventricle (LV) is crucial for evaluating myocardial perfusion SPECT (MPS) and assessing LV functions. In this study, a novel method combining deep learning with shape priors was developed and validated to extract the LV myocardium and automatically measure LV functional parameters. The method integrates a three-dimensional (3D) V-Net with a shape deformation module that incorporates shape priors generated by a dynamic programming (DP) algorithm to guide its output during training. A retrospective analysis was performed on an MPS dataset comprising 31 subjects without or with mild ischemia, 32 subjects with moderate ischemia, and 12 subjects with severe ischemia. Myocardial contours were manually annotated as the ground truth. A 5-fold stratified cross-validation was used to train and validate the models. The clinical performance was evaluated by measuring LV end-systolic volume (ESV), end-diastolic volume (EDV), left ventricular ejection fraction (LVEF), and scar burden from the extracted myocardial contours. There were excellent agreements between segmentation results by our proposed model and those from the ground truth, with a Dice similarity coefficient (DSC) of 0.9573 ± 0.0244, 0.9821 ± 0.0137, and 0.9903 ± 0.0041, as well as Hausdorff distances (HD) of 6.7529 ± 2.7334 mm, 7.2507 ± 3.1952 mm, and 7.6121 ± 3.0134 mm in extracting the LV endocardium, myocardium, and epicardium, respectively. Furthermore, the correlation coefficients between LVEF, ESV, EDV, stress scar burden, and rest scar burden measured from our model results and the ground truth were 0.92, 0.958, 0.952, 0.972, and 0.958, respectively. The proposed method achieved a high accuracy in extracting LV myocardial contours and assessing LV functions.

Militant Leadership and Terrorism in Armed Conflict

ABSTRACT The lethality and the frequency of terrorist attacks conducted by militant organizations vary considerably. Some militants perpetrate extreme and systemic levels of terrorist violence, some only do so occasionally, and others never conduct terror operations. To predict group-level differences in this outcome, we draw on conceptual frameworks in the political science and psychology fields to argue that prior military experience shapes leaders’ willingness to engage in terrorist violence. But not all experiences are created equal. We argue that formal experiences such as those associated with military training serve as a restricting influence on terrorist violence, while less formal and less structured experiences such as time in active combat are associated with increases in future terrorist violence. Using new data on the individual backgrounds of militant leaders active between 1989 and 2013, we find support for our hypotheses. This study contributes to the academic literature by bringing the leader back into explanations of wartime terrorist activity. Our findings also carry notable policy implications for policy makers, warfighters, and counterterrorism practitioners.

Near-infrared vascular image segmentation using improved level set method

Near-infrared vascular images are crucial for vascular disease diagnosis and treatment. To solve the problems of difficult segmentation and inaccurate computation issues, we propose a series of image processing methods. First, the images are preprocessed, including background removal, contrast stretching, and noise suppression. After that, the images are enhanced with a two-stage image enhancement method, which successfully combines the benefits of convolutional neural network and traditional image enhancement method. Finally, the images are segmented by an Adaptive Prior Shape Level Set Evolution (APSLSE) method, which effectively improves the common problems of initial contour sensitivity and unidirectional variation of area terms. A shape restriction item is designed to the level set function to improve its suitability for the segmentation of blood vessels. A dataset of 360 images is collected for the research and validation of above algorithms. Extensive experimental results show that the proposed algorithms can effectively process the poor quality near-infrared blood vessel images and segment their vascular shape. Compared the segmentation results with those manually labelled by experts, the False Negative Rate (FNR) is 0.1930 and False Positive Rate (FPR) is 0.04633.

The perception divide Prior cannabis use shapes perceived outcomes of its effects

The perception divide Prior cannabis use shapes perceived outcomes of its effects

Adversarial-based latent space alignment network for left atrial appendage segmentation in transesophageal echocardiography images

Left atrial appendage (LAA) is a leading cause of atrial fibrillation and thrombosis in cardiovascular disease. Clinicians can rely on LAA occlusion (LAAO) to effectively prevent and treat ischaemic strokes attributed to the LAA. The correct selection of the LAAO is one of the most critical stages in the successful surgical process, which relies on the quantification of the anatomical structure of the LAA for successful intervention in LAAO. In this paper, we propose an adversarial-based latent space alignment framework for LAA segmentation in transesophageal echocardiography (TEE) images by introducing prior knowledge from the label. The proposed method consists of an LAA segmentation network, a label reconstruction network, and a latent space alignment loss. To be specific, we first employ ConvNeXt as the backbone of the segmentation and reconstruction network to enhance the feature extraction capability of the encoder. The label reconstruction network then encodes the prior shape features from the LAA labels to the latent space. The latent space alignment loss consists of the adversarial-based alignment and the contrast learning losses. It can motivate the segmentation network to learn the prior shape features of the labels, thus improving the accuracy of LAA edge segmentation. The proposed method was evaluated on a TEE dataset including 1,783 images and the experimental results showed that the proposed method outperformed other state-of-the-art LAA segmentation methods with Dice coefficient, AUC, ACC, G-mean, and Kappa of 0.831, 0.917, 0.989, 0.911, and 0.825, respectively.

Challenges of mismatching timescales in longitudinal studies of collective behaviour.

How individuals' prior experience and population evolutionary history shape emergent patterns in animal collectives remains a major gap in the study of collective behaviour. One reason for this is that the processes that can shape individual contributions to collective actions can happen over very different timescales from each other and from the collective actions themselves, resulting in mismatched timescales. For example, a preference to move towards a specific patch might arise from phenotype, memory or physiological state. Although providing critical context to collective actions, bridging different timescales remains conceptually and methodologically challenging. Here, we briefly outline some of these challenges, and discuss existing approaches that have already generated insights into the factors shaping individual contributions in animal collectives. We then explore a case study of mismatching timescales-defining relevant group membership-by combining fine-scaled GPS tracking data and daily field census data from a wild population of vulturine guineafowl (Acryllium vulturinum). We show that applying different temporal definitions can produce different assignments of individuals into groups. These assignments can then have consequences when determining individuals' social history, and thus the conclusions we might draw on the impacts of the social environment on collective actions. This article is part of a discussion meeting issue 'Collective behaviour through time'.

Progressive DeepSSM: Training Methodology for Image-To-Shape Deep Models.

Statistical shape modeling (SSM) is an enabling quantitative tool to study anatomical shapes in various medical applications. However, directly using 3D images in these applications still has a long way to go. Recent deep learning methods have paved the way for reducing the substantial preprocessing steps to construct SSMs directly from unsegmented images. Nevertheless, the performance of these models is not up to the mark. Inspired by multiscale/multiresolution learning, we propose a new training strategy, progressive DeepSSM, to train image-to-shape deep learning models. The training is performed in multiple scales, and each scale utilizes the output from the previous scale. This strategy enables the model to learn coarse shape features in the first scales and gradually learn detailed fine shape features in the later scales. We leverage shape priors via segmentation-guided multi-task learning and employ deep supervision loss to ensure learning at each scale. Experiments show the superiority of models trained by the proposed strategy from both quantitative and qualitative perspectives. This training methodology can be employed to improve the stability and accuracy of any deep learning method for inferring statistical representations of anatomies from medical images and can be adopted by existing deep learning methods to improve model accuracy and training stability.

SADIR: Shape-Aware Diffusion Models for 3D Image Reconstruction.

3D image reconstruction from a limited number of 2D images has been a long-standing challenge in computer vision and image analysis. While deep learning-based approaches have achieved impressive performance in this area, existing deep networks often fail to effectively utilize the shape structures of objects presented in images. As a result, the topology of reconstructed objects may not be well preserved, leading to the presence of artifacts such as discontinuities, holes, or mismatched connections between different parts. In this paper, we propose a shape-aware network based on diffusion models for 3D image reconstruction, named SADIR, to address these issues. In contrast to previous methods that primarily rely on spatial correlations of image intensities for 3D reconstruction, our model leverages shape priors learned from the training data to guide the reconstruction process. To achieve this, we develop a joint learning network that simultaneously learns a mean shape under deformation models. Each reconstructed image is then considered as a deformed variant of the mean shape. We validate our model, SADIR, on both brain and cardiac magnetic resonance images (MRIs). Experimental results show that our method outperforms the baselines with lower reconstruction error and better preservation of the shape structure of objects within the images.

Automatic Myotendinous Junction Identification In Ultrasound Images Based on Junction-based Template Measurements.

Tracking the myotendinous junction (MTJ) motion in consecutive ultrasound images is essential to assess muscle and tendon interaction and understand the mechanics' muscle-tendon unit and its pathological conditions during motion. However, the inherent speckle noises and ambiguous boundaries deter the reliable identification of MTJ, thus restricting their usage in human motion analysis. This study advances a fully automatic displacement measurement method for MTJ using prior shape knowledge on the Y-shape MTJ, precluding the influence of irregular and complicated hyperechoic structures in muscular ultrasound images. Our proposed method first adopts the junction candidate points using a combined measure of Hessian matrix and phase congruency, followed by a hierarchical clustering technique to refine the candidates approximating the position of the MTJ. Then, based on the prior knowledge of Y-shape MTJ, we finally identify the best matching junction points according to intensity distributions and directions of their branches using multiscale Gaussian templates and a Kalman filter. We evaluated our proposed method using the ultrasound scans of the gastrocnemius from 8 young, healthy volunteers. Our results present more consistent with the manual method in the MTJ tracking method than existing optical flow tracking methods, suggesting its potential in facilitating muscle and tendon function examinations with in vivo ultrasound imaging.

Cell segmentation and representation with shape priors

Cell segmentation is a fundamental problem of computational biology, for which convolutional neural networks yield the best results nowadays. This field is expanding rapidly, and in the recent years, shape-constrained segmentation models emerged as strong competitors to traditional, pixel-based segmentation methods for instance segmentation. These methods predict the parameters of the underlying shape model, so choosing the right shape representation is critical for the success of the segmentation. In this study, we introduce two new representation-based deep learning segmentation methods after a quantitative comparison of the most important shape descriptors in the literature. Our networks are based on Fourier coefficients and statistical shape models, both of which have proven to be reliable tools for cell shape modelling. Our results indicate that the methods are competitive alternatives to the most widely used baseline deep learning algorithms, especially when the number of parameters for the underlying shape model are low or the cells to be segmented have irregular morphologies.

Lens structure segmentation from AS-OCT images via shape-based learning

Background and objectivesThe lens is one of the important refractive media in the eyeball. Abnormality of the nucleus or cortex in the lens can lead to ocular disorders such as cataracts and presbyopia. To achieve an accurate diagnosis, segmentation of these ocular structures from anterior segment optical coherence tomography (AS-OCT) is essential. However, weak-contrast boundaries of the object in the images present a challenge for accurate segmentation. The state-of-the-art (SOTA) methods, such as U-Net, treat segmentation as a binary classification of pixels, which cannot handle pixels on weak-contrast boundaries well. MethodsIn this paper, we propose to incorporate shape prior into a deep learning framework for accurate nucleus and cortex segmentation. Specifically, we propose to learn a level set function, whose zero-level set represents the object boundary, through a convolutional neural network. Moreover, we design a novel shape-based loss function, where the shape prior knowledge can be naturally embedded into the learning procedure, leading to improvement in performance. We collect a high-quality AS-OCT image dataset with precise annotations to train our model. ResultsAbundant experiments are conducted to verify the effectiveness of the proposed framework and the novel shape-based loss. The mean Intersection over Unions (MIoUs) of the proposed method for lens nucleus and cortex segmentation are 0.946 and 0.957, and the mean Euclidean Distance (MED) measure, which can reflect the accuracy of the segmentation boundary, are 6.746 and 2.045 pixels. In addition, the proposed shape-based loss improves the SOTA models on the nucleus and cortex segmentation tasks by an average of 0.0156 and 0.0078 in the MIoU metric and 1.394 and 0.134 pixels in the MED metric. ConclusionWe transform the segmentation from a classification task to a regression task by making the model learn the level set function, and embed shape information in deep learning by designing loss functions. This allows the proposed method to be more efficient in the segmentation of the object with weak-contrast boundaries. Concise abstractWe propose to incorporate shape priors into a deep learning framework for accurate nucleus and cortex segmentation from AS-OCT images. Specifically, we propose to learn a level set function, where the zero-level set represents the boundary of the target. Meanwhile, we design a novel shape-based loss function in which additional convex shape prior can be embedded in the learning process, leading to an improvement in performance. The IOUs for nucleus and cortex segmentation are 0.946 and 0.957, while the MED that reflects the accuracy of the boundary are 6.746 and 2.045 pixels. The proposed shape-based loss improves the SOTA model for nucleus and cortex segmentation by an average of 0.0156 and 0.0078 in IOU, and 1.394 and 0.134 pixels in MED. We transform segmentation from classification to regression by making the model learn a level set function, resulting in improved performance at the boundary with weak contrast.

Variational Approach for Joint Kidney Segmentation and Registration from DCE-MRI Using Fuzzy Clustering with Shape Priors.

The dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) technique has great potential in the diagnosis, therapy, and follow-up of patients with chronic kidney disease (CKD). Towards that end, precise kidney segmentation from DCE-MRI data becomes a prerequisite processing step. Exploiting the useful information about the kidney's shape in this step mandates a registration operation beforehand to relate the shape model coordinates to those of the image to be segmented. Imprecise alignment of the shape model induces errors in the segmentation results. In this paper, we propose a new variational formulation to jointly segment and register DCE-MRI kidney images based on fuzzy c-means clustering embedded within a level-set (LSet) method. The image pixels' fuzzy memberships and the spatial registration parameters are simultaneously updated in each evolution step to direct the LSet contour toward the target kidney. Results on real medical datasets of 45 subjects demonstrate the superior performance of the proposed approach, reporting a Dice similarity coefficient of 0.94 ± 0.03, Intersection-over-Union of 0.89 ± 0.05, and 2.2 ± 2.3 in 95-percentile of Hausdorff distance. Extensive experiments show that our approach outperforms several state-of-the-art LSet-based methods as well as two UNet-based deep neural models trained for the same task in terms of accuracy and consistency.

BUS-Net: Breast Tumour Detection Network for Ultrasound Images Using Bi-directional ConvLSTM and Dense Residual Connections.

Breast ultrasound (BUS) imaging has become one of the key imaging modalities for medical image diagnosis and prognosis. However, the manual process of lesion delineation from ultrasound images can incur various challenges concerning variable shape, size, intensity, curvature, or other medical priors of the lesion in the image. Therefore, computer-aided diagnostic (CADx) techniques incorporating deep learning-based neural networks are automatically used to segment the lesion from BUS images. This paper proposes an encoder-decoder-based architecture to recognize and accurately segment the lesion from two-dimensional BUS images. The architecture is utilized with the residual connection in both encoder and decoder paths; bi-directional ConvLSTM (BConvLSTM) units in the decoder extract the minute and detailed region of interest (ROI) information. BConvLSTM units and residual blocks help the network weigh ROI information more than the similar background region. Two public BUS image datasets, one with 163 images and the other with 42 images, are used. The proposed model is trained with the augmented images (ten forms) of dataset one (with 163 images), and test results are produced on the second dataset and the testing set of the first dataset-the segmentation performance yielding comparable results with the state-of-the-art segmentation methodologies. Similarly, the visual results show that the proposed approach for BUS image segmentation can accurately identify lesion contours and can potentially be applied for similar and larger datasets.

Feasibility and flexibility regions estimation at TSO–DSO interconnection node using grid structure optimization

Major integration of renewable energy has raised the requirement of frequency and voltage control ancillary services. This necessitates to utilize distributed energy resources for grid flexibility solutions. Aggregated DERs’ flexibility at TSO–DSO node, identified by flexibility operating regions, can provide these services to the grid. Estimating the actual shape of the flexibility region for both slow and fast response systems, is important for real-time applications. However, correct estimation of this region is challenging due to its dependence on various factors, its non-convex shape, and high computational requirements. Proposed novel methodology named ‘Grid Structure Optimization’ can estimate the actual shape of the flexibility region for both slow and fast response systems without any prior shape approximation. The proposed approach is tested on modified IEEE 9 and 33 bus systems considering impacts of Q/P ratio, market dispatch point, cost and type of market. Results highlight the efficacy of this approach for real-time applications as the shape of the region completely avoids the possibility of infeasible points in the region. The results also indicate that the cost willingness is a factor of non-convexity on the feasibility region. Furthermore, the proposed approach is demonstrated to identify DER’s flexibility for contingency-based frequency control ancillary services.