Image Similarity Criteria Research Articles

Learning-Based Prostate Needle Position Prediction for HDR Brachytherapy

During prostate high-dose-rate (HDR) brachytherapy, physicians select needle positions in a peripheral loading technique based on experience but without knowledge of the achievable dose distribution. Sub-optimal needle placement may lead to unfavorable dose distributions even after plan optimization. We propose a new deep learning-based method to predict HDR needle position for prostate HDR brachytherapy.The proposed framework consists of three major steps: (1) deformable registration via registration network (Reg-Net), (2) multi-atlas ranking and (3) needle regression. To model the global spatial relationship among multiple organs, binary masks of the target and organs at risk are transformed into distance maps which describe the distance of each local voxel to the organ surfaces. Reg-Net is utilized to deformably register the distance maps from multi-atlas set to match a patient's distance map and then bring needle maps from multi-atlas to this patient via spatial transformation. Several criteria are used for multi-atlas ranking including prostate volume similarity, multi-organ semantic image similarity and needle position criteria. Finally, needle regression is used to refine the final needle positions. A retrospective study of consecutively treated patients was used to evaluate the feasibility of the proposed method. We performed a five-fold cross validation to evaluate the proposed method. We generated a new dose plan for each patient based on predicted needle location. Target and organ dose volume histogram (DVH) metrics were used to quantify the difference between the clinical and predicted needle plans. All plans were normalized to prostate V100 of 95%.Ninety predicted needle plans were compared to clinical plans. In each case, the needle prediction algorithm completed within 1 minute. See table for comparison of target and organ DVH metrics (mean and standard deviation). Predicted needle plans had slightly greater target dose heterogeneity. The RTOG constraints of bladder V75% < 1 cc, rectum V75% < 1cc, and urethra V125% < 1 cc were met in all predicted plans.It is feasible to use this novel deep-learning-based method to predict needle locations for HDR prostate brachytherapy a priori. This strategy merits further study to improve utilization and quality of prostate brachytherapy.

Evaluation of different atlas selection strategies for multi-atlas segmentation of low-dose computed tomographic images of whole-body positron emission tomography/computed tomography

ABSTRACT Background and Objectives: The increasing clinical use of torso positron emission tomography/computed tomography (PET/CT) demands automated segmentation of torso organs from PET/CT images. We attempt to use the multi-atlas segmentation approach for trunk organ segmentation from the low-dose CT images of PET/CT. Since atlas selection is a prerequisite step for multi-atlas segmentation, this study focuses on evaluating the performance of different atlas selection strategies for torso organ segmentation. Methods: We evaluated two criteria for atlas selection, including image similarity and body mass index (BMI) difference between the atlas and the target image. Based on the two criteria, ten atlases are selected and registered to the target image, followed by the label fusion step to achieve final segmentation. Results: The BMI criterion yields comparable segmentation accuracy to the image similarity criterion but with much less computation time. All the evaluated atlas selection methods have Dice &gt;0.9 for the lungs, heart, and liver and Dice &lt; 0.85 for the skeleton, spleen, and kidneys. The inter-method differences are not significant for the high-contrast and big-sized organs such as skeleton, lungs, heart, and liver. For the low-contrast and smaller-sized organs such as spleen and kidneys, none of the atlas selection methods significantly outperforms random atlas selection. Conclusions: BMI is an effective and efficient atlas selection criterion for low-dose torso CT images. The spleen and kidneys are difficult to get good segmentation, no matter which atlas selection method is used. It is important to develop more effective atlas selection methods for the spleen and kidneys.

A general framework to learn surrogate relevance criterion for atlas based image segmentation

Multi-atlas based image segmentation sees great opportunities in the big data era but also faces unprecedented challenges in identifying positive contributors from extensive heterogeneous data. To assess data relevance, image similarity criteria based on various image features widely serve as surrogates for the inaccessible geometric agreement criteria. This paper proposes a general framework to learn image based surrogate relevance criteria to better mimic the behaviors of segmentation based oracle geometric relevance. The validity of its general rationale is verified in the specific context of fusion set selection for image segmentation. More specifically, we first present a unified formulation for surrogate relevance criteria and model the neighborhood relationship among atlases based on the oracle relevance knowledge. Surrogates are then trained to be small for geometrically relevant neighbors and large for irrelevant remotes to the given targets. The proposed surrogate learning framework is verified in corpus callosum segmentation. The learned surrogates demonstrate superiority in inferring the underlying oracle value and selecting relevant fusion set, compared to benchmark surrogates.

Learning image based surrogate relevance criterion for atlas selection in segmentation

Picking geometrically relevant atlases from the whole training set is crucial to multi-atlas based image segmentation, especially with extensive data of heterogeneous quality in the Big Data era. Unfortunately, there is very limited understanding of how currently used image similarity criteria reveal geometric relevance, let alone the optimization of them. This paper aims to develop a good image based surrogate relevance criterion to best reflect the underlying inaccessible geometric relevance in a learning context. We cast this surrogate learning problem into an optimization framework, by encouraging the image based surrogate to behave consistently with geometric relevance during training. In particular, we desire a criterion to be small for image pairs with similar geometry and large for those with significantly different segmentation geometry. Validation experiments on corpus callosum segmentation demonstrate the improved quality of the learned surrogate compared to benchmark surrogate candidates.

Fast topology estimation for image mosaicing using adaptive information thresholding

Over the past decade, several image mosaicing methods have been proposed in robotic mapping and remote sensing applications. Owing to rapid developments in obtaining optical data from areas beyond human reach, there is a high demand from different science fields for creating large-area image mosaics, often using images as the only source of information. One of the most important steps in the mosaicing process is motion estimation between overlapping images to obtain the topology, i.e., the spatial relationships between images.In this paper, we propose a generic framework for feature-based image mosaicing capable of obtaining the topology with a reduced number of matching attempts and of getting the best possible trajectory estimation. Innovative aspects include the use of a fast image similarity criterion combined with a Minimum Spanning Tree (MST) solution, to obtain a tentative topology and information theory principles to decide when to update trajectory estimation. Unlike previous approaches for large-area mosaicing, our framework is able to naturally deal with the cases where time-consecutive images cannot be matched successfully, such as completely unordered sets. This characteristic also makes our approach robust to sensor failure. The performance of the method is illustrated with experimental results obtained from different challenging underwater image sequences.

A mutual information based face clustering algorithm for movie content analysis

This paper investigates facial image clustering, primarily for movie video content analysis with respect to actor appearance. Our aim is to use novel formulation of the mutual information as a facial image similarity criterion and, by using spectral graph analysis, to cluster a similarity matrix containing the mutual information of facial images. To this end, we use the HSV color space of a facial image (more precisely, only the hue and saturation channels) in order to calculate the mutual information similarity matrix of a set of facial images. We make full use of the similarity matrix symmetries, so as to lower the computational complexity of the new mutual information calculation. We assign each row of this matrix as feature vector describing a facial image for producing a global similarity criterion for face clustering. In order to test our proposed method, we conducted two sets of experiments that have produced clustering accuracy of more than 80%. We also compared our algorithm with other clustering approaches, such as the k-means and fuzzy c-means (FCM) algorithms. Finally, in order to provide a baseline comparison for our approach, we compared the proposed global similarity measure with another one recently reported in the literature.

Multifaceted image similarity criteria as revealed by sorting tasks

AbstractThis paper reports a study on the types of image categories constructed from magazine photographs. A novel sorting procedure was tested with the aim of providing more data on image similarity and possible category overlap. Expert and non‐expert participants were compared in their categorizations. The new similarity sorting procedure resulted in an average of 67%–111% increase in similarity data gathered compared to basic free sorting. Categories were constructed on various levels of similarity: image Function, main visual content (People, Objects and Scene), conceptual content (Theme) and descriptors (Story, Affective, Description, Photography and Visual). Most categories were based on the theme and people portrayed in the photograph, and in the case of the expert subjects, image function. Also abstract and syntactic similarity criteria were employed by the subjects. The categories created by each subject showed on average a 35%–53% overlap. Participants also demonstrated a tendency to use multiple similarity criteria simultaneously and to combine terms from different levels in a single category name. These results indicate a need for a multifaceted approach in image categorization.

Automatic segmentation of phase‐correlated CT scans through nonrigid image registration using geometrically regularized free‐form deformation

Conventional radiotherapy is planned using free-breathing computed tomography (CT), ignoring the motion and deformation of the anatomy from respiration. New breath-hold-synchronized, gated, and four-dimensional (4D) CT acquisition strategies are enabling radiotherapy planning utilizing a set of CT scans belonging to different phases of the breathing cycle. Such 4D treatment planning relies on the availability of tumor and organ contours in all phases. The current practice of manual segmentation is impractical for 4D CT, because it is time consuming and tedious. A viable solution is registration-based segmentation, through which contours provided by an expert for a particular phase are propagated to all other phases while accounting for phase-to-phase motion and anatomical deformation. Deformable image registration is central to this task, and a free-form deformation-based nonrigid image registration algorithm will be presented. Compared with the original algorithm, this version uses novel, computationally simpler geometric constraints to preserve the topology of the dense control-point grid used to represent free-form deformation and prevent tissue fold-over. Using mean squared difference as an image similarity criterion, the inhale phase is registered to the exhale phase of lung CT scans of five patients and of characteristically low-contrast abdominal CT scans of four patients. In addition, using expert contours for the inhale phase, the corresponding contours were automatically generated for the exhale phase. The accuracy of the segmentation (and hence deformable image registration) was judged by comparing automatically segmented contours with expert contours traced directly in the exhale phase scan using three metrics: volume overlap index, root mean square distance, and Hausdorff distance. The accuracy of the segmentation (in terms of radial distance mismatch) was approximately 2 mm in the thorax and 3 mm in the abdomen, which compares favorably to the accuracies reported elsewhere. Unlike most prior work, segmentation of the tumor is also presented. The clinical implementation of 4D treatment planning is critically dependent on automatic segmentation, for which is offered one of the most accurate algorithms yet presented.

Relevance feedback-based image retrieval interface incorporating region and feature saliency patterns as visualizable image similarity criteria

Region and feature saliency (RFSP) is proposed as: (1) a new image similarity model; and (2) a new, visualizable representation of the image similarity criteria. RFSP, coupled with the proposed genetic-algorithm (GA)-based relevance feedback mechanism, is incorporated in the image retrieval interface. By capturing the two fundamental properties of the human visual system-region and feature saliencies-in a context-dependent sense, RFSP more accurately approximates the human similarity perception. By representing the image similarity criteria as a pattern of feature combinations distributed over the image regions, each having a different importance, RFSP enables the visualization-in a concise form-of the complex, low-level similarity criteria associated with each query image. None of the representative image similarity models captures both region and feature saliencies in a context-dependent sense. Furthermore, very few of the representative works-dealing with the relevance feedback in image retrieval-consider the visualization of the similarity criteria, as a user interface aspect. Also, this paper presents one of the first applications of GAs to the relevance feedback mechanism in the image retrieval field. The retrieval performance of the RFSP, coupled with the proposed GA-based relevance feedback mechanism, is evaluated on five test databases, with around 2500 images, covering 62 semantic categories. Compared with 11 of the representative image similarity models, including three which employ relevance feedback, RFSP brings in average between 6%-30% increase in the retrieval precision. The relevance feedback-based retrieval interface incorporating RFSP is demonstrated as well. Experiment results suggest that: (1) capturing the region and feature saliencies in a context-dependent sense improves the retrieval performance; whereas (2) visualizing the similarity criteria makes the relevance feedback-based image retrieval interface more user friendly, aiding the user in the understanding and expression of the information needs.

A new approach to elastography using mutual information and finite elements

Historically, increased mechanical stiffness during tissue palpation exams has been associated with assessing organ health as well as with detecting the growth of a potentially life-threatening cell mass. As such, techniques to image elasticity parameters (i.e., elastography) have recently become of great interest to scientists. In this work, a new method of elastography will be introduced within the context of mammographic imaging. The elastography method proposed represents a non-rigid iterative image registration algorithm that varies material properties within a finite element model to improve registration. More specifically, regional measures of image similarity are used within an objective function minimization framework to reconstruct elasticity images of tissue stiffness. Numerical simulations illustrate: (1) the encoding of stiffness information within the context of a regional image similarity criterion, (2) the methodology for an iterative elastographic imaging framework and (3) elasticity reconstruction simulations. The real strength in this approach is that images from any modality (e.g., magnetic resonance, computed tomography, ultrasound, etc) that have sufficient anatomically-based intensity heterogeneity and remain consistent from a pre- to a post-deformed state could be used in this paradigm.

Automatic video data structuring through shot partitioning and key-frame computing

In video processing, a common first step is to segment the videos into physical units, generally called shots. A shot is a video segment that consists of one continuous action. In general, these physical units need to be clustered to form more semantically significant units, such as scenes, sequences, programs, etc. This is the so-called story-based video structuring. Automatic video structuring is of great importance for video browsing and retrieval. The shots or scenes are usually described by one or several representative frames, called key-frames. Viewed from a higher level, key frames of some shots might be redundant in terms of semantics. In this paper, we propose automatic solutions to the problems of: (i) video partitioning, (ii) key frame computing, (iii) key frame pruning. For the first problem, an algorithm called “net comparison” is devised. It is accurate and fast because it uses both statistical and spatial information in an image and does not have to process the entire image. For the last two problems, we develop an original image similarity criterion, which considers both spatial layout and detail content in an image. For this purpose, coefficients of wavelet decomposition are used to derive parameter vectors accounting for the above two aspects. The parameters exhibit (quasi-) invariant properties, thus making the algorithm robust for many types of object/camera motions and scaling variances. The novel “seek and spread” strategy used in key frame computing allows us to obtain a large representative range for the key frames. Inter-shot redundancy of the key-frames is suppressed using the same image similarity measure. Experimental results demonstrate the effectiveness and efficiency of our techniques.