Retinal Image Registration Research Articles

Retinal Image Registration Method with Affine Invariance

Retinal Image Registration Method with Affine Invariance

Image Registration Based on Camera Calibration for Dual-Wavelength Retinal Oximetry

Retinal images at 570nm and 600nm are simultaneously captured by retinal oximetry. However, there exists misalignment between the two images, which should be adjusted before they are used to analyze hemoglobin oxygen saturation (SO2). Having a robust, accurate and fast registration scheme can ensure accuracy and speed of SO2 calculation. However, the existing retinal image registration methods cannot meet demand for clinical application of retinal oximetry. For example, errors in the vessel segmentation from each image cause corresponding errors in the registration, point matches may become unreliable when the predefined matching cost is corrupted by pathological retinal images. And it is always a time-consuming task. In this paper, we first analyzed the misalignment reason between dual-wavelength images, and concluded that there only exists the rotation and translation deviation between the two images caused by a mounting deviation of CCD cameras. The parameters were only related to system itself. Based on this, we proposed a registration method based on camera calibration that benefits from feature extraction and affine transformation matrix calculation. The parameters obtained by the proposed method were tested on 80 pairs of retinal images of 5 healthy volunteers and the results showed that the proposed method can successfully register the dual-wavelength images. The proposed registration method can significantly improve SO2 calculation speed while ensuring accuracy.

Gaussian field estimator with manifold regularization for retinal image registration

Abstract Retinal image registration, which combines the information of retinal images from different modalities, viewpoints or times, plays an important role in diagnosing and treating many eye diseases. In this work, we propose an efficient algorithm called Gaussian field estimator with manifold regularization to address this requirement. We formulate the registration problem as a probabilistic model, that is, a Gaussian field estimator, to achieve a robust estimation, associated with a prior involving manifold regularization to preserve the intrinsic geometry of input data. A non-rigid transformation is specified in a reproducing kernel Hilbert space. Moreover, a sparse approximation is applied to the non-rigid transformation for a fast implementation. Extensive experiments on retinal images show the effectiveness of our approach, which can yield superior results compared with other state-of-the-art methods, especially in the case of severely degraded data.

A-RANSAC: Adaptive random sample consensus method in multimodal retinal image registration

Abstract In this paper, an adaptive Random Sample Consensus (A-RANSAC) method is proposed for multimodal retinal image registration. In this method, the features of two various images from images taken with different modalities such as FA (Fluorescein angiography) and RF (Red free) are extracted using a modified version of Scale Invariant Feature Transform method (SIFT) called SAR-SIFT which is originally used for Synthetic Aperture Radar images. Then, the matching performance between these images is enhanced using the proposed A-RANSAC. In the A-RANSAC method, the threshold value is chosen so that the Root Mean Square Error (RMSE) and the number of removed matches are optimized simultaneously. The efficiency of the proposed method has been investigated in other modes such as high resolution and low-quality retinal image registration in addition to multimodal registration. The simulation results on several retinal image datasets show that the proposed method improves the precision matching by 9.89% and rate of success by 25% on the average compared to the SAR-SIFT method.

Performance Evaluation of State-of-the-Art Local Feature Detectors and Descriptors in the Context of Longitudinal Registration of Retinal Images.

In this paper we systematically evaluate the performance of several state-of-the-art local feature detectors and descriptors in the context of longitudinal registration of retinal images. Longitudinal (temporal) registration facilitates to track the changes in the retina that has happened over time. A wide number of local feature detectors and descriptors exist and many of them have already applied for retinal image registration, however, no comparative evaluation has been made so far to analyse their respective performance. In this manuscript we evaluate the performance of the widely known and commonly used detectors such as Harris, SIFT, SURF, BRISK, and bifurcation and cross-over points. As of descriptors SIFT, SURF, ALOHA, BRIEF, BRISK and PIIFD are used. Longitudinal retinal image datasets containing a total of 244 images are used for the experiment. The evaluation reveals some potential findings including more robustness of SURF and SIFT keypoints than the commonly used bifurcation and cross-over points, when detected on the vessels. SIFT keypoints can be detected with a reliability of 59% for without pathology images and 45% for with pathology images. For SURF keypoints these values are respectively 58% and 47%. ALOHA descriptor is best suited to describe SURF keypoints, which ensures an overall matching accuracy, distinguishability of 83%, 93% and 78%, 83% for without pathology and with pathology images respectively.

Retinal Image Registration Based on Robust Non-Rigid Point Matching Method

Retinal Image Registration Based on Robust Non-Rigid Point Matching Method

Multi-modal and multi-vendor retina image registration

Multi-modal retinal image registration is often required to utilize the complementary information from different retinal imaging modalities. However, a robust and accurate registration is still a challenge due to the modality-varied resolution, contrast, and luminosity. In this paper, a two step registration method is proposed to address this problem. Descriptor matching on mean phase images is used to globally register images in the first step. Deformable registration based on modality independent neighbourhood descriptor (MIND) method is followed to locally refine the registration result in the second step. The proposed method is extensively evaluated on color fundus images and scanning laser ophthalmoscope (SLO) images. Both qualitative and quantitative tests demonstrate improved registration using the proposed method compared to the state-of-the-art. The proposed method produces significantly and substantially larger mean Dice coefficients compared to other methods (p<0.001). It may facilitate the measurement of corresponding features from different retinal images, which can aid in assessing certain retinal diseases.

Multimodal registration of retinal images using domain-specific landmarks and vessel enhancement

The analysis of different image modalities is frequently performed in ophthalmology as it provides complementary information for the diagnosis and follow-up of relevant diseases, like hypertension or diabetes. This work presents a hybrid method for the multimodal registration of color fundus retinography and fluorescein angiography. The proposed method combines a feature-based approach, using domain-specific landmarks, with an intensity-based approach that employs a domain-adapted similarity metric. The methodology is tested on a dataset of 59 image pairs containing both healthy and pathological cases. The results show a satisfactory performance of the proposed combined approach in this multimodal scenario, improving the registration accuracy achieved by the feature-based and the intensity-based approaches.

Feature guided Gaussian mixture model with semi-supervised EM and local geometric constraint for retinal image registration

Retinal image registration, which can be formulated by matching two sets of sparse features extracted from two observed retinal images, is a crucial step in the diagnosis and treatment of various eye diseases. Existing methods suffer from missing true correspondences or do not fully consider local appearance information, which causes difficulty in matching low-quality retinal images due to insufficient reliable features. In addition, the relationships between retinal image pairs are usually modeled by linear transformation, such as affine transformation, which cannot generate accurate alignments in large viewpoint changes due to the nonplanar eyeball surface. To address these issues, a feature guided Gaussian mixture model (GMM) is proposed for the non-rigid registration of retinal images. We formulate the problem as an estimation of a feature guided mixture of densities: a GMM is fitted to one point set in which the centers of the Gaussian densities characterized by spatial positions associated with local appearance descriptors are constrained to coincide with the other point set. The problem is solved under a maximum-likelihood framework, and semi-supervised expectation-maximization is used to iteratively estimate the feature correspondence and spatial transformation, which is initialized by a set of confidential feature matches obtained previously. Non-rigid transformation is specified in a reproducing kernel Hilbert space, and a local geometric constraint is imposed to establish the transformation estimation for obtaining a meaningful solution. A fast implementation based on sparse approximation is also provided and reduces the time complexity from cubic to quadratic. Moreover, we use the edge map, which can extract more reliable features, as a uniform representation of retinal images. Experimental results on publicly available retinal images show that our approach is robust in different registration tasks and outperforms several competing approaches, especially when data is severely degraded.

FIRE: Fundus Image Registration dataset

Purpose: Retinal image registration is a useful tool for medical professionals. However, performance evaluation of registration methods has not been consistently assessed in the literature. To address that, a dataset comprised of retinal image pairs annotated with ground truth and an evaluation protocol for registration methods is proposed.Methods: The dataset is comprised by 134 retinal fundus image pairs. These pairs are classified into three categories, according to characteristics that are relevant to indicative registration applications. Such characteristics are the degree of overlap between images and the presence/absence of anatomical differences. Ground truth in the form of corresponding image points and a protocol to evaluate registration performance are provided.Results: The proposed protocol is shown to enable quantitative and comparative evaluation of retinal registration methods under a variety of conditions.Conclusion: This work enables the fair comparison of retinal registration methods. It also helps researchers to select the registration method that is most appropriate given a specific target use.

An experimental evaluation of the accuracy of keypoints-based retinal image registration.

This work regards an investigation of the accuracy of a state-of-the-art, keypoint-based retinal image registration approach, as to the type of keypoint features used to guide the registration process. The employed registration approach is a local method that incorporates the notion of a 3D retinal surface imaged from different viewpoints and has been shown, experimentally, to be more accurate than competing approaches. The correspondences obtained between SIFT, SURF, Harris-PIIFD and vessel bifurcations are studied, either individually or in combinations. The combination of SIFT features with vessel bifurcations was found to perform better than other combinations or any individual feature type, alone. The registration approach is also comparatively evaluated against representative methods of the state-of-the-art in retinal image registration, using a benchmark dataset that covers a broad range of cases regarding the overlap of the acquired images and the anatomical characteristics of the imaged retinas.

Automatic Retinal Image Registration Using Blood Vessel Segmentation and SIFT Feature

Automatic retinal image registration is still a great challenge in computer aided diagnosis and screening system. In this paper, a new retinal image registration method is proposed based on the combination of blood vessel segmentation and scale invariant feature transform (SIFT) feature. The algorithm includes two stages: retinal image segmentation and registration. In the segmentation stage, the blood vessel is segmented by using the guided filter to enhance the vessel structure and the bottom-hat transformation to extract blood vessel. In the registration stage, the SIFT algorithm is adopted to detect the feature of vessel segmentation image, complemented by using a random sample consensus (RANSAC) algorithm to eliminate incorrect matches. We evaluate our method from both segmentation and registration aspects. For segmentation evaluation, we test our method on DRIVE database, which provides manually labeled images from two specialists. The experimental results show that our method achieves 0.9562 in accuracy (Acc), which presents competitive performance compare to other existing segmentation methods. For registration evaluation, we test our method on STARE database, and the experimental results demonstrate the superior performance of the proposed method, which makes the algorithm a suitable tool for automated retinal image analysis.

Adjusted Fireworks Algorithm Applied to Retinal Image Registration

Adjusted Fireworks Algorithm Applied to Retinal Image Registration

Feature-Based Retinal Image Registration Using D-Saddle Feature.

Retinal image registration is important to assist diagnosis and monitor retinal diseases, such as diabetic retinopathy and glaucoma. However, registering retinal images for various registration applications requires the detection and distribution of feature points on the low-quality region that consists of vessels of varying contrast and sizes. A recent feature detector known as Saddle detects feature points on vessels that are poorly distributed and densely positioned on strong contrast vessels. Therefore, we propose a multiresolution difference of Gaussian pyramid with Saddle detector (D-Saddle) to detect feature points on the low-quality region that consists of vessels with varying contrast and sizes. D-Saddle is tested on Fundus Image Registration (FIRE) Dataset that consists of 134 retinal image pairs. Experimental results show that D-Saddle successfully registered 43% of retinal image pairs with average registration accuracy of 2.329 pixels while a lower success rate is observed in other four state-of-the-art retinal image registration methods GDB-ICP (28%), Harris-PIIFD (4%), H-M (16%), and Saddle (16%). Furthermore, the registration accuracy of D-Saddle has the weakest correlation (Spearman) with the intensity uniformity metric among all methods. Finally, the paired t-test shows that D-Saddle significantly improved the overall registration accuracy of the original Saddle.

A Two-Step Approach for Longitudinal Registration of Retinal Images.

This paper presents a novel two step approach for longitudinal (over time) registration of retinal images. Longitudinal registration is an important preliminary step to analyse longitudinal changes on the retina including disease progression. While potential overlap and minimal geometric distortion are likely in longitudinal images, identification of reliable features over time is a potential challenge for longitudinal registration. Relying on the widely accepted phenomenon that retinal vessels are more reliable over time, the proposed method aims to accurately match bifurcation and cross-over points between different timestamp images. Binary robust independent elementary features (BRIEF) are computed around bifurcation points which are then matched based on Hamming distance. Prior to computing BRIEF descriptors, a preliminary registration is performed relying on SURF key-point matching. Experiments are conducted on different image datasets containing 109 longitudinal image pairs in total. The proposed method has been found to produce accurate registration (i.e. registration with zero alignment error) for 97% cases, which is significantly higher than the other methods in comparison. The paper also reveals the finding that both the number and distributions of accurately matching key-points pairs are important for successful registration of image pairs.

Removing mismatches for retinal image registration via multi-attribute-driven regularized mixture model

In order to address the problem of retinal image registration, this paper proposes and analyzes a novel and general matching algorithm called Multi-Attribute-Driven Regularized Mixture Model (MAD-RMM). Mismatches removal can play a key role in image registration, which refers to establish reliable matches between two point sets. Here the presented approach starts from multi-feature attributes which are used to guide the feature matching to identify inliers (correct matches) from outliers (incorrect matches), and then estimates the spatial transformation. In this paper, motivated by the problem of feature matching that the initial correspondence is always contaminated by outliers, thereby we formulate this issue as a probability deformable mixture model which consists of Gaussian components for inliers and uniform components for outliers. Moreover, the algorithm takes full advantage of using multiple attributes for better general matching performance. Here we are assuming all inliers are mapped into a high-dimensional feature space, namely reproducing kernel Hilbert space (RKHS), and the closed-form solution to the mapping function is given by the representation theorem with L2 norm regularization under the Expectation-Maximization (EM) algorithm. Finally, we evaluate the performance of the algorithm by applying it to retinal image registration on several datasets, where experimental results demonstrate that the MAD-RMM outperforms current state-of-the-art methods and shows the robustness to outliers on real retinal images.

Retinal image registration through simultaneous camera pose and eye shape estimation.

In this paper, a retinal image registration method is proposed. The approach utilizes keypoint correspondences and assumes that the human eye has a spherical or ellipsoidal shape. The image registration problem amounts to solving a camera 3D pose estimation problem and, simultaneously, an eye 3D shape estimation problem. The camera pose estimation problem is solved by estimating the relative pose between the views from which the images were acquired. The eye shape estimation problem parameterizes the shape and orientation of an ellipsoidal model for the eye. Experimental evaluation shows 17.91% reduction of registration error and 47.52% reduction of the error standard deviation over state of the art methods.

Retinal image registration under the assumption of a spherical eye

We propose a method for registering a pair of retinal images. The proposed approach employs point correspondences and assumes that the human eye has a spherical shape. The image registration problem is formulated as a 3D pose estimation problem, solved by estimating the rigid transformation that relates the views from which the two images were acquired. Given this estimate, each image can be warped upon the other so that pixels with the same coordinates image the same retinal point. Extensive experimental evaluation shows improved accuracy over state of the art methods, as well as robustness to noise and spurious keypoint matches. Experiments also indicate the method's applicability to the comparative analysis of images from different examinations that may exhibit changes and its applicability to diagnostic support.

Retinal image registration via feature-guided Gaussian mixture model.

Registration of retinal images taken at different times, from different perspectives, or with different modalities is a critical prerequisite for the diagnoses and treatments of various eye diseases. This problem can be formulated as registration of two sets of sparse feature points extracted from the given images, and it is typically solved by first creating a set of putative correspondences and then removing the false matches as well as estimating the spatial transformation between the image pairs or solved by estimating the correspondence and transformation jointly involving an iteration process. However, the former strategy suffers from missing true correspondences, and the latter strategy does not make full use of local appearance information, which may be problematic for low-quality retinal images due to a lack of reliable features. In this paper, we propose a feature-guided Gaussian mixture model (GMM) to address these issues. We formulate point registration as the estimation of a feature-guided mixture of densities: A GMM is fitted to one point set, such that both the centers and local features of the Gaussian densities are constrained to coincide with the other point set. The problem is solved under a unified maximum-likelihood framework together with an iterative expectation-maximization algorithm initialized by the confident feature correspondences, where the image transformation is modeled by an affine function. Extensive experiments on various retinal images show the robustness of our approach, which consistently outperforms other state-of-the-art methods, especially when the data is badly degraded.

Retinal slit lamp video mosaicking.

PurposeTo this day, the slit lamp remains the first tool used by an ophthalmologist to examine patient eyes. Imaging of the retina poses, however, a variety of problems, namely a shallow depth of focus, reflections from the optical system, a small field of view and non-uniform illumination. For ophthalmologists, the use of slit lamp images for documentation and analysis purposes, however, remains extremely challenging due to large image artifacts. For this reason, we propose an automatic retinal slit lamp video mosaicking, which enlarges the field of view and reduces amount of noise and reflections, thus enhancing image quality.MethodsOur method is composed of three parts: (i) viable content segmentation, (ii) global registration and (iii) image blending. Frame content is segmented using gradient boosting with custom pixel-wise features. Speeded-up robust features are used for finding pair-wise translations between frames with robust random sample consensus estimation and graph-based simultaneous localization and mapping for global bundle adjustment. Foreground-aware blending based on feathering merges video frames into comprehensive mosaics.ResultsForeground is segmented successfully with an area under the curve of the receiver operating characteristic curve of 0.9557. Mosaicking results and state-of-the-art methods were compared and rated by ophthalmologists showing a strong preference for a large field of view provided by our method.ConclusionsThe proposed method for global registration of retinal slit lamp images of the retina into comprehensive mosaics improves over state-of-the-art methods and is preferred qualitatively.