Image-level Fusion Research Articles

ITFuse: An interactive transformer for infrared and visible image fusion

Infrared and visible image fusion (IVIF) has attracted increasing attention from the community because of its pleasing results in downstream applications. However, most existing deep fusion models are either feature–level fusion or image–level fusion, leading to information loss. In this paper, we propose an interactive transformer for IVIF, termed ITFuse. In contrast to previous algorithms, ITFuse consists of feature interactive modules (FIMs) and a feature reconstruction module (FRM) to alternatively extract and integrate important features. Specifically, to adequately exploit the common properties of different source images, we design a residual attention block (RAB) for mutual feature representation. To aggregate the distinct characteristics that existed in the corresponding input images, we leverage interactive attention (ITA) to incorporate the complementary information for comprehensive feature preservation and interaction. In addition, cross-modal attention (CMA) and transformer block (TRB) are presented to fully merge the capitalized features and construct long-range relationships. Furthermore, we devise a pixel loss and a structural loss to train the proposed deep fusion model in an unsupervised manner for excess performance amelioration. Massive experiments on popular databases illustrate that our ITFuse performs better than other representative state-of-the-art methods in terms of both qualitative and quantitative assessments. The source code of the proposed method is available at https://github.com/tthinking/ITFuse.

Learning SAR-Optical Cross Modal Features for Land Cover Classification

Synthetic aperture radar (SAR) and optical images provide highly complementary ground information. The fusion of SAR and optical data can significantly enhance semantic segmentation inference results. However, the fusion methods for multimodal data remains a challenge for current research due to significant disparities in imaging mechanisms from diverse sources. Our goal was to bridge the significant gaps between optical and SAR images by developing a dual-input model that utilizes image-level fusion. To improve most existing state-of-the-art image fusion methods, which often assign equal weights to multiple modalities, we employed the principal component analysis (PCA) transform approach. Subsequently, we performed feature-level fusion on shallow feature maps, which retain rich geometric information. We also incorporated a channel attention module to highlight channels rich in features and suppress irrelevant information. This step is crucial due to the substantial similarity between SAR and optical images in shallow layers such as geometric features. In summary, we propose a generic multimodal fusion strategy that can be attached to most encoding–decoding structures for feature classification tasks, designed with two inputs. One input is the optical image, and the other is the three-band fusion data obtained by combining the PCA component of the optical image with the SAR. Our feature-level fusion method effectively integrates multimodal data. The efficiency of our approach was validated using various public datasets, and the results showed significant improvements when applied to several land cover classification models.

Rethinking the necessity of image fusion in high-level vision tasks: A practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity

Image fusion aims to integrate complementary characteristics of source images into a single fused image that better serves human visual observation and machine vision perception. However, most existing image fusion algorithms primarily focus on improving the visual appeal of fused images. Although there are some semantic-driven methods that consider semantic requirements of downstream applications, none of them have demonstrated the potential of image-level fusion compared to feature-level fusion, which fulfills high-level vision tasks directly on multi-modal features rather than on a fused image. To overcome these limitations, this paper presents a practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity constraints, termed PSFusion. First of all, the sparse semantic perception branch extracts sufficient semantic features, which are then progressively integrated into the fusion network using the semantic injection module to fulfill the semantic requirements of high-level vision tasks. The scene fidelity path within the scene restoration branch is devised to ensure that the fusion features contain complete information for reconstructing the source images. Additionally, the contrast mask and salient target mask are employed to construct the fusion loss to maintain impressive visual effects of fusion results. In particular, we provide quantitative and qualitative analyses to demonstrate the potential of image-level fusion compared to feature-level fusion for high-level vision tasks. With the rapid advancement of large-scale models, image-level fusion can expeditiously leverage the advantages of multi-modal data and state-of-the-art (SOTA) unimodal segmentation to achieve superior performance. Furthermore, extensive comparative experiments demonstrate the superiority of our PSFusion over SOTA image-level fusion alternatives in terms of visual appeal and high-level semantics. Even under harsh circumstances, our method offers satisfactory fusion results to serve subsequent high-level vision applications. The source code is available at https://github.com/Linfeng-Tang/PSFusion.

Deep versus Handcrafted Tensor Radiomics Features: Prediction of Survival in Head and Neck Cancer Using Machine Learning and Fusion Techniques.

Although handcrafted radiomics features (RF) are commonly extracted via radiomics software, employing deep features (DF) extracted from deep learning (DL) algorithms merits significant investigation. Moreover, a "tensor'' radiomics paradigm where various flavours of a given feature are generated and explored can provide added value. We aimed to employ conventional and tensor DFs, and compare their outcome prediction performance to conventional and tensor RFs. 408 patients with head and neck cancer were selected from TCIA. PET images were first registered to CT, enhanced, normalized, and cropped. We employed 15 image-level fusion techniques (e.g., dual tree complex wavelet transform (DTCWT)) to combine PET and CT images. Subsequently, 215 RFs were extracted from each tumor in 17 images (or flavours) including CT only, PET only, and 15 fused PET-CT images through the standardized-SERA radiomics software. Furthermore, a 3 dimensional autoencoder was used to extract DFs. To predict the binary progression-free-survival-outcome, first, an end-to-end CNN algorithm was employed. Subsequently, we applied conventional and tensor DFs vs. RFs as extracted from each image to three sole classifiers, namely multilayer perceptron (MLP), random-forest, and logistic regression (LR), linked with dimension reduction algorithms. DTCWT fusion linked with CNN resulted in accuracies of 75.6 ± 7.0% and 63.4 ± 6.7% in five-fold cross-validation and external-nested-testing, respectively. For the tensor RF-framework, polynomial transform algorithms + analysis of variance feature selector (ANOVA) + LR enabled 76.67 ± 3.3% and 70.6 ± 6.7% in the mentioned tests. For the tensor DF framework, PCA + ANOVA + MLP arrived at 87.0 ± 3.5% and 85.3 ± 5.2% in both tests. This study showed that tensor DF combined with proper machine learning approaches enhanced survival prediction performance compared to conventional DF, tensor and conventional RF, and end-to-end CNN frameworks.

PET/MR fusion texture analysis for the clinical outcome prediction in soft-tissue sarcoma

BackgroundVarious fusion strategies (feature-level fusion, matrix-level fusion, and image-level fusion) were used to fuse PET and MR images, which might lead to different feature values and classification performance. The purpose of this study was to measure the classification capability of features extracted using various PET/MR fusion methods in a dataset of soft-tissue sarcoma (STS).MethodsThe retrospective dataset included 51 patients with histologically proven STS. All patients had pre-treatment PET and MR images. The image-level fusion was conducted using discrete wavelet transformation (DWT). During the DWT process, the MR weight was set as 0.1, 0.2, 0.3, 0.4, …, 0.9. And the corresponding PET weight was set as 1- (MR weight). The fused PET/MR images was generated using the inverse DWT. The matrix-level fusion was conducted by fusing the feature calculation matrix during the feature extracting process. The feature-level fusion was conducted by concatenating and averaging the features. We measured the predictive performance of features using univariate analysis and multivariable analysis. The univariate analysis included the Mann-Whitney U test and receiver operating characteristic (ROC) analysis. The multivariable analysis was used to develop the signatures by jointing the maximum relevance minimum redundancy method and multivariable logistic regression. The area under the ROC curve (AUC) value was calculated to evaluate the classification performance.ResultsBy using the univariate analysis, the features extracted using image-level fusion method showed the optimal classification performance. For the multivariable analysis, the signatures developed using the image-level fusion-based features showed the best performance. For the T1/PET image-level fusion, the signature developed using the MR weight of 0.1 showed the optimal performance (0.9524(95% confidence interval (CI), 0.8413–0.9999)). For the T2/PET image-level fusion, the signature developed using the MR weight of 0.3 showed the optimal performance (0.9048(95%CI, 0.7356–0.9999)).ConclusionsFor the fusion of PET/MR images in patients with STS, the signatures developed using the image-level fusion-based features showed the optimal classification performance than the signatures developed using the feature-level fusion and matrix-level fusion-based features, as well as the single modality features. The image-level fusion method was more recommended to fuse PET/MR images in future radiomics studies.

Self-Supervised SAR-Optical Data Fusion of Sentinel-1/-2 Images

The effective combination of the complementary information provided by huge amount of unlabeled multisensor data (e.g., synthetic aperture radar (SAR) and optical images) is a critical issue in remote sensing. Recently, contrastive learning methods have reached remarkable success in obtaining meaningful feature representations from multiview data. However, these methods only focus on image-level features, which may not satisfy the requirement for dense prediction tasks such as land-cover mapping. In this work, we propose a self-supervised framework for SAR-optical data fusion and land-cover mapping tasks. SAR and optical images are fused by using a multiview contrastive loss at image level and super-pixel level according to one of those possible strategies: in the early, intermediate, and late strategies. For the land-cover mapping task, we assign each pixel a land-cover class by the joint use of pretrained features and spectral information of the image itself. Experimental results show that the proposed approach not only achieves a comparable accuracy but also reduces the dimension of features with respect to the image-level contrastive learning method. Among three fusion strategies, the intermediate fusion strategy achieves the best performance. The combination of the pixel-level fusion approach and the self-training on spectral indices leads to further improvements in the land-cover mapping task with respect to the image-level fusion approach, especially with sparse pseudo labels. The code to reproduce our results will be found at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/yusin2it/SARoptical_fusion</uri> .

Overall Survival Prognostic Modelling of Non-small Cell Lung Cancer Patients Using Positron Emission Tomography/Computed Tomography Harmonised Radiomics Features: The Quest for the Optimal Machine Learning Algorithm

AimsDespite the promising results achieved by radiomics prognostic models for various clinical applications, multiple challenges still need to be addressed. The two main limitations of radiomics prognostic models include information limitation owing to single imaging modalities and the selection of optimum machine learning and feature selection methods for the considered modality and clinical outcome. In this work, we applied several feature selection and machine learning methods to single-modality positron emission tomography (PET) and computed tomography (CT) and multimodality PET/CT fusion to identify the best combinations for different radiomics modalities towards overall survival prediction in non-small cell lung cancer patients. Materials and methodsA PET/CT dataset from The Cancer Imaging Archive, including subjects from two independent institutions (87 and 95 patients), was used in this study. Each cohort was used once as training and once as a test, followed by averaging of the results. ComBat harmonisation was used to address the centre effect. In our proposed radiomics framework, apart from single-modality PET and CT models, multimodality radiomics models were developed using multilevel (feature and image levels) fusion. Two different methods were considered for the feature-level strategy, including concatenating PET and CT features into a single feature set and alternatively averaging them. For image-level fusion, we used three different fusion methods, namely wavelet fusion, guided filtering-based fusion and latent low-rank representation fusion. In the proposed prognostic modelling framework, combinations of four feature selection and seven machine learning methods were applied to all radiomics modalities (two single and five multimodalities), machine learning hyper-parameters were optimised and finally the models were evaluated in the test cohort with 1000 repetitions via bootstrapping. Feature selection and machine learning methods were selected as popular techniques in the literature, supported by open source software in the public domain and their ability to cope with continuous time-to-event survival data. Multifactor ANOVA was used to carry out variability analysis and the proportion of total variance explained by radiomics modality, feature selection and machine learning methods was calculated by a bias-corrected effect size estimate known as ω2. ResultsOptimum feature selection and machine learning methods differed owing to the applied radiomics modality. However, minimum depth (MD) as feature selection and Lasso and Elastic-Net regularized generalized linear model (glmnet) as machine learning method had the highest average results. Results from the ANOVA test indicated that the variability that each factor (radiomics modality, feature selection and machine learning methods) introduces to the performance of models is case specific, i.e. variances differ regarding different radiomics modalities and fusion strategies. Overall, the greatest proportion of variance was explained by machine learning, except for models in feature-level fusion strategy. ConclusionThe identification of optimal feature selection and machine learning methods is a crucial step in developing sound and accurate radiomics risk models. Furthermore, optimum methods are case specific, differing due to the radiomics modality and fusion strategy used.

Multi-level multi-modality (PET and CT) fusion radiomics: prognostic modeling for non-small cell lung carcinoma

We developed multi-modality radiomic models by integrating information extracted from 18F-FDG PET and CT images using feature- and image-level fusions, toward improved prognosis for non-small cell lung carcinoma (NSCLC) patients. Two independent cohorts of NSCLC patients from two institutions (87 and 95 patients) were cycled as training and testing datasets. Fusion approaches were applied at two levels, namely feature- and image-levels. For feature-level fusion, radiomic features were extracted individually from CT and PET images and concatenated. Alternatively, radiomic features extracted separately from CT and PET images were averaged. For image-level fusion, wavelet fusion was utilized and tuned with two parameters, namely CT weight and Wavelet Band Pass Filtering Ratio. Clinical and combined clinical + radiomic models were developed. Gray level discretization was performed at 3 different levels (16, 32 and 64) and 225 radiomics features were extracted. Overall survival (OS) was considered as the endpoint. For feature reduction, correlated (redundant) features were excluded using Spearman’s correlation, and best combination of top ten features with highest concordance-indices (via univariate Cox model) were selected in each model for further multivariate Cox model. Moreover, prognostic score’s median, obtained from the training cohort, was used intact in the testing cohort as a threshold to classify patients into low- versus high-risk groups, and log-rank test was applied to assess differences between the Kaplan–Meier curves. Overall, while models based on feature-level fusion strategy showed limited superiority over single-modalities, image-level fusion strategy significantly outperformed both single-modality and feature-level fusion strategies. As such, the clinical model (C-index = 0.656) outperformed all models from single-modality and feature-level strategies, but was outperformed by certain models from image-level fusion strategy. Our findings indicated that image-level fusion multi-modality radiomics models outperformed single-modality, feature-level fusion, and clinical models for OS prediction of NSCLC patients.

Dual Siamese network for RGBT tracking via fusing predicted position maps

Visual object tracking is a basic task in the field of computer vision. Despite the rapid development of visual object tracking, it is not reliable to use only visible light images for object tracking in some cases. Since visible light and thermal infrared images have complementary advantages in imaging, and the use of them as a joint input for tracking becomes more noted, this kind of tracking is RGBT tracking. The existing RGBT tracking can be divided into image-level fusion tracking, feature-level fusion tracking, and response-level fusion tracking. Compared with the first two, response-level fusion tracking can use deeper dual-mode image information, but most of them use traditional tracking methods and introduce weights at inappropriate stages. Based on the above, we propose a response-level fusion tracking algorithm that employed deep learning. And the weight distribution is placed in the feature extraction stage, for which we design the joint modal channel attention module. We adopt the Siamese framework and expand it into a dual Siamese subnetwork. In the meantime, we improve the regional proposal subnetwork and propose the strategy for fusing two modal predicted position maps. To verify the performance of our algorithm, we conducted experiments on two tracking benchmarks. After testing, our algorithm has very good performance and runs at 116 frames per second, which far exceeds the real-time requirement of 25 frames per second.

Deep Learning for Automatic Differential Diagnosis of Primary Central Nervous System Lymphoma and Glioblastoma: Multi-Parametric Magnetic Resonance Imaging Based Convolutional Neural Network Model.

Differential diagnosis of primary central nervous system lymphoma (PCNSL) and glioblastoma (GBM) is useful to guide treatment strategies. To investigate the use of a convolutional neural network (CNN) model for differentiation of PCNSL and GBM without tumor delineation. Retrospective. A total of 289 patients with PCNSL (136) or GBM (153) were included, the average age of the cohort was 54 years, and there were 173 men and 116 women. 3.0 T Axial contrast-enhanced T1 -weighted spin-echo inversion recovery sequence (CE-T1 WI), T2 -weighted fluid-attenuation inversion recovery sequence (FLAIR), and diffusion weighted imaging (DWI, b=0 second/mm2 , 1000 seconds/mm2 ). A single-parametric CNN model was built using CE-T1 WI, FLAIR, and the apparent diffusion coefficient (ADC) map derived from DWI, respectively. A decision-level fusion based multi-parametric CNN model (DF-CNN) was built by combining the predictions of single-parametric CNN models through logistic regression. An image-level fusion based multi-parametric CNN model (IF-CNN) was built using the integrated multi-parametric MR images. The radiomics models were developed. The diagnoses by three radiologists with 6 years (junior radiologist Y.Y.), 11 years (intermediate-level radiologist Y.T.), and 21 years (senior radiologist Y.L.) of experience were obtained. The 5-fold cross validation was used for model evaluation. The Pearson's chi-squared test was used to compare the accuracies. U-test and Fisher's exact test were used to compare clinical characteristics. The CE-T1 WI, FLAIR, and ADC based single-parametric CNN model had accuracy of 0.884, 0.782, and 0.700, respectively. The DF-CNN model had an accuracy of 0.899 which was higher than the IF-CNN model (0.830, P=0.021), but had no significant difference in accuracy compared to the radiomics model (0.865, P=0.255), and the senior radiologist (0.906, P=0.886). A CNN model can differentiate PCNSL from GBM without tumor delineation, and comparable to the radiomics models and radiologists. 4 TECHNICAL EFFICACY: Stage 2.

Palmprint-Palmvein Fusion Recognition Based on Deep Hashing Network

Palmprint has attracted increasing attention due to its several advantages in the biometrics field. Deep learning has achieved remarkable performance in the computer vision area, so a large number of deep-learning-based methods have been proposed by the research community for palmprint recognition. The outputs of a deep hashing network (DHN) can be represented as a binary bit string, so DHN can reduce the storage and accelerate the matching/retrieval speed. In this paper, DHN is employed to extract the binary template for palmprint and palmvein verification. Spatial transformer network is used to overcome the rotation and dislocation. Palmprint and palmvein can be acquired from visible-light spectrums, including red (R), green (G), blue (B), and near infrared (NIR) spectrum, respectively. Since the features in different spectrums are different, their complementary advantages can be exploited to the full by fusion. Image-level fusion and score-level fusion are developed for palmprint-palmvein fusion recognition. The experiments demonstrate that score-level fusion can improve the accuracy efficiently.

Mass detection in mammograms by bilateral analysis using convolution neural network

Background and objectiveAutomatic detection of the masses in mammograms is a big challenge and plays a crucial role to assist radiologists for accurate diagnosis. In this paper, a bilateral image analysis method based on Convolution Neural Network (CNN) is developed for mass detection in mammograms. MethodsThe proposed bilateral mass detection method consists of two networks: a registration network for registering bilateral mammograms and a Siamese-Faster-RCNN network for mass detection using a pair of registered mammograms. In the first step, self-supervised learning network is built to learn the spatial transformation between bilateral mammograms. This network can directly estimate spatial transformation by maximizing an image-wise similarity metric and corresponding points labeling is not needed. In the second step, an end-to-end network combining the Region Proposal Network (RPN) and a Siamese Fully Connected (Siamese-FC) network is designed. Different from existing methods, the designed network integrates mass detection on single image with registered bilateral images comparison. ResultsThe proposed method is evaluated on three datasets (publicly available dataset INbreast and private dataset BCPKUPH and TXMD). For INbreast dataset, the proposed method achieves 0.88 true positive rate (TPR) with 1.12 false positives per image (FPs/I). For BCPKUPH dataset, the proposed method achieves 0.85 TPR with 1.86 FPs/I. For TXMD dataset, the proposed method achieves 0.85 TPR with 2.70 FPs/I. ConclusionsRegistration experimental result shows that the proposed method is suitable for bilateral mass detection. Mass detection experimental results show that the proposed method performs better than unilateral mass detection method, different bilateral connection schemes and image level fusion bilateral schemes.

Multi-Level Multi-Modality Fusion Radiomics: Application to PET and CT Imaging for Prognostication of Head and Neck Cancer.

To characterize intra-tumor heterogeneity comprehensively, we propose a multi-level fusion strategy to combine PET and CT information at the image-, matrix-and feature-levels towards improved prognosis. Specifically, we developed fusion radiomics in the context of 3 prognostic outcomes in a multi-center setting (4 centers) involving 296 head & neck cancer patients. Eight clinical parameters were first utilized to build a (1) clinical model. We also built models by extracting 127 radiomics features from (2) PET images alone; (3-8) PET and CT images fused via wavelet-based fusion (WF) using CT-weights of 0.2, 0.4, 0.6 and 0.8, gradient transfer fusion (GTF), and guided filtering-based fusion (GFF); (9) fused matrices (sumMat); (10-11) fused features constructed via feature averaging (avgFea) and feature concatenation (conFea); and finally, (12) CT images alone; above models were also expanded to include both clinical and radiomics features. Seven variations of training and testing partitions were investigated. Highest performance in 5, 6 and 5 partitions was achieved by image-level fusion strategies for RFS, MFS and OS prediction, respectively. Among all partitions, WF0.6 and WF0.8 showed significantly higher performance than CT model for RFS (C-index: 0.60±0.04vs. 0.56±0.03, p-value: 0.015) and MFS (C-index: 0.71±0.13vs. 0.62±0.08, p-value: 0.020) predictions, respectively. In partition CER 23vs. 14, WF0.6 significantly outperformed Clinical model for RFS prediction (C-index: 0.67vs. 0.53, p-value: 0.003); both avgFea and WF0.6 showed C-index of 0.64 and significantly higher than that of PET only (C-index: 0.51, p-value: 0.018 and 0.031, respectively) for OS prediction. Fusion radiomics modeling showed varying improvements compared to single modality models for different outcome predictions in different partitions, highlighting the importance of generalizing radiomics models. Image-level fusion holds potential to capture more useful characteristics.

An Improved Recognition Approach for Noisy Multispectral Palmprint by Robust L2 Sparse Representation with a Tensor-Based Extreme Learning Machine.

For the past decades, recognition technologies of multispectral palmprint have attracted more and more attention due to their abundant spatial and spectral characteristics compared with the single spectral case. Enlightened by this, an innovative robust L2 sparse representation with tensor-based extreme learning machine (RL2SR-TELM) algorithm is put forward by using an adaptive image level fusion strategy to accomplish the multispectral palmprint recognition. Firstly, we construct a robust L2 sparse representation (RL2SR) optimization model to calculate the linear representation coefficients. To suppress the affection caused by noise contamination, we introduce a logistic function into RL2SR model to evaluate the representation residual. Secondly, we propose a novel weighted sparse and collaborative concentration index (WSCCI) to calculate the fusion weight adaptively. Finally, we put forward a TELM approach to carry out the classification task. It can deal with the high dimension data directly and reserve the image spatial information well. Extensive experiments are implemented on the benchmark multispectral palmprint database provided by PolyU. The experiment results validate that our RL2SR-TELM algorithm overmatches a number of state-of-the-art multispectral palmprint recognition algorithms both when the images are noise-free and contaminated by different noises.

Multispectral image fusion for illumination-invariant palmprint recognition.

Multispectral palmprint recognition has shown broad prospects for personal identification due to its high accuracy and great stability. In this paper, we develop a novel illumination-invariant multispectral palmprint recognition method. To combine the information from multiple spectral bands, an image-level fusion framework is completed based on a fast and adaptive bidimensional empirical mode decomposition (FABEMD) and a weighted Fisher criterion. The FABEMD technique decomposes the multispectral images into their bidimensional intrinsic mode functions (BIMFs), on which an illumination compensation operation is performed. The weighted Fisher criterion is to construct the fusion coefficients at the decomposition level, making the images be separated correctly in the fusion space. The image fusion framework has shown strong robustness against illumination variation. In addition, a tensor-based extreme learning machine (TELM) mechanism is presented for feature extraction and classification of two-dimensional (2D) images. In general, this method has fast learning speed and satisfying recognition accuracy. Comprehensive experiments conducted on the PolyU multispectral palmprint database illustrate that the proposed method can achieve favorable results. For the testing under ideal illumination, the recognition accuracy is as high as 99.93%, and the result is 99.50% when the lighting condition is unsatisfied.

Application of Data Fusion to External Insulation Strength of Contaminated-Insulator Assessment

An external insulation of contaminated-insulator assessment method is proposed based on a tri-level data fusion model of combining principal component analysis (PCA) method, artificial neural net (ANN) method and evidence theory in this paper. When contaminated-insulators partial discharge (PD) occur, much effective information obtained from the sound emitted with PD are synthesized to evaluate the external insulation strength of insulators in operation by the studied method. Firstly, nine characteristic parameters that can rapidly reflect the PD process are selected for image-level fusion of PCA to reduce dimension, which gets two new parameters. Then the new parameters are inputted to ANN for feature-level fusion. Finally, the feature-level fusion output is used as the input of decision-level fusion and fused by means of D-S evidence theory for further reducing the uncertainty of assessment. The artificial contamination experiments were explored to verify the proposed method. The result indicates that the proposed model is more precise than the ANN model under the same conditions.

High-Speed Multispectral Iris Capture System Design

Multispectral iris recognition is one of the most reliable biometrics in terms of recognition performance. This paper describes the design and implementation of a high-speed multispectral iris capture device, which consists of four parts, i.e., capture, illumination, interaction, and control units. A multispectral iris image database is created by the proposed capture device, and then, we use the iris image-level fusion to further investigate the effectiveness of the proposed capture device by the 1-D log-Gabor wavelet filter approach. Experimental results are also presented in this paper.

Hierarchical fusion of multi-spectral face images for improved recognition performance

This paper presents a two level hierarchical fusion of face images captured under visible and infrared light spectrum to improve the performance of face recognition. At image level fusion, two face images from different spectrums are fused using DWT based fusion algorithm. At feature level fusion, the amplitude and phase features are extracted from the fused image using 2D log polar Gabor wavelet. An adaptive SVM learning algorithm intelligently selects either the amplitude or phase features to generate a fused feature set for improved face recognition. The recognition performance is observed under the worst case scenario of using single training images. Experimental results on Equinox face database show that the combination of visible light and short-wave IR spectrum face images yielded the best recognition performance with an equal error rate of 2.86%. The proposed image-feature fusion algorithm also performed better than existing fusion algorithms.

Guest Editorial: Special Section on Sensor Fusion 1998: A Case for Intelligent Fusion

Guest Editorial: Special Section on Sensor Fusion 1998: A Case for Intelligent Fusion