Score Fusion Research Articles

Quality based adaptive score fusion approach for multimodal biometric system

Multimodal Biometric Systems are extensively employed over unimodal counterparts for user authentication in the digital world. However, the application of multimodal systems to security-critical applications is limited mainly due to non-adaptiveness of these systems to the dynamic environment and inability to distinguish between spoofing attack and the noisy input image. In order to address these issues, a multimodal biometric system, which adaptively combines the scores from individual classifiers is proposed. For this, three modalities viz. face, finger, and iris are used to extract individual classifier scores. These classifier scores are adaptively fused considering that concurrent modalities are boosted and discordant modalities are suppressed. The conflicting belief among classifiers is resolved not only to achieve optimum fusion of classifier scores but also to cater dynamic environment. The proposed quality based score fusion also distinguish between spoofing attacks and noisy inputs as well. The performance of the proposed multimodal biometric system is experimentally validated using three chimeric multimodal databases. On an average, the proposed system achieves an accuracy of 99.5%, an EER of 0.5% and also outperforms state-of-the-art methods.

BLAN: Bi-directional ladder attentive network for facial attribute prediction

Deep facial attribute prediction has received considerable attention with a wide range of real-world applications in the past few years. Existing works almost extract abstract global features at high levels of deep neural networks to make predictions. However, local features at low levels, which contain detailed local attribute information, are not well exploited. In this paper, we propose a novel Bi-directional Ladder Attentive Network (BLAN) to learn hierarchical representations, covering the correlations between feature hierarchies and attribute characteristics. BLAN adopts layer-wise bi-directional connections based on the autoencoder framework from low to high levels. In this way, hierarchical features with local and global attribute characteristics could be correspondingly interweaved at each level via multiple designed Residual Dual Attention Modules (RDAMs). Besides, we derive a Local Mutual Information Maximization (LMIM) loss to further incorporate the locality of facial attributes to high-level representations at each hierarchy. Multiple attribute classifiers receive hierarchical representations to produce local and global decisions, followed by a proposed adaptive score fusion module to merge these decisions for yielding the final prediction result. Extensive experiments on two facial attribute datasets, CelebA and LFWA, demonstrate that our BLAN outperforms state-of-the-art methods.

Regenerating face images from multi-spectral palm images using multiple fusion methods

This paper established a relationship between multi-spectral palm images and a face image based on multiple fusion methods. The first fusion method to be considered is a feature extraction between different multi-spectral palm images, where multi-spectral CASIA database was used. The second fusion method to be considered is a score fusion between two parts of an output face image. Our method suggests that both right and left hands are used, and that each hand aims to produce a significant part of a face image by using a Multi-Layer Perceptron (MLP) network. This will lead to the second fusion part to reconstruct the full-face image, in order to examine its appearance. This topology provided interesting results of Equal Error Rate (EER) equal to 1.99%.

Relationship between cognitive fusion, empirical avoidance and obsessive-compulsive symptoms in patients with obsessive-compulsive disorder

Objective To compare the difference of cognitive fusion, empirical avoidance between patients with obsessive-compulsive disorder and healthy control group, and to explore the relationship between obsessive-compulsive score and cognitive fusion, empirical avoidance. Methods The cognitive fusion questionnaire(CFQ), acceptance and action questionnaire-2nd edition(AAQ-II) and Yale-Brown scale for obsessive-compulsive symptoms (Y-BOCS) were used to investigate 100 subjects with obsessive-compulsive disorder (OCD) and 166 healthy controls.And the differences in cognitive fusion and empirical avoidance scores were compared between the two groups.The relationship of obsessive-compulsive symptoms and cognitive fusion, empirical avoidance was analyzed using multiple linear regression analysis. Results The scores of cognitive fusion ((49.89±10.62) vs (33.88±11.44), t=-11.345, P<0.01)and empirical avoidance ((29.75±9.53) vs (21.59±7.03), t=-7.995, P<0.01) in obsessive-compulsive disorder group were significantly higher than those in healthy control group.There were significant differences in cognitive fusion and empirical avoidance in age variables (F=8.63, P<0.01). In AAQ-Ⅱ, item 2 (r=0.246, P<0.05), item 6 (r=0.223, P<0.05) and total score (r=0.240, P<0.05) were positively correlated with the total score of Y-BOCS.Item 2 (r=0.311, P<0.01), item 3 (r=0.286, P<0.05), item 6 (r=0.248, P<0.05) and total score (r=0.229, P<0.05) were positively correlated with the scores of obsessive-thinking.In CFQ, item 2 (r=0.231, P<0.0) 5), item 4 (r=0.242, P<0.05), item 7 (r=0.308, P<0.05), item 8 (r=0.277, P<0.05) and item 9 (r=0.249, P<0.05) were positively correlated with the total score of Y-BOCS.Item 8 (r=0.261, P<0.05) was positively correlated with the scores of obsessive-thinking, item 7 (r=0.237, P<0.05) and item 9 (r=0.238, P<0.05) were positively correlated with scores of obsessive-compulsive behavior.When predicting total obsessive-compulsive scores, only CF item 7 of Q (B=1.827, P<0.01), item 3 (B=0.956, P<0.05), and item 6 of AAQ-Ⅱ (B=0.584, P<0.05) entered the equation with a joint explanatory variation of 19%.When predicting the score of obsessive-thinking, only item 2 of AAQ-Ⅱ (B=0.446, P<0.01) entered the equation, explaining the variance was 9.7%.In the prediction of obsessive-compulsive behavior, only item 9 (B=0.815, P<0.05) of CFQ entered the equation, and the explanatory variation was 5.6%. Conclusion Cognitive fusion in the patients with obsessive-compulsive disorder and the high level of empirical avoidance may be an important factor for the maintenance of the symptoms. Key words: Obsessive-compulsive disorder; Cognitive fusion; Empirical avoidance; Acceptance and commitment therapy

A New Time\u2013Frequency Attention Tensor Network for Language Identification

In this paper, we aim to improve traditional DNN x-vector language identification performance by employing wide residual networks (WRN) as a powerful feature extractor which we combine with a novel frequency attention network. Compared with conventional time attention, our method learns discriminative weights for different frequency bands to generate weighted means and standard deviations for utterance-level classification. This mechanism enables the architecture to direct attention to important frequency bands rather than important time frames, as in traditional time attention methods. Furthermore, we then introduce a cross-layer frequency attention tensor network (CLF-ATN) which exploits information from different layers to recapture frame-level language characteristics that have been dropped by aggressive frequency pooling in lower layers. This effectively restores fine-grained discriminative language details. Finally, we explore the joint fusion of frame-level and frequency-band attention in a time–frequency attention network. Experimental results show that firstly, WRN can significantly outperform a traditional DNN x-vector implementation; secondly, the proposed frequency attention method is more effective than time attention; and thirdly, frequency–time score fusion can yield further improvement. Finally, extensive experiments on CLF-ATN demonstrate that it is able to improve discrimination by regaining dropped fine-grained frequency information, particularly for low-dimension frequency features.

Features-Enhanced Multi-Attribute Estimation with Convolutional Tensor Correlation Fusion Network

To achieve robust facial attribute estimation, a hierarchical prediction system referred to as tensor correlation fusion network (TCFN) is proposed for attribute estimation. The system includes feature extraction, correlation excavation among facial attribute features, score fusion, and multi-attribute prediction. Subnetworks (Age-Net, Gender-Net, Race-Net, and Smile-Net) are used to extract corresponding features while Main-Net extracts features not only from an input image but also from corresponding pooling layers of subnetworks. Dynamic tensor canonical correlation analysis (DTCCA) is proposed to explore the correlation of different targets’ features in the F7 layers. Then, for binary classifications of gender, race, and smile, corresponding robust decisions are achieved by fusing the results of subnetworks with those of TCFN while for age prediction, facial image into one of age groups, and then ELM regressor performs the final age estimation. Experimental results on benchmarks with multiple face attributes (MORPH-II, Adience Benchmark datasets, LAP-2016, and CelebA) show that the proposed approach has superior performance compared to state of the art.

Synthesis Score Level Fusion Based Multifarious Classifier for Multi-Biometrics Applications

In this paper, we are presenting a face and signature recognition method from a large dataset with the different pose and multiple features. Initially, Face and corresponding signature are detected from devices for further pre-processing. Face recognition is the first stage of a system then the signature verification will be done. The proposed Legion feature based verification method will be developed using four important steps like, (i) feature extraction from face and data glove signals using feature Extraction. The various Features like Local binary pattern, shape and geometrical features of face, then the global and local features of the signatures were extracted. (ii) Score match normalization is used to enhance the recognition accuracy using min–max and median estimations. (iii) Then the match scores are evaluated using synthesis score level fusion based feature matching through Euclidean distance, (iv) Recognition based on the final score. Finally based on the feature library the face image and signature can be recognized. The similarity measurement is done by using Synthesis score level fusion (SSF) based multifarious Neural network (MNN) Classifier with weighted summation formulae where two weights will be optimally found out using Adapted motion search optimization algorithm. Finally SSF-MNN based matching score fusion based decision classifier to determine recognized and non-recognized biometrics. Moreover, in comparative analysis, a proposed technique is compared with the existing method by several performance metrics and the proposed SSF-MNN technique efficiently recognize the face images and corresponding signature from the input databases than the existing technique.

Clustering Algorithms and Validation Indices for a Wide mmWave Spectrum

Radio channel propagation models for the millimeter wave (mmWave) spectrum are extremely important for planning future 5G wireless communication systems. Transmitted radio signals are received as clusters of multipath rays. Identifying these clusters provides better spatial and temporal characteristics of the mmWave channel. This paper deals with the clustering process and its validation across a wide range of frequencies in the mmWave spectrum below 100 GHz. By way of simulations, we show that in outdoor communication scenarios clustering of received rays is influenced by the frequency of the transmitted signal. This demonstrates the sparse characteristic of the mmWave spectrum (i.e., we obtain a lower number of rays at the receiver for the same urban scenario). We use the well-known k-means clustering algorithm to group arriving rays at the receiver. The accuracy of this partitioning is studied with both cluster validity indices (CVIs) and score fusion techniques. Finally, we analyze how the clustering solution changes with narrower-beam antennas, and we provide a comparison of the cluster characteristics for different types of antennas.

A Novel Score Fusion Approach for Breast Cancer Diagnosis

Meme kanseri tüm dünyada yaygın bir hastalık olması sebebiyle hastalığın erken teşhisi, hastaların bu hastalıktan tamamen kurtulabilmeleri açısından kritik öneme sahiptir. Hastalığın teşhisini kolaylaştırmak için tıp doktorları bilgisayar destekli uzman sistemlerden yararlanabilmektedir. Bu çalışmada meme kanseri veri örneklerini iyi huylu veya kötü huylu sınıflarına ayırmak için genel regresyon sinir ağı (Generalized Regression Neural Network-GRNN) ve ileri beslemeli sinir ağı (Feed Forward Neural Network-FFNN) temelli bir skor füzyon yöntemi önerilmiştir. Önerilen yöntem Wisconsin Teşhis Meme Kanseri (Wisconsin Diagnostic Breast Cancer-WDBC) veri seti üzerinde test edilmiştir. Bu iki temel ağın ve önerilen yöntemin kullanışlılığı incelenmiş ve performans sonuçları karşılaştırmalı olarak sunulmuştur. Önerilen yöntem sınıflandırma doğruluğu bakımından literatürde WDBC veri setini kullanarak yapılan mevcut çalışmalar ile kıyaslanmıştır. Elde edilen deneysel sonuçlar önerilen yöntemin, meme kanseri teşhisi için umut vadettiğini ve tıp uzmanlarının hastalığa ilişkin karar vermelerinde yardımcı bir araç olarak kullanılabileceğini göstermektedir.

A two-step idle-state detection method for SSVEP BCI.

The role of detecting work/idle state in asynchronous Steady-state visual evoked potential (SSVEP) Brain-computer interface (BCI) or a self-paced SSVEP BCI has received increased attention in recent years. This study proposed a tree structure method which identifies the work/idle state based on the frequency recognition to detect work/idle state. Firstly, a frequency recognition estimated with task-related component analysis (TRCA). Then, the work/idle state is classified with step-wise linear discriminant analysis (SWLDA) using the data fusion of TRCA scores and power spectral density (PSD) as features. This method was evaluated by Electroencephalography (EEG) data from fourteen healthy participants with eight frequencies as work states and three idle state conditions. The averaged AUC of this method achieved 0.89 with data lengths of one second, which was significantly higher than that of the conventional power spectrum-based algorithm. The proposed method could identify the work/idle state fast and accurately, making the SSVEP BCI better suited for practical application.

Preliminary fusion of EEG and MRI with phenotypic scores in children with epilepsy based on the Canonical Polyadic Decomposition.

Cognitive and behavioural impairments in early-onset epilepsy affect the children and families' quality of life. Our ability to detect these impairments is limited, and it requires laborious questionnaires. Here, we describe a pilot study exploring the fusion of resting-state EEG, volumetric MRI, and phenotypic scores of child development based on the Canonical Polyadic Decomposition, expanding the recently presented Joint EEG-Development Inference (JEDI) model. Pilot data fusion was performed on functional, structural and developmental brain features of 29 preschool children diagnosed with epilepsy. The results suggest that combining multimodal brain data towards a comprehensive analysis of brain development in young children is plausible.

Differentiating between morphologically similar species in genus Cinnamomum (Lauraceae) using deep convolutional neural networks

Plant adulteration is a rising problem in the forest industry. Cinnamomum osmophloeum Kanehira (Lauraceae) is an evergreen plant that yields cinnamaldehyde. Although two other species—C. burmannii (Nees & T. Nees) Blume and C. insularimontanum Hayata— morphologically resemble C. osmophloeum, they produce a minimum amount of cinnamaldehyde, and thus have lower economical values. The adulteration of C. osmophloeum using C. burmannii has been reported. However, even for experts, differentiating between the three species on the basis of their appearance is challenging due to their high degree of similarity in appearance. This study proposed to identify the three Cinnamomum species using leaf images and deep convolutional neural networks (CNN). In this study, leaf images of the three species were acquired using flatbed scanners. Leaf patches around the blade center of the leaves were extracted. Classifiers based on deep CNN models, VGG16, Inception-V3, and NASNet, were then developed using the leaf images or patches as inputs. Score fusion was then applied to improve the performance of the developed CNN classifiers. The fused CNN classifiers reached a test accuracy as high as 96.7%. A performance comparison indicated that three developed deep CNN classifiers outperformed support vector machine classifiers.

AFIF4: Deep gender classification based on AdaBoost-based fusion of isolated facial features and foggy faces

Gender classification aims at recognizing a person’s gender. Despite the high accuracy achieved by state-of-the-art methods for this task, there is still room for improvement in generalized and unrestricted datasets. In this paper, we advocate a new strategy inspired by the behavior of humans in gender recognition. Instead of dealing with the face image as a sole feature, we rely on the combination of isolated facial components and a contextual feature which we call the foggy face. Then, we use these features to train deep convolutional neural networks followed by an AdaBoost-based score fusion to infer the final gender class. We evaluate our method on four challenging datasets to demonstrate its efficacy in achieving better or on-par accuracy with state-of-the-art methods. In addition, we present a new face dataset that intensifies the challenges of occluded faces and illumination changes, which we believe to be a much-needed resource for gender classification research.

Toward efficient indexing structure for scalable content-based music retrieval

With advancement of various information processing and storage techniques, the scale of digital music collections has been growing at very fast speed during recent decades. To support high-quality content-based retrieval over such a large volume of music data, how to develop indexing structure with good effectiveness, efficiency and scalability becomes an important research issue. However, existing techniques mainly focus on improving query efficiency. Very few approaches have been proposed to address issues related to scalability and accuracy. In this study, we address the problem via introducing a novel indexing technique called effective music indexing framework (EMIF) to facilitate scalable and accurate music retrieval. It is designed based on a “classification-and-indexing” principle and consists of two main functionality modules: (1) music classification—a novel semantic-sensitive classification to identify an input song’s category and (2) indexing module—multiple local indexing structures, one for each semantic category to reduce query response time significantly. In particular, the classification model combining linear discriminative mixture model (LDMM) and advanced score fusion scheme has been applied to estimate category of music accurately. Layered architecture enables EMIF to enjoy superior scalability and efficiency. To evaluate the approach, a set of experimental studies has been carried out using two large music test collections and the results demonstrate various advantages of EMIF over state-of-the-art approaches including efficiency, scalability and effectiveness.

Integrating Feature Extractors for the Estimation of Human Facial Age

ABSTRACTFacial feature extraction algorithms play an important role in many applications of face biometrics such as face recognition for person identification, classification of emotions by facial expression recognition and age estimation using facial images. In this paper, an integration of different type of feature extraction algorithms is applied on facial images for accurate age estimation. This integration is performed by using two-level fusion of features and scores with the help of feature-level and score-level fusion techniques. In our proposed method, the advantage of using different types of features such as biologically inspired features, texture-based features, and appearance-based features is used. Feature-level fusion of biologically inspired and texture-based methods is integrated into the proposed method and their combination is fused with an appearance-based method using score-level fusion. Experiments on the publicly available MORPH and FG-NET databases prove the effectiveness of the proposed method and the proposed method outperforms many of the state-of-the-art systems.

Multiclass Alpha Integration of Scores from Multiple Classifiers.

Alpha integration methods have been used for integrating stochastic models and fusion in the context of detection (binary classification). Our work proposes separated score integration (SSI), a new method based on alpha integration to perform soft fusion of scores in multiclass classification problems, one of the most common problems in automatic classification. Theoretical derivation is presented to optimize the parameters of this method to achieve the least mean squared error (LMSE) or the minimum probability of error (MPE). The proposed alpha integration method was tested on several sets of simulated and real data. The first set of experiments used synthetic data to replicate a problem of automatic detection and classification of three types of ultrasonic pulses buried in noise (four-class classification). The second set of experiments analyzed two databases (one publicly available and one private) of real polysomnographic records from subjects with sleep disorders. These records were automatically staged in wake, rapid eye movement (REM) sleep, and non-REM sleep (three-class classification). Finally, the third set of experiments was performed on a publicly available database of single-channel real electroencephalographic data that included epileptic patients and healthy controls in five conditions (five-class classification). In all cases, alpha integration performed better than the considered single classifiers and classical fusion techniques.

Secure multimodal biometric system based on diffused graphs and optimal score fusion

Biometric authentication systems offer potential advantages over traditional knowledge-based methods. However, most of the biometric systems which are extensively used lack template security and robustness. In order to address these issues, in this study, the authors have proposed a multimodal biometric system based on the combination of multiple modalities and optimal score level fusion. In addition, key features are introduced for each modality for generating cancellable biometric features. Features from individual traits are combined with corresponding key features to provide feature transformation. A robust template is generated by diffusion of individual transformed matrices using graph-based random walk cross-diffusion. In addition, individual classifier score is optimally fused using proposed multistage score level fusion model. Optimal belief masses for individual classifier are determined using cuckoo search optimisation. Wherein, optimal classifier beliefs are fused using DSmT based proportional conflict redistribution (PCR-6) rules. Experimental results demonstrate that optimal score fusion applied on cross-diffused features produce better results than existing state-of-the-art multimodal fusion schemes. On average of the outcome, equal error rate and accuracy achieved using the proposed method on four chimeric benchmarked datasets, are 2.32 and 98.316%.

Machine learning based soft biometrics for enhanced keystroke recognition system

The proposed work investigates the performance enhancement of keystroke biometric recognition using soft biometric with filter and Score Boost Weighting (SBW) scheme. Usually, Keystroke recognition performance is lower due to user’s emotional behaviour or distraction, typing patterns vary from user normal position which causes recognition error of genuine user for degrading the recognition accuracy. To address this problem, this work presents Dual Matcher with fusion to reduce the false rejection of genuine user to improve the accuracy of keystroke recognition. In this paper, soft biometric is used as secondary information to improve the recognition accuracy for primary keystroke biometric system. Specifically, soft biometrics provides additional support for keystroke biometric recognition at the combination approach. The performance of keystroke system can be further improved using SVM as machine learning under the score level fusion in the combination approach. Lastly, the fusion technique is used to combine the primary and secondary biometric. The new approach with score fusion enhances the overall performance of keystroke biometric system with 99% accuracy. Maximum of 2% improvement is achieved compared to existing works.

An Improved Two-Step Face Recognition Algorithm Based on Sparse Representation

Weighted score fusion is a widely used score fusion scheme, but the weights need to be set manually. The results generally vary greatly when the weights are different, so it is difficult to find the optimal weights. This is why it is necessary to constantly set different weights for experimental comparisons to find the optimal weights. In this paper, an improved fusion method is proposed for above shortcoming, that is, multiplication fusion applicable to sparse representation. The fusion scheme not only is easy to use but also does not need to be artificially set weights. Moreover, it is consistent with the correlation between the classification error and the score obtained by the experimental analysis. In the field of face recognition, it has been shown that the two-step face recognition (TSFR) based on representation using the original training samples and the generated “symmetric face” training samples can achieve excellent face recognition performance. Face recognition based on multiplication fusion and TSFR proposed in this paper can further improve the recognition accuracy.

Detection of Double Compressed HEVC Videos Using GOP-Based PU Type Statistics

Double compression detection is of great significance to recover the compression history of suspicious videos and help analyzers locate tampering points in forged videos. However, the lack of efficient methods still haunts researchers when it comes to double HEVC compression detection with a shifted group of pictures (GOP) structures. To tackle this problem, we first analyze the properties of re-encoded frames in double compressed HEVC videos. It is found that the abnormal statistics of prediction unit (PU) types in relocated I-frames can be used as the clue to expose double compression. Relocated I-frames are re-encoded P-frames, which were I-frames in the first compression. Different from the existing works, we designed the GOP-based features instead of frame-based features, enabling more efficient utilization of temporal variation patterns of PU-type statistics within each GOP unit. In the proposed method, the ratio of I-PUs and S-Pus is first computed to construct PU sequences for each GOP unit. The three-tap median filter is applied to PU sequences in order to mitigate the influence of various video contents. Several first-order statistics, including average, variation, and maximum/minimum, are selected to generate a 6-D detection feature. Then, multi-layer perceptron (MLP) is used to classify GOP-based features. Finally, the score fusion strategy is applied to obtain the detection result of the input video. Several publicly available YUV sequences are used to construct the double compression dataset with various coding parameters. In experiments, the proposed method outperforms several state-of-the-art methods under different settings of bitrates and GOP structures. The proposed method has more robust detection capability even when tampered videos are generated by combining two video clips with different GOP sizes.