Feature-level Fusion Framework Research Articles

A novel characterisation method for asphalt pavement structural performance assessment based on vehicle vibration data fusion

ABSTRACT Rapid structural performance assessment is essential to ensure asphalt pavement condition and efficient road network operation. However, extracting deeper pavement structural performance information based on vehicle vibration remains a major challenge. This study proposes a vibration data fusion method to realise the task of accurate structural performance characterisation. A 2D reconstruction algorithm transforms raw vibration signals into vibration images. Based on reconstruction images generated from different sensors and sensing directions, a pixel-level fusion framework and a feature-level fusion framework were put to improve the characterization accuracy. The study extracted two effective image texture features to characterise asphalt pavement’s structural strength. Fitting with deflection data, the average goodness-of-fit of optimised indicators combination under a feature-level fusion framework can reach 0.827. It is 32.5% and 15.2% higher than two unfused single-index accuracy. The data fusion framework shows the practicality in evaluating pavement structural performance rapidly and accurately through the vehicle vibration response.

A Bayesian CNN-based fusion framework of sensor fault diagnosis

Sensors equipped on the high-speed train provide large amounts of data which contributes to its state monitoring. However, it is challenging to distinguish whether the fault originates from the mechanical component or the sensors themselves. The main difficulties lie in the biased amount of normal and fault data as well as the deficiency of multi-source data’s inherent correlation. In this paper, we propose a Bayesian convolutional neural networks (CNN)-based fusion framework to enhance the ability to identify sensor errors. The framework utilizes wavelet time–frequency maps to extract abnormal features, employs a Bayesian CNN to obtain spatial features from a single sensor, integrates multi-source features via bidirectional long short-term memory network and enhances the acquired spatial and temporal features using an attention mechanism. The enhanced information finally generated leads to precise identification of the sensor faults. The proposed feature-level fusion framework and the associated attention mechanism facilitate discovering the inherent correlation and filtering of irrelevant information. Results indicate that our proposed method achieves 95.4% in terms of accuracy, which outperforms methods relying on feature extraction with single-source sensors by 7.8%.

A Novel Feature-Level Fusion Framework Using Optical and SAR Remote Sensing Images for Land Use/Land Cover (LULC) Classification in Cloudy Mountainous Area

Remote sensing data plays an important role in classifying land use/land cover (LULC) information from various sensors having different spectral, spatial and temporal resolutions. The fusion of an optical image and a synthetic aperture radar (SAR) image is significant for the study of LULC change and simulation in cloudy mountain areas. This paper proposes a novel feature-level fusion framework, in which the Landsat operational land imager (OLI) images with different cloud covers, and a fully polarized Advanced Land Observing Satellite-2 (ALOS-2) image are selected to conduct LULC classification experiments. We take the karst mountain in Chongqing as a study area, following which the features of the spectrum, texture, and space of the optical and SAR images are extracted, respectively, supplemented by the normalized difference vegetation index (NDVI), elevation, slope and other relevant information. Furthermore, the fused feature image is subjected to object-oriented multi-scale segmentation, subsequently, an improved support vector machine (SVM) model is used to conduct the experiment. The results showed that the proposed framework has the advantages of multi-source data feature fusion, high classification performance and can be applied in mountain areas. The overall accuracy (OA) was more than 85%, with the Kappa coefficient values of 0.845. In terms of forest, gardenland, water, and artificial surfaces, the precision of fusion image was higher compared to single data source. In addition, ALOS-2 data have a comparative advantage in the extraction of shrubland, water, and artificial surfaces. This work aims to provide a reference for selecting the suitable data and methods for LULC classification in cloudy mountain areas. When in cloudy mountain areas, the fusion features of images should be preferred, during the period of low cloudiness, the Landsat OLI data should be selected, when no optical remote sensing data are available, and the fully polarized ALOS-2 data are an appropriate substitute.

Local Oppugnant Color Texture Pattern for image retrieval system

The current scenario of image retrieval pays attention to local texture patterns. The recently proposed Local Tetra Pattern (LTrP) represents the image by the directional information and gives promising results. This paper proposes Local Oppugnant Color Texture Pattern (LOCTP), an enhancement of LTrP, which is able to discriminate the information derived from spatial inter-chromatic texture patterns of different spectral channels within a region. It determines the relationship in terms of the intensity and directional information between the referenced pixels and their oppugnant neighbors. The LOCTP strives to use the harmonized link between color and texture, which helps the system to incorporate the human perception. The experimental analysis of the proposed method is done with state-of-art techniques by using standard image databases Brodatz texture database (DB1) and Corel database (DB2). Also, the evaluation has been done in various color models like YCbCr, HSV, Lab, and RGB. In addition, a feature-level fusion framework is used to combine the Colored Pattern Appearance Model (CPAM) and the LOCTP for getting better result in natural images. The experimental results show considerable improvement in terms of average precision, average recall and average retrieval rate when compared with the previous works.

Facial Feature Extraction Based on Local Color and Texture for Face Recognition using Neural Network

For the purpose of face recognition (FR), the new color local texture features, i.e., color local Gabor wavelets (CLGWs) and color local binary pattern (CLBP), are being proposed. The proposed color local texture features are able to exploit the discriminative information derived from spatiochromatic texture patterns of different spectral channels within a certain local face region. Furthermore, in order to maximize a complementary effect taken by using color and texture information, the opponent color texture features that capture the texture patterns of spatial interactions between spectral channels are also incorporated into the generation of CLGW and CLBP. In addition, to perform the final classification, multiple color local texture features (each corresponding to the associated color band) are combined within a feature-level fusion framework using Neural Network. Particularly, compared with gray scale texture features, the proposed color local texture features are able to provide excellent recognition rates for face images taken under severe variation in illumination, as well as some variations in face images.

Multibiometric Cryptosystems Based on Feature-Level Fusion

Multibiometric systems are being increasingly de- ployed in many large-scale biometric applications (e.g., FBI-IAFIS, UIDAI system in India) because they have several advantages such as lower error rates and larger population coverage compared to unibiometric systems. However, multibiometric systems require storage of multiple biometric templates (e.g., fingerprint, iris, and face) for each user, which results in increased risk to user privacy and system security. One method to protect individual templates is to store only the secure sketch generated from the corresponding template using a biometric cryptosystem. This requires storage of multiple sketches. In this paper, we propose a feature-level fusion framework to simultaneously protect multiple templates of a user as a single secure sketch. Our main contributions include: (1) practical implementation of the proposed feature-level fusion framework using two well-known biometric cryptosystems, namery,fuzzy vault and fuzzy commitment, and (2) detailed analysis of the trade-off between matching accuracy and security in the proposed multibiometric cryptosystems based on two different databases (one real and one virtual multimodal database), each containing the three most popular biometric modalities, namely, fingerprint, iris, and face. Experimental results show that both the multibiometric cryptosystems proposed here have higher security and matching performance compared to their unibiometric counterparts.

Color Local Texture Features for Color Face Recognition

This paper proposes new color local texture features, i.e., color local Gabor wavelets (CLGWs) and color local binary pattern (CLBP), for the purpose of face recognition (FR). The proposed color local texture features are able to exploit the discriminative information derived from spatiochromatic texture patterns of different spectral channels within a certain local face region. Furthermore, in order to maximize a complementary effect taken by using both color and texture information, the opponent color texture features that capture the texture patterns of spatial interactions between spectral channels are also incorporated into the generation of CLGW and CLBP. In addition, to perform the final classification, multiple color local texture features (each corresponding to the associated color band) are combined within a feature-level fusion framework. Extensive and comparative experiments have been conducted to evaluate our color local texture features for FR on five public face databases, i.e., CMU-PIE, Color FERET, XM2VTSDB, SCface, and FRGC 2.0. Experimental results show that FR approaches using color local texture features impressively yield better recognition rates than FR approaches using only color or texture information. Particularly, compared with grayscale texture features, the proposed color local texture features are able to provide excellent recognition rates for face images taken under severe variation in illumination, as well as for small- (low-) resolution face images. In addition, the feasibility of our color local texture features has been successfully demonstrated by making comparisons with other state-of-the-art color FR methods.