Fusion-based Classification Research Articles

A Deep Learning Model for Neural Network Optimization for Glaucoma Classification Using Fundus and Oct Feature Fusion

Irreversible vision loss is a common consequence of glaucoma, demands accurate and timely diagnosis for effective management. This research aims to enhance glaucoma classification accuracy by fusing information from two distinct imaging modalities like using deep learning explores the fusion of these modalities through an innovative neural network architecture with optimization. This approach combines Deep Stochastic Variational Autoencoder Convolution Neural Networks (DSVAECNN) and Adam optimization techniques to enable robust and accurate classification of glaucoma. A multi-path architecture is designed to accommodate both imaging as optical coherence tomography (OCT) radiomics features and fundus morphological features simultaneously. To ensure the effectiveness of the model, this study investigates the implementation of advanced optimization algorithm such as Adam, to expedite convergence and alleviate the risk of over fitting. The resulting model demonstrates improved generalization capabilities, critically for accurate diagnosis across diverse patient populations. A comprehensive OCT and fundus images dataset is used to evaluate the proposed approach from a representative cohort of glaucoma patients and healthy individuals. Quantitative various metrics including sensitivity, accuracy and specificity are employed toward appraise the accomplishment of the fusion-based classification model. Comparisons with current techniques show the dominance of the proposed move toward in accurately detecting glaucoma cases. This research provides the advancement of glaucoma diagnosis by effectively harnessing the synergy between fundus image and OCT scans and findings helpful information for clinics, ultimately facilitating early detection and personalized management of glaucoma, thus preserving vision for affected individuals.

Research on quantitative identification method for wire rope wire breakage damage signals based on multi-decomposition information fusion

Steel wire ropes are widely used in various fields, such as mining, elevators, and cable cars. However, their long-term use can lead to wire breakage, posing safety risks. The detection of wire breakages in steel wire ropes is crucial. This study addresses the shortcomings of existing quantitative identification methods for steel wire rope damage detection and proposes a novel model for fusion-based classification and recognition of wire rope damage. This model first combines the continuous wavelet transform and variational mode decomposition for feature extraction. Subsequently, it utilized convolutional neural networks to learn data features and introduced an attention mechanism to weigh and select the fused data. The final output provides the classification results, aiming to enhance the classification accuracy. Comparative experiments and ablation studies were conducted using the memory networks, autoencoder, and support vector machine models. The experimental results demonstrate the superiority of the proposed model regarding feature extraction, classification accuracy, and automation. The model achieved an accuracy rate of 94.44 % when classifying the nine types of wire breakages. This study presents an effective approach for signal processing and damage classification in steel wire rope damage detection, which is crucial for improving the reliability of wire breakage detection in steel wire ropes.

Sensor Fusion-based Classification Fault-tolerant System for Detected Objects in Autonomous Cars

Sensor Fusion-based Classification Fault-tolerant System for Detected Objects in Autonomous Cars

Feature-Decision Level Collaborative Fusion Network for Hyperspectral and LiDAR Classification

The fusion-based classification of hyperspectral (HS) and light detection and ranging (LiDAR) images has become a prominent research topic, as their complementary information can effectively improve classification performance. The current methods encompass pixel-, feature- and decision-level fusion. Among them, feature- and decision-level fusion have emerged as the mainstream approaches. Collaborative fusion of these two levels can enhance classification accuracy. Although various methods have been proposed, some shortcomings still exist. On one hand, current methods ignore the shared advanced features between HS and LiDAR images, impeding the integration of multimodal features and thereby limiting the classification performance. On the other hand, the existing methods face difficulties in achieving a balance between feature- and decision-level contributions, or they simply overlook the significance of one level and fail to utilize it effectively. In this paper, we propose a novel feature-decision level collaborative fusion network (FDCFNet) for hyperspectral and LiDAR classification to alleviate these problems. Specifically, a multilevel interactive fusion module is proposed to indirectly connect hyperspectral and LiDAR flows to refine the spectral-elevation information. Moreover, the fusion features of the intermediate branch can further enhance the shared-complementary information of hyperspectral and LiDAR to reduce the modality differences. In addition, a dynamic weight selection strategy is meticulously designed to adaptively assign weight to the output of three branches at the decision level. Experiments on three public benchmark datasets demonstrate the effectiveness of the proposed methods.

Madhubani Art Classification using transfer learning with deep feature fusion and decision fusion based techniques

Some traditional Indian art forms enjoy widespread popularity across the world. One of the most prominent among these is the Madhubani style. This art form’s rich heritage and beauty enthrall the connoisseurs and continue to inspire new designs catering to the changing tastes of prevalent fashion. Preservation of these traditional art forms is the need of the hour. Modern technological advances can be utilized with great advantage for this purpose. Since a database of Madhubani art forms was hitherto unavailable, an attempt is made in this work to create one from scratch. Five different classes of Madhubani art, i.e., Bharni, Godna, Kachni, Kohbar, and Tantrik, are identified, and the collected images are annotated with these classes. Classification of the art images is attempted using the handcrafted Local Binary Pattern (LBP) texture descriptors and state-of-the-art Convolutional Neural Networks (CNNs). The Transfer Learning approach with CNNs is employed to classify the designs. An attempt is made to obtain a better classification accuracy than the one provided by standard CNNs. Towards this end, the current work proposes a fusion of features extracted from several deep CNNs, decision fusion-based classification based on averaging prediction score (FAVG), and maximum vote score (FMAX). The proposed method’s performance is tested on our Madhubani art dataset and compared against several standard pre-trained CNNs available in the literature. The proposed approaches provide significantly higher classification accuracy for Madhubani art patterns, with decision fusion based on averaging prediction score (FAVG) approach being the best. The maximum accuracy, specificity, and error rate scores are 98.82%, 99.72%, and 1.18%, respectively. This is the first such attempt, and the excellent results motivate further work to develop content-based image retrieval tools and evolutionary design-based tools for automating the development of new designs. These endeavors are expected to go a long way in preserving precious art heritage and fostering its rapid growth in the world market. The dataset will be made publically available for further experimentation.

Integrating Zhuhai-1 Hyperspectral Imagery With Sentinel-2 Multispectral Imagery to Improve High-Resolution Impervious Surface Area Mapping

Mapping impervious surface area (ISA) in an accurate and timely manner is essential for a variety of fields and applications, such as urban heat islands, hydrology, waterlogging, and urban planning and management. However, the large and complex urban landscapes pose great challenges in retrieving ISA information. Spaceborne hyperspectral (HS) remote sensing imagery provides rich spectral information with short revisit cycles, making it an ideal data source for ISA extraction from complex urban scenes. Nevertheless, insufficient single-band energy, the involvement of modulation transfer function (MTF), and the low signal-to-noise ratio (SNR) of spaceborne HS imagery usually result in poor image clarity and noises, leading to inaccurate ISA extraction. To address this challenge, we propose a new deep feature fusion-based classification method to improve 10 m resolution ISA mapping by integrating Zhuhai-1 HS imagery with Sentinel-2 multispectral (MS) imagery. We extract deep features that include spectral and spatial features, respectively, from MS and HS imagery via a 2-D convolutional neural network (CNN), aiming to increase feature diversity and improve the model’s recognition capability. The Sentinel-2 imagery is used to enhance the spatial information of the Zhuhai-1 HS image, improving the urban ISA retrieval by reducing the impact of noises. By combining the deep spatial features and deep spectral features, we obtain joint spatial-spectral features, leading to high classification accuracy and robustness. We test the proposed method in two highly urbanized study areas that cover Foshan city and Wuhan city, China. The results reveal that the proposed method obtains an overall accuracy of 96.72% and 96.75% in the two study areas, 18.78% and 8.66% higher than classification results with only HS imagery as input. The final ISA extraction overall accuracy is 95.42% and 95.50% in the two study areas, the highest among the comparison methods.

OptFus: Optical Sensor Fusion for the Classification of Multisource Data: Application to Mineralogical Mapping

We propose a new fusion-based classification technique for optical multisource remote-sensing images called OptFus. OptFus is developed to merge and process optical imagery having different spatial and spectral resolutions. The spatial features are extracted using morphological filters from the RGB data containing high spatial resolution. A feature fusion technique is developed to combine all the sensor data in a subspace using a common set of representative features. Finally, the fused features are classified using a support vector machine to ensure a robust supervised spectral classification. The proposed method is designed to allocate varying weights to the data from various imaging sensors in the fusion process. OptFus is applied to two multisource optical datasets captured from geological drill-core samples. The classification accuracy demonstrates considerable improvements compared to the state-of-the-art. A MATLAB implementation of OptFus is available online: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/BehnoodRasti/OptFus</uri> .

Multimodal Earth observation data fusion: Graph-based approach in shared latent space

Multiple and heterogenous Earth observation (EO) platforms are broadly used for a wide array of applications, and the integration of these diverse modalities facilitates better extraction of information than using them individually. The detection capability of the multispectral unmanned aerial vehicle (UAV) and satellite imagery can be significantly improved by fusing with ground hyperspectral data. However, variability in spatial and spectral resolution can affect the efficiency of such dataset's fusion. In this study, to address the modality bias, the input data was projected to a shared latent space using cross-modal generative approaches or guided unsupervised transformation. The proposed adversarial networks and variational encoder-based strategies used bi-directional transformations to model the cross-domain correlation without using cross-domain correspondence. It may be noted that an interpolation-based convolution was adopted instead of the normal convolution for learning the features of the point spectral data (ground spectra). The proposed generative adversarial network-based approach employed dynamic time wrapping based layers along with a cyclic consistency constraint to use the minimal number of unlabeled samples, having cross-domain correlation, to compute a cross-modal generative latent space. The proposed variational encoder-based transformation also addressed the cross-modal resolution differences and limited availability of cross-domain samples by using a mixture of expert-based strategy, cross-domain constraints, and adversarial learning. In addition, the latent space was modelled to be composed of modality independent and modality dependent spaces, thereby further reducing the requirement of training samples and addressing the cross-modality biases. An unsupervised covariance guided transformation was also proposed to transform the labelled samples without using cross-domain correlation prior. The proposed latent space transformation approaches resolved the requirement of cross-domain samples which has been a critical issue with the fusion of multi-modal Earth observation data. This study also proposed a latent graph generation and graph convolutional approach to predict the labels resolving the domain discrepancy and cross-modality biases. Based on the experiments over different standard benchmark airborne datasets and real-world UAV datasets, the developed approaches outperformed the prominent hyperspectral panchromatic sharpening, image fusion, and domain adaptation approaches. By using specific constraints and regularizations, the network developed was less sensitive to network parameters, unlike in similar implementations. The proposed approach illustrated improved generalizability in comparison with the prominent existing approaches. In addition to the fusion-based classification of the multispectral and hyperspectral datasets, the proposed approach was extended to the classification of hyperspectral airborne datasets where the latent graph generation and convolution were employed to resolve the domain bias with a small number of training samples. Overall, the developed transformations and architectures will be useful for the semantic interpretation and analysis of multimodal data and are applicable to signal processing, manifold learning, video analysis, data mining, and time series analysis, to name a few.

Multiobjective Task Scheduling for Energy-Efficient Cloud Implementation of Hyperspectral Image Classification

Cloud computing has become a promising solution to efficient processing of remotely sensed big data, due to its high-performance and scalable computing capabilities. However, existing cloud solutions generally involve the problems of low resource utilization and high energy consumption when processing large-scale remote sensing datasets, affecting the quality-of-service of the cloud system. Aiming at hyperspectral image classification applications, this article proposes an energy-efficient cloud implementation by employing a multiobjective task scheduling algorithm. We first present a parallel computing mechanism for a fusion-based classification method based on Apache Spark. With the general classification flow represented by a workflow model, we formulate a multiobjective scheduling framework that jointly minimizes the total execution time as well as energy consumption. We further develop an effective scheduling algorithm to solve the multiobjective optimization problem and produce a set of Pareto-optimal solutions, providing the tradeoff between computational efficiency and energy efficiency. Experimental results demonstrate that the multiobjective scheduling approach proposed in this work can substantially reduce the execution time and energy consumption for performing large-scale hyperspectral image classification on Spark. In addition, our proposed algorithm can generate better tradeoff solutions to the multiobjective scheduling problem as compared to competing scheduling algorithms.

Nuclei-Based Features for Uterine Cervical Cancer Histology Image Analysis With Fusion-Based Classification.

Cervical cancer, which has been affecting women worldwide as the second most common cancer, can be cured if detected early and treated well. Routinely, expert pathologists visually examine histology slides for cervix tissue abnormality assessment. In previous research, we investigated an automated, localized, fusion-based approach for classifying squamous epithelium into Normal, CIN1, CIN2, and CIN3 grades of cervical intraepithelial neoplasia (CIN) based on image analysis of 61 digitized histology images. This paper introduces novel acellular and atypical cell concentration features computed from vertical segment partitions of the epithelium region within digitized histology images to quantize the relative increase in nuclei numbers as the CIN grade increases. Based on the CIN grade assessments from two expert pathologists, image-based epithelium classification is investigated with voting fusion of vertical segments using support vector machine and linear discriminant analysis approaches. Leave-one-out is used for the training and testing for CIN classification, achieving an exact grade labeling accuracy as high as 88.5%.

Breast cancer detection using MRF-based probable texture feature and decision-level fusion-based classification using HMM on thermography images

Breast cancer is one of the major causes of death for women in the last decade. Thermography is a breast imaging technique that can detect cancerous masses much faster than the conventional mammography technology. In this paper, a breast cancer detection algorithm based on asymmetric analysis as primitive decision and decision-level fusion by using Hidden Markov Model (HMM) is proposed. In this decision structure, by using primitive decisions obtained from extracted features from left and right breasts and also asymmetric analysis, final decision is determined by a new application of HMM. For this purpose, a novel texture feature based on Markov Random Field (MRF) model that is named MRF-based probable texture feature and another texture feature based on a new scheme in Local Binary Pattern (LBP) of the images are extracted. In the MRF-based probable texture feature, we try to capture breast texture information by using proper definition of neighborhood system and clique and also determination of new potential functions. Ultimately, our proposed breast cancer detection algorithm is evaluated on a variety dataset of thermography images and false negative rate of 8.3% and false positive rate of 5% are obtained on test image dataset.