Classification Experiments Research Articles

Research on Library Resource Management Based on Modern Information Technology and Reconfigurable Mobile Information System

Library resource library management based on modern information technology and reconfigurable mobile information systems have attracted much attention from researchers of academia. This paper introduces the LSTM model into the field of Chinese library resource management, and tries to propose a classification system with practical application value. The main research work:The Chinese library resource is designed LSTM model, and three experiments are designed on this basis: the coarse-grained book single classification experiment, different measures on the classification results; the fine-grained book single classification experiment, to compare The advantages and disadvantages of direct classification and layer-by-layer classification; coarse-grained book multi-classification experiments to explore the feasibility of multi-classification. After analyzing the experimental results, the model has achieved a good classification effect, and overcomes some deficiencies in the library resource library predecessors, and has a strong practical application value.

Learning real-world heterogeneous noise models with a benchmark dataset

Noise modeling is an important research field in computer vision; the design of an accurate model for imaging sensor noise depends on not only a comprehensive benchmark dataset of the real world, but also a precise design of the noise modeling algorithm. However, due to the inaccurate estimation method of noise-free images and limited shooting scenes, the current realistic datasets could not describe the diverse noise properties sufficiently. Moreover, popular parametric noise models are not sophisticated enough to characterize the real-world noise exactly. In this work, we first construct a more comprehensive dataset of the real world by capturing more indoor and outdoor scenes under different lighting conditions using diverse smartphones, then we propose a non-parametric noise estimation method capable of modeling the spatial heterogeneity of real-world noise patterns. Specifically, in order to characterize the spatial heterogeneity of real-world noise, we assume a non-i.i.d Gaussian distribution and propose a deep convolutional neural network (DCNN)-based approach for learning pixel-wise noise variance maps. To learn the pixel-wise variance map, we have constructed a variance estimation network mapping from the conditional signals (clean image, ISO, and camera model) to surrogate labels obtained from the nonlocal search of similar patches from the clean-noisy image pair. Finally, we conducted denoising and classification experiments using different kinds of simulated noisy images, compared to the Poisson-Gaussian and Noise Flow noise models, the proposed method achieves denoising performance improvements (PSNR) of 1.13 dB and 2.51 dB respectively on the proposed real-world test dataset, denoising and classification results on the real noisy data captured by mobile phones have verified that our approach is more accurate than current noise modeling methods.

An underwater wake sensing method based on Micro Thermoelectric Generator for enhancing recognition accuracy

The accidental entry and poaching of foreign boats and autonomous underwater vehicles (AUVs) have a detrimental impact on the ecological environment of marine ranching. Currently, there is a need for a monitoring method specifically designed to monitor underwater moving objects (UMOs), such as foreign boats and AUVs, in real-time within marine ranching. Therefore, this paper creatively proposes a method for monitoring UMOs based on thermal tactile of Micro Thermoelectric Generator (MTEG). A thermal tactile perceptual model for propeller wake identification is established and the wake sensing sensor (WSS) is designed. Furthermore, the performance verification experiment of wake sensing and the target classification experiment based on machine learning are carried out to prove the feasibility of the method. The experimental results demonstrate that the propeller wakes are identified with an accuracy of more than 90 % within 30 s in the monitoring terminal. Besides, the method can accurately distinguish between propeller wakes and fish wakes, thereby avoiding false alarms. The reported underwater wake sensing method is a new solution to real-time monitoring and safety early warning of marine ranching.

Research on automated rapid diagnosis of distal radius fractures using X-ray images

The primary diagnostic method for distal radius fractures remains conventional X-ray imaging. In emergency situations, rapid assessment and decision-making are often required from non-orthopedic physicians or junior radiologists in the emergency room. Therefore, an accurate and efficient auxiliary diagnostic technology for distal radius fractures is of great significance. This paper combines deep learning with image analysis techniques to propose an effective classification method for types of distal radius fractures. Firstly, an extended U-Net three-level cascade segmentation network is used for precise segmentation of the most critical joint surface and non-joint surface areas for fracture identification. The edge is further optimized using the central pixel image block classification method. Then, fracture identification is performed separately on the joint surface and non-joint surface areas. Finally, based on the classification results of the two types of images, a comprehensive judgment is made to determine normal or ABC fracture types. The experimental X-ray data comes from three tertiary hospitals, with a total of 12,000 images in the training set, 3,000 of each type, and 1,200 images in the test set, 300 of each type. To further demonstrate the generalization performance of the classification experiment, an additional 500 images were added for fracture classification testing, coming from collaborative medical institutions in the United States and Germany. All images were annotated by orthopedic medical experts with more than ten years of experience. The accuracy rates for normal, type A, type B, and type C fractures in the test set were 0.99, 0.92, 0.91, and 0.82, respectively. For orthopedic medical experts, the average recognition accuracy rates were 0.98, 0.90, 0.87, and 0.81. The proposed automatic recognition method overall performs better than experts and can be used for preliminary auxiliary diagnosis of distal radius fractures without expert participation.

Research on automated rapid diagnosis of distal radius fractures using X-ray images

The primary diagnostic method for distal radius fractures remains conventional X-ray imaging. In emergency situations, rapid assessment and decision-making are often required from non-orthopedic physicians or junior radiologists in the emergency room. Therefore, an accurate and efficient auxiliary diagnostic technology for distal radius fractures is of great significance. This paper combines deep learning with image analysis techniques to propose an effective classification method for types of distal radius fractures. Firstly, an extended U-Net three-level cascade segmentation network is used for precise segmentation of the most critical joint surface and non-joint surface areas for fracture identification. The edge is further optimized using the central pixel image block classification method. Then, fracture identification is performed separately on the joint surface and non-joint surface areas. Finally, based on the classification results of the two types of images, a comprehensive judgment is made to determine normal or ABC fracture types. The experimental X-ray data comes from three tertiary hospitals, with a total of 12,000 images in the training set, 3,000 of each type, and 1,200 images in the test set, 300 of each type. To further demonstrate the generalization performance of the classification experiment, an additional 500 images were added for fracture classification testing, coming from collaborative medical institutions in the United States and Germany. All images were annotated by orthopedic medical experts with more than ten years of experience. The accuracy rates for normal, type A, type B, and type C fractures in the test set were 0.99, 0.92, 0.91, and 0.82, respectively. For orthopedic medical experts, the average recognition accuracy rates were 0.98, 0.90, 0.87, and 0.81. The proposed automatic recognition method overall performs better than experts and can be used for preliminary auxiliary diagnosis of distal radius fractures without expert participation.

A High-Performance Anti-Noise Algorithm for Arrhythmia Recognition.

In recent years, the incidence of cardiac arrhythmias has been on the rise because of changes in lifestyle and the aging population. Electrocardiograms (ECGs) are widely used for the automated diagnosis of cardiac arrhythmias. However, existing models possess poor noise robustness and complex structures, limiting their effectiveness. To solve these problems, this paper proposes an arrhythmia recognition system with excellent anti-noise performance: a convolutionally optimized broad learning system (COBLS). In the proposed COBLS method, the signal is convolved with blind source separation using a signal analysis method based on high-order-statistic independent component analysis (ICA). The constructed feature matrix is further feature-extracted and dimensionally reduced using principal component analysis (PCA), which reveals the essence of the signal. The linear feature correlation between the data can be effectively reduced, and redundant attributes can be eliminated to obtain a low-dimensional feature matrix that retains the essential features of the classification model. Then, arrhythmia recognition is realized by combining this matrix with the broad learning system (BLS). Subsequently, the model was evaluated using the MIT-BIH arrhythmia database and the MIT-BIH noise stress test database. The outcomes of the experiments demonstrate exceptional performance, with impressive achievements in terms of the overall accuracy, overall precision, overall sensitivity, and overall F1-score. Specifically, the results indicate outstanding performance, with figures reaching 99.11% for the overall accuracy, 96.95% for the overall precision, 89.71% for the overall sensitivity, and 93.01% for the overall F1-score across all four classification experiments. The model proposed in this paper shows excellent performance, with 24 dB, 18 dB, and 12 dB signal-to-noise ratios.

An Arduino-Powered Device for the Study of White Perception beyond the Visual Chromatic Critical Flicker Fusion Frequency.

Arduino microcontrollers are used for a wide range of technological and biomedical applications, such as image classification, computer vision, brain-computer interaction and vision experiments. Here, we present a new cost-effective mini-device based on RGB LED flicker stimulation for the assessment of the chromatic temporal resolution of the visual function based on the concept of critical flicker fusion frequency (CFF). The assembly of the device and its testing in thirty young subjects demonstrate the steady white visual perception of a trichromatic flicker stimulus (mixture of red, green and blue stimuli) beyond the CFF. Macular function as measured by photo-stress recovery time (PRT) was found to be independent of the CFF measurements for red, green and blue lights. However, a statistical correlation was found between the contrast modulation for CFF for red and green stimuli and PRT. Finally, wavefront measurements demonstrate that high-order aberrations improve the temporal resolution of the visual function.

Rescaling large datasets based on validation outcomes of a pre-trained network

In fact, several categories in a large dataset are not difficult for recent advanced deep neural networks to recognize. Eliminating them for a challenging smaller subset will assist the early network proposals in taking a quick trial of verification. To this end, we propose an efficient rescaling method based on the validation outcomes of a pre-trained model. Firstly, we will take out the sensitive images of the lowest-accuracy classes of the validation outcomes. Each of such images is then considered to identify which label it was confused with. Gathering the lowest-accuracy classes along with the most confused ones can produce a smaller subset with a higher challenge for quick validation of an early network draft. Finally, a rescaling application is introduced to rescale two popular large datasets (ImageNet and Places365) for different tiny subsets (i.e., ReINΩ and RePLΩ respectively). Experiments for image classification have proved that neural networks obtaining good performance on the original datasets also achieve good results on their rescaled subsets. For instance, MobileNetV1 and MobileNetV2 with 70.6% and 72% on ImageNet respectively obtained 46.53% and 47.47% on its small subset ReIN30, which only contains about 39000 images. It can be observed that the better performance of MobileNetV2 on ImageNet correspondingly leads to the better rate on its rescaled subset. Appropriately, utilizing these rescaled sets would help researchers save time and computational costs in the way of designing deep neural architectures. All codes related to the rescaling proposal and the resultant subsets are available at http://github.com/nttbdrk25/ImageNetPlaces365.

Siamese Graph Convolutional Network quantifies increasing structure-function discrepancy over the cognitive decline continuum

Background and Objective:Alzheimer’s disease dementia (ADD) is well known to induce alterations in both structural and functional brain connectivity. However, reported changes in connectivity are mostly limited to global/local network features, which have poor specificity for diagnostic purposes. Following recent advances in machine learning, deep neural networks, particularly Graph Neural Network (GNN) based approaches, have found applications in brain research as well. The majority of existing applications of GNNs employ a single network (uni-modal or structure/function unified), despite the widely accepted view that there is a nontrivial interdependence between the brain’s structural connectivity and the neural activity patterns, which is hypothesized to be disrupted in ADD. This disruption is quantified as a discrepancy score by the proposed “structure-function discrepancy learning network” (sfDLN) and its distribution is studied over the spectrum of clinical cognitive decline. The measured discrepancy score is utilized as a diagnostic biomarker and is compared with state-of-the-art diagnostic classifiers. Methods:sfDLN is a GNN with a siamese architecture built on the hypothesis that the mismatch between structural and functional connectivity patterns increases over the cognitive decline spectrum, starting from subjective cognitive impairment (SCI), passing through a mid-stage mild cognitive impairment (MCI), and ending up with ADD. The structural brain connectome (sNET) built using diffusion MRI-based tractography and the novel, sparse (lean) functional brain connectome (ℓNET) built using fMRI are input to sfDLN. The siamese sfDLN is trained to extract connectome representations and a discrepancy (dissimilarity) score that complies with the proposed hypothesis and is blindly tested on an MCI group. Results:The sfDLN generated structure-function discrepancy scores show high disparity between ADD and SCI subjects. Leave-one-out experiments of SCI-ADD classification over a cohort of 42 subjects reach 88% accuracy, surpassing state-of-the-art GNN-based classifiers in the literature. Furthermore, a blind assessment over a cohort of 46 MCI subjects confirmed that it captures the intermediary character of the MCI group. GNNExplainer module employed to investigate the anatomical determinants of the observed discrepancy confirms that sfDLN attends to cortical regions neurologically relevant to ADD. Conclusion:In support of our hypothesis, the harmony between the structural and functional organization of the brain degrades with increasing cognitive decline. This discrepancy, shown to be rooted in brain regions neurologically relevant to ADD, can be quantified by sfDLN and outperforms state-of-the-art GNN-based ADD classification methods when used as a biomarker.

Hybrid Aquila optimizer for efficient classification with probabilistic neural networks

The model of a probabilistic neural network (PNN) is commonly utilized for classification and pattern recognition issues in data mining. An approach frequently used to enhance its effectiveness is the adjustment of PNN classifier parameters through the outcomes of metaheuristic optimization strategies. Since PNN employs a limited set of instructions, metaheuristic algorithms provide an efficient way to modify its parameters. In this study, we have employed the Aquila optimizer algorithm (AO), a contemporary algorithm, to modify PNN parameters. We have proposed two methods: Aquila optimizer based probabilistic neural network (AO-PNN), which uses both local and global search capabilities of AO, and hybrid Aquila optimizer and simulated annealing based probabilistic neural network (AOS-PNN), which integrates the global search abilities of AO with the local search mechanism of simulated annealing (SA). Our experimental results indicate that both AO-PNN and AOS-PNN perform better than the PNN model in terms of accuracy across all datasets. This suggests that they have the potential to generate more precise results when utilized to improve PNN parameters. Moreover, our hybridization technique, AOS-PNN, is more effective than AO-PNN, as evidenced by classification experiments accuracy, data distribution, convergence speed, and significance. We have also compared our suggested approaches with three different methodologies, namely Coronavirus herd immunity optimizer based probabilistic neural network (CHIO-PNN), African buffalo algorithm based probabilistic neural network (ABO-PNN), and β-hill climbing. We have found that AO-PNN and AOS-PNN have achieved significantly higher classification accuracy rates of 90.68 and 93.95, respectively.

TadaStride: Using time adaptive strides in audio data for effective downsampling

In this paper, we introduce a new downsampling method for audio data called TadaStride, which can adaptively adjust the downsampling ratios across an audio data instance. Unlike previous methods using a fixed downsampling ratio, TadaStride can preserve more information from task-relevant parts of a data instance by using smaller strides for those parts and larger strides for less relevant parts. Additionally, we also introduce TadaStride-F, which is developed as a more efficient version of TadaStride while maintaining minimal performance loss. In experiments, we evaluate our TadaStride, primarily focusing on a range of audio processing tasks. Firstly, in audio classification experiments, TadaStride and TadaStride-F outperform other widely used standard downsampling methods, even with comparable memory and time usage. Furthermore, through various analyses, we provide an understanding of how TadaStride learns effective adaptive strides and how it leads to improved performance. In addition, through additional experiments on automatic speech recognition and discrete speech representation learning, we demonstrate that TadaStride and TadaStride-F consistently outperform other downsampling methods and examine how the adaptive strides are learned in these tasks.

Proto-Adapter: Efficient Training-Free CLIP-Adapter for Few-Shot Image Classification.

Large vision-language models, such as Contrastive Vision-Language Pre-training (CLIP), pre-trained on large-scale image-text datasets, have demonstrated robust zero-shot transfer capabilities across various downstream tasks. To further enhance the few-shot recognition performance of CLIP, Tip-Adapter augments the CLIP model with an adapter that incorporates a key-value cache model constructed from the few-shot training set. This approach enables training-free adaptation and has shown significant improvements in few-shot recognition, especially with additional fine-tuning. However, the size of the adapter increases in proportion to the number of training samples, making it difficult to deploy in practical applications. In this paper, we propose a novel CLIP adaptation method, named Proto-Adapter, which employs a single-layer adapter of constant size regardless of the amount of training data and even outperforms Tip-Adapter. Proto-Adapter constructs the adapter's weights based on prototype representations for each class. By aggregating the features of the training samples, it successfully reduces the size of the adapter without compromising performance. Moreover, the performance of the model can be further enhanced by fine-tuning the adapter's weights using a distance margin penalty, which imposes additional inter-class discrepancy to the output logits. We posit that this training scheme allows us to obtain a model with a discriminative decision boundary even when trained with a limited amount of data. We demonstrate the effectiveness of the proposed method through extensive experiments of few-shot classification on diverse datasets.

Trajectory-driven computational analysis for element characterization in Trypanosoma cruzi video microscopy.

Optical microscopy videos enable experts to analyze the motion of several biological elements. Particularly in blood samples infected with Trypanosoma cruzi (T. cruzi), microscopy videos reveal a dynamic scenario where the parasites' motions are conspicuous. While parasites have self-motion, cells are inert and may assume some displacement under dynamic events, such as fluids and microscope focus adjustments. This paper analyzes the trajectory of T. cruzi and blood cells to discriminate between these elements by identifying the following motion patterns: collateral, fluctuating, and pan-tilt-zoom (PTZ). We consider two approaches: i) classification experiments for discrimination between parasites and cells; and ii) clustering experiments to identify the cell motion. We propose the trajectory step dispersion (TSD) descriptor based on standard deviation to characterize these elements, outperforming state-of-the-art descriptors. Our results confirm motion is valuable in discriminating T. cruzi of the cells. Since the parasites perform the collateral motion, their trajectory steps tend to randomness. The cells may assume fluctuating motion following a homogeneous and directional path or PTZ motion with trajectory steps in a restricted area. Thus, our findings may contribute to developing new computational tools focused on trajectory analysis, which can advance the study and medical diagnosis of Chagas disease.

Contrastive and uncertainty-aware nuclei segmentation and classification

Nuclei segmentation and classification play a crucial role in pathology diagnosis, enabling pathologists to analyze cellular characteristics accurately. Overlapping cluster nuclei, misdetection of small-scale nuclei, and pleomorphic nuclei-induced misclassification have always been major challenges in the nuclei segmentation and classification tasks. To this end, we introduce an auxiliary task of nuclei boundary-guided contrastive learning to enhance the representativeness and discriminative power of visual features, particularly for addressing the challenge posed by the unclear contours of adherent nuclei and small nuclei. In addition, misclassifications resulting from pleomorphic nuclei often exhibit low classification confidence, indicating a high level of uncertainty. To mitigate misclassification, we capitalize on the characteristic clustering of similar cells to propose a locality-aware class embedding module, offering a regional perspective to capture category information. Moreover, we address uncertain classification in densely aggregated nuclei by designing a top-k uncertainty attention module that leverages deep features to enhance shallow features, thereby improving the learning of contextual semantic information. We demonstrate that the proposed network outperforms the off-the-shelf methods in both nuclei segmentation and classification experiments, achieving the state-of-the-art performance.

Intelligent intrusion detection through deep autoencoder and stacked long short-term memory

In the realm of network intrusion detection, the escalating complexity and diversity of cyber threats necessitate innovative approaches to enhance detection accuracy. This study introduces an integrated solution leveraging deep learning techniques for improved intrusion detection. The proposed framework consists on a deep autoencoder for feature extraction, and a stacked long short-term memory (LSTM) network ensemble for classification. The deep autoencoder compresses raw network data, extracting salient features and mitigating noise. Subsequently, the stacked LSTM ensemble captures intricate temporal dependencies, correcting anomaly detection precision. Experiments conducted on the UNSW-NB15 dataset, and a benchmark in intrusion detection validate the effectiveness of the approach. The solution achieves an accuracy of 90.59%, with precision, recall, and F1-Score metrics reaching 90.65, 90.59, and 90.57, respectively. Notably, the framework outperforms standalone models and demonstrates the advantage of synergizing deep autoencoder-driven feature extraction with the stacked LSTM ensemble. Furthermore, a binary classification experiment attains an accuracy of about 90.59%, surpassing the multiclass classification and affirming the model's potential for binary threat identification. Comparative analyses highlight the pivotal role of feature extraction, while experimentation illustrates the enhancement achieved by incorporating the synergistic deep autoencoder-Stacked LSTM approach.

MPSA: Multi-Position Supervised Soft Attention-based convolutional neural network for histopathological image classification

In recent years, significant achievements have been made in the field of histopathological image analysis using convolutional neural networks (CNNs). However, existing CNNs fail to fully capture the important local structures and regional information in histopathological images due to the complex tissue structures and variable pathological features present in these images. They often treat all regions equally, which further exacerbates the challenge of accurately analyzing such images. Current network model can’t extract deep layer features efficiently without guiding. To alleviate this problem, we propose a novel network model called Multi-Position Supervised Soft Attention (MPSA). MPSA adds regions of interest (RoI) labels at multiple feature layers for deep supervision, and then uses the supervised layers as soft attention to guide the learning of the classification network, enabling the network to accurately extract features of the lesion target. Additionally, we design a Multi-level Attention Feature Enhancement Module (MAFEM), which combines multiple levels of attention mechanisms to enhance the performance of the convolutional neural network in histopathological image classification. MAFEM includes spatial attention, soft attention of the main branch, and our proposed soft attention for multi-branch feature fusion. The proposed soft attention for multi-branch feature fusion aims to enhance the predictive performance of the classification model by activating relevant neurons in the diagnostic area in a highly activated state, while effectively avoiding noise activation. This innovative approach ensures that the model can focus on the most pertinent information, leading to improved classification accuracy. We conducted classification experiments on the liver cancer histopathological images dataset and the results showed that our method achieved a classification accuracy of 95.79%, indicating that it is very effective in the analysis of liver histopathological images. Our proposed network architecture has also demonstrated good generalization ability in other medical datasets, achieving a classification accuracy of 84.41% on the ultrasound carotid plaque dataset.

MCCM: multi-scale feature extraction network for disease classification and recognition of chili leaves.

Currently, foliar diseases of chili have significantly impacted both yield and quality. Despite effective advancements in deep learning techniques for the classification of chili leaf diseases, most existing classification models still face challenges in terms of accuracy and practical application in disease identification. Therefore, in this study, an optimized and enhanced convolutional neural network model named MCCM (MCSAM-ConvNeXt-MSFFM) is proposed by introducing ConvNeXt. The model incorporates a Multi-Scale Feature Fusion Module (MSFFM) aimed at better capturing disease features of various sizes and positions within the images. Moreover, adjustments are made to the positioning, activation functions, and normalization operations of the MSFFM module to further optimize the overall model. Additionally, a proposed Mixed Channel Spatial Attention Mechanism (MCSAM) strengthens the correlation between non-local channels and spatial features, enhancing the model's extraction of fundamental characteristics of chili leaf diseases. During the training process, pre-trained weights are obtained from the Plant Village dataset using transfer learning to accelerate the model's convergence. Regarding model evaluation, the MCCM model is compared with existing CNN models (Vgg16, ResNet34, GoogLeNet, MobileNetV2, ShuffleNet, EfficientNetV2, ConvNeXt), and Swin-Transformer. The results demonstrate that the MCCM model achieves average improvements of 3.38%, 2.62%, 2.48%, and 2.53% in accuracy, precision, recall, and F1 score, respectively. Particularly noteworthy is that compared to the original ConvNeXt model, the MCCM model exhibits significant enhancements across all performance metrics. Furthermore, classification experiments conducted on rice and maize disease datasets showcase the MCCM model's strong generalization performance. Finally, in terms of application, a chili leaf disease classification website is successfully developed using the Flask framework. This website accurately identifies uploaded chili leaf disease images, demonstrating the practical utility of the model.

Advantages and Pitfalls of Dataset Condensation: An Approach to Keyword Spotting with Time-Frequency Representations

With the exponential growth of data, the need for efficient techniques to extract relevant information from datasets becomes increasingly imperative. Reducing the training data can be useful for applications wherein storage space or computational resources are limited. In this work, we explore the concept of data condensation (DC) in the context of keyword spotting systems (KWS). Using deep learning architectures and time-frequency speech representations, we have obtained condensed speech signal representations using gradient matching with Efficient Synthetic-Data Parameterization. From a series of classification experiments, we analyze the models and condensed data performances in terms of accuracy and number of data per class. We also present results using cross-model techniques, wherein models are trained with condensed data obtained from a different architecture. Our findings demonstrate the potential of data condensation in the context of the speech domain for reducing the size of datasets while retaining their most important information and maintaining high accuracy for the model trained with the condensed dataset. We have obtained an accuracy of 80.75% with 30 condensed speech representations per class with ConvNet, representing an addition of 24.9% in absolute terms when compared to 30 random samples from the original training dataset. However, we demonstrate the limitations of this approach in the cross-model tests. We also highlight the challenges and opportunities for further improving the accuracy of condensed data obtained and trained with different neural network architectures.

HGR Correlation Pooling Fusion Framework for Recognition and Classification in Multimodal Remote Sensing Data

This paper investigates remote sensing data recognition and classification with multimodal data fusion. Aiming at the problems of low recognition and classification accuracy and the difficulty in integrating multimodal features in existing methods, a multimodal remote sensing data recognition and classification model based on a heatmap and Hirschfeld–Gebelein–Rényi (HGR) correlation pooling fusion operation is proposed. A novel HGR correlation pooling fusion algorithm is developed by combining a feature fusion method and an HGR maximum correlation algorithm. This method enables the restoration of the original signal without changing the value of transmitted information by performing reverse operations on the sample data. This enhances feature learning for images and improves performance in specific tasks of interpretation by efficiently using multi-modal information with varying degrees of relevance. Ship recognition experiments conducted on the QXS-SROPT dataset demonstrate that the proposed method surpasses existing remote sensing data recognition methods. Furthermore, land cover classification experiments conducted on the Houston 2013 and MUUFL datasets confirm the generalizability of the proposed method. The experimental results fully validate the effectiveness and significant superiority of the proposed method in the recognition and classification of multimodal remote sensing data.

Full-scale semantic segmentation of hyperspectral imaging based on spatial spatial-spectral joint network

Abstract. Hyperspectral images contain dozens or even hundreds of spectral bands, which contain rich spectral information and help distinguish different ground objects. Hyperspectral images have a wide range of applications in urban planning, environmental monitoring, and other fields. The semantic segmentation of hyperspectral images is one of the current research hotspots. The difficulty lies in the rich spectral information and strong correlation of hyperspectral images. Traditional semantic segmentation methods cannot fully extract information, which affects the accuracy of classification. This article utilizes an encoding decoding structure to simultaneously extract deep and shallow features of images. A REGCS convolution module was constructed using the idea of group convolution to extract spectral and spatial features of images. We compared the Salinas Valley dataset and MUUFL dataset with various classification algorithms. The experimental results show that compared with other classification models, the RESSU model has achieved stable and excellent results in hyperspectral image classification experiments. Among them, in the classification experiment of the Salinas Valley dataset, the accuracy of single class classification reached over 92%. In the effectiveness analysis experiment, we calculated different model parameter quantities to verify the performance of our method, and ultimately achieved good results.