Small Sample Learning Research Articles

Siamese Neural Network and Multimodal Data Fusion Approach for Small-Sample Learning in Industrial Soft Sensor Modeling

Siamese Neural Network and Multimodal Data Fusion Approach for Small-Sample Learning in Industrial Soft Sensor Modeling

STID-Prompt: Prompt learning for sentiment-topic-importance detection in financial news

With the development of the Internet and the financial industry, the analysis and judgement of financial news has become increasingly important. Common tasks in this area include sentiment analysis, topic classification, and importance judgement. Existing approaches use fine-tuning paradigms to address these tasks. However, the lack of labeled data in the field of financial news poses a small sample learning problem. The new paradigm represented by prompt learning provides a new way and means to improve the performance of small-sample classification. In addition, existing methods do not focus on research on sentiment, topic, and importance simultaneously, which is also an important factor in evaluating financial news. In practical applications, joint judgment of sentiment, topic, and importance is more effective. Finally, these methods require different tuning for each specific task. They do not consider relationships between individual tasks and cannot utilize information across tasks. This limits their application in complex situations. In this work, we propose a prompt-based financial news classification model (STID-Prompt) to address these issues. For the first two problems, we solve the sentiment analysis task, the topic classification task, the importance judgment task, and the 〈sentiment, topic, importance〉 classification task by designing complex prompt templates. For the third problem, we propose a unique prompt-based joint multi-task learning approach. It learns knowledge from multiple tasks and integrates it into the target task, and the multi-task learning approach further improves the performance of the model. Experimental results show the effectiveness of our approach even with less training data.

Cotton Fusarium wilt diagnosis based on generative adversarial networks in small samples.

This study aimed to explore the feasibility of applying Generative Adversarial Networks (GANs) for the diagnosis of Verticillium wilt disease in cotton and compared it with traditional data augmentation methods and transfer learning. By designing a model based on small-sample learning, we proposed an innovative cotton Verticillium wilt disease diagnosis system. The system uses Convolutional Neural Networks (CNNs) as feature extractors and applies trained GAN models for sample augmentation to improve classification accuracy. This study collected and processed a dataset of cotton Verticillium wilt disease images, including samples from normal and infected plants. Data augmentation techniques were used to expand the dataset and train the CNNs. Transfer learning using InceptionV3 was applied to train the CNNs on the dataset. The dataset was augmented using GAN algorithms and used to train CNNs. The performances of the data augmentation, transfer learning, and GANs were compared and analyzed. The results have demonstrated that augmenting the cotton Verticillium wilt disease image dataset using GAN algorithms enhanced the diagnostic accuracy and recall rate of the CNNs. Compared to traditional data augmentation methods, GANs exhibit better performance and generated more representative and diverse samples. Unlike transfer learning, GANs ensured an adequate sample size. By visualizing the images generated, GANs were found to generate realistic cotton images of Verticillium wilt disease, highlighting their potential applications in agricultural disease diagnosis. This study has demonstrated the potential of GANs in the diagnosis of cotton Verticillium wilt disease diagnosis, offering an effective approach for agricultural disease detection and providing insights into disease detection in other crops.

Dynamic Model-Embedded Intelligent Machine Fault Diagnosis Without Fault Data

Intelligent machine fault diagnosis technique has recently exploded interest in digital health, energy power, and industrial maintenance. Collecting machine fault data in engineering practice is usually costly, causing big challenges in the intelligent diagnosis of most fresh-from-the-factory machines that are missing fault data. Inspired by the main idea of the digital twin, this paper proposes a dynamic model-embedded intelligent machine fault diagnosis framework. Specifically, a machine dynamic modeling and parameter identification approach is introduced for building the machine digital model. The digital model is used to predict machine fault data from the measured healthy vibration signals. Furthermore, a parameterized convolutional neural network structure is designed for learning optimization features and recognizing the machine's healthy state. Experimental investigation demonstrates the effectiveness of the framework. The results show that the proposed framework enables intelligent machine fault diagnosis without fault data. The diagnosis effect outperforms supervised, unsupervised and small-sample learning approaches and can be generalized to another load and speed with acceptable accuracy.

Blind light field image quality assessment based on deep meta-learning.

In recent years, the use of deep convolutional neural networks (DCNNs) for light field image quality assessment (LFIQA) has gained significant attention. Despite their notable successes, it is widely accepted that training DCNNs heavily depends on a large amount of annotated data. Additionally, convolutional network-based LFIQA methods show a limitation in capturing long-range dependencies. Unfortunately, LFIQA is essentially a typical small-sample problem, leading to existing DCNN-based LFIQA metrics requiring data augmentation but with unsatisfactory performance. To address these issues, this study proposes utilizing the self-attention capability of the Swin Transformer to efficiently capture spatial-angular information while employing meta-learning for small-sample learning in the LFIQA task. Specifically, a collection of LFIQA tasks is gathered, representing different distortions. Then, meta-learning is employed to acquire shared prior knowledge across diverse distortions. Finally, the quality prior model is fine-tuned on a target LFIQA task to obtain the final LFIQA model quickly. Experimental results show that the proposed LFIQA metric achieves high consistency with subjective scores, and outperforms several state-of-the-art LFIQA approaches.

Research advanced in deep learning-based licence plate recognition and localization

The purpose of licence plate recognition is to analyze pictures or videos of moving vehicles to read the plate and identify the vehicle's owner. Traffic data management and smart transportation systems rely heavily on licence plate reading technology. Initial picture capture, image preprocessing, licence plate analysis, character segmentation, and recognition are the building blocks of licence plate recognition. The present analysis centres on the examination of the above key steps. In this paper, we introduce the latest research progress in the implementation of licence plate recognition utilizing deep learning techniques, including the classic framework of licence plate location and character recognition, representative methods, and their advantages and disadvantages. We also perform a quantitative comparison of existing representative methods. Finally, we summarize the challenges in the research domain of licence plate recognition and discuss the future development direction from the aspects of neural network interpretability, more general small sample learning methods, and incremental learning.

Efficient ship noise classification with positive incentive noise and fused features using a simple convolutional network

Ship noise analysis is a critical area of research in hydroacoustic remote sensing due to its practical implications in identifying ship direction, type, and even specific ship identities. However, the limited availability of data poses challenges in developing accurate ship noise classification models. Previous studies have mainly focused on small-sample learning approaches, resulting in complex network structures. Nonetheless, underwater robots often have limited computing power, making it essential to develop simpler recognition networks. In this paper, we address the issue of data scarcity by introducing positive incentive noise. We propose a CNN-based hydroacoustic signal recognition method that achieves comparable or superior performance to previous studies, using a simple network structure as a back-end decision system. We describe the feature extraction process using a dataset with added noise and compare the performance of various features. Additionally, we compare our proposed method with previous studies. Experimental results demonstrate that simple neural networks can achieve high performance and excellent generalizability without the need for complex network structures like adversarial learning models.

Deep Learning Methods for Semantic Segmentation in Remote Sensing with Small Data: A Survey

The annotations used during the training process are crucial for the inference results of remote sensing images (RSIs) based on a deep learning framework. Unlabeled RSIs can be obtained relatively easily. However, pixel-level annotation is a process that necessitates a high level of expertise and experience. Consequently, the use of small sample training methods has attracted widespread attention as they help alleviate reliance on large amounts of high-quality labeled data and current deep learning methods. Moreover, research on small sample learning is still in its infancy owing to the unique challenges faced when completing semantic segmentation tasks with RSI. To better understand and stimulate future research that utilizes semantic segmentation tasks with small data, we summarized the supervised learning methods and challenges they face. We also reviewed the supervised approaches with data that are currently popular to help elucidate how to efficiently utilize a limited number of samples to address issues with semantic segmentation in RSI. The main methods discussed are self-supervised learning, semi-supervised learning, weakly supervised learning and few-shot methods. The solution of cross-domain challenges has also been discussed. Furthermore, multi-modal methods, prior knowledge constrained methods, and future research required to help optimize deep learning models for various downstream tasks in relation to RSI have been identified.

Weakly Supervised Cross-Domain Person Re-Identification Algorithm Based on Small Sample Learning

This paper proposes a weakly supervised cross-domain person re-identification (Re-ID) method based on small sample data. In order to reduce the cost of data collection and annotation, the model design focuses on extracting and abstracting the information contained in the data under limited conditions. In this paper, we focus on the problems of strong data dependence, weak cross-domain capability and low accuracy in Re-ID in weakly supervised scenarios. Our contributions are as follows: first, we implement a joint training framework with the help of small sample learning and cross-domain migration for Re-ID. Second, with the help of residual compensation and fusion attention module, the RCFA module is designed, and the model framework is built on this basis to improve the cross-domain ability of the model. Third, to solve the problem of low accuracy caused by insufficient data coverage of small samples, a fusion of shallow features and deep features is designed to enable the model to weighted fusion of shallow detail information and deep semantic information. Finally, by selecting different camera images in Market1501 dataset and DukeMTMC-reID dataset as small samples, respectively, and introducing another dataset data for joint training, we demonstrate the feasibility of this joint training framework, which can perform weakly supervised cross-domain Re-ID based on small sample data.

An Underwater Side-Scan Sonar Transfer Recognition Method Based on Crossed Point-to-Point Second-Order Self-Attention Mechanism

Recognizing targets through side-scan sonar (SSS) data by deep learning-based techniques has been particularly challenging. The primary challenge stems from the difficulty and time consumption associated with underwater acoustic data acquisition, which demands systematic explorations to obtain sufficient training samples for accurate deep learning-based models. Moreover, if the sample size of the available data is small, the design of effective target recognition models becomes complex. These challenges have posed significant obstacles to developing accurate SSS-based target recognition methods via deep learning models. However, utilizing multi-modal datasets to enhance the recognition performance of sonar images through knowledge transfer in deep networks appears promising. Owing to the unique statistical properties of various modal images, transitioning between different modalities can significantly increase the complexity of network training. This issue remains unresolved, directly impacting the target transfer recognition performance. To enhance the precision of categorizing underwater sonar images when faced with a limited number of mode types and data samples, this study introduces a crossed point-to-point second-order self-attention (PPCSSA) method based on double-mode sample transfer recognition. In the PPCSSA method, first-order importance features are derived by extracting key horizontal and longitudinal point-to-point features. Based on these features, the self-supervised attention strategy effectively removes redundant features, securing the second-order significant features of SSS images. This strategy introduces a potent low-mode-type small-sample learning method for transfer learning. Classification experiment results indicate that the proposed method excels in extracting key features with minimal training complexity. Moreover, experimental outcomes underscore that the proposed technique enhances recognition stability and accuracy, achieving a remarkable overall accuracy rate of 99.28%. Finally, the proposed method maintains high recognition accuracy even in noisy environments.

An innovative approach based on meta-learning for real-time modal fault diagnosis with small sample learning

The actual multimodal process data usually exhibit non-linear time correlation and non-Gaussian distribution accompanied by new modes. Existing fault diagnosis methods have difficulty adapting to the complex nature of new modalities and are unable to train models based on small samples. Therefore, this paper proposes a new modal fault diagnosis method based on meta-learning (ML) and neural architecture search (NAS), MetaNAS. Specifically, the best performing network model of the existing modal is first automatically obtained using NAS, and then, the fault diagnosis model design is learned from the NAS of the existing model using ML. Finally, when generating new modalities, the gradient is updated based on the learned design experience, i.e., new modal fault diagnosis models are quickly generated under small sample conditions. The effectiveness and feasibility of the proposed method are fully verified by the numerical system and simulation experiments of the Tennessee Eastman (TE) chemical process.

Fault diagnosis for wind turbines with graph neural network model based on one-shot learning

Because of the harsh working environment, there is usually a lack of effective data from the gearboxes of wind turbines for fault classification. In this paper, a fault-diagnosis model based on graph neural networks and one-shot learning is proposed to solve the problem of fault classification with limited data. In the proposed method, the short-time Fourier transform is used to convert one-dimensional vibration signals into two-dimensional data, then feature vectors are extracted from the two-dimensional data, and small-sample learning is achieved. An experimental rig was built to simulate the real working scenario of a wind turbine, and the results indicate the high classification accuracy of the proposed method. Furthermore, its effectiveness is verified in comparisons with Siamese, matching and prototypical networks, with the proposed method outperforming all of them.

An unsupervised STDP-based spiking neural network inspired by biologically plausible learning rules and connections

The backpropagation algorithm has promoted the rapid development of deep learning, but it relies on a large amount of labeled data and still has a large gap with how humans learn. The human brain can quickly learn various conceptual knowledge in a self-organized and unsupervised manner, accomplished through coordinating various learning rules and structures in the human brain. Spike-timing-dependent plasticity (STDP) is a general learning rule in the brain, but spiking neural networks (SNNs) trained with STDP alone is inefficient and perform poorly. In this paper, taking inspiration from short-term synaptic plasticity, we design an adaptive synaptic filter and introduce the adaptive spiking threshold as the neuron plasticity to enrich the representation ability of SNNs. We also introduce an adaptive lateral inhibitory connection to adjust the spikes balance dynamically to help the network learn richer features. To speed up and stabilize the training of unsupervised spiking neural networks, we design a samples temporal batch STDP (STB-STDP), which updates weights based on multiple samples and moments. By integrating the above three adaptive mechanisms and STB-STDP, our model greatly accelerates the training of unsupervised spiking neural networks and improves the performance of unsupervised SNNs on complex tasks. Our model achieves the current state-of-the-art performance of unsupervised STDP-based SNNs in the MNIST and FashionMNIST datasets. Further, we tested on the more complex CIFAR10 dataset, and the results fully illustrate the superiority of our algorithm. Our model is also the first work to apply unsupervised STDP-based SNNs to CIFAR10. At the same time, in the small-sample learning scenario, it will far exceed the supervised ANN using the same structure.

Global Model Calibration of High-Temperature Gas-Cooled Reactor Pebble-Bed Module Using an Adaptive Experimental Design

The world’s first high-temperature gas-cooled reactor pebble-bed module (HTR-PM) nuclear power plant adopts an innovative reactor type and a modular structure design. Parameter estimation and model calibration are of great significance prior to the implementation of model-based control and optimization. This paper focuses on identifying the thermal hydraulic parameters of HTR-PM over the global operating domain. The process technology and model mechanism of HTR-PM are reviewed. A parameter submodel named global parameter mapping is presented to quantify the relationship between an unknown model parameter and different operating conditions in a data-driven manner. The ideal construction of such a mapping requires reliable estimates, a well-poised sample set and an appropriate global surrogate. An adaptive model calibration scheme is designed to tackle these three issues correspondingly. First, a systematic parameter estimation approach is developed to ensure reliable estimates via heuristic subset selection consisting of estimability analysis and reliability evaluation. To capture the parameter behavior among the multiple experimental conditions and meanwhile reduce the operating cost, an adaptive experimental design is employed to guide condition testing. Experimental conditions are sequentially determined by comprehensively considering the criteria of sampling density, local nonlinearity and parameter uncertainty. Support vector regression is introduced as the global surrogate due to its capability of small-sample learning. Finally, the effectiveness of the model calibration scheme and its application performance in HTR-PM are validated by the simulation results.

Research on Financial Field Integrating Artificial Intelligence: Application Basis, Case Analysis, and SVR Model-Based Overnight

ABSTRACT The integration of Artificial Intelligence (AI) into the financial industry has witnessed significant growth, transitioning from an academic concept to widespread adoption in the industrial sector. This trend has given rise to various AI technologies, presenting both novel opportunities and potential risks within the financial landscape. In light of this development, the present research article aims to investigate the expanding role of AI in the financial sector, focusing on its applications and impact across financial products, channels, and service methodologies. To accomplish this objective, a specific AI algorithm called Support Vector Machine for Regression (SVR) has been selected for analysis. The SVR algorithm is particularly well-suited for small sample learning, making it an appropriate choice for examining trends in Shibor. By employing a combination of theoretical analysis, case studies, and risk assessment, this article contributes to fostering a profound and robust integration of AI and finance. Consequently, it delivers both theoretical insights and practical significance, offering valuable knowledge for industry practitioners and academic researchers alike.

Recognition of bird nests on transmission lines based on YOLOv5 and DETR using small samples

The safety and reliability of transmission line operations are critical to the delivery of electricity. Bird nests are one of the frequent potential factors affecting the safety of transmission lines. The main objective of this paper is to propose a small-sample learning method to detect bird nests on transmission lines. The method combines the two latest deep learning models, YOLOv5 and detection transformer (DETR). Inspired by biological vision, this method transfers the learning of bird nests in daily scenes to the recognition of bird nests on transmission lines. The proposed method is evaluated by two public datasets. The test on the first one presents a recognition rate of 95.50%, whereas the training set only contains ten homologous data and 80 non-homologous data. The second test shows that 85.54% of the samples are recognized by generalization without homologous training data. The results show that our method provides a way to identify bird nests on transmission lines with the help of bird nests of daily scenes under small sample conditions.

Improved automatic kernel construction for Gaussian process regression in small sample learning for predicting lift body aerodynamic performance

A Gaussian process regression (GPR) model based on an improved automatic kernel construction (AKC) algorithm using beam search is proposed to establish a surrogate model between lift body shape parameters and aerodynamic coefficients with various training sets sizes. The precision of our proposed surrogate model is assessed through tenfold cross-validation. The improved AKC-GPR algorithm, polynomial regression, and support vector regression (SVR) are employed to construct the regression model. The interpolation and extrapolation capabilities of the model, as generated by the improved AKC-GPR algorithm, are examined using six shapes beyond the sample set. The results show that the three models perform similarly with a large training set. However, when the training set size is less than 40% sample dataset, the model constructed by the improved AKC-GPR algorithm has better fitting and prediction capabilities than the other models. Specifically, the max relative error of the improved model is one-fourth of that of SVR and one-half of that of polynomial regression with the training set size of 8% of the sample dataset. Furthermore, the lift-to-drag ratio relative error of interpolation is only 3%, and extrapolation error is 6%. In terms of the fitting and prediction abilities for small samples, the lift-to-drag ratio model outperforms the drag coefficient model, while the lift coefficient model performs the poorest. These findings suggest that the proposed AKC-GPR algorithm can be an effective approach for building a surrogate model in the field of aerodynamics.

Small-sample learning reveals propionylation in determining global protein homeostasis

Proteostasis is fundamental for maintaining organismal health. However, the mechanisms underlying its dynamic regulation and how its disruptions lead to diseases are largely unclear. Here, we conduct in-depth propionylomic profiling in Drosophila, and develop a small-sample learning framework to prioritize the propionylation at lysine 17 of H2B (H2BK17pr) to be functionally important. Mutating H2BK17 which eliminates propionylation leads to elevated total protein level in vivo. Further analyses reveal that H2BK17pr modulates the expression of 14.7–16.3% of genes in the proteostasis network, and determines global protein level by regulating the expression of genes involved in the ubiquitin-proteasome system. In addition, H2BK17pr exhibits daily oscillation, mediating the influences of feeding/fasting cycles to drive rhythmic expression of proteasomal genes. Our study not only reveals a role of lysine propionylation in regulating proteostasis, but also implements a generally applicable method which can be extended to other issues with little prior knowledge.

Improved Physics-Informed Neural Networks Combined with Small Sample Learning to Solve Two-Dimensional Stefan Problem

With the remarkable development of deep learning in the field of science, deep neural networks provide a new way to solve the Stefan problem. In this paper, deep neural networks combined with small sample learning and a general deep learning framework are proposed to solve the two-dimensional Stefan problem. In the case of adding less sample data, the model can be modified and the prediction accuracy can be improved. In addition, by solving the forward and inverse problems of the two-dimensional single-phase Stefan problem, it is verified that the improved method can accurately predict the solutions of the partial differential equations of the moving boundary and the dynamic interface.

Automatic Detection of Small Sample Apple Surface Defects Using ASDINet.

The appearance quality of apples directly affects their price. To realize apple grading automatically, it is necessary to find an effective method for detecting apple surface defects. Aiming at the problem of a low recognition rate in apple surface defect detection under small sample conditions, we designed an apple surface defect detection network (ASDINet) suitable for small sample learning. The self-developed apple sorting system collected RGB images of 50 apple samples for model verification, including non-defective and defective apples (rot, disease, lacerations, and mechanical damage). First, a segmentation network (AU-Net) with a stronger ability to capture small details was designed, and a Dep-conv module that could expand the feature capacity of the receptive field was inserted in its down-sampling path. Among them, the number of convolutional layers in the single-layer convolutional module was positively correlated with the network depth. Next, to achieve real-time segmentation, we replaced the flooding of feature maps with mask output in the 13th layer of the network. Finally, we designed a global decision module (GDM) with global properties, which inserted the global spatial domain attention mechanism (GSAM) and performed fast prediction on abnormal images through the input of masks. In the comparison experiment with state-of-the-art models, our network achieved an AP of 98.8%, and a 97.75% F1-score, which were higher than those of most of the state-of-the-art networks; the detection speed reached 39ms per frame, achieving accuracy-easy deployment and substantial trade-offs that are in line with actual production needs. In the data sensitivity experiment, the ASDINet achieved results that met the production needs under the training of 42 defective pictures. In addition, we also discussed the effect of the ASDINet in actual production, and the test results showed that our proposed network demonstrated excellent performance consistent with the theory in actual production.