Small Sample Datasets Research Articles

A New Method of Time-Series Event Prediction Based on Sequence Labeling

In the existing research on time-series event prediction (TSEP) methods, most of the work is focused on improving the algorithm for classifying subsequence sets (sets composed of multiple adjacent subsequences). However, these prediction methods ignore the timing dependence between the subsequence sets, nor do they capture the mutual transition relationship between events, the prediction effect on a small sample data set is very poor. Meanwhile, the sequence labeling problem is one of the common problems in natural language processing and image segmentation. To solve this problem, this paper proposed a new framework for time-series event prediction, which transforms the event prediction problem into a labeling problem, to better capture the timing relationship between the subsequence sets. Specifically, the framework used a sequence clustering algorithm for the first time to identify representative patterns in the time series, then represented the set of subsequences as a weighted combination of patterns, and used the eXtreme gradient boosting algorithm (XGBoost) for feature selection. After that, the selected pattern feature was used as the input of the long-term short-term memory model (LSTM) to obtain the preliminary prediction value. Furthermore, the fully-linked conditional random field (CRF) was used to smooth and refine the preliminary prediction value to obtain the final prediction result. Finally, the experimental results of event prediction on five real data sets show that the CX-LC method has a certain improvement in prediction accuracy compared with the other six models.

Origin Identification of Saposhnikovia divaricata by CNN Embedded with the Hierarchical Residual Connection Block

This paper proposes a method for recognizing the origin of Saposhnikovia divaricata using the IResNet model to achieve computer vision-based classification. Firstly, we created a small sample dataset and applied data augmentation techniques to enhance its diversity. After that, we introduced the hierarchical residual connection block in the early stage of the original model to expand the perceptual field of the neural network and enhance the extraction of scale features. Meanwhile, a depth-separable convolution operation was adopted in the later stage of the model to replace the conventional convolution operation and further reduce the time cost of the model. The experimental results demonstrate that the improved network model achieved a 5.03% improvement in accuracy compared to the original model while also significantly reducing the number of parameters required for the model. In our experiments, we compared the accuracy of the proposed model with several classical convolutional neural network models, including ResNet50, Resnest50, Res2net50, RepVggNet_B0, and ConvNext_T. The results showed that our proposed model achieved an accuracy of 93.72%, which outperformed ResNet50 (86.68%), Resnest50 (89.38%), Res2net50 (91.83%), RepVggNet_B0 (88.68%), and ConvNext_T (92.18%). Furthermore, our proposed model achieved the highest accuracy among the compared models, with a transmission frame rate of 158.9 fps and an inference time of only 6.29 ms. The research methodology employed in this study has demonstrated the ability to reduce potential errors caused by manual observation, effectively improving the recognition ability of Saposhnikovia divaricata based on existing data. Furthermore, the findings of this study provide valuable reference and support for future efforts to develop lightweight models in this area.

Evaluating thermal expansion in fluorides and oxides: Machine learning predictions with connectivity descriptors

Open framework structures (e.g., ScF3, Sc2W3O12, etc.) exhibit significant potential for thermal expansion tailoring owing to their high atomic vibrational degrees of freedom and diverse connectivity between polyhedral units, displaying positive/negative thermal expansion (PTE/NTE) coefficients at a certain temperature. Despite the proposal of several physical mechanisms to explain the origin of NTE, an accurate mapping relationship between the structural–compositional properties and thermal expansion behavior is still lacking. This deficiency impedes the rapid evaluation of thermal expansion properties and hinders the design and development of such materials. We developed an algorithm for identifying and characterizing the connection patterns of structural units in open-framework structures and constructed a descriptor set for the thermal expansion properties of this system, which is composed of connectivity and elemental information. Our developed descriptor, aided by machine learning (ML) algorithms, can effectively learn the thermal expansion behavior in small sample datasets collected from literature-reported experimental data (246 samples). The trained model can accurately distinguish the thermal expansion behavior (PTE/NTE), achieving an accuracy of 92%. Additionally, our model predicted six new thermodynamically stable NTE materials, which were validated through first-principles calculations. Our results demonstrate that developing effective descriptors closely related to thermal expansion properties enables ML models to make accurate predictions even on small sample datasets, providing a new perspective for understanding the relationship between connectivity and thermal expansion properties in the open framework structure. The datasets that were used to support these results are available on Science Data Bank, accessible via the link https://doi.org/10.57760/sciencedb.j00113.00100.

A diagnosis model of soybean leaf diseases based on improved residual neural network

To solve the problems of high complexity and large computation of current disease diagnosis models based on deep learning, which are difficult to be applied to commonly used portable mobile terminals, a lightweight soybean disease diagnosis model based on attention mechanism and residual neural network was proposed. The residual attention layer (RAL) was constructed by using the attention mechanism, continuous convolution layer and shortcut connection on the traditional residual neural network, and was inserted into the residual neural network (ResNet18) in order to replace the residual structural layer. And a novel residual attention network model (RANet18) was built for soybean disease diagnosis. Based on 4301 images of soybean brown leaf spot, soybean frogeye leaf spot and soybean phyllosticta leaf spot, the RANet18 was used to conduct simulation experiments. The test result showed that the recognition accuracy was 96.50%, while the recognition time was 0.047184s, and the model size was only 40.64 MB with the F1 value was 96.43. To further verify the performance of the RANet18 on the above small sample data set, a comparative test was conducted between RANet18 and the original model, and the training time of RANet18 was 49.76% less than that of ResNet18. The model size was simplified by 4.94% and the average recognition time was saved by 45.60%. This achievement constructed a fast, efficient and accurate lightweight disease recognition model suitable for small sample data sets, and provided relevant basis and guarantee for rapid diagnosis and accurate control of crop diseases.

ConvNeXt-Haze: A Fog Image Classification Algorithm for Small and Imbalanced Sample Dataset Based on Convolutional Neural Network

Aiming at the different abilities of the defogging algorithms in different fog concentrations, this paper proposes a fog image classification algorithm for a small and imbalanced sample dataset based on a convolution neural network, which can classify the fog images in advance, so as to improve the effect and adaptive ability of image defogging algorithm in fog and haze weather. In order to solve the problems of environmental interference, camera depth of field interference and uneven feature distribution in fog images, the CutBlur-Gauss data augmentation method and focal loss and label smoothing strategies are used to improve the accuracy of classification. It is compared with the machine learning algorithm SVM and classical convolution neural network classification algorithms alexnet, resnet34, resnet50 and resnet101. This algorithm achieves 94.5% classification accuracy on the dataset in this paper, which exceeds other excellent comparison algorithms at present, and achieves the best accuracy. It is proved that the improved algorithm has better classification accuracy.

ADAPTATION OF DOMAIN-SPECIFIC TRANSFORMER MODELS WITH TEXT OVERSAMPLING FOR SENTIMENT ANALYSIS OF SOCIAL MEDIA POSTS ON COVID-19 VACCINE

Covid-19 has spread across the world and many different vaccines have been developed to counter its surge. To identify the correct sentiments associated with the vaccines from social media posts, this paper aims to fine-tune pre-trained transformer models on tweets associated with different Covid vaccines, specifically RoBERTa, XLNet and BERT which are recently introduced state-of-the-art bi-directional transformer models, and domain-specific transformer models BERTweet and CT-BERT that are pre-trained on Covid-19 tweets. We further explore the option of data augmentation by text oversampling using LMOTE to improve the accuracies of these models, specifically, for small sample datasets where there is an imbalanced class distribution among the positive, negative and neutral sentiment classes. Our results summarize our findings on the suitability of text oversampling for imbalanced, small sample datasets that are used to fine-tune state-of-the-art pre-trained transformer models, and the utility of having domain-specific transformer models for the classification task.

Tool wear assessment and life prediction model based on image processing and deep learning

Drilling is one of the most classical machining operations. Real-time monitoring of drill wear can effectively testify whether the product fails to meet the specifications due to drill failure. This paper proposes a tool wear assessment and life prediction model based on image processing and deep learning methods, which works effectively for small sample datasets and for low-quality images. The normal areas and worn areas of the drill bits are extracted using the U-Net network and traditional image processing methods, respectively. Moreover, the original dataset is classified using the migration learning technique. The wear level of a drill bit can be accurately evaluated through experimental tests. Testing results show that the proposed method is more convenient and efficient than previous methods using manual measurements. These results can be applied to real-time drill wear monitoring, thus reducing part damage caused by tool wear.

Self-Attention Based Neural Network for Predicting RNA-Protein Binding Sites.

Proteins binding to Ribonucleic Acid (RNA) inside cells are called RNA-binding proteins (RBP), which play a crucial role in gene regulation. The identification of RNA-protein binding sites helps to understand the function of RBP better. Although many computational methods have been developed to predict RNA-protein binding sites, their prediction accuracy on small sample datasets needs improvement. To overcome this limitation, we propose a novel model called SA-Net, which utilizes k-mer embedding to encode RNA sequences and a self-attention-based neural network to extract sequence features. K-mer embedding assists the model to discover significant subsequence fragments associated with binding sites. The self-attention mechanism captures contextual information from the entire input sequence globally, performing well in small sample sequence learning. Experimental results demonstrate that SA-Net attains state-of-the-art results on the RBP-24 dataset. We find that 4-mer embedding aids the model to achieve optimal performance. We also show that the self-attention network outperforms the commonly used CNN and CNN-BLSTM models in sequence feature extraction.

The effects of station density in geostatistical prediction of air temperatures in Sweden: A comparison of two interpolation techniques

High fidelity gridded temperature datasets are difficult to obtain for areas with sparse coverage of meteorological stations given that sparsity of stations is known to introduce uncertainty in the interpolation of climatic variables generally. Inspired by their potential for optimal results especially for small sample datasets, the accuracy of interpolation results of Empirical Bayesian Kriging (EBK) and EBK-Regression Prediction (EBKRP) spatial prediction techniques were assessed and compared under varying sampling density scenarios, using monthly maximum temperature normals (1991–2020) for the entire area of Sweden. The objective of the was to understand how EBK and EBKRP interpolation techniques perform in different sampling density scenarios and in a sparse data setting, and the possible difference in the prediction accuracy between the two techniques. The 708 sampled stations obtained from the historical climatology database of the Swedish Meteorological and Hydrological Institute (SMHI) were split into seven sampling density subsets, ranging from 1 sample per 63,614 km 2 to 1 sample per 634,350 km 2 and representing both low and high sampling density scenarios. EBK interpolation technique was implemented using temperature data while land use land cover (LULC) and digital elevation model (DEM) were used as temperature covariates for the EBKRP interpolation models. The prediction accuracy assessment was based on five robust prediction performance indicators - mean error, mean absolute error, mean square error, root mean square error and Pearson correlation (R) - obtained from independent validation/cross-validation operations. Prediction accuracy was found to be generally positively related to sampling density and sampling density accounted for 85%–87% of interpolation accuracy for both EBK and EBKRP techniques. Although sampling density increased linearly, the rate of change in accuracy from one sampling density step to the next was not particularly proportional. A rapid increase in accuracy was observed when transiting from 40% to 60% sampling density scenario, but the rate of change in accuracy remained fairly stable before and after this (40% to 60%) sampling density scenario. For equivalent sampling density set-ups, EBKRP consistently performed better than EBK in all the accuracy metrics and EBKRP proved to be about 40% better than EBK. However, the two interpolation techniques produced generally very low prediction biases at all the sampling density scenarios investigated. Our study suggests that potential effects of low sampling density and non-stationarity of temperature data can be significantly reduced by applying EBK but especially EBKRP when coupled with relevant covariates. We also concluded that EBK and EBKRP can be applied to generate accurate prediction surfaces even in a widely varying sampling density setting including environments with moderately low sample density. This is especially true for continuous and slowly varying phenomena such as temperature and similar variables. • Station density is positively related to interpolation accuracy of air temperature but in non-linear fashion. • Univariate and multivariate Empirical Bayesian Kriging interpolation techniques can produce high prediction accuracy result even when data density is small or locally non-stationary. • Multivariate Empirical Bayesian Kriging technique performs better than the univariate technique in different sampling density settings when predicting air temperature. • Normally distributed data can generally improve prediction results even when data density is relatively small or sparse.

A U-Shaped Network Based on Multi-level Feature and Dual-Attention Coordination Mechanism for Coronary Artery Segmentation of CCTA Images.

Computed tomography coronary angiography (CCTA) images provide optimal visualization of coronary arteries to aid in diagnosing coronary heart disease (CHD). With the deep convolutional neural network, this work aims to develop an intelligent and lightweight coronary artery segmentation algorithm that can be deployed in hospital systems to assist clinicians in quantitatively analyzing CHD. With the multi-level feature fusion, we proposed Dual-Attention Coordination U-Net (DAC-UNet) that achieves automated coronary artery segmentation in 2D CCTA images. The coronary artery occupies a small region, and the foreground and background are extremely unbalanced. For this reason, the more original information can be retained by fusing related features between adjacent layers, which is conducive to recovering the small coronary artery area. The dual-attention coordination mechanism can select valid information and filter redundant information. Moreover, the complementation and coordination of double attention factors can enhance the integrity of features of coronary arteries, reduce the interference of non-coronary arteries, and prevent over-learning. With gradual learning, the balanced character of double attention factors promotes the generalization ability of the model to enhance coronary artery localization and contour detail segmentation. Compared with existing related segmentation methods, our method achieves a certain degree of improvement in 2D CCTA images for the segmentation accuracy of coronary arteries with a mean Dice index of 0.7920. Furthermore, the method can obtain relatively accurate results even in a small sample dataset and is easy to implement and deploy, which is promising. The code is available at: https://github.com/windfly666/Segmentation . Our method can capture the coronary artery structure end-to-end, which can be used as a fundamental means for automatic detection of coronary artery stenosis, blood flow reserve fraction analysis, and assisting clinicians in diagnosing CHD.

Physical-priors-guided DehazeFormer

Single-image dehazing is a challenging task in several machine-vision applications. Methods based on physical models and prior knowledge fail under certain conditions, resulting in defects such as color distortion. Transformer-based methods have a strong representation ability owing to their self-attention mechanism that can effectively obtain global information. However, this approach is computationally expensive, and its weak inductive bias capability increases the risk of overfitting on small-sample datasets. To address these problems, in this study, we propose a novel DehazeFormer guided by physical priors, named SwinTD-Net, which is trained according to supervised and self-supervised learning, and combines the advantages of physical priors and transformers. The proposed DehazeFormer learns features guided by physical priors, which improves the generalization ability of the network and enables it to achieve good restoration effects on both synthetic and real-world hazy images. In addition, we propose a more appropriate prior input to better use physical priors, and we design a multi-scale dark-light enhancement algorithm for image restoration post-processing, which can improve the visual perception quality for human observers while performing some local enhancements. Extensive experiments illustrate that the proposed method outperforms state-of-the-art methods. The code and pre-trained models are available to academics so that they can reproduce our results and test them (https://github.com/hocking-cloud/SwinTD_Net).

Research on arc fault detection using ResNet and gamma transform regularization

Series arc fault is the main cause of electrical fire in low-voltage distribution system. A fast and accurate detection system can reduce the risk of fire effectively. In this paper, series arc experiment is carried out for different kinds of electrical load. The time-domain current is analyzed by Morlet wavelet. Then, the multiscale wavelet coefficients are expressed as the coefficient matrix. In order to meet the data dimension requirements of neural networks, a color domain transformation method is used to transform the feature matrix into an image. A regularization method based on gamma transform is proposed for small sample data sets. The results showed that the proposed regularization method improved the validation set accuracy of ResNet50 from 66.67% to 96.53%. The overfitting problem of neural network was solved. In addition, this method fused fault features of 64 different scales, and provided a valuable manually labeled arc fault dataset. Compared with the threshold detection method, this method was more objective. The use of image features increased intuitiveness and generality. Compared with other typical lightweight networks, this method had the best detection performance.

Pedestrian gender classification on imbalanced and small sample datasets using deep and traditional features

Pedestrian gender classification on imbalanced and small sample datasets using deep and traditional features

A Time Series Forecasting Model Selection Framework using CNN and Data Augmentation for Small Sample Data

The key to the accuracy of time series forecasting is to find the most appropriate forecasting method. Therefore, the forecasting model selection of time series has become a new research hotspot in the data analysis field. However, most of the existing meta learning forecasting model selection methods rely on manual selection of features, which leads to low efficiency and lack of objectivity. Therefore, this paper proposes an improved meta learning framework for deep learning time series forecasting model selection. Inspired by computer vision, we transform one -dimensional time series into two-dimensional images, and use convolution neural network to train and classify time series images (model selection). Moreover, in order to deal with the over fitting problem caused by small sample datasets, the sliding window data augmentation method is used to improve the accuracy of small datasets model selection. The large-scale empirical study on M3 data sets shows that the framework has better model selection accuracy and smaller forecasting error than the recurrent neural network (RNN), back propagation neural network and traditional time series image algorithms. In addition, compared with the traditional time series image method, RNN and BP, the classification rate (model selection accuracy) of this algorithm is improved by 6.5%, 4.4% and 3.2%, respectively.

Hydrogen yield prediction for supercritical water gasification based on generative adversarial network data augmentation

Machine learning methods can accurately predict the hydrogen yield of supercritical water gasification (SCWG), which can provide a reference for SCWG optimization. However, the time cost of obtaining SCWG data by experiments is high, and the prediction accuracy of the SCWG model is low when the sample number is small. For this problem, a novel sample generation method based on the generative adversarial network (GAN) and the hybrid strategy is proposed to achieve data augmentation for SCWG hydrogen yield prediction. The hybrid strategy that combines the cross-validation and looping iteration methods can reduce the randomness of the GAN, thereby improving the generated sample quality. In addition, the least squares support vector regression is adopted to construct a hydrogen yield prediction model. A case study was conducted on a small sample SCWG dataset of Yimin lignite with reaction temperatures in the range of 650–850 °C. The results show that after expanding the small sample dataset by the single GAN, the average RMSE and MAE of the prediction model are reduced by 20.43% and 11.58%, respectively. The proposed sample generation method is applied to the SCWG hydrogen yield prediction. The average RMSE and MAE are as low as 1.1737 mol/kg and 1.0171 mol/kg, respectively, while the average R2 reaches 0.9457. The average RMSE and MAE are further reduced by 18.44% and 15.29% based on the single GAN model, respectively. It indicates that the hybrid strategy improves the quality and reliability of the generated samples, and the proposed method can effectively achieve data augmentation for SCWG hydrogen yield prediction, which can provide ideas for the small sample SCWG modeling.

Diagnosis and Mobile Application of Apple Leaf Disease Degree Based on a Small-Sample Dataset.

The accurate segmentation of apple leaf disease spots is the key to identifying the classification of apple leaf diseases and disease severity. Therefore, a DeepLabV3+ semantic segmentation network model with an actors spatial pyramid pool module (ASPP) was proposed to achieve effective extraction of apple leaf lesion features and to improve the apple leaf disease recognition and disease severity diagnosis compared with the classical semantic segmentation network models PSPNet and GCNet. In addition, the effects of the learning rate, optimizer, and backbone network on the performance of the DeepLabV3+ network model with the best performance were analyzed. The experimental results show that the mean pixel accuracy (MPA) and mean intersection over union (MIoU) of the model reached 97.26% and 83.85%, respectively. After being deployed into the smartphone platform, the detection time of the detection system was 9s per image for the portable and intelligent diagnostics of apple leaf diseases. The transfer learning method provided the possibility of quickly acquiring a high-performance model under the condition of small datasets. The research results can provide a precise guide for the prevention and precise control of apple diseases in fields.

Non-contact diagnosis for gearbox based on the fusion of multi-sensor heterogeneous data

Non-contact sensing technology plays an important role in the health monitoring of the gearbox. However, a single non-contact measurement is challenging to achieve the simultaneous monitoring of both structural and non-structural damages. In order to explore the fusion mechanism of multi-sensor heterogeneous measurements, acoustic and thermal characteristics of the gearbox under typical fault states are analyzed, and it is verified the fusion of infrared thermal (IRT) images and acoustic data integrates complementary fault information. In this paper, an attention-enhanced information fusion diagnosis network (AIFN-IA) is proposed for the complementary fusion of IRT images and acoustic data. Firstly, the acoustic data is converted into images by the non-hyperparameter encoding method and then fused with IRT images in data-level. Secondly, the limited shuffle attention module is designed to adaptively focus on the fault elements hidden in the complex fusion features. Finally, experimental data verify the effectiveness of the proposed AIFN-IA method in recognizing six structural and non-structural damages of the gearbox. Compared with seven state-of-the-art methods, the proposed AIFN-IA method performs best in extracting discriminating features with the highest diagnosis accuracy. Moreover, the proposed AIFN-IA method can still achieve satisfactory results under the challenges of small sample datasets and strong noise interference, which is more competitive in real industrial applications.

Application Of Machine Learning Algorithms to Free Cash Flows Growth Rate Estimation

Machine learning (ML) demonstrates superior accuracy in financial time-series forecasting compared to traditional statistical models. While most studies focus on applying ML algorithms to high-frequency pricing data, the availability of fundamental financial data is limited as it is generated quarterly. This paper investigates the performance of nine ML algorithms in small sample data sets, against an ARIMA model — frequently used for financial time-series forecasting — serving as a benchmark. Results obtained from 100 US companies indicate that the majority of ML algorithms exhibit low error rates on the test set, outperforming benchmark results. Notably, the k-nearest neighbor algorithm achieves the highest prediction accuracy among the algorithms considered, even with only 33 data observations, while avoiding overfitting.

Classification and recognition of milk somatic cell images based on PolyLoss and PCAM-Reset50.

Somatic cell count (SCC) is a fundamental approach for determining the quality of cattle and bovine milk. So far, different classification and recognition methods have been proposed, all with certain limitations. In this study, we introduced a new deep learning tool, i.e., an improved ResNet50 model constructed based on the residual network and fused with the position attention module and channel attention module to extract the feature information more effectively. In this paper, macrophages, lymphocytes, epithelial cells, and neutrophils were assessed. An image dataset for milk somatic cells was constructed by preprocessing to increase the diversity of samples. PolyLoss was selected as the loss function to solve the unbalanced category samples and difficult sample mining. The Adam optimization algorithm was used to update the gradient, while Warm-up was used to warm up the learning rate to alleviate the overfitting caused by small sample data sets and improve the model's generalization ability. The experimental results showed that the classification accuracy, precision rate, recall rate, and comprehensive evaluation index F value of the proposed model reached 97%, 94.5%, 90.75%, and 92.25%, respectively, indicating that the proposed model could effectively classify the milk somatic cell images, showing a better classification performance than five previous models (i.e., ResNet50, ResNet18, ResNet34, AlexNet andMobileNetv2). The accuracies of the ResNet18, ResNet34, ResNet50, AlexNet, MobileNetv2, and the new model were 95%, 93%, 93%, 56%, 37%, and 97%, respectively. In addition, the comprehensive evaluation index F1 showed the best effect, fully verifying the effectiveness of the proposed method in this paper. The proposed method overcame the limitations of image preprocessing and manual feature extraction by traditional machine learning methods and the limitations of manual feature selection, improving the classification accuracy and showing a strong generalization ability.

Fault States Diagnosis of Marine Diesel Engine Valve Based on a Modified VGG16 Transfer Learning Method

The marine diesel engine is an important power machine for ships. Traditional machine learning methods for diesel engine fault diagnosis usually require a large amount of labeled training data, and the diagnosis performance may decline when encounters vibrational and environmental interference. A transfer learning convolutional neural network model based on VGG16 is introduced for diesel engine valve leakage fault diagnosis. The acquired diesel engine cylinder head vibration signal is first converted to time domain, frequency domain, and wavelet decomposition images. Secondly, the VGG16 deep convolutional neural network is pretrained using the ImageNet dataset. Subsequently, fine tuning the network based on the pretrained basic parameters and image enhancement methods. Finally, the well‐trained model is adopted to train and test the target dataset. In addition, the cosine annealing learning rate setting method is used to make the learning rate close to the global optimal solution. Experimental results show that the proposed method has higher accuracy and better robustness against noise with a small sample dataset than traditional methods and deep learning models. This study not only demonstrates a novel view for the diagnosis of marine diesel engine valve leakage, but also provides an applicable diagnosis method for other similar issues.