Small-scale Datasets Research Articles

Cross-Anatomy Transfer Learning via Shape-Aware Adaptive Fine-Tuning for 3D Vessel Segmentation.

Deep learning methods have recently achieved remarkable performance in vessel segmentation applications, yet require numerous labor-intensive labeled data. To alleviate the requirement of manual annotation, transfer learning methods can potentially be used to acquire the related knowledge of tubular structures from public large-scale labeled vessel datasets for target vessel segmentation in other anatomic sites of the human body. However, the cross-anatomy domain shift is a challenging task due to the formidable discrepancy among various vessel structures in different anatomies, resulting in the limited performance of transfer learning. Therefore, we propose a cross-anatomy transfer learning framework for 3D vessel segmentation, which first generates a pre-trained model on a public hepatic vessel dataset and then adaptively fine-tunes our target segmentation network initialized from the model for segmentation of other anatomic vessels. In the framework, the adaptive fine-tuning strategy is presented to dynamically decide on the frozen or fine-tuned filters of the target network for each input sample with a proxy network. Moreover, we develop a Gaussian-based signed distance map that explicitly encodes vessel-specific shape context. The prediction of the map is added as an auxiliary task in the segmentation network to capture geometry-aware knowledge in the fine-tuning. We demonstrate the effectiveness of our method through extensive experiments on two small-scale datasets of coronary artery and brain vessel. The results indicate the proposed method effectively overcomes the discrepancy of cross-anatomy domain shift to achieve accurate vessel segmentation for these two datasets.

PlantBiCNet: A new paradigm in plant science with bi-directional cascade neural network for detection and counting

Deep learning is increasingly popular for precise plant detection and counting in computer vision applications. Despite the rising cross-disciplinary research in this field, an important issue has been neglected: due to the inherent constraints imposed by the natural growth patterns of plants, the costly and time-consuming process of collecting plant image data, resulting in a scarcity of real-world available data. Consequently, researchers are often constrained to exploring large-scale data representations or limited applications. Currently, still no generalized network architecture excels in demanding environments. To address this, we introduce two new bi-directional cascade neural network paradigms, PlantBiCNet and its lightweight variant, PlantBiCNet-Lite. Unlike previous research, we place greater emphasis on the network decoding process itself rather than the encoder. Our bi-directional cascade and weight fusion decoding approach maximizes high-level semantics and low-level spatial data, enhancing the localization signal in a way that respects gradient diversity. We evaluate PlantBiCNet and PlantBiCNet-Lite on five public plant detection and counting datasets and a novel plant dataset. The average precision AP50 on these six small-scale datasets reached 0.905, with a high coefficient of determination R2 of 0.935, while the reported frame rate was 196 FPS. These results significantly outperform state-of-the-art computer vision methods. Moreover, we found that the introduction of the Dropout mechanism markedly enhances the model's performance and generalization ability. Further analysis supports our experiments and provides valuable insights into network optimization. Overall, our organization's open datasets and network modeling details provide a solid foundation for future research in the field of plant science.

Few-shot Sentiment Analysis Based on Adaptive Prompt Learning and Contrastive Learning

Traditional deep learning-based strategies for sentiment analysis rely heavily on large-scale labeled datasets for model training, but these methods become less effective when dealing with small-scale datasets. Fine-tuning large pre-trained models on small datasets is currently the most commonly adopted approach to tackle this issue. Recently, prompt-based learning has gained significant attention as a promising research area. Although prompt-based learning has the potential to address data scarcity problems by utilizing prompts to reformulate downstream tasks, the current prompt-based methods for few-shot sentiment analysis are still considered inefficient. To tackle this challenge, an adaptive prompt-based learning method is proposed, which includes two aspects. Firstly, an adaptive prompting construction strategy is proposed, which can capture the semantic information of texts by utilizing a dot-product attention structure, improving the quality of the prompt templates. Secondly, contrastive learning is applied to the implicit word vectors obtained twice during the training stage to alleviate over-fitting in few-shot learning processes. This improves the model’s generalization ability by achieving data enhancement while keeping the semantic information of input sentences unchanged. Experimental results on the ERPSTMT datasets of FewCLUE demonstrate that the proposed method have great ability to construct suitable adaptive prompts and outperforms the state-of-the-art baselines.

Data-driven propagation prediction of subsonic and supersonic turbulent jets by combining self-similarity analysis model and artificial neural network

Previous self-similarity analysis models for turbulent jet flames (TJF) have inherent limitations in the flame tip location and velocity prediction based on experimental data. A novel model (BP-TJF) to predict the propagation process of subsonic and supersonic TJF is proposed by combining developed self-similarity analysis modeling and back propagation neural network (BPNN). The BP-TJF model is trained from three datasets of initial temperature, initial pressure, and oxygen content. The results show that the pressure difference prediction error was only 0.46 % for subsonic jets and 5 % for supersonic jets. The overall correlation coefficients (R) and mean squared errors (MSE) range from 0.95–0.97 and 0.01–0.1, respectively. The model optimized by genetic algorithm (GA) significantly improved the prediction stability. However, there is scope for improvement in the overpressure peak prediction. Due to the small-scale datasets and parameter errors of self-similar model for jet propagation, the model cannot provide real-time feedback on the interaction between the shock wave and the flame front. Jet tip locations and velocities obtained from the BP-TJF model and experiments are generally consistent in magnitude and overall trends. Without considering the flame structure, the prediction framework developed in this paper can calculate the jet tip propagation characteristics with little difference from experiments and CFD, which is a great advantage, especially in the calculation of subsonic jets.

PE-Net: a parallel framework for 3D inferior mesenteric artery segmentation.

The structural morphology of mesenteric artery vessels is of significant importance for the diagnosis and treatment of colorectal cancer. However, developing automated vessel segmentation methods for this purpose remains challenging. Existing convolution-based segmentation methods have limitations in capturing long-range dependencies, while transformer-based models require large datasets, making them less suitable for tasks with limited training samples. Moreover, over-segmentation, mis-segmentation, and vessel discontinuity are common challenges in vessel segmentation tasks. To address these issues, we propose a parallel encoding architecture that combines transformers and convolutions to retain the advantages of both approaches. The model effectively learns position deviations and enhances robustness for small-scale datasets. Additionally, we introduce a vessel edge capture module to improve vessel continuity and topology. Extensive experimental results demonstrate the improved performance of our model, with Dice Similarity Coefficient and Average Hausdorff Distance scores of 81.64% and 7.7428, respectively.

Obfuscating Community Structure in Complex Network With Evolutionary Divide-and-Conquer Strategy

As the number of social network users grows exponentially with increasingly complex profiles, community detection algorithms play a critical role in user portrait analysis. The associated privacy concerns, however, have not sufficiently received the attention that it deserves. In this work, we investigate methods for obfuscating the original community structure by modifying a small number of connections imperceptibly so as to protect the privacy of users. The existing evolutionary models have some successes in this type of NP-hard problem but can only be applied to small-scale datasets, rendering them inadequate for real-world applications. To alleviate this problem, we propose an original and novel CoeCo, a cooperative evolutionary community obfuscation model. In CoeCo, we leverage the divide-and-conquer strategy and put forward a co-evolutionary optimization algorithm suitable for community structure, in which two different fitness functions promote each other to find the optimal edge set. In addition, the motif hypergraph and permanence are used to improve population initialization. The experimental results indicate that our proposed method can achieve excellent efficacy in obfuscating community structure and also greatly reduces running time.

Mitochondrial genome diversity across the subphylum Saccharomycotina.

Eukaryotic life depends on the functional elements encoded by both the nuclear genome and organellar genomes, such as those contained within the mitochondria. The content, size, and structure of the mitochondrial genome varies across organisms with potentially large implications for phenotypic variance and resulting evolutionary trajectories. Among yeasts in the subphylum Saccharomycotina, extensive differences have been observed in various species relative to the model yeast Saccharomyces cerevisiae, but mitochondrial genome sampling across many groups has been scarce, even as hundreds of nuclear genomes have become available. By extracting mitochondrial assemblies from existing short-read genome sequence datasets, we have greatly expanded both the number of available genomes and the coverage across sparsely sampled clades. Comparison of 353 yeast mitochondrial genomes revealed that, while size and GC content were fairly consistent across species, those in the genera Metschnikowia and Saccharomyces trended larger, while several species in the order Saccharomycetales, which includes S. cerevisiae, exhibited lower GC content. Extreme examples for both size and GC content were scattered throughout the subphylum. All mitochondrial genomes shared a core set of protein-coding genes for Complexes III, IV, and V, but they varied in the presence or absence of mitochondrially-encoded canonical Complex I genes. We traced the loss of Complex I genes to a major event in the ancestor of the orders Saccharomycetales and Saccharomycodales, but we also observed several independent losses in the orders Phaffomycetales, Pichiales, and Dipodascales. In contrast to prior hypotheses based on smaller-scale datasets, comparison of evolutionary rates in protein-coding genes showed no bias towards elevated rates among aerobically fermenting (Crabtree/Warburg-positive) yeasts. Mitochondrial introns were widely distributed, but they were highly enriched in some groups. The majority of mitochondrial introns were poorly conserved within groups, but several were shared within groups, between groups, and even across taxonomic orders, which is consistent with horizontal gene transfer, likely involving homing endonucleases acting as selfish elements. As the number of available fungal nuclear genomes continues to expand, the methods described here to retrieve mitochondrial genome sequences from these datasets will prove invaluable to ensuring that studies of fungal mitochondrial genomes keep pace with their nuclear counterparts.

A data-driven model of drop size prediction based on artificial neural networks using small-scale data sets

An artificial neural network (ANN) method is introduced to predict drop size in two kinds of pulsed columns with small-scale data sets. After training, the deviation between calculate and experimental results are 3.8% and 9.3%, respectively. Through ANN model, the influence of interfacial tension and pulsation intensity on the droplet diameter has been developed. Droplet size gradually increases with the increase of interfacial tension, and decreases with the increase of pulse intensity. It can be seen that the accuracy of ANN model in predicting droplet size outside the training set range is reach the same level as the accuracy of correlation obtained based on experiments within this range. For two kinds of columns, the drop size prediction deviations of ANN model are 9.6% and 18.5% and the deviations in correlations are 11% and 15%.

Cost Estimation of Manufacturing Enterprises based on BP Neural Network and Big Data Analysis

The manufacturing industry is the pillar industry of modern industry, and the cost estimation of manufacturing enterprises is an important management means of the manufacturing industry. Aiming at the cost estimation problem of manufacturing enterprises, this research proposes a cost estimation method based on Back Propagation (Back Propagation) neural network and big data analysis. In the process, the Lambda architecture was used to construct the big data analysis architecture of manufacturing enterprises, the K-means clustering algorithm was introduced for data clustering, and then the genetic algorithm was combined with the Back Propagation neural network to estimate the cost. In the estimation accuracy test, the accuracy of the research method can reach 94.7% after 240 iterations; in the calculation time test, the calculation time of the research method is 403 Ks when the data size is 500 Gb in a large-scale data set; in the call data volume test, the call data volume of the research method is 164 Kb when the research method is carried out to the seventh step in the small-scale data set; when the application analysis is carried out, the research method completes accurate cost estimation for 9 target parts. This research method has good model performance and calculation accuracy, and can effectively estimate manufacturing enterprises’ costs.

Bayesian variational transformer: A generalizable model for rotating machinery fault diagnosis

Transformer has been widely applied in the research of rotating machinery fault diagnosis due to its ability to explore the internal correlation of vibration signals. However, challenges still exist despite the countless efforts. Generally, Transformer is more prone to overfitting than CNN on small-scale datasets. In practical engineering, collecting sufficient fault samples for training is difficult, resulting in poor generalization of Transformer. In addition, the measured signals are often accompanied with severe noise, further reducing the generalization performance of the model. Meanwhile, the collected signals often follow different distributions due to the changing operating conditions, which places higher demands on the generalizability of Transformer. This paper proposes a Bayesian variational Transformer (Bayesformer) to cope with the abovementioned problems. In Bayesformer, all the attention weights are treated as latent random variables, rather than determined values as the previous studies. This allows to train an ensemble of networks, instead of a single one, enhancing the generalizability of the model. Three experimental studies are conducted to illustrate the developed model and superior diagnostic performance is showed throughout the experiments.

DS-Former: A dual-stream encoding-based transformer for 3D medical image segmentation

Models that utilize self-attention mechanisms, including but not limited to Vision Transformers (ViTs), have shown promising performance in visual tasks like semantic segmentation. This is attributed to their capacity to capture global features of images, enabling them to learn more comprehensive representations. However, transformer-based models typically demand a considerable amount of training data to achieve satisfactory performance, while being deficient in the ability to efficiently extract local image features. As a result, these models may not be as effective in some computer vision tasks that involve small-scale datasets, like medical image segmentation. To address these issues, this paper proposes a dual-stream encoding-based transformer dubbed as Dual-stream Transformer (DS-Former). The dual-stream module in DS-Former can simultaneously acquire local and global features in the image and construct relation between the two kinds of features via self-attention. Compared with the simple splicing or serial connection, the dual-stream module can extract more comprehensive and hierarchical feature information from the fusion interaction of the two features. Our method is evaluated on the UK Biobank (UKBB) cardiac magnetic resonance imaging (CMR) dataset and The Beyond the Cranial Vault (BTCV) abdominal challenge dataset. The experimental results indicate that our DS-Former outperforms other state-of-the-art approaches on both datasets, indicating its potential for medical images semantic segmentation.

A physics-constrained neural network for crystal plasticity modelling of FCC materials

In the current work, a physics-constrained neural network is coupled with the crystal plasticity theory to predict the grain-level responses in FCC material. Based on the crystal plasticity, the shear strain rate of slip system is identified as the key feature, and the physical constitutive equations of crystal plasticity are encoded into the loss function. A data augmentation considering the slip shear direction enables the model to learn the reverse loading in constitutive relations. The introduced physics-constraints accelerate neural network model convergence and promotes prediction accuracy, especially for small-scale dataset. The transfer learning is performed on the model by leveraging the constitutive equations learned from the base dataset with linear biaxial loading to complex strain paths with a small-scale extended dataset. This approach significantly reduces the requirement of data quantity and accurately captures the complex in-plane deformation of crystals with any initial orientations, including cyclic loading and arbitrary non-monotonic loading.

CCCS: Contrastive Cross-Language Code Search Using Code Graph Information

Developers often search and reuse existing code snippets to improve software development efficiency during software development. Currently, researchers have proposed many code search methods. However, the search intent of existing methods is basically a natural language query. In order to support code migration and code refactoring, it is necessary to search relevant code snippets of another programming language with code snippets of one programming language. In this paper, we propose a C ontrastive C ross-language C ode S earch method using code graph information, called CCCS . CCCS first converts code snippets into high-dimensional vectors using pre-trained CodeBERT to extract the sequence features of code snippets. Next, the structural features of code snippets are extracted using a graph convolutional neural network. Finally, the model is trained using the contrastive learning method to optimize the vector representation of cross-language code snippets, enabling the model to distinguish code snippets from different programming languages with the same functionality. To evaluate the effectiveness of our method, we conducted comparison experiments and ablation experiments on a small-scale dataset and a large-scale dataset, respectively. The experimental results show that our method far outperforms the state-of-the-art baseline model in terms of MRR metrics.

Self-supervised Learning for Expression Recognition on Small-scale Data Set

In this paper, we propose a facial micro-expression recognition method utilizing self-supervised learning and Vision Transformer. We employ a contrast learning approach for self-supervision and extract image features using an attention mechanism. Various data augmentation techniques were utilized, and we specifically designed an enhancement method for facial recognition. By combining the strengths of the Vision Transformer and CNN models for feature extraction, our approach achieves improved recognition accuracy, even with limited labeled data. Experimental evaluation shows that our proposed method has good results in facial micro-expression recognition tasks.

Polarimetric image denoising on small datasets using deep transfer learning

Although deep learning-based methods have achieved great success in various polarimetric imaging tasks, the performance and the generalization ability are strongly dependent on massive training data, which is a critical limitation and limits the practical applications. In this paper, for the first time to our knowledge, we present a deep transfer learning-based solution for polarimetric image denoising. This solution performs the transfer learning by fine-tuning a denoising model pre-trained on a large-scale color image dataset and using a small-scale polarimetric dataset. The experimental results show that, based on a small-scale dataset, the proposed network can achieve almost the same denoising performance as that with a large-scale dataset. The polarization parameters, i.e., the degree of polarization and the angle of polarization, can be reconstructed simultaneously. In addition, serials of experiments demonstrate the generalization ability of the method for different materials and noise levels.

GcForest-based compound-protein interaction prediction model and its application in discovering small-molecule drugs targeting CD47.

Identifying compound-protein interaction plays a vital role in drug discovery. Artificial intelligence (AI), especially machine learning (ML) and deep learning (DL) algorithms, are playing increasingly important roles in compound-protein interaction (CPI) prediction. However, ML relies on learning from large sample data. And the CPI for specific target often has a small amount of data available. To overcome the dilemma, we propose a virtual screening model, in which word2vec is used as an embedding tool to generate low-dimensional vectors of SMILES of compounds and amino acid sequences of proteins, and the modified multi-grained cascade forest based gcForest is used as the classifier. This proposed method is capable of constructing a model from raw data, adjusting model complexity according to the scale of datasets, especially for small scale datasets, and is robust with few hyper-parameters and without over-fitting. We found that the proposed model is superior to other CPI prediction models and performs well on the constructed challenging dataset. We finally predicted 2 new inhibitors for clusters of differentiation 47(CD47) which has few known inhibitors. The IC50s of enzyme activities of these 2 new small molecular inhibitors targeting CD47-SIRPα interaction are 3.57 and 4.79μM respectively. These results fully demonstrate the competence of this concise but efficient tool for CPI prediction.

Explainable few-shot learning with visual explanations on a low resource pneumonia dataset

The data hungry nature of neural networks acts as a barrier to their application in data-scarce scenarios like medical domain. Few-shot learning has attracted much interest recently in handling small-scale datasets. Along with the need to handle the issue of limited annotated data in each class, we also need to address the black-box nature of neural nets which hampers their application in risk-sensitive areas. Pneumonia identification in low resource scenarios along with explanations is a challenging task. Therefore, in this paper, we reveal a few-shot learning-based pneumonia classification technique along with deriving visual explanations to gain insights into the model’s decision-making process. Interpretation for models’ decisions is generated by computing the importance scores based on a comparison between the values of activated neurons and the reference values. We achieve a classification accuracy of 87% and 82% for covid-chestxray-dataset and RSNA Pneumonia dataset respectively, thereby utilising 20 images per category for our experiments. Evaluation of importance scores and the change in log odds scores obtained by image morphing task demonstrate that the proposed framework outperformed the other popular gradient-based methods.

Ensemble extended belief rule-based systems with different similarity measures for classification problems

The extended belief rule-based (EBRB) system is shown to have the potential to handle both quantitative and qualitative information under uncertainty, and it has been used as an effective tool for decision support and classification problems. However, despite these advances, several drawbacks have emerged recently, and the most significant one is caused by its similarity measure using Euclidean distance, which could lead to counterintuitive individual matching degrees, while other widely used similarity measures have not been studied for their application in the EBRB system. To this end, seventeen similarity measures are investigated and applied in the EBRB system in this paper, and based on the analysis, an ensemble method for EBRB system with different similarity measures is proposed. Firstly, the problem of the similarity measure of the conventional EBRB system is investigated. Then, a variety of similarity measures are analyzed and their application in the EBRB system is studied. Next, the ensemble method for EBRB systems with different similarity measures is proposed, which consists of two parts, the adaptive weight learning method for determining the weight of each EBRB system with different similarity measures, and the evidential reasoning (ER)-based combination method for combining the inferential results of different methods. Finally, 25 classification datasets are studied to test the performance of EBRB systems with different similarity measures as well as the proposed method, and the results are compared with existing works. The comparison results show that the proposed method could not only achieve better results than any other EBRB systems with different similarity measures, but also outperform other conventional classifiers on some classification datasets, especially small-scale datasets.

Acoustic bird species classification under low SNR and small-scale dataset conditions

Bird species classification based on acoustic monitoring has attracted increasing attention. Two common methods, deep learning and feature extraction-based, are widely applied for bird sound classification. Although deep learning based methods have shown great success, they usually require an enormous amount of labeled data to train models, which could make these approaches prohibitive since collecting enough labeled samples of various species, especially rare birds, is a difficult task. On the other hand, feature extraction based methods are less dependent on the number of samples and have achieved remarkable classification performance for small-scale datasets, however they prove susceptible to signal-to-noise ratio (SNR). To address this problem, a new method is proposed for acoustic bird species classification in this work, which is well suited to low SNR and small-scale dataset conditions. Firstly, an adaptive threshold scheme based on the constant false alarm rate (CFAR) criterion is designed to more effectively detect potential bird sound segments in field recordings. Then, the local wavelet acoustic pattern (LWAP) and Mel-frequency cepstral coefficients (MFCC) features are extracted, which are further processed using improved vector of locally aggregated descriptors (VLAD) encoding. Finally, encoded LWAP and MFCC features are fused with dimensionality reduction and a classifier with weighted distance is used for classification. Corresponding feature space analysis upon feature fusion is also provided demonstrating explicit improvement in bird sounds discrimination under low SNR conditions. Experimental results on the dataset consisting of 11 bird species under low SNR and small-scale conditions reveal that the proposed method achieves considerable improvement in classification performance as compared with other feature extraction based methods. Meanwhile, our approach also outperforms state-of-the-art deep learning based methods.

3D Face Recognition: Two Decades of Progress and Prospects

Three-dimensional (3D) face recognition has been extensively investigated in the last two decades due to its wide range of applications in many areas, such as security and forensics. Numerous methods have been proposed to deal with the challenges faced by 3D face recognition, such as facial expressions, pose variations, and occlusions. These methods have achieved superior performances on several small-scale datasets, including FRGC v2.0, Bosphorus, BU-3DFE, and Gavab. However, deep learning–based 3D face recognition methods are still in their infancy due to the lack of large-scale 3D face datasets. To stimulate future research in this area, we present a comprehensive review of the progress achieved by both traditional and deep learning–based 3D face recognition methods in the last two decades. Comparative results on several publicly available datasets under different challenges of facial expressions, pose variations, and occlusions are also presented.