Small Datasets Research Articles

Analysis of small three-dimensional fluorescence spectroscopy dataset using migration learning an example of phenol concentration prediction in wastewater

Three-dimensional fluorescence spectroscopy has been widely used to detect organic pollutants in water. However, the amount of data required for three-dimensional fluorescence spectroscopy analysis is relatively large, and the time cost of sample collection is high. The amount of data has become an unavoidable limitation of spectral analysis. This study takes the detection of phenol in industrial discharge wastewater as an example and proposes a transfer learning method for small fluorescence spectroscopy datasets. First, fluorescence spectra are generated by splitting them into linear combinations of positively and negatively distributed spectra. Then, based on the idea of transfer learning, the generated fluorescence spectra are used to train a task-specific pre-trained model, which is then transferred to the collected spectral dataset. Experimental results show that the prediction performance of the transfer learning method is improved by 50.08 % compared with that obtained by directly training the model using a small amount of spectral data. In addition, when the spectral data remains unchanged, the accuracy of the model can be improved to a certain extent by increasing the amount of spectral data used for pre-training. The transfer learning method proposed in this study further improves the prediction accuracy when data is limited, and the results of verification in real environments are also satisfactory. It provides a feasible solution to the problem of data limitations in three-dimensional fluorescence spectroscopy.

Real-time disease detection on bean leaves from a small image dataset using data augmentation and deep learning methods

Real-time disease detection on bean leaves from a small image dataset using data augmentation and deep learning methods

Prediction of CO2 emissions to achieve net-zero objectives in the iron and steel sectors of North America

ABSTRACT Predictive models are widely used to create effective plans for reducing CO2 emissions in manufacturing. The Iron and Steel (I&S) industries play a crucial role in meeting international commitments to achieve Net-zero emissions by 2050. The objective of this study is to forecast carbon dioxide emissions from the I&S industries in North America through the utilization of a Multi-Objective Mathematical model. The proposed data-driven approach is integrated with various machine learning algorithms capable of accurately predicting future values with a small dataset. Additionally, sensitivity analyses under different scenarios are conducted to evaluate the impact of implementing proposed solutions by the research community. Results show a significant improvement in accuracy through the employment of the Whale Optimization Algorithm (WOA). Forecasts reveal a sustained increment of 0.7 MtCO2 every year spanning between 2022 and 2050. This study provides valuable information for stakeholders and policymakers as it allows a more precise evaluation to integrate new technologies to abate forthcoming CO2 emissions.

Neural network–based transfer learning to improve stiffness modeling of industrial robots with small experimental data sets

AbstractStiffness modeling is an essential subject for the composition of robot control. Accurate stiffness modeling is helpful for improving the control accuracy of industrial robots, particularly under dynamic load circumstances. The classic virtual joint modeling (VJM) method is challenging in predicting the deformation of the end-effector throughout the full workspace due to the nonlinear deformation of the robot joint and its serial articulated structure. This paper proposes a full-space stiffness modeling method for robots based on the integration of a multi-layer perceptual (MLP) model and VJM. To provide enough training data for the MLP model, VJM is used to build a stiffness model with a small set of experimental data to generate 106,400 training data. A model-based transfer learning approach is proposed to improve the model’s accuracy and generalization regarding the difference between generated training data and actual experimental data. The VJM stiffness model is compared with the MLP stiffness model and the existing CNN-based transfer learning model based on the same experimental data. Considering the deformation prediction in the three directions in Cartesian space, the mean absolute error, standard deviation, and maximum error of the MLP model are decreased by at least 24.90%, 14.20%, and 8.50%, respectively, than the VJM. These prediction results demonstrate that the proposed modeling technique can significantly increase the accuracy of robot stiffness modeling, which is essential for position compensation in precise motion control of robots under dynamic load.

Density interface prediction using neural networks: Incorporating fluctuation features from observed gravity data

Predicting density interfaces in the subsurface using neural networks is a promising research area in deep learning-based gravity exploration. Constructing a comprehensive training dataset that captures the fluctuation characteristics of the density interface is vital for predictions. However, large datasets often introduce extraneous features and significantly increase the computational time. To address these challenges, we have developed a method that enhances the accuracy of the prediction by integrating fluctuation features of observed gravity data, specifically the z-component gravity data, into the training dataset. This is achieved by perturbing and deforming the approximate spectrum of the density interface derived from a physics-driven algorithm, thereby generating training samples that closely match the prediction target. Such an approach enables the neural network to concentrate on learning relevant fluctuation features associated with the target. Experimental results underscore the efficacy of our method in accurately discerning density interface features and its robustness against noise interference. Furthermore, our method maintains high prediction accuracy with smaller datasets, outperforming the conventional approaches. This advantage enhances the efficiency and practicality of neural networks in addressing density interface problems by reducing the computational costs for predictions with high accuracy.

GeM-LR: Discovering predictive biomarkers for small datasets in vaccine studies.

Despite significant progress in vaccine research, the level of protection provided by vaccination can vary significantly across individuals. As a result, understanding immunologic variation across individuals in response to vaccination is important for developing next-generation efficacious vaccines. Accurate outcome prediction and identification of predictive biomarkers would represent a significant step towards this goal. Moreover, in early phase vaccine clinical trials, small datasets are prevalent, raising the need and challenge of building a robust and explainable prediction model that can reveal heterogeneity in small datasets. We propose a new model named Generative Mixture of Logistic Regression (GeM-LR), which combines characteristics of both a generative and a discriminative model. In addition, we propose a set of model selection strategies to enhance the robustness and interpretability of the model. GeM-LR extends a linear classifier to a non-linear classifier without losing interpretability and empowers the notion of predictive clustering for characterizing data heterogeneity in connection with the outcome variable. We demonstrate the strengths and utility of GeM-LR by applying it to data from several studies. GeM-LR achieves better prediction results than other popular methods while providing interpretations at different levels.

Evaluating the Impact of Retinal Vessel Segmentation Metrics on Retest Reliability in a Clinical Setting: A Comparative Analysis Using AutoMorph.

Current research on artificial intelligence-based fundus photography biomarkers has demonstrated inconsistent results. Consequently, we aimed to evaluate and predict the test-retest reliability of retinal parameters extracted from fundus photography. Two groups of patients were recruited for the study: an intervisit group (n = 28) to assess retest reliability over a period of 1 to 14days and an intravisit group (n = 44) to evaluate retest reliability within a single session. Using AutoMorph, we generated test and retest vessel segmentation maps; measured segmentation map agreement via accuracy, sensitivity, F1 score and Jaccard index; and calculated 76 metrics from each fundus image. The retest reliability of each metric was analyzed in terms of the Spearman correlation coefficient, intraclass correlation coefficient (ICC), and relative percentage change. A linear model with the input variables contrast-to-noise-ratio and fractal dimension, chosen by a P-value-based backward selection process, was developed to predict the median percentage difference on retest per image based on image-quality metrics. This model was trained on the intravisit dataset and validated using the intervisit dataset. In the intervisit group, retest reliability varied between Spearman correlation coefficients of 0.34 and 0.99, ICC values of 0.31 to 0.99, and mean absolute percentage differences of 0.96% to 223.67%. Similarly, in the intravisit group, the retest reliability ranged from Spearman correlation coefficients of 0.55 and 0.96, ICC values of 0.40 to 0.97, and mean percentage differences of 0.49% to 371.23%. Segmentation map accuracy between test and retest never dropped below 97%; the mean F1 scores were 0.85 for the intravisit dataset and 0.82 for the intervisit dataset. The best retest was achieved with disc-width regarding the Spearman correlation coefficient in both datasets. In terms of the Spearman correlation coefficient, the worst retests of the intervisit and intravisit groups were tortuosity density and artery tortuosity density, respectively. The intravisit group exhibited better retest reliability than the intervisit group (P < 0.001). Our linear model, with the two independent variables contrast-to-noise ratio and fractal dimension predicted the median retest reliability per image on its validation dataset, the intervisit group, with an R2 of 0.53 (P < 0.001). Our findings highlight a considerable volatility in the reliability of some retinal biomarkers. Improving retest could allow disease progression modeling in smaller datasets or an individualized treatment approach. Image quality is moderately predictive of retest reliability, and further work is warranted to understand the reasons behind our observations better and thus ensure consistent retest results.

Part-of-Speech (POS) Tagging of Low-Resource Language (Limbu) with Deep learning

POS tagging is a basic Natural Language Processing (NLP) task that tags the words in an input text according to its grammatical values. Although POS Tagging is a fundamental application for very resourced languages, such as Limbu, is still unknown due to only few tagged datasets and linguistic resources. This research project uses deep learning techniques, transfer learning, and the BiLSTM-CRF model to develop an accurate POS-tagging system for the Limbu language. Using annotated and unannotated language data, we progress in achieving a small yet informative dataset of Limbu text. Skilled multilingual tutoring was modified to enhance success on low-resource language tests. The model as propose attains 90% accuracy, which is very much better than traditional rule-based and machine learning methods for Limbu POS tagging. The results indicate that deep learning methods can address linguistic issues facing low-resource languages even with limited data. In turn, this study provides a cornerstone for follow up NLP-based applications of Limbu and similar low-resource languages, demonstrating how deep learning can fill the gap where data is scarce.

Assessing Project Resilience Through Reference Class Forecasting and Radial Basis Function Neural Network

A project needs to be able to anticipate potential threats, respond effectively to adverse events, and adapt to environmental changes. This overall capability is known as project resilience. In order to make efficient project decisions when the project is subjected to disruption, such as adjusting the project budget, reformulating the work plan, and rationalizing the allocation of resources, it is necessary to quantitatively understand the level of project resilience. Therefore, this paper develops a novel approach for forecasting project performance, illustrating the changes in performance levels during the disruption and recovery phases of a project and thus quantitatively assessing project resilience. While there are several methods for assessing project resilience in existing research, the majority of assessment approaches originate from within projects and are highly subjective, which makes it difficult to objectively reflect the level of project resilience. Moreover, the availability of project samples is limited, which makes it difficult to forecast the level of project performance. In view of the fact that the Reference Class Forecasting (RCF) technique avoids subjectivity and the Radial Basis Function (RBF) neural network is known to be better at forecasting small sample datasets, this paper therefore combines the RCF technique and the RBF neural network to construct a model that forecasts the project performance of the current project after experiencing a disruption, further assessing the level of the project resilience. Specifically, this paper first presents a conceptual model of project resilience assessment; subsequently, an RBF neural network model that takes into account project budget, duration, risk level of disruption, and performance before disruption based on the RCF technique is developed to forecast project performance after experiencing disruption; and finally, the level of project resilience is assessed through calculating the ratio of recovery to loss of project performance. The model is trained and validated using 64 completed construction projects with disruptions as the datasets. The results show that the average relative errors between the forecast results of schedule performance index (SPI) and the real values are less than 5%, and the R2 of the training set and the testing set is 0.991 and 0.964, respectively, and the discrepancy between the forecasted and real values of project resilience is less than 10%. These illustrate that the model performs well and is feasible for quantifying the level of project resilience, clarifying its impact on project disruption and recovery situations, and facilitating the decision-makers of the project to make reasonable decisions.

Predicting and Accelerating Nanomaterial Synthesis Using Machine Learning Featurization.

Materials synthesis optimization is constrained by serial feedback processes that rely on manual tools and intuition across multiple siloed modes of characterization. We automate and generalize feature extraction of reflection high-energy electron diffraction (RHEED) data with machine learning to establish quantitatively predictive relationships in small sets (∼10) of expert-labeled data, saving significant time on subsequently grown samples. These predictive relationships are evaluated in a representative material system (W1-xVxSe2 on c-plane sapphire (0001)) with two aims: 1) predicting grain alignment of the deposited film using pregrowth substrate data and 2) estimating vanadium dopant concentration using in situ RHEED as a proxy for ex situ methods (e.g., X-ray photoelectron spectroscopy). Both tasks are accomplished using the same materials-agnostic features, avoiding specific system retraining and leading to a potential 80% time saving over a 100-sample synthesis campaign. These predictions provide guidance to avoid doomed trials, reduce follow-on characterization, and improve control resolution for materials synthesis.

Tomato Fungal Disease Diagnosis Using Few-Shot Learning Based on Deep Feature Extraction and Cosine Similarity

Tomato fungal diseases can cause significant economic losses to farmers. Advanced disease detection methods based on symptom recognition in images face challenges when identifying fungal diseases in tomatoes, especially with limited training images. This study utilized novel techniques designed for limited data scenarios, such as one-shot and few-shot learning, to identify three tomato fungal diseases, i.e., Alternaria solani, Alternaria alternata, and Botrytis cinerea. Automated feature extraction was performed using the ResNet-12 deep model, and a cosine similarity approach was employed during shot learning. The accuracy of diagnosing the three diseases and healthy leaves using the 4-way 1-shot learning method was 91.64, 92.37, 92.93, and 100%. For the 4-way 3-shot learning method, the accuracy improved to 92.75, 95.07, 96.63, and 100%, respectively. These results demonstrate that the proposed method effectively reduces the dependence on experts labeling images, working well with small datasets and enhancing plant disease identification.

Data-driven pressure prediction for self-pressurization liquid hydrogen tank using transfer learning method

Prediction of pressure evolution in liquid hydrogen (LH2) tanks utilizing data-driven technology has gradually garnered significant attention. However, training an accurate data-driven model for pressure prediction in LH2 tanks is a challenge due to inadequate experimental data. To end this, this paper presents a Backpropagation Neural Network model for pressure prediction in LH2 tanks based on transfer learning on pre-trained models, using experimental data and results from thermodynamics models. Pre-trained models are firstly optimized by Bayesian optimization algorithms with large sample datasets from the thermal multi-zone model program and then performed the fine-tuning method with small sample datasets from experiments. The effectiveness of the proposed approach is validated by experimental data and numerical results. Compared with baseline models trained only with experimental data, the results demonstrate better model performance, with a maximum of 10.51% accuracy improvement. The proposed approach demonstrates considerable potential in facilitating the application of advanced machine learning algorithms for LH2 tank pressure prediction, significantly reducing the dependence on experimental data collection.

Machine learning-based EEG analysis for enhanced schizophrenia classification and diagnosis

Abstract. Schizophrenia can have a significant impact on patients' lives, studies, and work. Patients may experience delusions, hallucinations, disorganized thinking, and abnormal behavior, among other symptoms. Therefore, research related to schizophrenia is of great importance. This paper proposes the design of a classifier based on machine learning for diagnosing schizophrenia. The classifier extracts N100 features, P300 features, and power features across different frequency bands from both time-domain and frequency-domain characteristics from Electroencephalography (EEG) signals. Given the small dataset, a Support Vector Machine (SVM) machine learning algorithm was chosen to process and classify the selected features. To address the limitations of SVM in handling high-dimensional data and nonlinear problems, this study introduces a comprehensive improvement method based on Bayesian optimization, Recursive Feature Elimination (RFE), and data augmentation. Bayesian optimization was used to find the best combination of hyperparameters for the model, thereby improving its performance; RFE was employed to assess feature importance and remove the least important features, enhancing the model's training efficiency and generalization capability. Additionally, data augmentation was included to increase the sample size and introduce diversity, thereby improving the model's robustness. The study found that these methods effectively improved classification accuracy and generalization ability.

Transfer Learning Approaches for Brain Metastases Screenings

Background: In this study, we examined the effectiveness of transfer learning in improving automatic segmentation of brain metastases on magnetic resonance imaging scans, with potential applications in preventive exams and remote diagnostics. Methods: We trained three deep learning models on a public dataset from the ASNR-MICCAI Brain Metastasis Challenge 2024, fine-tuned them on a small private dataset, and compared their performance to models trained from scratch. Results: Results showed that models using transfer learning performed better than scratch-trained models, though the improvement was not statistically substantial. The custom Tversky and Binary Cross-Entropy loss function helped manage class imbalance and reduce false negatives, limiting missed tumor regions. Medical experts noted that, while fine-tuned models worked well with larger, well-defined tumors, they struggled with tiny, scattered tumors in complex cases. Conclusions: This study highlights the potential of transfer learning and tailored loss functions in medical imaging, while also pointing out the models’ limitations in detecting very small tumors in challenging cases.

SFFNet: Shallow Feature Fusion Network Based on Detection Framework for Infrared Small Target Detection

Infrared small target detection (IRSTD) is the process of recognizing and distinguishing small targets from infrared images that are obstructed by crowded backgrounds. This technique is used in various areas, including ground monitoring, flight navigation, and so on. However, due to complex backgrounds and the loss of information in deep networks, infrared small target detection remains a difficult undertaking. To solve the above problems, we present a shallow feature fusion network (SFFNet) based on detection framework. Specifically, we design the shallow-layer-guided feature enhancement (SLGFE) module, which guides multi-scale feature fusion with shallow layer information, effectively mitigating the loss of information in deep networks. Then, we design the visual-Mamba-based global information extension (VMamba-GIE) module, which leverages a multi-branch structure combining the capability of convolutional layers to extract features in local space with the advantages of state space models in the exploration of long-distance information. The design significantly extends the network’s capacity to acquire global contextual information, enhancing its capability to handle complex backgrounds. And through the effective fusion of the SLGFE and VMamba-GIE modules, the exorbitant computation brought by the SLGFE module is substantially reduced. The experimental results on two publicly available infrared small target datasets demonstrate that the SFFNet surpasses other state-of-the-art algorithms.

Application of Prompt Learning Models in Identifying the Collaborative Problem Solving Skills in an Online Task

Collaborative problem solving (CPS) competence is considered one of the essential 21st-century skills. To facilitate the assessment and learning of CPS competence, researchers have proposed a series of frameworks to conceptualize CPS and explored ways to make sense of the complex processes involved in collaborative problem solving. However, encoding explicit behaviors into subskills within the frameworks of CPS skills is still a challenging task. Traditional studies have relied on manual coding to decipher behavioral data for CPS, but such coding methods can be very time-consuming and cannot support real-time analyses. Scholars have begun to explore approaches for constructing automatic coding models. Nevertheless, the existing models built using machine learning or deep learning techniques depend on a large amount of training data and have relatively low accuracy. To address these problems, this paper proposes a prompt-based learning pre-trained model. The model can achieve high performance even with limited training data. In this study, three experiments were conducted, and the results showed that our model not only produced the highest accuracy, macro F1 score, and kappa values on large training sets, but also performed the best on small training sets of the CPS behavioral data. The application of the proposed prompt-based learning pre-trained model contributes to the CPS skills coding task and can also be used for other CSCW coding tasks to replace manual coding.

Adaptive AI-based surrogate modelling via transfer learning for DEM simulation of multi-component segregation

Segregation of granular materials is a critical challenge in many industries, often aimed at being controlled or minimised. The discrete element method (DEM) offers valuable insights into this phenomenon. However, calibrating DEM models is a crucial, albeit time-consuming, step. Recently, using machine learning (ML)-based surrogate models (SMs) in the calibration process has emerged as a promising solution. Nevertheless, developing such SMs is challenging due to the high number of DEM simulations required for training. Additionally, choosing a suitable ML model is not trivial. This study aims to develop SMs that effectively link particle-particle and particle-wall DEM interaction parameters to segregation of a multi-component mixture. We evaluate several ML models, ranging from artificial neural networks to ensemble learning, that are trained on a very cost-effective dataset, employing Bayesian optimisation with cross-validation to tune their hyperparameters. Next, we introduce a novel transfer learning (TL)-based approach that leverages knowledge from a few scenarios to handle new “unseen” ones. This method enables the construction of adaptive SMs for unseen scenarios, such as a new initial configuration (IC) of granular mixtures, without the need for a full-sized dataset. Our findings indicate that Gaussian process regression (GPR) efficiently builds accurate SMs on a very small dataset. We also demonstrate that only a few samples are required to build an accurate SM for the unseen IC, which significantly reduces the data preparation burden. By incorporating one and five samples from unseen scenarios to update the TL-GPR-based surrogate model, the SM’s performance (based on \\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\:{R}^{2}$$\\end{document}) on unseen scenarios improves by 17 and 47%, respectively. The insights and methodology presented in this study will facilitate and accelerate the development of accurate SMs for DEM calibration, assisting in developing reliable DEM models in a shorter timeframe.

Classification of salivary gland biopsies in Sjögren's syndrome by a convolutional neural network using an auto-machine learning platform.

The histopathological analysis of minor salivary gland biopsies, particularly through the quantification of the Focus Score (FS), is pivotal in the diagnostic workflow for Sjögren's Syndrome (SS). AI-based image recognition using deep learning models has demonstrated potential in enhancing diagnostic accuracy and efficiency in preclinical research. The primary aim of this investigation was to utilize an auto-machine learning (autoML) platform for the automated segmentation and quantification of FS on histopathological slides, aiming to augment diagnostic precision and speed in SS. A cohort comprising 86 patients with sicca syndrome (37 diagnosed with SS based on the 2016 ACR/EULAR Classification Criteria and 49 non-SS) was selected for an in-depth histological examination. A repository of 172 slides (two per patient) was assembled, encompassing 74 slides meeting the classificatory thresholds for SS (FS ≥ 1, indicative of lymphocytic infiltration) and 98 slides showcasing normal salivary gland histology. The autoML platform utilized (Giotto, L2F, Lausanne Switzerland) employed a Convolutional Neural Network (CNN) architecture (ResNet-152) for the training and validation phases, using a dataset of 172 slides. The developed model exhibited a reliability score of 0.88, proficiently distinguishing SS cases, with a sensitivity of 89.47% (95% CI: 66.86% to 98.70%) and a specificity of 88.24% (95% CI: 63.56% to 98.54%). The model found histological slides of suboptimal quality (e.g., those compromised during fixation or staining processes) to be the most challenging for accurate classification. AutoML platforms offer a rapid and flexible approach to developing machine learning models, even with smaller datasets, as demonstrated in this study for SS. These platforms hold significant potential for enhancing diagnostic precision and efficiency in both clinical and research settings. Multicentric studies with larger patient cohorts are essential for thorough evaluation and validation of this innovative diagnostic approach.

Establishing an AI model and application for automated capsule endoscopy recognition based on convolutional neural networks (with video).

Although capsule endoscopy (CE) is a crucial tool for diagnosing small bowel diseases, the need to process a vast number of images imposes a significant workload on physicians, leading to a high risk of missed diagnoses. This study aims to develop an artificial intelligence (AI) model and application based on convolutional neural networks that can automatically recognize various lesions in small bowel capsule endoscopy. Three small bowel capsule endoscopy datasets were used for AI model training, validation, and testing, encompassing 12 categories of images. The model's performance was evaluated using metrics such as AUC, sensitivity, specificity, precision, accuracy, and F1 score to select the best model. A human-machine comparison experiment was conducted using the best model and endoscopists with varying levels of experience. Model interpretability was analyzed using Grad-CAM and SHAP techniques. Finally, a clinical application was developed based on the best model using PyQt5 technology. A total of 34,303 images were included in this study. The best model, MobileNetv3-large, achieved a weighted average sensitivity of 87.17%, specificity of 98.77%, and an AUC of 0.9897 across all categories. The application developed based on this model performed exceptionally well in comparison with endoscopists, achieving an accuracy of 87.17% and a processing speed of 75.04 frames per second, surpassing endoscopists of varying experience levels. The AI model and application developed based on convolutional neural networks can quickly and accurately identify 12 types of small bowel lesions. With its high sensitivity, this system can effectively assist physicians in interpreting small bowel capsule endoscopy images.Future studies will validate the AI system for video evaluations and real-world clinical integration.

Multi-Objective Optimization for Controlling Conflicts in Roadway Surrounding Rock Induced by Floor Stress-Relief Groove

This paper studies the effectiveness of the stress-relief groove on the floor of deep coal roadway and determines the influence of the stress-relief groove parameters on the surrounding rock through qualitative analysis. Based on the displacement conflict problem, evaluation indicators were established, and the optimal solution set was obtained. The innovations of this research include: 1. For geotechnical numerical simulations, novel stress monitoring and plastic zone monitoring techniques have been introduced to accurately reflect the condition of the surrounding rock; 2. The effects of floor relief grooves in deep roadway on surrounding rock have been analyzed, and the advantages and utilities of central and corner relief grooves have been determined; 3. The usability of small datasets has been enhanced by applying SEGA to optimize machine learning models with data augmentation techniques; 4. Multi-objective optimization algorithms have been applied to geotechnical engineering, providing valuable references for decision making. The results demonstrate that multi-objective optimization can significantly enhance the effectiveness of surrounding rock control, resolve conflicts, and achieve more reasonable construction plans. This research provides new theoretical foundations and practical guidance for deep mine roadway-surrounding rock control.